Obiettivi formativi
This course offers a rigorous and practical introduction to modern Natural Language Processing (NLP), from text preprocessing and tokenization through word embeddings, neural sequence models, the Transformer architecture, and Large Language Models (LLMs). It is designed for students with a background in statistics: every method is connected to its probabilistic and linear-algebra foundations, and each theoretical block is paired with hands-on implementation in Python. By the end of the course students will be able to design, build, and critically evaluate end-to-end NLP systems, from classical baselines to fine-tuned and retrieval-augmented LLM applications.
Objectives
At the end of the course, students are expected to be able to:
Knowledge and understanding
understand the core concepts of NLP and how text is represented and modeled computationally;
master tokenization strategies — word, character, and subword (BPE, WordPiece, SentencePiece) — and their trade-offs;
understand word and contextual embeddings, and their distributional and geometric foundations;
understand the main neural architectures for text: feedforward networks, RNN/LSTM/GRU, and the Transformer;
understand how Large Language Models are pretrained, adapted, aligned, and prompted;
understand agentic systems: tool use and function calling, the agent loop, and the design of an agent harness (orchestration, memory, control flow, guardrails).
Applying knowledge and understanding
build complete text-processing pipelines in Python (NLTK, spaCy, scikit-learn, Hugging Face);
train and evaluate text classifiers, from Naive Bayes and logistic regression to fine-tuned Transformers;
train and inspect word embeddings, and implement the attention mechanism from scratch;
fine-tune pretrained models (including parameter-efficient methods) and build retrieval-augmented (RAG) applications;
design and build a tool-using LLM agent with a minimal harness, and evaluate its behaviour.
Making judgements
select appropriate representations, models, and evaluation metrics for a given task and dataset;
critically assess model performance, including bias, robustness, and the limitations of generative systems, drawing on a sound statistical understanding of evaluation.
Communication skills
present and defend modeling choices and experimental results;
document experiments reproducibly and discuss conclusions with peers and stakeholders.
Learning skills
independently consult library and model documentation;
read and interpret foundational research papers;
adopt an iterative, experimental approach in which practice reinforces theory.
Programma
The 60 hours are delivered as 15 lessons of 4 hours, twice a week. Each lesson combines a theory block and a hands-on lab, separated by a short break. The course is organized in two blocks separated by a mid-term exam.
#
Topic
Theory block (≈ 2h)
Lab block (≈ 2h)
MODULE 1 — Foundations of Statistical NLP (Lessons 1–7 · 28 hours)
1
Introduction to NLP & working environment
What NLP is and why language is hard; ambiguity and structure; the NLP/LLM landscape and state of the art; language modeling as a probabilistic task (framing for statisticians).
Setting up Python, Jupyter/Colab, and core libraries (NLTK, spaCy, scikit-learn, Hugging Face); first text manipulation.
2
Text as data: preprocessing & normalization
Corpora, documents, tokens vs. types; Unicode, regex, case-folding, stopwords; stemming vs. lemmatization; part-of-speech tagging, sentence segmentation.
Building a reusable preprocessing pipeline with spaCy on a real corpus.
3
Tokenization: from words to subwords
Why subword tokenization (OOV, vocabulary size, morphology); Byte-Pair Encoding (BPE), WordPiece, Unigram/SentencePiece, byte-level BPE; vocabulary trade-offs.
Training a BPE tokenizer with the Hugging Face tokenizers library; comparing vocabularies and segmentations.
4
Vector space models: BoW, TF-IDF, n-grams
Document-term matrices; TF-IDF weighting; n-gram language models, smoothing and perplexity; sparse representations (links to frequency statistics).
Building TF-IDF representations and an n-gram language model on a text dataset.
5
Text classification with classical ML
Naive Bayes, logistic regression and linear SVM for text; the bias-variance view; evaluation: precision/recall/F1, confusion matrix, ROC, cross-validation.
End-to-end scikit-learn classification pipeline (e.g. sentiment / topic) with rigorous evaluation.
6
Word embeddings I: Word2Vec & friends
The distributional hypothesis; Word2Vec (CBOW, skip-gram), negative sampling; GloVe and FastText; cosine similarity and analogies.
Training and querying Word2Vec; exploring the embedding space via nearest neighbours.
7
Word embeddings II + Module 1 synthesis
Intrinsic vs. extrinsic evaluation of embeddings; bias in embeddings; using embeddings as features; consolidation and review for the midterm.
Using pretrained embeddings as features in a downstream classifier; guided review session.
— MID-TERM EXAM (prova intermedia) — break between the two blocks —
MODULE 2 — Neural NLP and Large Language Models (Lessons 8–15 · 32 hours)
8
Neural networks for NLP
From logistic regression to feedforward networks (GLM → MLP); loss functions, backpropagation, optimization (SGD, Adam), regularization; neural language models.
PyTorch fundamentals; building and training a feedforward text classifier.
9
Sequence models: RNN, LSTM, GRU
Modeling sequences and recurrence; vanishing/exploding gradients; LSTM and GRU gating; encoder-decoder (seq2seq).
Implementing an LSTM/GRU model for classification or language modeling.
10
Attention & the Transformer
Attention and self-attention (Query/Key/Value); scaled dot-product, multi-head attention, positional encoding; the full Transformer (links to linear algebra).
Implementing scaled dot-product attention from scratch; inspecting attention weights.
11
Encoder models: BERT & fine-tuning
Pretraining objectives (masked LM, NSP); BERT, RoBERTa, DistilBERT; transfer learning and fine-tuning for classification, NER, QA.
Fine-tuning a BERT model with Hugging Face Transformers on a downstream task.
12
Decoder models: GPT & text generation
Autoregressive language modeling; the GPT family; decoding strategies (greedy, beam, top-k, top-p, temperature); scaling laws and emergent abilities.
Text generation with a pretrained decoder model; comparing decoding strategies.
13
LLMs: alignment, adaptation & prompting
The LLM landscape (GPT, LLaMA, Mistral, …); instruction tuning and alignment (RLHF / DPO); parameter-efficient adaptation (LoRA / QLoRA); in-context learning, few-shot prompting, chain-of-thought.
Prompt engineering and structured prompting; a parameter-efficient (LoRA) fine-tune.
14
Grounding & tools: RAG and function calling
Embeddings for retrieval and vector databases; Retrieval-Augmented Generation; hallucination and grounding; tool use and function calling as the bridge from generation to action.
Building a RAG pipeline and adding tool use / function calling to an LLM.
15
Agentic systems, harnesses & evaluation
From a single call to an agent loop; ReAct (reason + act); harness design — context assembly, tool routing, memory, control flow, guardrails, error handling and observability; planning and multi-agent systems; agent and generation evaluation (BLEU, ROUGE, BERTScore, human eval); safety, bias and compute cost.
Building a tool-using agent with a minimal harness; agent evaluation; project consolidation and course wrap-up.
The course is held twice a week according to the academic calendar
Each lesson pairs a theory block with a hands-on lab separated by a break.
Students are required to participate actively; a laptop is needed for the labs.
Teaching support includes slides, notebooks (Jupyter / Google Colab), datasets, case studies, and exercises.
The course is organized in two blocks separated by a mid-term, marking the transition from statistical to neural / LLM-based NLP.
Modalità d'esame
The project work counts for at most 40% of the final grade. Two assessment schemes are available; the recommended option splits the written assessment into a mid-term and a final exam, aligned with the two blocks of the course.
Recommended scheme — with mid-term
Component
Weight
Coverage
Mid-term exam (prova intermedia)
30%
Lessons 1–7 (Module 1): preprocessing, tokenization, vector models, classical ML, embeddings.
Final exam (prova finale)
30%
Lessons 8–15 (Module 2): neural models, Transformer, BERT/GPT, LLMs, RAG, agentic systems and harnesses.
Project work
40%
Applied NLP project (notebook + report) with oral discussion of theory, methods, code, and results.
Alternative scheme — single final
Component
Weight
Coverage
Final exam (prova finale)
60%
Whole programme (Lessons 1–15).
Project work
40%
Applied NLP project with oral discussion.
Mid-term exam — format
The mid-term is an in-class written test held at the break between the two blocks (after Lesson 7) and covers Module 1 (Lessons 1–7). It is closed-device: no laptops, phones or internet.
It combines two question types:
closed questions (multiple choice) covering the breadth of the module: text preprocessing, tokenization, vector representations, classical models and word embeddings;
open questions: short quantitative problems (TF-IDF, n-gram perplexity with smoothing, precision / recall / F1 from a confusion matrix, BPE merges) and a short reasoning question on choosing and justifying a representation, model and evaluation metric for a given task.
To preserve integrity with a large cohort, several equivalent variants are used, differing in the numeric values of the problems and in the order of the options. Partial credit is awarded on the open questions for correct method.
The final exam combines theory questions with short practical tasks. The project work is carried out in small teams and discussed in an oral interview that probes each member on theory, methods, code and results.
Testi di riferimento
D. Jurafsky, J. H. Martin — Speech and Language Processing (3rd ed., online draft). Main reference text.
L. Tunstall, L. von Werra, T. Wolf — Natural Language Processing with Transformers, O’Reilly.
A. Vaswani et al. (2017) — Attention Is All You Need.
T. Mikolov et al. (2013) — Efficient Estimation of Word Representations in Vector Space (Word2Vec).
J. Pennington et al. (2014) — GloVe: Global Vectors for Word Representation.
R. Sennrich et al. (2016) — Neural Machine Translation of Rare Words with Subword Units (BPE).
J. Devlin et al. (2019) — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
T. Brown et al. (2020) — Language Models are Few-Shot Learners (GPT-3).
E. Hu et al. (2021) — LoRA: Low-Rank Adaptation of Large Language Models.
P. Lewis et al. (2020) — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.
S. Yao et al. (2023) — ReAct: Synergizing Reasoning and Acting in Language Models.
T. Schick et al. (2023) — Toolformer: Language Models Can Teach Themselves to Use Tools.
Hugging Face — The NLP Course (online). Plus selected arXiv papers shared during the lessons.
Codice insegnamento artificial-intelligence
