Skip to content

ISE

Jump to bottom
Paolo Ferragina edited this page Oct 25, 2026 · 6 revisions

Instructor: Paolo Ferragina (paolo.ferragina@santannapisa.it)

Language: English

Duration: 20h, February-March 2027.

Room: They will be decided on an individual lecture basis. Lectures are strictly in presence.

Exam: A set of questions and exercises on the topics of the course

Description: The goal of this course is to introduce, with a light technical yet rigorous educational approach, the main methodologies, algorithms, and AI techniques that underlie the design of modern search engines and, more generally, Information Retrieval systems. The lectures will describe, study, and analyze the main components of a search engine: Crawler, Parser, Indexer, Query Resolver, Query and Document Annotator, and Results Ranker. We will also delve deeply into the novel Generative AI tools and how they contribute to shaping a new search task. The structure of the lectures will also allow us to talk about text mining and text analytics in general, thus stimulating multi-disciplinary discussions that pertain to other applications, not limited to (web) search.

Research opportunities on these topics: If interested, you can contact the teacher because there is the possibility of a single credit of research (1 CFU) developing AI-based tools for Software Heritage Archive or Health Science.

Materials:

Topics of the lectures [TO BE UPDATED]:

  • Topic 1 (Introduction, slides) [MRS: Chap 1, and Sect. 2.3-2.4]:

    • Modern IR, not just search engines!
    • Web search engine's structure.
    • A history lasting 30 years: from AltaVista to ChatGPT and Perplexity.

  • Topic 2 (Parsing, slides) [MRS: Sect 2.1, 2.2, 5.1, 19.2]

    • Sketch of crawling: properties of the Web graph (a note on the Bloom Filter).
    • Sketch of document parsing: tokenization, normalization, lemmatization, stemming, thesauri.
    • The Zipf, Heaps, and Luhn laws.

  • Topic 3 (Storage, slides LSH, slides IL and docs) [F: Chap 11, Sect 5.3, 19.6]:

    • Document deduplication (exact or approximate): Classic versus Locality-sensitive hashing (LSH).
    • LSH: basics, shingling & Jaccard similarity (min-hashing, with proof), Hamming distance (proof of the probability bounds), cosine similarity.
    • LSH: use in offline and online settings.
    • Posting list compression and its codes: gamma, variable bytes (t-nibble), Elias-Fano.
    • Document compression: LZ-parsing, delta-compression, compressing a group of files.

  • Topic 4 (Indexing) [MRS: Chap 3 and Sect 5.2]

    • The Inverted List (IL): dictionary + postings, building the IL.
    • The boolean retrieval model. How to implement AND, OR, NOT queries, skip pointers.
    • The IL with positional information: solving phrase queries and proximity queries. Zone indexes.
    • Sketch of how to search over the dictionary

  • Topic 5 (Content-based Ranking, slides) [sect 6.2, 6.3, 7.1, 8.1, 9.1]

    • Text-based ranking: TF-IDF.
    • Vector space model and cosine similarity doc-doc and query-doc. Sketch of fast top-k retrieval (high IDF, clustering).
    • Performance measures: precision, recall, F1, DCG, and NDCG.

  • Topic 6 (Network-based ranking, slides) [Chap 21].

    • Random Walks. Link-based ranking: PageRank.
    • Topic-specific PageRank. Personalized PageRank.

  • Topics 7 (Advanced topics, slide annotators, slide GenAI):

Clone this wiki locally