Skip to content

Add Real-World Project: Build a Hybrid Search Demo #286

Description

@abderrahim-lectures

Scope

  • Lesson file: docs/projects/hybrid-search/index.md (per rag-notes skeleton: 5 steps, StepChecklist + Socratic questions per step, 6-provider table, opencode blurb, badges, ProjectProgressCheckbox).
  • Example folder: examples/hybrid-search/
    • main.py — CLI with BM25-style KeywordScorer (from scratch, stopword filtering), semantic_scores() (all-MiniLM-L6-v2 dot product, lazy model load), hybrid_scores() (per-query min-max normalize + alpha), rank(), retrieve_all(), run_evaluation(); --build, --evaluate, --top-k, --alpha, --reuse-index.
    • data/corpus/ — 11 committed .txt passages (neptune, telescope, espresso, coldbrew, deepsea, piano, tomatoes, cycling, sourdough, 2 paraphrases).
    • data/test_queries.json — 10 test queries (5 keyword exact-match, 5 semantic paraphrase) with expected, kind, note.
    • notebook.ipynb (14 cells), pyproject.toml, uv.lock, .gitignore (ignores .venv, __pycache__, *.pyc, data/index.npy), .python-version, README.md.
  • Registration: docs/projects/hybrid-search/_category_.json (position 23), plus first-entry inserts in src/data/projects.ts, docs/projects/index.mdx, and src/pages/index.tsx (tags: AI Agents, Retrieval-Augmented Generation, Developer Tools).
  • Badges/notebook: Colab/Kaggle/Binder badges point at main paths in the lesson.

Framing note

Teaches that no single retriever is best: BM25-style keyword matching wins on vocabulary-exact queries, embeddings win on paraphrase/semantic queries, and a hybrid alpha-weighted blend keeps both. No LLM and no API key — evaluation over the 10-query set shows keyword 5/10, semantic 10/10, hybrid 10/10, which makes the "keyword fails on paraphrase, hybrid recovers it" lesson concrete.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:capstoneCapstone project content, examples, or architectureprojectBacklog: a candidate Real-World Project ideatopic:ragRAG, vector search & knowledge graph projectstype:content

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions