README.md

Vector Search

3.1 Introduction to Vector Search

• Slides

3.2 Semantic Search with Elasticsearch

• Notebook: demo_es.ipynb

3.2.2 Advanced Semantic Search

3.3 Evaluating Retrieval

3.3.1 Introduction

Plan for the section:

• Why do we need evaluation
• Evaluation metrics
• Ground truth / gold standard data
• Generating ground truth with LLM
• Evaluating the search resuls

3.3.2 Getting ground truth data

• Approaches for getting evaluation data
• Using OpenAI to generate evaluation data

Links:

• notebook
• documents with ids
• queries generated by OpenAI (pickle)
• ground truth dataset

3.3.3 Ranking evaluation: text search

• Elasticsearch with text results
• minsearch

Links:

• Notebook

3.3.4 Ranking evaluation: vector search

• Elasticsearch with vector search
• Ranking with question, answer, question+answer embeddings

Links:

• Notebook

Homework

See here

Notes

• Did you take notes? Add them above this line (Send a PR with links to your notes)