Name		Name	Last commit message	Last commit date
parent directory ..
README.md		README.md
demo_es.ipynb		demo_es.ipynb
documents-with-ids.json		documents-with-ids.json
documents.json		documents.json
evaluate-text.ipynb		evaluate-text.ipynb
evaluate-vector.ipynb		evaluate-vector.ipynb
evaluation-metrics.md		evaluation-metrics.md
ground-truth-data.csv		ground-truth-data.csv
ground-truth-data.ipynb		ground-truth-data.ipynb
homework-3.ipynb		homework-3.ipynb
homework.md		homework.md
results.bin		results.bin

README.md

Vector Search

3.1 Introduction to Vector Search

Slides

3.2 Semantic Search with Elasticsearch

Notebook: demo_es.ipynb

3.2.2

Video

3.3 Evaluating Retrieval

3.3.1 Introduction

Plan for the section:

Why do we need evaluation
Evaluation metrics
Ground truth / gold standard data
Generating ground truth with LLM
Evaluating the search resuls

3.3.2 Getting ground truth data

Approaches for getting evaluation data
Using OpenAI to generate evaluation data

3.3.3 Ranking evaluation: text search

Elasticsearch with text results
minsearch

3.3.4 Ranking evaluation: vector search

Elasticsearch with vector search
Ranking with question, answer, question+answer embeddings

Homework

See here

Notes

Did you take notes? Add them above this line (Send a PR with links to your notes)