Repository navigation
v2.1.5
What's Changed
Proper Hybrid Search with Reciprocal Rank Fusion (RRF)
Implemented industry-standard hybrid retrieval using RRF algorithm:
How it works:
- Stage 1: Run k-NN search (semantic similarity) and BM25 search (lexical matching) independently
- Stage 2: Fuse results using RRF:
score = dense_weight/(k+rank_dense) + sparse_weight/(k+rank_sparse) - Stage 3: Return top results sorted by fused score
Why RRF?
- Robust to score distribution differences between retrieval methods
- No need for score normalization (uses ranks, not raw scores)
- Industry standard (used by Elasticsearch, Cohere, etc.)
- Works with OpenSearch Serverless (no scripting required)
New methods:
text_search()- BM25 lexical search on text_contenthybrid_search()- RRF-fused k-NN + BM25
Parameters:
rrf_k=60- Standard RRF constant from literaturedense_weight=0.7- Weight for semantic resultssparse_weight=0.3- Weight for lexical results
Full Changelog: v2.1.4...v2.1.5