Skip to content

v1.0.0-beta.9

Choose a tag to compare

@KikeVen KikeVen released this 20 Jun 20:25
· 21 commits to main since this release

Add configurable FETCH_CAP for ChromaDB pre-rerank pool size

Replaces the hardcoded n_results=5 ChromaDB query limit with a configurable FETCH_CAP (default 75), allowing the lexical reranker to draw from a wider candidate pool before final results are selected.

Changes:

  • .env: add FETCH_CAP=75
  • config.py: load FETCH_CAP from environment
  • main.py: use FETCH_CAP for the initial ChromaDB n_results query; final result count remains capped at 5 post-rerank (relevant[:5]) as a separate, intentional safety limit on answer scope
  • todo.md: mark fetch_cap item complete

Behavior note: with FETCH_CAP=5 this is a no-op versus prior behavior (identical retrieval). The functional change only takes effect if FETCH_CAP is raised above 5, which widens the reranking candidate pool at the cost of additional ChromaDB/rerank compute per query.

Known follow-up: the post-rerank relevant[:5] cutoff is still hardcoded and should likely become its own named config value (e.g. FINAL_RESULT_COUNT), separate from FETCH_CAP, so both the pre-rerank pool size and the final answer scope are configurable rather than one knob exposed and one fixed.