You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
v0.29.0 — BM25 rescue lane was silenced, not down-weighted
The BM25 lane was silenced, not down-weighted, and is now audible. FTS_LANE_WEIGHT moves 0.3 -> 0.9. At 0.3 the lane could not surface a single
row the vector lane had not already found — arithmetic, not corpus luck: with k=60 and fetch_limit=3x every vector-only RRF score sits in [1/90, 1/61]
while an FTS-only hit at BM25 rank 1 scores w/61, so reaching position p
needs w > (k+1)/(k+p) (0.87 for position 10, 0.97 for the top 3). At 0.3 an
FTS-only row scored 0.0049 against a worst-case vector 0.0111 and could only
reorder rows both lanes already shared.
Measured on 300 identifier queries over a real note corpus: Hit@1 35.3% -> 49.0%, Hit@5 46.3% -> 64.7%, MRR 0.402 -> 0.561.
Measured on 179 doc-block queries: unchanged (Hit@1 95.5%, MRR 0.973).
0.9 is Pareto-dominant on both measured corpora — no regression found.
No migration, no re-embed, no schema change: applied at query time, so
existing installs pick this up without reindexing. Result ORDER will change,
which is the point.
Known limit, surfaced by an independent review before release: at w=0.9
an FTS-only row can reach rank 8 at best, so a caller requesting fewer than
8 results still won't see a BM25-only hit. This is strictly non-regressive
versus 0.3 (which never surfaced one at any rank) — flagged as a target for
a future confidence-scaled weight, not attempted in this release.
Added
scripts/build_paraphrase_fixture.py, scripts/sweep_ranking_constant.py, scripts/probe_rerank_headroom.py — the unbiased benchmark fixtures and
sweep tooling used to calibrate the constants below.
Fixed
Constant documentation now records what was measured. VECTOR_GATE_THRESHOLD
stays 0.82 (validated). RECENCY_BONUS_MAX stays 0.05 (peak of a unimodal
curve; upside is noise, downside is real). PROVENANCE_HUMAN_BONUS is
documented as UNCALIBRATABLE (898/905 active notes are agent-provenance).
RRF k is documented as not a useful dial in this range.