Skip to content

v0.29.0 — BM25 rescue lane was silenced, not down-weighted

Choose a tag to compare

@Lexus2016 Lexus2016 released this 20 Sep 11:03
· 84 commits to main since this release

Changed

  • The BM25 lane was silenced, not down-weighted, and is now audible.
    FTS_LANE_WEIGHT moves 0.3 -> 0.9. At 0.3 the lane could not surface a single
    row the vector lane had not already found — arithmetic, not corpus luck: with
    k=60 and fetch_limit=3x every vector-only RRF score sits in [1/90, 1/61]
    while an FTS-only hit at BM25 rank 1 scores w/61, so reaching position p
    needs w > (k+1)/(k+p) (0.87 for position 10, 0.97 for the top 3). At 0.3 an
    FTS-only row scored 0.0049 against a worst-case vector 0.0111 and could only
    reorder rows both lanes already shared.
    • Measured on 300 identifier queries over a real note corpus:
      Hit@1 35.3% -> 49.0%, Hit@5 46.3% -> 64.7%, MRR 0.402 -> 0.561.
    • Measured on 179 doc-block queries: unchanged (Hit@1 95.5%, MRR 0.973).
      0.9 is Pareto-dominant on both measured corpora — no regression found.
    • No migration, no re-embed, no schema change: applied at query time, so
      existing installs pick this up without reindexing. Result ORDER will change,
      which is the point.
    • Known limit, surfaced by an independent review before release: at w=0.9
      an FTS-only row can reach rank 8 at best, so a caller requesting fewer than
      8 results still won't see a BM25-only hit. This is strictly non-regressive
      versus 0.3 (which never surfaced one at any rank) — flagged as a target for
      a future confidence-scaled weight, not attempted in this release.

Added

  • scripts/build_paraphrase_fixture.py, scripts/sweep_ranking_constant.py,
    scripts/probe_rerank_headroom.py — the unbiased benchmark fixtures and
    sweep tooling used to calibrate the constants below.

Fixed

  • Constant documentation now records what was measured. VECTOR_GATE_THRESHOLD
    stays 0.82 (validated). RECENCY_BONUS_MAX stays 0.05 (peak of a unimodal
    curve; upside is noise, downside is real). PROVENANCE_HUMAN_BONUS is
    documented as UNCALIBRATABLE (898/905 active notes are agent-provenance).
    RRF k is documented as not a useful dial in this range.