Skip to content

v0.10.0 — Measured Retrieval

Choose a tag to compare

@Rushour0 Rushour0 released this 07 Jul 16:51

Turns memory retrieval from unmeasured into measured, gated, observable, and better by default.

Highlights

  • Offline retrieval eval + CI gate (M4)python -m fabri.benchmarks.retrieval_eval drives the real retrieval path over a labeled fixture (recall@k / MRR / precision@k), zero API credits; a pytest gate locks the shipped defaults at measured − 0.05.
  • Default flipped densehybrid (M5/D3) — eval-backed, degrades gracefully to dense where BM25 is unavailable, so never worse.
  • Two eval-driven quality fixes make hybrid the best strategy on every metric: RRF k 60→20 (memory.rrf_k; recall@3 0.60→0.90) and success_pattern slot back-load (recall@1 0.13→0.58, MRR 0.45→0.84 — relevance owns rank 1, the guarantee fills reserved tail slots).
  • BM25 FTS5 no-op fixed — space-join → implicit-AND meant sparse/hybrid silently ran as dense; found by the eval.
  • Retrieval-decision observability (M3) — one structured retrieval trace event per call (strategy, pool sizes, BM25-fired-or-fell-back, per-candidate inclusion_reason). Trace-only, zero prompt cost.
  • Memory-health section in fabri report (M6) across md/json/html, offline-safe.

Fixed

  • Root start event now emits before retrieval (nesting invariant).
  • Per-test Qdrant isolation kills the order-dependent CI flake.

New tuning knob memory.rrf_k. First-user tuning guide: docs/retrieval-tuning.md. Full details in CHANGELOG.md.