Repository navigation
Reproducible retrieval benchmark
npm run eval:reportruns matha's existing golden-set eval standalone against a naive substring-match baseline (matha's own pre-scoring behaviour), and writesBENCHMARKS.md.- Real, measured lift over the naive baseline on the in-house fixture: recall@5 0.846 → 0.957, precision@5 0.694 → 0.797.
- Deterministic and reproducible — the fixture clock is fixed, so recall/precision are exactly reproducible on any machine.
- Not a public academic benchmark claim: there's no published dataset for "retrieve the right decision for a given file + intent," so the report says so directly rather than compare against an unrelated task.
No breaking changes — .matha/ schema unchanged since v1.2.0.