v2.1.0 — Scale the eval
Milestone: scale the eval. The Opus judge — validated against the human gold labels at 94.4% / 94.4% agreement — grades 300 fresh real DVIDS snippets, disjoint from both the corpus and the gold set, with 95% Wilson confidence intervals: category 93.3% [89.9, 95.6], domain 90.3% [86.5, 93.2] — half the n=54 CI width, corroborating the gold-set numbers rather than replacing them.
This tag also ships everything accumulated under [Unreleased] since v2.0.1:
- Evals-as-CI capability gate (ADR-007) — a free offline gate on every push/PR grading the committed snapshots against
thresholds.tomlfloors, plus a paid weekly live gate. - Rung-1 prompt-optimization loop (ADR-005, autonomy ladder L3) — agent-driven prompt revision with a structural Goodhart guard and an explicit done-signal, held for this release per the spec's sequencing.
- Prompt refinement (tech-vs-ops +
land) — measured on the gold set: category 90.7% → 94.4%, domain 90.7% → 92.6%,technologyrecall to 1.000. - BM25 grounding retired (ADR-012) — a fair same-prompt re-measure showed grounding no longer pays: neutral on category, 0 fixed / 4 broken on domain. Removed from the shipped path; kept dormant as the record.
- OpenTelemetry tracing over
classify(), the v2.0.2 orchestration-test backfill (src/ coverage 90% → 97%), and a parityJenkinsfile.
Full detail: CHANGELOG 2.1.0