v0.28.0-beta — The empty-rate gate understated every arm it measured
Pre-release
0.27.0-beta shipped a per-arm empty-response statistic so that "a reasoning deployment burned its completion budget" would fail at authoring time instead of being found forensically. The instrument itself was mis-measuring, in two ways, and both understated the defect it exists to expose.
The two defects
A fallback that borrowed a real arm's identity. Arm tokens were parsed as v + digits, with return "v1" on no match. v9strip is a real arm, not a malformed v9, so its 700 calls were filed under v1: v1's denominator read 920 against a true 220, v9strip's empties landed in v1's numerator, and v9strip had no row and therefore no ceiling — it cleared the gate by not being in it.
Judge grades pooled into probe denominators. Judge calls are a different population, and a healthy one (0 empty of 1246), so including them mechanically dragged every rate down.
Corrected figures (probe answers only)
| arm | published in 0.27.0-beta | corrected |
|---|---|---|
| v3 | 258/387 (66.7%) | 258/330 (78.2%) |
| v6 | 182/861 (21.1%) | 182/675 (27.0%) |
| v9 | 8/212 (3.8%) | 8/110 (7.3%) |
| v1 | 4/920 (0.4%) | 0/110 (0.0%) |
| v9strip | not measured | 4/352 (1.1%) |
V9's true rate breaches the 5% ceiling and always did — pooling 102 judge grades into 110 probe calls is the only reason it read as passing. It is now a ratchet entry: visible, and able only to shrink. Its direction is conservative (silence scores as a failure on V9), so the published retrieval ceiling remains a lower bound.
Every error runs the same way — understated, never overstated. No conclusion flips: V3/V6 remain uncitable pending re-run, by a wider margin than first stated.
Hardening
The gate now asserts the recorded arm set equals a C# list, rather than checking only that present arms are under their ceilings — the same pass-by-absence defence the V7 separability test uses. An unattributable key becomes unknown, which the gate fails on, instead of borrowing a real arm's identity.
Blast radius
Corpus bytes are untouched. Only the metadata stamp changed, so every probed_corpus_sha256 still matches and no probe re-run was required. Recomputed offline from the same call cache via --restamp-empty-rates-from-cache, which reuses the runner's own attribution rather than reimplementing it.
Consumers pinned to 0.27.0-beta need no re-run and no data migration — the corpora they load are byte-identical. Upgrade to correct the published statistic, not to change results.
Verification
Falsified three ways (removing an arm, injecting an unknown bucket, regressing a rate — each fails the gate). 985/985 memory tests pass. Six attribution cases added to the CI self-test. Stamped data re-derived by an independent parse across all 7 corpora.
Also fixes a latent CI flake: a LongMemEval percent-rendering guard asserted "5000" appeared nowhere in a transcript that prints a random temp path, and failed on a hex coincidence.