A measurement release. One v0.8.0 feature is removed because an A/B showed it changed nothing, and another is now backed by evidence rather than argument.
Removed: the CLAUDE.md staleness rule
Two independent fixture designs, N=3, both arms. The first named a symbol the diff changed — reachable by the consistency agent's existing stale-reference grep, so it proved nothing. The second was built specifically to be out of that mechanic's reach: an architectural invariant stated in prose sharing no identifier with the diff, falsified by a fetch() added inside a request handler. The v0.7.1 baseline caught it anyway, PASS 3/3, exactly as the treatment did.
An instruction that changes no outcome is prompt weight in every consistency-agent spawn, so it is gone: the persona section, the stale-docs root-cause category, tests/check-doc-staleness.sh, and the README paragraph. Eval case 91 stays, reclassified as a regression guard — the behavior is real and emergent, it just was not caused by the rule.
Measured: REVIEW.md support works
Case 90 is the suite's first genuine evidence for a v0.8.0 feature: baseline FAIL (1/3), treatment PASS (3/3). Both directions of the rule require the feature — a DEBT-tier rounding duplication promoted to CRITICAL, and a real unvalidated-input defect under a suppressed path staying unreported.
Its first version was non-discriminating (it used a defect default calibration already rated CRITICAL, and bait the baseline already ignored), which is what prompted the rewrite.
Honestly unmeasured
- Per-agent diff ordering. Case 02 is the only fixture that exercises it, and it returned
PASS (2/3)thenFAIL (1/3)on identical baseline code, with no timeouts or errors in the raw logs. It cannot resolve a difference, so ordering is neither proven nor regressing. An earlier apparent regression from this case was that flicker. <gate_results>. Case 92 passes on both arms: a failing gate is already caught by Phase 1 without the block. It is kept because it is the only fixture whose gates genuinely run and fail — infrastructure the suite did not previously have.
The rule this produced
A case proves nothing until the baseline arm has been shown to fail it. Run the baseline on every new case, not just the treatment. A case both arms pass is a regression guard; only a case the baseline fails and the treatment passes is evidence. evals/README.md now records this along with the full A/B table.
Suite: 92 cases, 73 unit tests, 9 gates, shellcheck clean.