SSOT execution contract
Outcome
Version, Node floor, headline metrics, provenance vocabulary, and compatibility facts come from one generated/checkable source shared by English and Korean docs.
Acceptance
- conflicting denominators cannot pass CI
- every product-written provenance value appears in the docs
- checker states exactly which facts it owns
What the README claims
It prints 2.8% (16/580) against 18.8% (109/579), then says a statistic retyped into prose drifts and that scripts/check-readme-numbers.mjs gates against exactly that (README:257-273).
What the checker covers
Only the <!-- BENCH:BEGIN -->…<!-- BENCH:END --> block, plus four stray shapes outside it: p =, pp, "odds ratio", "Fisher exact" (scripts/check-readme-numbers.mjs:105-110). Rates, N/N fractions, milliseconds and run counts are invisible to it.
So the paragraph explaining why the numbers can be trusted is describing a guarantee that does not extend to the numbers next to it.
And they have already drifted
Same shards, three denominators, in one file:
| source |
on |
off |
prose (README:257-264, matches bench/VERDICT-M5.md:13-14) |
16/580 = 2.8% |
109/579 = 18.8% |
| generated block (README:547-552) |
16/585 = 0.027 |
110/584 = 0.188 |
| generated Fisher row |
16/569 |
110/474 |
The off-arm count is 109 in prose and 110 in the block. The headline figure is consistent with a document that is not the one the gate regenerates.
Also outside the block and unchecked: 81.7% / 42.0% / 92.3%, 496 ms / 185.85 ms / 102.40 ms, 44.8% / 22.0%, 10,002, 7.4 seconds, 100,000 commits. v0.8.1 and Node.js 22.13 happen to match package.json today; nothing keeps them matching.
Related: the README's Provenance vocabulary omits what the product writes
README:502 lists authored | inherited <sha> | reconstructed. The spec and the schema also accept drafted and unknown — and unattended capture stamps Provenance: drafted on every record it produces (src/core/capture-verify.ts:390-396). docs/protocol.md calls that README table the working summary of SPEC §3.
spec/schema/readme-vocab-check.mjs compares keys only, never value grammars, so this cannot fail CI. The same omission is in the validator's repair text (src/core/schema.ts:71), which tells an author that drafted is not a legal value.
What closes it
Either move the headline table inside the generated block so the gate actually produces it, or stop claiming the checker prevents prose drift and say plainly which numbers are hand-maintained. Reconcile the 580/585/569 denominators either way — three of them cannot all be the analysis set. Extend readme-vocab-check.mjs to value grammars so the vocabulary table cannot omit a value the product writes.
Found by an audit of the README's claims against the code. Reproduced.
SSOT execution contract
Outcome
Version, Node floor, headline metrics, provenance vocabulary, and compatibility facts come from one generated/checkable source shared by English and Korean docs.
Acceptance
What the README claims
It prints 2.8% (16/580) against 18.8% (109/579), then says a statistic retyped into prose drifts and that
scripts/check-readme-numbers.mjsgates against exactly that (README:257-273).What the checker covers
Only the
<!-- BENCH:BEGIN -->…<!-- BENCH:END -->block, plus four stray shapes outside it:p =,pp, "odds ratio", "Fisher exact" (scripts/check-readme-numbers.mjs:105-110). Rates,N/Nfractions, milliseconds and run counts are invisible to it.So the paragraph explaining why the numbers can be trusted is describing a guarantee that does not extend to the numbers next to it.
And they have already drifted
Same shards, three denominators, in one file:
bench/VERDICT-M5.md:13-14)The off-arm count is 109 in prose and 110 in the block. The headline figure is consistent with a document that is not the one the gate regenerates.
Also outside the block and unchecked: 81.7% / 42.0% / 92.3%, 496 ms / 185.85 ms / 102.40 ms, 44.8% / 22.0%, 10,002, 7.4 seconds, 100,000 commits.
v0.8.1andNode.js 22.13happen to matchpackage.jsontoday; nothing keeps them matching.Related: the README's Provenance vocabulary omits what the product writes
README:502 lists
authored | inherited <sha> | reconstructed. The spec and the schema also acceptdraftedandunknown— and unattended capture stampsProvenance: draftedon every record it produces (src/core/capture-verify.ts:390-396).docs/protocol.mdcalls that README table the working summary of SPEC §3.spec/schema/readme-vocab-check.mjscompares keys only, never value grammars, so this cannot fail CI. The same omission is in the validator's repair text (src/core/schema.ts:71), which tells an author thatdraftedis not a legal value.What closes it
Either move the headline table inside the generated block so the gate actually produces it, or stop claiming the checker prevents prose drift and say plainly which numbers are hand-maintained. Reconcile the 580/585/569 denominators either way — three of them cannot all be the analysis set. Extend
readme-vocab-check.mjsto value grammars so the vocabulary table cannot omit a value the product writes.Found by an audit of the README's claims against the code. Reproduced.