v4.4.0 — finding a leak was creating one
check-hygiene.sh printed the offending line. run-checks.sh stores each
check's output in checks-result.json, which is committed. So a detected leak
was written verbatim into the repo, and the next run found its own report and
recorded that too — a loop that fed itself.
It now reports by location and pattern class and never quotes the match: the
same rule the delegated review agents already follow for injected text, which
had not been applied to the check that needed it most. It still detects; it
just names the file and line and lets the reader open it.
evals/results/ is gitignored. Run artifacts are regenerated per run and carry
the absolute paths of whoever ran them — the leak that shipped in v4.2.0, in a
second costume. The corpus stays tracked; the runs do not.
The measured result they produced is recorded instead, on B10: end to end the
classifier catches 16 of 16 on the 26-case corpus at 0.94 precision, against 13
of 16 for the same model with no plugin. The three it uniquely catches are
domain entailment, an unquantified adjective against a numeric bound, and
vocabulary drift — the semantic tail the deterministic checker cannot reach.
n is small, the corpus is ours, and the harness is early-access.
The view:
- "N things need you" links to the banners it is counting.
- The acceptance-row banner's copy-prompt used to tell an agent to add the
missing rows, which it would have done by inventing test names — turning
"unknown" into a confident "yes" in the table whose whole job is recording
whether a test really asserts a requirement. It now interrogates the
maintainer instead, writes a row only from an answer, records "nothing
asserts this yet" as a real answer, and names the ids left unanswered. - Opt-in
--stale-after: the page notices it is old and prints the command to
regenerate itself, rebuilt from real argv. It does not reload — re-serving
identical bytes teaches the reader that nothing changed when nothing was
re-measured. It says the page is old, never that it is wrong: an old page
over an untouched repo is exactly right. Off by default, because embedding a
generation time trades away byte-identical output.