feat(asvs): promote the scorecard writer to scripts/asvs/apply.py, with tests - #298
Merged
Conversation
…h tests P0 of the ASVS tracking rework. The only tool that WRITES the record of record lived at docs/security/asvs-apply-cells.py in the vault: hardcoded absolute path, no argparse, zero tests, zero references, and outside the CI path filter. THE OLD LOCATION WAS WORSE THAN UNTESTED. ci.yml's docs-only detector treats ^docs/ as non-code, so a PR touching ONLY the writer set code=false and skipped install, lint, type-check and the entire pytest suite. The tool that can silently un-close an owner-closed cell was classified as documentation. Verified by running the workflow's own regex against all three paths. The hardcoded path pointed at the SHARED vault checkout, which several sessions edit at once, so running it from a worktree rewrote a record the operator was not looking at -- I hit that this session and worked around it with a patched copy. --scorecard is now REQUIRED with no default: the one thing a writer must never guess is which record it is rewriting. BOTH SHIPPED INVARIANTS KEPT VERBATIM, as directed, and now proved rather than asserted: - The non-allowlist. render() enumerates only what it ORDERS; every other key on the live cell survives by default. Test: a payload omitting decision_closed and decision_closed_by leaves both intact. That is the 7818991d incident, where an ALLOWLIST silently un-closed two owner-closed cells while every gate stayed green, because an absent decision_closed is a valid False. - The set(was) - set(now) backstop, MUTATION-PROVED: render is replaced with one that drops the decision_* keys, reproducing the historical defect in the one function that could reintroduce it, and the write must be refused. The test asserts the refusal names those keys, because a non-zero exit is not evidence -- several guards return 1, and a mutation proof that trips an unrelated one proves nothing about the invariant it claims to test. Also proved to fire: anchor_repair byte-identity on prose and verdict, the owner-closed rescore refusal, the glyph fail-closed check (which fired for real on 13.3.4 this session), unknown-cell refusal, and dry-run-by-default. Two mypy errors were sitting in this file and are fixed here -- it had never been type-checked, because of the path filter above. Not in scope: deleting the vault copy. That is the two-repo consolidation and it needs the mirror settled first.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
The one refusal in this writer against a WELL-FORMED payload. Every other guard rejects malformed input; this rejects input that is valid and means more than its author intended -- a verdict moving during a pass whose stated purpose was mechanical (an anchor repair, a re-render, a bulk transform). That is this writer's whole failure mode, so the safe thing is now the default and the dangerous thing is explicit: --allow-verdict-change. The refusal names the cell and BOTH verdicts, per review. A refusal that says only 'verdict changed' leaves the operator's actual next question -- which cell, and to what -- unanswered, and an unanswerable refusal gets re-run with the override reflexively, which converts the guard into a speed bump. BOTH HALVES TESTED, because the first commit of this guard passed all 11 existing tests while nothing exercised it. Adding a guard no test drives is how a guard that cannot fire ships looking green. So: the move is refused and the message names 1.1.1 and 'partial' -> 'pass'; AND the flag actually lifts it. Without the second, the flag could be misspelled, unwired or shadowed and the refusal test would still pass.
wshallwshall
enabled auto-merge (squash)
August 9, 2026 08:39
… range CodeQL flagged the class as an overly permissive range: it analyses the character class in UTF-16, where 1f000-1f2ff and 1f300-1faff both decompose to ranges sharing a high surrogate, and reads that as an overlap. The two ranges were adjacent (0x1F2FF + 1 == 0x1F300), so their union is exactly 1f000-1faff. Proved rather than asserted: compared old against new over every one of the 1,112,064 non-surrogate codepoints -- zero behavioural differences, both matching the same 2,940 -- with the comparison itself guarded against passing degenerately. The seam is now a test. A later edit that re-splits the range and mistypes a bound, or truncates it, leaves a hole exactly where the two halves used to meet, and a hole in a FAIL-CLOSED guard goes red nowhere: nothing fails, a glyph simply starts getting written into the security record. Mutation-proved in both directions -- truncating the range trips the inside-bounds assertion, widening it trips the outside-bounds one. The first mutation run was a false green. A heredoc collapsed the double backslash, so the search string became the literal U+1F000 CHARACTER while the source carries the escape SEQUENCE; the replace matched nothing, the file was never mutated, and the test "passed" over unmodified code. The harness now refuses a no-op replacement and prints the before/after line, because a mutation test that cannot prove its mutation landed is not evidence.
wshallwshall
added a commit
that referenced
this pull request
Aug 10, 2026
… as the engine (#309) test_connscale_smoke_end_to_end red on #298 with fd_count_monotonic: "fixed_per_conn@N=24: 344 < prior 3.56e+04 * 0.75". The suspect number is the PRIOR. Calibrated against this repo's own published at-scale run: ~3.0 handles/connection over an ~833 base (2333 @ N=500, 3835 @ N=1000, 5335 @ N=1500), so N=12 predicts ~870. The observed 344 at N=24 is in band. 35,600 at N=12 is 6.7x the reading for a FIFTEEN HUNDRED connection engine, and no leak produces both minutes apart in one sweep from two separately-spawned engines. Mechanism: _walk_descendants validates nothing -- no creation time, no image name, no cardinality bound. Windows keeps a stale ParentProcessId after the parent exits and recycles PIDs, so any live process pointing at the engine's recycled PID is adopted with its whole subtree. Measured on this box, walking one such stale ppid summed 144,688 handles. And in the connscale smoke the engine spawns NO children, so every descendant the walk finds is by construction not the engine. max() then latches the poisoned tick permanently -- a mean would have diluted it. My first reading was "handles has no PID-set gate like CPU does". An independent pass sharpened it and corrected the remedy: copying #220's gate would be WRONG. Differencing sums over different PID sets is arithmetically invalid, but an INSTANTANEOUS gauge over a genuinely larger subtree is correct. The gap is that the number carries no record of the set it covered. Validating the walk (reject a descendant older than its root) is the fix; gating the aggregate is not. Also recorded: #220's gate is a change-detector, not a binding check -- a first-and-only resolution that adopts a wrong-but-stable subtree passes it. And the test that shipped #220's fix asserts this exposure as correct, on a fixture that is physically unrealizable: it varies cpu_pids while pinning handles=61, though the Windows probe reads both from the same Get-Process rows. Diagnosis only. The fix shape is proposed and ranked but not applied.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P0 of the ASVS tracking rework, per the design on the parallel session's branch.
The only tool that writes the record of record lived at
docs/security/asvs-apply-cells.pyin the vault: hardcoded absolute path, no argparse, zero tests, zero references, outside the CI path filter.The old location was worse than untested
ci.yml's docs-only detector treats^docs/as non-code. So a PR touching only the writer setcode=falseand skipped install, lint, type-check and the entire pytest suite. The tool that can silently un-close an owner-closed cell was classified as documentation. Verified by running the workflow's own regex:docs/security/asvs-apply-cells.pyscripts/asvs/apply.pyThe hardcoded path pointed at the shared vault checkout that several sessions edit at once, so running it from a worktree rewrote a record the operator wasn't looking at. I hit that this session and worked around it with a patched copy.
--scorecardis now required with no default — the one thing a writer must never guess is which record it is rewriting.Both shipped invariants kept verbatim, and now proved
render()enumerates only what it orders; every other key survives by default. Test: a payload omittingdecision_closed/decision_closed_byleaves both intact. That's the7818991dincident, where an allowlist silently un-closed two owner-closed cells while every gate stayed green — an absentdecision_closedis a validFalse.set(was) - set(now)backstop, mutation-proved.renderis replaced with one that drops thedecision_*keys, reproducing the historical defect in the one function that could reintroduce it. The write must be refused, and the test asserts the refusal names those keys — a non-zero exit isn't evidence on its own, since several guards return 1 and a proof that trips an unrelated one proves nothing.Also proved to fire:
anchor_repairbyte-identity on prose and verdict, the owner-closed rescore refusal, the glyph fail-closed check (which fired for real on 13.3.4 this session), unknown-cell refusal, dry-run-by-default.Two mypy errors were sitting in this file and are fixed — it had never been type-checked, for the reason above.
Not in scope: deleting the vault copy. That's the two-repo consolidation and it needs the mirror (#297) settled first.