Skip to content

fix(ci): an executable file is CODE wherever it lives, including under docs/ (#1200) - #299

Merged
wshallwshall merged 3 commits into
mainfrom
claude/ci-docs-only-executable-exempt
Aug 9, 2026
Merged

fix(ci): an executable file is CODE wherever it lives, including under docs/ (#1200)#299
wshallwshall merged 3 commits into
mainfrom
claude/ci-docs-only-executable-exempt

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

ci.yml's docs-only short-circuit skips install, lint, type-check and the whole of pytest when every changed path is docs-only. ^docs/ is an alternation branch, so it matched a .py under docs/ and short-circuited before the stated *.py rule was ever reached.

A defect here does not fail loudly — it removes the thing that would have failed.

Measured, with the workflow's own regex under real grep -E

PRE-FIX (noncode only):
  docs/security/asvs-apply-cells.py      -> NON-CODE (suite skipped)
  docs/benchmarks/.../b5_microbench.py   -> NON-CODE (suite skipped)
POST-FIX (alwayscode checked first):
  docs/security/asvs-apply-cells.py      -> code
  docs/SECURITY.md                       -> NON-CODE (still short-circuits)
  .gitignore                             -> code (via the noncode branch, #327)

That first file is the tool that writes the ASVS record of record and can silently un-close an owner-closed cell — exempt from lint, mypy and the entire suite by virtue of its directory. Two mypy errors had been sitting in it since it was written; they could not have survived a single check. That's the corroboration the exemption was real.

Two things make this worse than a missing test

The comment and the regex disagree, and the comment is what people read. ci.yml states the intent in as many words — "any *.py … counts as CODE and runs the full suite" — and the regex does not implement that sentence. An auditor reads the comment, agrees, and moves on.

The precedent sits four lines above the defect. #327 fixed exactly this shape for .gitignore and wrote the lesson down in place. The identical defect for docs/**/*.py was in the regex immediately below that paragraph. Instance fixed, class left open, with the reasoning that would have closed it preserved alongside. Hence an extension rule rather than another one-path exception.

The fix, and what it preserves

alwayscode='\.(py|ps1|sh|ts|js|yml|yaml|toml|lock|cfg|ini)$', evaluated before the noncode allowlist. The docs-only optimisation is deliberately kept for actual documents — simply deleting ^docs/ would run the full suite on every prose edit, which is the cost the short-circuit exists to avoid.

The test drives the detector, and reads its regexes out of ci.yml

A test carrying its own copy of the pattern passes forever while the workflow drifts underneath it — this defect, one level up.

It asserts the regression in both directions in a single test: the pre-fix logic classifies docs/x.py as non-code and the post-fix logic does not. Asserting only the new behaviour cannot distinguish a fixed detector from a deleted one — return True passes that. A negative control is included so a regex that accidentally matched everything cannot make every assertion pass vacuously.

Blast radius: 2 files in this repo (benchmark scripts, low risk), 3 in the vault including the writer above.

Class escalated from my instance by the parallel asvs-tracking-rework session, which measured both repos and identified the #327 precedent.

…r docs/ (#1200)

ci.yml's docs-only short-circuit skips install, lint, type-check and the whole of
pytest when every changed path is docs-only. `^docs/` is an alternation branch,
so it matched a .py under docs/ and short-circuited before the stated *.py rule
was ever reached.

A defect here does not fail loudly -- it REMOVES the thing that would have
failed. Measured with the workflow's own regex under real grep -E:
docs/security/asvs-apply-cells.py classified NON-CODE. That is the tool that
WRITES the ASVS record of record and can silently un-close an owner-closed cell,
exempt from lint, mypy and the entire suite by virtue of its directory. Two mypy
errors had been sitting in it since it was written; they could not have survived
a single check.

TWO THINGS MAKE THIS WORSE THAN A MISSING TEST. The comment and the regex
disagree, and the comment is what people read -- ci.yml states "any *.py ...
counts as CODE" and the regex does not implement that sentence. And the precedent
is FOUR LINES ABOVE the defect: #327 fixed this exact shape for .gitignore and
wrote the lesson down in place, while the identical defect for docs/**/*.py sat
in the regex immediately below the paragraph explaining it. Instance fixed, class
left open. Hence an EXTENSION rule rather than another one-path exception.

The docs-only optimisation is preserved for actual documents. Deleting `^docs/`
would fix the defect and run the full suite on every prose edit, which is the
cost the short-circuit exists to avoid.

THE TEST DRIVES THE DETECTOR AND READS ITS REGEXES OUT OF ci.yml. A test carrying
its own copy of the pattern passes forever while the workflow drifts underneath
it -- this defect, one level up. It asserts the regression in BOTH directions in
one test (pre-fix says non-code, post-fix says code), because asserting only the
new behaviour cannot tell a fixed detector from a deleted one: `return True`
passes that. Negative control included so a regex that matched everything cannot
make every assertion pass vacuously.

Class escalated from my instance by the parallel asvs-tracking-rework session,
which measured the blast radius in both repos and found the #327 precedent.
…ed (#1200)

Amends this PR rather than following it up, because it is still open.

THE GAP: #1200's rule keys on the EXTENSION, and `.gitattributes` has none. It
was still in the docs-only allowlist, so a `.gitattributes`-only PR skipped lint,
mypy and the entire suite. Not cosmetic -- the vault's own asvs-scorecard.yml
records that a change there "silently alters how the corpus is materialized,
which is exactly what makes the digest differ", so it is an input to the ASVS
corpus pin. Found by the CI-hygiene stream; verified here against the shipped
regex under real grep before acting.

PRECEDENCE, NOT DELETION. `alwayscodepath` is a positive list checked FIRST;
`noncode` keeps its historical entry and loses. Two reasons, and the second is
what changed my first attempt: "is code" should not depend on the ABSENCE of a
line elsewhere, and keeping `noncode` intact is what lets the test reconstruct
the OLD classification by disabling this rule alone -- so the regression is
asserted in BOTH directions. My first version deleted the entry, which made the
pre-amendment simulation impossible and the regression test meaningless. The test
caught that.

`.gitignore` is named here too, though #327 already made it code: it was code
only by FALLING THROUGH, and a future edit re-adding it to `noncode` would
silently undo #327 with nothing to say so.

THE COMMENT AT :836 IS THE THIRD INSTANCE IN THIS FILE of the comment stating the
opposite of its code -- it listed `.gitignore` as docs-only for weeks after #327
removed it from the regex, directly below the paragraph explaining why #327 was
necessary. The first two instances are the "any *.py counts as CODE" line this PR
already fixes, and #327 itself.

Also removes three glyphs (two U+26A0 with variation selectors, one U+2717)
against CLAUDE.md section 11. Found because printing them to a cp1252 console
raised UnicodeEncodeError -- the exact failure mode section 11 documents as the
reason for the rule.
BACKLOG.md conflicted because both sides append. Verified collision-free before
resolving -- this branch contributes #1200, main contributes #1201 (from #301) --
so keeping both is the semantically correct resolution and not merely the
mechanical one. Ordered numerically; 277 open items, hygiene green.
@wshallwshall
wshallwshall merged commit 1d56b6b into main Aug 9, 2026
37 of 38 checks passed
@wshallwshall
wshallwshall deleted the claude/ci-docs-only-executable-exempt branch August 9, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant