Skip to content

feat(gate): resolve every cited CLAUDE.md section number, and list the four missing subpackages - #355

Closed
wshallwshall wants to merge 3 commits into
mainfrom
claude/codemag-article-insights-525408
Closed

feat(gate): resolve every cited CLAUDE.md section number, and list the four missing subpackages#355
wshallwshall wants to merge 3 commits into
mainfrom
claude/codemag-article-insights-525408

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

CLAUDE.md's numbered sections are a de facto API -- 653 citations across 260 tracked files -- and nothing validated a section number. link_check.py resolves the PATH and skips the fragment; the two doc guards that read CLAUDE.md check SDS identifiers and link paths. A renumber landed entirely green. This closes that.

5 files, +366/-2. CLAUDE.md is insertion-only, no section renumbered.

Verified here, not accepted on report

The authoring session's figures and mutation evidence were independently reproduced against a tree that includes main's newest CLAUDE.md change:

guard on the MERGED tree     rc=0    "sections [0..12]; scanned 1523 tracked files;
                                      653 section citations across 260 files"
renumber section 11 -> 13    rc=1    names 61 broken citations, by file and line
after revert                 rc=0    CLAUDE.md byte-identical

Their 653/260 and 61 figures reproduce exactly. The guard goes red on the defect, names the citers, and leaves no residue -- it is not green because it checks nothing.

One caution about that measurement, recorded because it nearly fooled me: my first attempt piped the guard through tail and read $?, which captured tail's exit code and reported rc=0 on a run that had plainly failed. Re-measured with no pipe. That is the trap SDS-3.8 names, and this PR's own subject -- a check that silently doesn't check -- is the same family one level up.

Why this matters beyond the file

A markdown-only PR is classified non-code and runs only the DOC_GUARDS list. This PR touches .py and .yml so it takes the full-suite path -- but if it were ever split, the CLAUDE.md-only half would run only those 15 modules. That is the blind mode this repo already records: test_link_resolution is on the list and caught a broken link on a PR earlier today, while a doc guard that was not on the list reached main. Same blind mode, two outcomes, decided purely by list membership -- and this PR adds the 15th module to that list.

Provenance

Authored by the CLAUDE.md-split session; routed here for push and PR. Verified before pushing: both commits present, genuinely unpushed, 5 files +366/-2 exactly as stated, 3 behind / 2 ahead of current main and still merge-tree rc=0. Leak scan of all 366 added lines: clean, negative control 366.

Their reported local verification, labelled as theirs: ruff + ruff format + mypy strict clean; the full 15-module DOC_GUARDS set at 219 passed / 89 skipped; all 7 pre-commit hooks green on the real commit; and the SDS nested-anchor mutation red-then-green.

Unblocks a third session: the ASVS tracking lane is recording the section-citation exposure as closed pending merge of e7e825ed rather than closed -- correctly, since a control on an unpushed branch is not a shipped control.

The repository-layout tree listed 10 subpackages while messagefoundry/ has
14. security/, support/, verify/ and tray/ were absent, so CLAUDE.md
contained ZERO occurrences of "tray" despite ADR 0113 and a 17-module
package shipping at HEAD.

Cause: the tray, verify and support packages landed in the private lineage
between 2026-07-13 and 07-25, and that content never reached the repo of
record at publication -- five commits dropped by omission, absent from HEAD.
The tree has been stale since.

Descriptions taken from each package's own __init__ docstring. The tray line
records what it wraps, verified rather than assumed: tray/ imports only
messagefoundry.service, messagefoundry.service_status and its own submodules
-- no api/, no engine internals -- so section 4's dependency rule needs no
carve-out for it.

Insertion-only, no section renumbered. That is deliberate: 281 tracked files
cite this file by section number, nothing validates a section number
(link_check.py:21 validates the path and skips the fragment), and the same
rot has already landed once -- test_sds_rule_ids_are_stable.py:7-11 records
four security citations that still resolve to the wrong section today.

Verified: tests/test_link_resolution.py and
tests/test_sds_rule_ids_are_stable.py, 34 passed. The second was confirmed to
read this worktree's file by injecting a bogus SDS-9.9 citation and watching
it go red, then removing it.
…nested anchor escaping the SDS gate

CLAUDE.md's numbered sections are a de facto API and NOTHING validated a
section number. Measured: 281 tracked files name the file, 646 citations name
one of its sections, and tests/test_dependency_boundaries.py cites section 4
in its own docstring, so this reaches code and not only prose.

Every existing gate stays green through a renumber. link_check.py:21 says so
in its own header -- it resolves the PATH and skips the #fragment -- and the
two doc guards that already read CLAUDE.md check other things: one checks
SDS-N.N identifiers, one checks link paths. Path resolves, identifiers
resolve, only the meaning moves. The checkable half stays green.

The rot is not hypothetical. The sibling guard
tests/test_sds_rule_ids_are_stable.py:5-11 records it landing on the Secure
Development Standards -- a new section 5 pushed 5-9 to 6-10, and citations
still resolve to the wrong section today. The instance was fixed and the
class left open. This closes it for the anchor.

MEASURED BLAST RADIUS: renumbering section 11 alone breaks 61 citations.

scripts/docs/claude_section_check.py, stdlib only, beside its siblings:
  - parses the anchor's own `## N.` headings as the source of truth
  - scans tracked .md/.py/.ps1/.yml via git ls-files
  - requires the anchor's NAME on the same line within a bounded window, so a
    section number belonging to another document is never attributed to it --
    false positives are how a gate gets disabled
  - ALWAYS prints what it scanned; a finding count alone cannot distinguish a
    clean scan from one that examined nothing
  - REFUSES to run when zero headings parse or the anchor is missing, rather
    than reporting every citation broken or the corpus clean

ADDED TO DOC_GUARDS in ci.yml. Without that it would be born unable to fire:
a markdown-only PR is classified non-code and runs only that list, so a check
landed outside it goes green by absence on exactly the PR shape it exists to
catch. That enumeration hole has already been fixed twice for other paths
(BACKLOG #327, #1200) and left open both times; ci.yml:1050 documents its own
recurrence.

Also widens _citation_sources() in the SDS guard to include nested CLAUDE.md
files. Root is a literal path and the globs are rooted at docs/,
messagefoundry/ and scripts/, so a subpackage anchor -- which CLAUDE.md
section 3 invites -- would take its SDS citations out of coverage silently.
Resolved through git ls-files, not rglob, because this repository nests
worktrees under .claude/worktrees/ and an rglob would scan sibling checkouts.
There are no nested anchors today; this is the prerequisite, not a response.

Verified. Both guards were made to FAIL ON PURPOSE and the injected defect
confirmed to land:
  - renumber section 11 to 13        -> rc=1, 61 citations; revert -> rc=0
  - nested CLAUDE.md citing SDS-9.9  -> SDS guard red; remove  -> green
  - 8 self-bite tests, including both refuse-to-run guards and a
    false-positive case (another document's section 9 is not claimed)
  - ruff check, ruff format, mypy strict clean
  - full DOC_GUARDS set, 15 modules: 219 passed, 89 skipped
@wshallwshall
wshallwshall enabled auto-merge (squash) August 12, 2026 23:08
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Superseded by #356, which carries the identical commits (trees match at aa4933a395a8) on a neutrally-named branch.

This PR's branch name referenced an unrelated research topic and did not belong on the public repo. The change itself carries no trace of it -- verified across all 366 added lines: zero references to that topic in the code, the tests, the CLAUDE.md edit, or the commit messages. Only the branch name did, and both it and this PR are being retired rather than renamed in place, so nothing points at the old name.

No work is lost. #356 is the same tree.

auto-merge was automatically disabled August 13, 2026 00:21

Pull request was closed

@wshallwshall
wshallwshall deleted the claude/codemag-article-insights-525408 branch August 13, 2026 00:21
wshallwshall added a commit that referenced this pull request Aug 13, 2026
…ot repeating (#358)

#1211 said a recurrence gets evidence rather than another re-run. This is the
evidence. Two excursions on windows-2025, same test, same SLO, hours apart on
unrelated changes:

  PR #343  fixed_per_conn@N=24: 36   < prior 48.4 * 0.75 (36.30)   short by 0.30
  PR #355  fixed_per_conn@N=24: 34.9 < prior 48.2 * 0.75 (36.15)   short by 1.25

NEITHER CHANGE COULD REACH THE SUBSYSTEM. #343 was the worktree gate and its tests;
#355 was a docs guard, ci.yml and four CLAUDE.md lines. Zero overlap with connscale,
the store or the pipeline in either case, so attribution to the diff is ruled out on
blast radius rather than on "flake".

THE NEW INFORMATION IS THE TREND, not the repeat. The miss went 0.30 -> 1.25 against a
prior that barely moved (48.4 -> 48.2). One excursion is the band being tight; two
excursions drifting further out is the ratio behaving differently, which is exactly
what this item was left open to catch.

AND IT IS NOT THE CLOCK: the suite grew 11,961 -> 12,025 passing between the runs, and
the failing step took 40:46 against a 55:00 cap, so the leg had headroom and was not
being killed. Slowness would have shown up as a cap kill, not as an SLO miss.

I did NOT re-run to make it green, and I did not file a new number -- this item already
exists for this exact metric, and a second instance belongs on it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant