Execute CLAUDE.md's architecture tree instead of trusting it (#460)

[realmarcin]

Closes #460.

Why this file specifically

CLAUDE.md is loaded as authoritative context at the start of every session and described as instructions that override default behaviour. Nothing checked it. It had drifted two ways at once, and both were found by accident while working #410 — not by anyone reading it:

• It named validators/reference_validator.py as the sole occupant of that directory. That file was deleted in 4dd299a ("Replace custom validators with official LinkML validators"); only an untracked .pyc survives, so a naive find appears to succeed. Meanwhile the seven validators that are there — the ones the CI gate runs — went undocumented. Fixed in Say in the file that these five scripts have never run (#410) #461.
• It said kb/communities/ holds 60 files. It holds 312. Off by a factor of five, fixed here.

The test

Parses the tree rather than grepping it — tracking parents, since the tree is nested and a bare cli.py means src/communitymech/cli.py — and asserts every path resolves. A separate check pins the record count the prose quotes.

There's a guard on the parser itself, so reformatting the tree can't silently empty the check that depends on it. (My first attempt at this was a regex over path-like tokens; it reported 8 false positives because it ignored nesting. Worth mentioning because that version would have passed while checking nothing meaningful.)

Verified by injecting a fake path:

FAILED test_every_path_named_in_the_architecture_tree_exists[src/communitymech/nonexistent_module.py]

Scope

Deliberately narrow: it checks what is mechanically checkable and says nothing about whether the prose is any good, which no test can. 14 tests. just qc green.

🤖 Generated with Claude Code

[realmarcin]

Review round 1 — the parser was blind to exactly the entries it should catch

The hole

The test checked 12 of the 15 paths the tree declares, and the three it missed were the top-level file entries: conf/oak_config.yaml, NEXT_TASKS.md, NEXT_TASKS_LOOP.md. The reviewer demonstrated it — replacing two with nonexistent files left the suite green, including NEXT_TASKS.md, which the same doc calls the backlog source of truth.

A test written to catch a stale path couldn't see the stale paths most likely to appear. Three line shapes carry a path (nested child, bare directory, top-level entry) and the parser recognised two. Now 15 paths, 0 unconsumed; faking both top-level entries produces two failures.

The guard was the other half

>= 10 is a lower bound with two of slack, so it reported healthy at 12 while three went unchecked — and it cannot distinguish "the tree shrank" from "the parser went blind". Replaced with the invariant that actually holds: every non-blank, non-comment line is classified, or the test names the ones that weren't.

Also

• Dropped the ternary special-casing children starting with kb/ or src/. It was unreachable for the current tree, hardcoded two directory names, and produced wrong paths when I made it fire. A child is relative to its parent — that's what a tree means.
• CLAUDE.md said "Python 3.9+ target" while pyproject.toml requires >=3.10 and black/ruff/mypy are all on 3.10. Fixed and pinned: a PR premised on executing this file's checkable facts shouldn't leave one wrong.

Verified clean by the review

312 is correct (ls and git ls-files agree; the 21 untracked .bak files don't match *.yaml). Every command CLAUDE.md names exists in the justfile — including validate-taxa and validate-cross-repo-ids from the tree comments — and tests/test_datamodel.py, its "single test" example, exists. The validators comment is accurate. The count regex matches only the intended sentence and fails loud if reworded. And the empty-parametrize concern is real but covered: pytest reports an empty parameter set as skipped, not failed, so the sibling guard is genuinely load-bearing — which is why it needed to be a real invariant rather than a floor.

just qc green · 18 tests here, 1611 in the suite.