Skip to content

fix(drift): check paths in every scaffold file, not only ROUTER.md - #170

Merged
Yashasvi2229 merged 5 commits into
mainfrom
fix/107-claim-extraction
Sep 4, 2026
Merged

fix(drift): check paths in every scaffold file, not only ROUTER.md#170
Yashasvi2229 merged 5 commits into
mainfrom
fix/107-claim-extraction

Conversation

@Yashasvi2229

Copy link
Copy Markdown
Collaborator

What

Makes checkPaths trustworthy enough to run on the whole scaffold, then runs it there.

#80 scoped path checking to ROUTER.md because inline code in the other files produced false MISSING_PATH errors. That stopped the noise but left ten of the eleven scaffold files unchecked — a path could rot in context/ or patterns/ and nothing said so.

Widening the scope alone does not work. Done on its own it takes a real scaffold from 75/100 to 0/100 with 56 errors, because the failures are about resolution, not scope. So this change does both halves:

Extraction stops claiming values that were never file references — home locations (~/.claude/projects), globs (.mex/graph.db*), anything containing whitespace (nodemon src/index.ts), and elided shapes (api/...).

Resolution learns where documentation actually points:

  • a scaffold file naming another scaffold file, since the project search deliberately skips .mex/ and so never found INDEX.md or a pattern
  • a path written from a subproject's own root — routes/quiz.ts when the file is at server/src/routes/quiz.ts
  • generated state the repository ignores, which is absent from a clean checkout by design and documented precisely because mex writes it
  • API routes and placeholders (documents/upload, owner/repo), which are shaped like relative paths but whose first segment names nothing
  • naming-convention examples such as PascalCase.tsx

Dependencies get the same treatment. Every bold span under a stack heading became a dependency claim, and only the first text child was read, so an entry naming its packages in code claimed a package called "Radix UI + " and dropped the real ones. Now only bold that opens a list item declares a dependency, the packages it names in code are the claim, and a prose description like Supabase (Postgres + Auth) claims nothing. Package managers are not dependencies, and a second application in a subdirectory has its own manifest, which is now read.

Why

Closes #107.

This PR carries @miloquinn's #83 as its first commit, with the change requested in review applied on top: DOTTED_KEY_WITH_SLASH allowed the dotted segment to be empty, so it matched .github/CODEOWNERS, .github/workflows and .mex/ROUTER.md as readily as argocd.argoproj.io/sync-wave — which would have dropped mex's own scaffold paths out of the check. Requiring a real character before the dot fixes it; the regression test @theDakshJaitly asked for is included.

Type of change

  • Bug fix
  • New feature
  • Refactor
  • Docs
  • CI/Tooling

How to test

Run mex check on a populated scaffold before and after. On a real v0.8 scaffold with a Next.js frontend and an Express backend in a subdirectory:

Before: 54/100 — 1 error, 12 warnings. The error is youtubei.js, an npm package named in prose, reported as a missing file. The warnings are dependency claims made of connective punctuation: ", ", " + ", "Groq (", "Radix UI + ".

After: 100/100 — 0 errors, 0 warnings, while checking paths in all eleven scaffold files instead of one.

To see the widened scope find something real, run it on a scaffold whose docs have drifted — expect a MISSING_PATH for a package documented under a name nothing declares, reported from a file that is not ROUTER.md.

Checklist

  • Tests pass (npm test) — see note below
  • No breaking changes (or documented below)
  • Tested locally with a real project

Note on npm test: the drift suites pass in full — 117 tests across claims and checkers, 23 in public-api — and npm run typecheck is clean. The full suite does not pass on this branch, but it does not pass on main either. Measured with a hermetic TMPDIR, main fails 47 and this branch fails 41; comparing failures by name, the only branch-only failure is cli.test.ts :: built CLI main-module guard …, which passes in isolation and exercises Team JSON parsing, not drift. The same test flaked on the previous drift PR and on runs of main.

Behaviour change: runDriftCheck now reports MISSING_PATH from any scaffold file. public-api.test.ts pinned the old ROUTER.md-only contract and has been updated rather than removed; its other two cases now assert something stronger — a context file is checked, and its config values are correctly not read as paths.

Verified against three real scaffolds, main build vs this branch:

Scaffold main this branch
fresh scaffold from templates/ 100/100 100/100
populated v0.8 scaffold 54/100 (1E, 12W) 100/100 (0, 0)
populated v0.7 scaffold 78/100 (1E, 4W) 60/100 (4E, 0W)
this repo 8E, 11W 8E, 12W

The v0.7 scaffold drops because the widened check finds real drift that ROUTER.md-only scoping hid: it documents a workspace package under a name nothing declares, and points at a .mex/sync.sh that no longer exists. Every one of the v0.8 scaffold's thirteen findings, by contrast, was false.

Code-graph changes

  • This PR targets main
  • A linked issue agrees on the bounded extractor/resolver scope
  • The change follows the frozen LanguageExtractor or FrameworkResolver interface
  • A focused fixture and assertions for the expected node/edge shape are included
  • Any new grammar WASM, extension mapping, extractor, or resolver is registered
  • No graph identity, reconciliation, schema, or drift-semantics changes are included, or a core / discuss-first issue is linked above

miloquinn and others added 5 commits September 4, 2026 11:25
- Skip inline IP/CIDR values, file extensions, shell commands, and dotted annotation keys when extracting path claims
- Add regression coverage for Kubernetes/GitOps-style inline code snippets
DOTTED_KEY_WITH_SLASH allowed the dotted segment to be empty, so it
matched a leading-dot directory as readily as a Kubernetes annotation:
.github/CODEOWNERS, .github/workflows and .mex/ROUTER.md all stopped
being extracted as path claims, which would have dropped mex's own
scaffold paths out of checkPaths entirely.

Require a real character before the dot. argocd.argoproj.io/sync-wave
and k8s.io/api are still skipped; the hidden directories are checked
again.

Addresses the review on #83.
Every bold span under a stack heading became a dependency claim, and
getStrongText returned only the first text child. An entry written as
`- **Radix UI + ` + `** — primitives` (with the two package names in
inline code) therefore claimed a package called "Radix UI + ", and the
packages it actually named were dropped. Other entries produced ", ",
" + " and "Groq (" — twelve unactionable warnings on one real scaffold.

Read the declaration the way the docs write it. Only bold that opens a
list item declares a dependency, so emphasis inside prose — the
**service-role** key — is no longer a package. When the entry names its
packages in code, those are the claim. A plain-text name is claimed only
if it could be an installed package, so a description like
"Supabase (Postgres + Auth)" makes no claim at all.

A package named in a dependency entry is also no longer extracted as a
path, so youtubei.js stops being reported as a missing file.

Measured on two real scaffolds: 54/100 to 75/100 and 78/100 to 87/100,
with the remaining warnings naming packages that genuinely are absent
from the root manifest.

Refs #107
#80 narrowed checkPaths to ROUTER.md because inline code in the other
files produced false MISSING_PATH errors. That stopped the noise but
left ten of the eleven scaffold files unchecked: a path could rot in
context/ or patterns/ and nothing said so. Widening the scope alone is
not enough — done naively it takes a real scaffold from 75/100 to 0/100
with 56 errors — because the failures are about resolution, not scope.

Extraction now rejects values that were never file references: home
locations (~/.claude/projects), globs (.mex/graph.db*), anything with
whitespace (`nodemon src/index.ts`), and elided shapes (`api/...`).

Resolution now understands where documentation actually points:

- a scaffold file naming another scaffold file, since the project search
  deliberately skips .mex/ and so never found INDEX.md or a pattern
- a path written from a subproject's root, `routes/quiz.ts`, when the
  file is at server/src/routes/quiz.ts
- generated state the repository ignores, which is absent from a clean
  checkout by design and is documented precisely because mex writes it
- API routes and placeholders — `documents/upload`, `owner/repo` — which
  are shaped like relative paths but whose first segment names nothing
- naming-convention examples such as PascalCase.tsx

The dependency side gains the same treatment: package managers are not
dependencies, and a second application in a subdirectory declares its
packages in its own manifest, so reading only the root one reported
every one of them missing.

Measured against three real scaffolds, comparing main to this branch
with paths now checked in all eleven files rather than one:

  fresh scaffold   100/100  ->  100/100
  populated v0.8    54/100  ->  100/100   (13 findings, all false)
  populated v0.7    78/100  ->   60/100

The v0.7 scaffold drops because the widened check finds real drift that
ROUTER.md-only scoping hid: it documents a package under a name nothing
declares, and points at a .mex/sync.sh that no longer exists.

Closes #107
The public API suite pinned the ROUTER.md-only scope that #107 removes.
The first two cases still hold and now say something stronger -- a
context file is checked, and its config values are correctly not read as
paths -- so only the third needed rewriting: both files make a path
claim, so both are answered for.
@Yashasvi2229
Yashasvi2229 merged commit c99eb9e into main Sep 4, 2026
9 checks passed
@Yashasvi2229
Yashasvi2229 deleted the fix/107-claim-extraction branch September 4, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

checkPaths: ROUTER.md prose still yields MISSING_PATH false positives for runtime paths (follow-up to #79)

2 participants