fix(ci-coverage): run the 87 unwired vitest tests, and gate the include-allowlist so it cannot drift again

[artyhoo]

What

packages/core/vitest.config.ts include: is the real test population; the principles-meta-tests job invoked a hand-written list of npm scripts and paths. Nothing tied the two together, so the list fell behind: 87 *.test.ts files across 12 directories had zero CI invocation, plus partial coverage of synthesizer (3/23), install (2/5), audit-self (3/6). audit-self.yml already admitted part of it in prose — a comment is not a mechanism.

Reproduced independently before acting: the new gate's own resolver, run against the pre-wiring tree, reported exactly those 12 directories at exactly the sweep's counts (skills 22, research 22, validator 16, eslint-rules 7, detector 7, installer 4, scenario-generator 2, diff 2, diagnostics 2, detector-v0 1, spec-validation 1, python-starter 1).

1 — What now runs, and what does not

Wired (all green, run exactly as CI runs them):

Step	Result	Why its own step
test:units	905 passed / 39 skipped, 7s	deterministic, offline: audit-self detector detector-v0 diagnostics diff eslint-rules install installer python-starter research scenario-generator validator
test:skills	211 passed, 39s	every case spawns a real bash helper against a sandbox tree — the slow arm, kept attributable
test:spec-validation	9 passed	the only network suite (shells to gh); GH_TOKEN passed so assertions are real, not uniformly rate-limited into the tolerated exit-2 branch
test:synthesizer	186 passed / 23 files	supersedes the narrow 3-file test:live-generation step

Deliberately excluded: none. COVERAGE_ALLOWLIST ships empty — every gap the sweep found is wired. It exists for files that genuinely are not merge gates (a toolchain CI lacks, a manual probe). "Slow" alone is not a reason: it earns its own step, as skills/ did.

Placement is load-bearing, and is what made the old scoping unnecessary. validator/gate-*.ts, install/rule-bootstrap-cli.ts and synthesizer/generate-react-spa.test.ts import @rules-as-tests/preset-next-15-canonical — a workspace sibling absent from packages/core/package.json and its standalone lock, so it resolves only after the root npm install links it into the repo-root node_modules. The old comment blamed scoping and called wiring the rest "a separate task"; the real fix was placement. All new steps run after that install.

Pre-existing failures found by actually running them

15 tests across 6 files failed on the vitest 5000ms default while the underlying script succeeded — the #1363 defect class, in the four skills/ files #1363 named as deliberate residue plus two it did not reach. Fixed at 30_000 (the existing SLOW_SHELL_MS convention), per #1363's precedent:

• skills/plan-currency-check{,-digest,-kickoff}.test.ts, skills/planner-discovery.test.ts, skills/run-helper.test.ts
• synthesizer/run-generated-rule-mutation-skip.test.ts (surfaced only under parallel load)
• spec-validation/validate-batch-spec.test.ts:118,133 — its two untimed --soft cases; the three integration siblings at :162,183,207 already carried 30_000 for the same gh round-trip

Timeout configuration only: git diff contains zero touched expect/toBe/toMatch/toContain/it( lines.

2 — The mechanism

packages/core/principles/38-vitest-include-ci-coverage.test.ts (ADAPT of tests/install-sh/meta-all-wired.test.sh:22-43 + principles/36-ci-needs-completeness.test.ts onto a third population/registry pair).

Per-file, not per-directory — a directory named by one step is no evidence its other 19 files run (the synthesizer 3/23 case). Runs at pre-push and in CI, so it fails at the earliest reachable channel.

Paired-negative: arm (d) deletes the real test:skills wiring from the real workflow text and re-runs the real resolver, asserting RED and that only skills/ files drop out. Arm (e) covers a new unwired directory; arm (f) proves a restructured include: block parses to [] so arm (b) fails loudly instead of passing vacuously.

Filter semantics are modelled on the runner, not assumed. vitest positional args are substring matches and the trailing slash is load-bearing — verified against the real binary: vitest list hooks/ selects 0 hooks-tree-guard files, vitest list hooks selects 6. My first draft normalised hooks/→hooks and thereby credited the hooks step with covering audit-self/hooks-tree-guard.test.ts — false coverage, the exact bug this gate exists to catch. Arm (c) pins it. (vitest list tor/ selecting 180 tests from validator/+detector/ confirms substring, not prefix.)

Also wired into scripts/run-local-ci-sweep.sh (self-test passes): CI-only wiring would reproduce the same one-channel defect a layer up, and the sweep is the pre-push mirror.

Fidelity verdict

FIDELITY: skipped — no cold fidelity-auditor round was run for this PR, so claiming GO would be unbacked; this is not a stage PR against a spec substrate (no ## Provenance section), and every quantitative claim below is backed by a command output reproduced in this body.

Ships as claimed. Every number above is measured, not estimated. Verified on the merged tree (origin/staging moved 4 commits mid-work; merged forward, SSOT conflict resolved keeping both rows): test:principles 375 passed / 1 skipped, typecheck clean, format:check clean, actionlint clean, sweep self-test ALL PASS.

Collisions caught and resolved across two merge-forwards (staging moved 7 commits during the work; the PR also went CONFLICTING once, which blocks Actions entirely): principle slot 37 was taken by #1368 and again by #1366 — mine is 38, still unique — and SSOT id 245 then 246 were taken, so mine is 247.

The wiring immediately earned its keep. The first CI run reddened on research/ecosystem-cargo.test.ts: two containment paired-negatives died with EACCES: permission denied, mkdir '/etc/foo'. Root cause was tmpdir depth, not the guard — makeRoot() placed root at mkdtempSync(os.tmpdir()) and the tests plant a real manifest at root/../.., which is writable under macOS /var/folders/<x>/<y>/T/ but resolves to / under Linux /tmp. Those two tests could never have passed on a Linux runner and nothing noticed, because research/ ran at no CI channel. Fixed by nesting root two levels inside the sandbox; intent preserved (the planted manifest is still outside root). Non-vacuity re-proven rather than assumed: neutering resolvedWithinRoot behind a temporary env flag turned 8 containment tests RED, restoring it turned them green. The whole wired set was then swept under TMPDIR=/tmp to reproduce the Linux shape — units 905, synthesizer 186, skills 211, spec-validation 9, all green, no other tmpdir-depth assumption found.

Known-red, NOT caused by this PR and NOT fixed here (no drive-by per CLAUDE.md «PR strategy»): hooks/getff-work.test.ts fails 3/7 locally (expected 1 to be +0 — the script exits 1). Proven pre-existing by checking out clean origin/staging b41bece884 into a throwaway worktree and running the same file there: identical 3 failures, none of my changes present. The clean-temp-repo case (FRESH-CONSUMER-SMOKE) passes while the three real-repo cases fail, so it looks environment-sensitive (nested worktree) and may well be green on an ubuntu runner. hooks/ is already CI-wired; this PR neither touches it nor changes its wiring.

§1.7 Forward-check applied

Complies with no-paid-llm-in-ci.md: the new gate is deterministic file parsing, zero API-billed calls; the one network suite shells to gh, not to an LLM. Complies with ci-tool-pinning.md §1: no new tool install is added — the new steps invoke npm scripts already backed by npm ci --prefix packages/core at .github/workflows/audit-self.yml:219. Complies with attention-is-not-a-mechanism.md §1: the prose admission at .github/workflows/audit-self.yml:692 was #warning-nobody-reads; it is replaced by a deterministic gate at the earliest reachable channel, not by a louder comment. Complies with build-first-reuse-default.md §1: verdict ADAPT, not BUILD — packages/core/principles/38-vitest-include-ci-coverage.test.ts:1 ports two existing in-repo implementations rather than inventing a third shape; zero new dependencies. Complies with the CLAUDE.md capability-commit gate: SSOT #247 added in the same commit as the artifact, Prior-art: trailer present.

§1.7 Backward-check applied

Class of this change: a declared test population whose CI invocation is a hand-written list that can silently drift from it.

Surfaces where that class occurs, enumerated mechanically (for f in <dir>/*.test.sh; do grep -qF "$f" .github/workflows/*.yml; done, plus a workspace scan for test scripts) — not the diff's own file list:

• packages/core/vitest.config.ts:9-32 ↔ audit-self.yml — GAP-FOUND, this PR (87 + 26 files); closed by principle 38.
• tests/install-sh/*.test.sh (98 files) ↔ workflow — SWEPT-CLEAN, 0 unwired; gated by tests/install-sh/meta-all-wired.test.sh:22.
• tests/hooks/*.test.sh (7) — SWEPT-CLEAN, 0 unwired.
• tests/dispatcher/*.test.sh (1) — SWEPT-CLEAN, 0 unwired.
• tests/agnosticism/*.test.sh (1) — SWEPT-CLEAN, 0 unwired.
• scripts/*.test.sh (8) — 1 unwired: scripts/probe-channels.test.sh. Already known and deliberately documented at scripts/run-local-ci-sweep.sh:117 («a test present in scripts/ but wired to NO CI step … correctly stays out — the sweep predicts CI, it does not invent gates»). Not silent, so not this PR's class of defect; left alone.
• workflow jobs ↔ ci-success.needs — SWEPT-CLEAN, gated by packages/core/principles/36-ci-needs-completeness.test.ts:180 (fix(ci-success): wire the two unwired jobs, and gate the wiring so it cannot drift again #1362).
• required contexts ↔ the two declared lists — SWEPT-CLEAN, gated by packages/core/principles/37-required-context-completeness.test.ts (fix(required-contexts): declare required status contexts at the job, and gate the two lists against it #1368).
• sibling workspace packages — GAP-FOUND, out of scope, surfaced not fixed: packages/preset-next-15-canonical (3 test files), packages/preset-react-spa (1), packages/runtime-bridge (15) each define a test script, but no workflow runs npm test --workspaces — grep -rnE "npm test --workspaces|--workspaces" .github/workflows/*.yml returns only a typecheck comment at .github/workflows/audit-self.yml:806. That is 19 test files at no CI channel, the same class one level up (package population instead of directory population). Principle 38 is scoped to packages/core's include-allowlist and does not see them. Worth its own task; not fixed here per CLAUDE.md «PR strategy».

Prior-art: prior-art-evaluations.md#247 (Vitest test.include/globTestFiles/collect; orphan finders git-orphaned-files + dependency-cruiser --orphan; in-repo meta-all-wired.test.sh + principles 36/27 — verdict ADAPT). DeepWiki vitest-dev/vitest 2026-08-10: «no explicit feature to reconcile the declared include globs with the files that are actually run»; FilesNotFoundError fires only when NO file is found, never on a covered subset. context7 /vitest-dev/vitest surfaces enumeration primitives only. WebSearch ×3 phrasings surfaced the class filed as a bug in the wild (rjwalters/loom#4769) rather than any tool that prevents it. Orphan finders reason about the import graph, to which a test file is a root by construction.