Skip to content

fix(verify): wire the test layer into typecheck and graduate its TEST_DEBT entry - #15623

Merged
os-litant merged 4 commits into
mainfrom
claude/issue-15145-verify-test-typecheck-wiring
Sep 5, 2026
Merged

fix(verify): wire the test layer into typecheck and graduate its TEST_DEBT entry#15623
os-litant merged 4 commits into
mainfrom
claude/issue-15145-verify-test-typecheck-wiring

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Fixes #15145

packages/verify/tsconfig.json excludes **/*.test.ts and the package's typecheck was a bare tsc --noEmit against that config, so the script was green over a layer no tsc program read. This wires the layer in the way 30 sibling packages already do, and the @objectstack/verify TEST_DEBT entry graduates in the same PR.

1. Baseline, then the fix, then the re-measure — in that order

Ruling on the card was that steps 1 and 2 be measured, not assumed: the three TS2835 make every symbol those imports name resolve to any, so fixing them can reveal diagnostics and the honest move on an upward count is to re-tally, never to delete.

step pnpm check:type-check-debt verdict, quoted from the gate
baseline, unmodified tree at f1d787294f0 OK — 13 ledger entr(ies) re-measured in 108.6s, 143 raw tsc error(s) total, none above its recorded number. / surplus: none — every entry sits exactly at its measurement
after the three .js extensions, at 1d671305858 ℹ @objectstack/verify: TEST_DEBT records 3, and tsc now reports 0 -- graduation candidate.
final head 02b1bdcde2d OK — 12 ledger entr(ies) re-measured in 142.9s, 140 raw tsc error(s) total, none above its recorded number. / surplus: none

The count went DOWN, to zero. So ruling 2's first branch applies and the entry is deleted here, not re-tallied. The prediction in the entry's own note ("the same one-line fix graduates this entry") turned out right — but it was a prediction, and only the middle row above could say so.

check:type-check-coverage moves consistently with that: test layer: 9 package(s) still hide their own tests from tsc (118 files hidden ..., 90 frozen raw errors in TEST_DEBT) becomes 8 package(s) ... (108 files hidden ..., 87 frozen raw errors). Minus one package, minus ten files, minus three errors.

2. The wiring route, and why route (b) was declined although it was available

Route (a), the #5286 sibling: packages/verify/tsconfig.test.json plus check:test-typecheck --package packages/verify --project tsconfig.test.json named in the typecheck chain. tsconfig.json is untouched.

Route (b) — dropping **/*.test.ts from tsconfig.json's excludewas measured available here, which is worth stating because the gate's own graduation text says to assume it is not ("Measured red on 14 of the 18 entries that have an exclusion to drop, so assume (b) is unavailable until that gate says otherwise"). Under a trap-restored mutation of tsconfig.json, pnpm check:type-source-resolution came back exit 0 (124 tsc program(s) across 78 packages), and the restore was proved by blob-hash equality against the HEAD blob plus an empty git diff HEAD. The #11491 note inside the ledger script already listed verify as one of the four that stay green; that still holds.

It was declined on module semantics, not availability. tsconfig.json inherits NodeNext from the repo root, so route (b) would hold the test layer to a resolver vitest never runs it under and demand .js on every relative specifier in it — the config-tier noise the shared gate's header attributes 108 of spec's 842 raw errors to. 28 of the 30 sibling configs override module/moduleResolution for exactly this reason; packages/runtime is the closest structural analogue (tests under src, tsup build, exclusion in the build config) and this config copies its shape.

3. The wiring is not vacuous — two independent proofs

--listFiles, on the tree at 1d671305858 with the workspace closure built:

program packages/verify/src/*.test.ts in it non-test src/** in it total files
tsc -p tsconfig.json (what typecheck used to be, alone) 0 of 10 9 of 9 981
tsc -p tsconfig.test.json 10 of 10 9 of 9 990

So the zero is the exclude line, not a probe that sees nothing.

Ablation. A deliberate type error was appended to src/derive.test.ts — a file already in both programs, compiled from source by each, so no rebuild leg applies. Predicted direction: the old program stays green, the wired script reddens. Measured, exactly that:

  • old program, tsc --noEmit -p tsconfig.json: exit 0, 0 errors
  • wired pnpm --filter @objectstack/verify typecheck: exit 1src/derive.test.ts: 1 type error(s) in a file the ledger does not cover.

The mutation was proved on disk before either run (injected marker counted 1, injected statement counted 1, git diff --numstat 4 added lines) and the restore was proved by blob hash against the HEAD blob and an empty git diff HEAD.

4. Findings

Zero phantom pins, and that is the finding. The card's sharpest line is that any ts-expect-error in those files is currently a phantom check. Grepped with a positive control (the same grep hits packages/spec/src), this package's test layer holds zero ts-expect-error and zero ts-ignore directives. So no pin was silently dead — the statement was about what would happen. That matches what packages/cli's graduation recorded for its own 115 files. What the gap actually cost was the other half: no way to write a type-level pin in this layer at all, and nothing reading it.

The card's file count was stale. Re-derived on origin/main: ten src/*.test.ts, not the card's nine. The tenth is artifact-collections.test.ts. Triage had already caught this; it is re-derived here rather than carried over.

Adding gen:test-typecheck-debt reds check:merge-driver, and the right answer is not to add a disposition. The first draft declared that generator alongside check:test-typecheck, and the gate refused: generator(s) with NO recorded merge disposition: gen:test-typecheck-debt [@objectstack/verify]. The pairing is 1:1 across all 30 sibling configs — 16 with a test-typecheck-debt.json declare the generator, 14 without one declare nothing — and #14062's note in scripts/regen-artifacts.mjs refuses to invent a disposition for a ledger that does not exist. The script was dropped rather than routed. If this layer ever measures non-zero, the ledger, the generator and its regen-artifacts row arrive together in that one PR.

5. No changeset — skip-changeset

Judged from the rule text in .github/workflows/pr-automation.yml, Check Changeset step: route 2, "It releases nothing ... tests-only, and the like". This diff is build-config and test-layer wiring plus repo tooling. It adds no exported symbol, no accepted key or value, and no runtime behaviour: tsconfig.test.json is not in the package's files, the three edited files are tests that tsup never bundles, and scripts/check-type-check-coverage.mjs is not published. Under the WHICH LEVEL rule a bump would have to be at least minor only for a purely additive widening of a published surface, and there is none — so the label, not a changeset written to be safe.

6. Verification

Gate union re-derived from the real changed paths with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (the tool reads its own change set from the merge base; the first derivation warned STALE and was re-run after merging origin/main).

Run to a verdict, all green, on head 02b1bdcde2d unless noted:

  • check:type-check-coverage, check:type-check-debt — both to a verdict, never left PREREQUISITE NOT MET; closure built first with pnpm exec turbo run build --filter=./packages/* --filter=./packages/*/*
  • pnpm --filter @objectstack/verify typecheckcheck:test-typecheck: OK — @objectstack/verify's test layer compiles under packages/verify/tsconfig.test.json; 0 file(s) / 0 error(s) / 0 pinned signature(s)
  • pnpm --filter @objectstack/verify testTest Files 10 passed (10) / Tests 58 passed (58)
  • the derived families: agent-test-spelling bash32-floor cli-command-ids cross-package-test-inputs doc-authoring dual-build-cjs-loads entry-guard logger-receiver-detach merge-driver objectql-double-limit org-identifier override-consistency page-declaration-shape parse-guard pnpm-filter-targets published-files ratchet-remedy-authority service-providers slot-lookup test-source-alias turbo-task-graph type-source-resolution where-matcher workspace-manifest-cycles, plus spec run check:llms-txt
  • the whole-tree kinds: nul-bytes engine-double-contract driver-memory-census refd-timer-probe watch-hint-literal query-options-erasure pm-dispatch-gates
  • the artifact-roster families the derivation flagged as scoring silent under a directory one of these paths is in — silence there is not a clearance, so they were run: published-list-mirrors (and its self-test) authz-resolver console-injection error-code-casing filter-alias-parity i18n-stale-fill published-readme-exports single-claim-paths partof-closing-keyword ci-filter-parity --self-test pm-governed-prose
  • pnpm lint — the full repo scan, eslint . --no-inline-config, exit 0 in 1m20s. No narrowing was needed, so none is claimed.

Exit codes were captured after redirection, never through a pipe.

🤖 Generated with Claude Code


Generated by Claude Code

Under `moduleResolution: NodeNext` a relative specifier without its `.js`
extension does not resolve, and every symbol it names becomes `any`. These
three are the whole of the `@objectstack/verify` TEST_DEBT entry (TS2835 x3).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
…T_DEBT entry

`packages/verify/tsconfig.json` excludes `**/*.test.ts` and the package's
`typecheck` was a bare `tsc --noEmit` against it, so the script was green over
a layer no tsc program read. Add the sibling `tsconfig.test.json` the other 30
packages use and name it in the `typecheck` chain via
`scripts/check-test-typecheck.mts`.

With the three `./harness` imports fixed the ledger re-measures the entry at 0
("TEST_DEBT records 3, and tsc now reports 0 -- graduation candidate"), so the
entry is deleted here rather than re-tallied, and the residue is zero: no
`test-typecheck-debt.json` is seeded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
…generator

`check:merge-driver` requires every manifest generator to carry a merge
disposition in `scripts/regen-artifacts.mjs`, and #14062's note there refuses to
invent one for a ledger that does not exist. The pairing is 1:1 across all 30
sibling configs: 16 with a `test-typecheck-debt.json` declare the generator, 14
without one declare nothing. `check:test-typecheck` is what names the project,
and it stays.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
@github-actions github-actions Bot added the size/m label Sep 5, 2026
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 6b8c677782019510f490ec7c9b51443addb1714bpackageMentionDocs.

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file tests labels Sep 5, 2026
@os-litant os-litant added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed dependencies Pull requests that update a dependency file tests labels Sep 5, 2026 — with Claude
@os-litant
os-litant marked this pull request as ready for review September 5, 2026 01:04
@os-litant
os-litant enabled auto-merge September 5, 2026 01:04
@os-litant
os-litant added this pull request to the merge queue Sep 5, 2026
Merged via the queue into main with commit 45a72b0 Sep 5, 2026
46 checks passed
@os-litant
os-litant deleted the claude/issue-15145-verify-test-typecheck-wiring branch September 5, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants