ci: wire the two dead cockpit spec files into targets CI actually runs - #936
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
blove
force-pushed
the
blove/wire-dead-specs
branch
from
September 1, 2026 21:22
65b6679 to
c4b48ae
Compare
Contributor
`cockpit/deep-agents/footprint.spec.ts` and `libs/cockpit-docs/src/lib/docs-bundle.spec.ts` were executed by nothing: no vitest project included the first, and `cockpit-docs` had no `test` target at all. Both are wired in here, following #916 (give the orphan a target) and #918 (glob out-of-project cockpit specs into `nx test cockpit`). Wiring surfaced three further gaps, all fixed: - `cockpit-docs` and `cockpit-registry` were both unreachable. `nx test` does not walk `^test`, and the `library` job runs a hardcoded LIBS list that excludes them, so cockpit-registry's three specs had never run either. The cockpit job now uses `nx run-many` over all three projects. - The footprint specs resolved paths from `process.cwd()`, which under `nx test cockpit` is `apps/cockpit`. Left as-is they would have asserted against `apps/cockpit/cockpit/...` and passed vacuously. They now resolve from `import.meta.url`. - `cockpit/<product>/*.spec.ts` sits outside every project root, so `nx affected` attributes it to the untagged `root` project and a PR touching only those specs skipped the job that runs them. ci-scope now maps them onto the cockpit scope by path. The website `.mdx` assertions in all three footprint specs asserted the five-segment docs shape #918 deleted — the one docs-links.ts records as having "produced a URL that 404s for every product". They are removed, not weakened: that coupling is a table checked against the website's real content tree by apps/cockpit/src/lib/docs-links.spec.ts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
blove
force-pushed
the
blove/wire-dead-specs
branch
from
September 1, 2026 21:55
c4b48ae to
78e800e
Compare
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two spec files were executed by nothing. Both are wired in — neither deleted.
cockpit/deep-agents/footprint.spec.tsnx test cockpit(glob, per #918)libs/cockpit-docs/src/lib/docs-bundle.spec.tsnx test cockpit-docs(new target, per #916)On
/docs/deep-agentsWhen I opened this PR the library did not exist. #937 landed it while this PR was in CI, so this branch is rebased onto it — and it confirms the call below rather than overturning it.
The new library is six pages shaped
deep-agents/capabilities/<topic>.mdxanddeep-agents/getting-started/introduction.mdx— the three-segment guide shape. The spec asserteddeep-agents/core-capabilities/<topic>/python/<page>.mdx. Those assertions would still fail against main today, now for the same reason as chat and render: the website is organised by guide, not by cockpit topic.#937 also rewired the six deep-agents sentinels to real targets, leaving
COCKPIT_TOPICS_WITHOUT_DOCSempty so that blanking anything now fails. Worth recording that I had drafted an inverted "no docs library yet" tripwire and dropped it before pushing — it would be red on main today. The generic guard #937 strengthened is the right home for this, not a footprint spec.Three further gaps surfaced while wiring
1.
cockpit-docsandcockpit-registrywere both unreachable. Adding atesttarget tocockpit-docsmirroring itscockpit-registrysibling would have been dead on arrival:nx testdoes not walk^test, and thelibraryjob runs a hardcodedLIBS: chat,langgraph,ag-ui,render,a2ui,telemetry.npx nx test cockpit-registryappears nowhere in any workflow, so its three specs (manifest,validate-manifest,resolve-language) had never run either. The cockpit job now usesnx run-many -t test --projects=cockpit,cockpit-docs,cockpit-registry.2. The footprint specs would have passed vacuously. They resolved paths from
process.cwd(), which undernx test cockpitisapps/cockpit, not the repo root. Wired in unchanged, the new deep-agents assertion pointed atapps/cockpit/apps/website/...and passed green while asserting nothing. They now resolve fromimport.meta.url.3. ci-scope could not see them.
cockpit/<product>/*.spec.tssits outside every project root, sonx affectedattributes it toroot, which carries onlynpm:private— noscope:tag. A PR touching only these specs produced an empty scope and skipped the job that runs them (inherited from #918, which affected the matrix specs too).ci-scope.mjsnow maps them onto thecockpitscope by path.The stale assertions, and why removing them is not weakening
All three footprint specs (chat, render, deep-agents — the latter two were also unwired) asserted a website page per topic at
content/docs/<product>/core-capabilities/<topic>/python/<page>.mdx. That is the five-segment shape #918 deleted. Fromlibs/cockpit-registry/src/lib/docs-links.ts:The website tree is organised by guide (
chat/{components,guides,getting-started}/*.mdx), not by cockpit topic. Every one of those ~90 assertions was describing a layout the website has never served. For deep-agents specifically,docs-links.tsmapped all six topics toNO_COCKPIT_DOCS_LINK; #937 has since pointed them at real pages — at the guide-shaped URLs, not the five-segment ones.I removed them rather than inverting them into a tripwire, because the coverage already exists and is stronger:
apps/cockpit/src/lib/docs-links.spec.ts(7 tests, already running) checks every non-sentinel link against the website's actual content tree and nav config, and asserts the blanked set equalsCOCKPIT_TOPICS_WITHOUT_DOCSso nobody can silently blank an entry. What remains in the footprint specs is what "footprint" means: the cockpit modules exist and are runnable. Those assertions were already true on main and are unchanged.Coverage lost: nothing that was running, and nothing another running spec does not cover better.
Verification
Mutation-tested — each broken, observed red through the CI-visible target, restored:
'planning'→'planning-BROKEN'nx test cockpitENOENT .../planning-BROKEN/python/project.json(path proves root resolution)5→6nx run-many(exact CI cmd)Failed tasks: cockpit-docs:testcockpit-docsfrom ci.ymlnode --test ci-workflow.spec.mjsnot ok 15 - runs the cockpit sibling libraries…COCKPIT_ROOTLESS_SPECnode --test ci-scope.spec.mjsGreen after restore:
nx test cockpit— 45 files / 426 tests, up from 42 / 421 (re-run after the rebase onto Add the /docs/deep-agents website library #937, which touchedcockpit-shell.spec.tsx,route-resolution.spec.tsand all five deep-agentssrc/index.ts)nx test cockpit-docs— 1 file / 4 tests (confirmed non-vacuous by direct vitest count)nx test cockpit-registry— passesnode --test ci-scope.spec.mjs ci-workflow.spec.mjs— 85 pass / 0 failnx build cockpit-docs— passes (spec still excluded fromtsconfig.lib.json)Reachability traced through the real classifier: footprint-only PR →
cockpit=true(cockpit_e2estays false); docs-bundle PR →cockpit=true.Pre-existing, unrelated:
cockpit-shell.spec.tsx"copies sanitized diagnostics…" times out at 5000 ms on my machine (needs ~7.5 s). It fails identically on unmodified main, and passes 19/19 with--testTimeout=30000. Not touched — that is a separate call about the timeout.Follow-up: a structural guard against this whole class
This repo has now yielded eight unwired suites (website e2e drift, root generator specs, mit-cutover, cockpit matrix specs, these two, the deploy gates measuring at the wrong layer, and cockpit-registry found here). Each was repaired one at a time, after drifting silently.
Proposing as a follow-up, not built here: a meta-spec that enumerates every
*.spec.*in the repo and asserts each is matched by some project's vitestincludeglobs (or an explicit, commented allowlist for thenode --testsuites, whichscripts/vite.config.mtsalready models). It would run inci-scope, cost seconds, and fail loudly the day a spec lands unowned instead of years later. The allowlist is the design question worth settling before building it — it must be narrow enough that "add it to the allowlist" is not the path of least resistance.