Skip to content

ci: run the coverage gate in a parallel job instead of serially after validate - #2160

Merged
cliffhall merged 3 commits into
v2/mainfrom
v2/chore/2159-parallel-coverage-job
Aug 27, 2026
Merged

ci: run the coverage gate in a parallel job instead of serially after validate#2160
cliffhall merged 3 commits into
v2/mainfrom
v2/chore/2159-parallel-coverage-job

Conversation

@cliffhall

Copy link
Copy Markdown
Member

Closes #2159

What

build ran npm run validate and npm run coverage as two sequential steps in one job. On a recent green run that was 17m 9s wall clock and those two steps were 84% of it (7m41 + 6m50). This splits coverage into its own job so the two run in parallel.

Why it's safe

coverage consumes nothing validate produces — each client's test:coverage is self-sufficient:

client test:coverage needs validate's build?
web test-servers:build && vitest run --project=unit --project=integration --coverage no — builds its own fixture server
cli test-servers:build && npm run build && vitest run --coverage no — builds its own bin
tui vitest run --coverage no — runs from source
launcher vitest run --coverage no

The only consumers of clients/*/build are the smokes, which stay in build alongside the validate that produces them.

Re-confirmed the two test files the issue flagged, and neither reads compiled output:

  • clients/launcher/__tests__/parse-launcher-argv.test.ts"/path/to/launcher/build/index.js" is an argv string fixture.
  • clients/tui/__tests__/tsupConfig.test.ts — reads package.json files and node_modules manifests only.

A repo-wide grep for tests referencing clients/*/build or dist turns up only those two plus clients/cli/__tests__/e2e.test.ts, which resolves ../build/index.js — and cli's own test:coverage builds it first, so it is covered either way.

Publish gating

Both publish jobs move from needs: build to needs: [build, coverage]. That is the whole safety property of the change, not a side effect: left as-is, a release could publish while the coverage gate is red.

Not done, deliberately

Backgrounding the two commands inside one job would skip the second install, but it puts two vitest fleets on one 4-core runner — a configuration already known to time tests out at the 5s default. Separate jobs get separate runners and separate CPU.

Splitting coverage per client (web is the bulk of the 6m50) is the next lever and stays out of scope, as the existing workflow comment says.

Expected result

  • build ≈ 24s install + 7m41 validate + ~2m gates/Playwright/smokes/Storybook ≈ ~10m 15s
  • coverage ≈ 24s install + 6m50 ≈ ~7m 15s, in parallel

~17m → ~10m. Cost: one extra runner and a duplicated ~24s install.

Docs

Updated the validate step's comment (it explained why coverage was a separate step) and the two AGENTS.md passages describing the CI step order. npm run ci — the local pre-push gate — is unchanged and stays sequential; the invariant that passing it locally means CI's gates pass still holds, it is just no longer the same order.

Verification

The diff touches only .github/workflows/main.yml and AGENTS.md — neither is an input to any format/lint/typecheck/test gate, so npm run ci has nothing to say about it. The real verification is this PR's own CI run: it exercises the new two-job layout directly, which is what the issue asks for ("verify on a real push that the jobs are genuinely independent"). I'll confirm both jobs go green before merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_0177mPHAdECD18nLLCTwR5rh

`build` ran `npm run validate` and `npm run coverage` back to back, and on a
recent green run those two steps were 84% of a 17m9s wall clock (7m41 + 6m50).
They are serial for no reason: `coverage` consumes nothing `validate` produces.
Every client's `test:coverage` is self-sufficient — web and cli build the test
servers themselves (cli its own bin too), tui and launcher run from source — and
the only consumers of `clients/*/build` are the smokes, which stay in `build`
alongside the `validate` that produces them.

So split `coverage` into a sibling job. Both publish jobs become
`needs: [build, coverage]`: that is the safety property, not a side effect —
left as `needs: build` a release could publish with the coverage gate red.

Deliberately NOT backgrounding the two inside one job: that puts two vitest
fleets on one 4-core runner, which already times tests out at the 5s default.

Wall clock ~17m -> ~10m, for one extra runner and a duplicated ~24s install.
Splitting `coverage` per client is the next lever and stays out of scope.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177mPHAdECD18nLLCTwR5rh
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall cliffhall added the v2 Issues and PRs for v2 label Aug 27, 2026
@cliffhall
cliffhall requested a balanced review from Copilot August 27, 2026 02:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Splits the coverage gate from build to reduce CI wall time while preserving release gating.

Changes:

  • Runs validate and coverage in parallel jobs.
  • Requires both jobs before npm or container publishing.
  • Updates CI documentation.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/main.yml Adds the parallel coverage job and publish dependencies.
AGENTS.md Documents the revised CI structure.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@cliffhall

Copy link
Copy Markdown
Member Author

@copilot review round 1 — no comments generated, so nothing to address. Noting for the record what the review did not need to catch, since the risk here is behavioral rather than in the diff text:

  • The publish gating moved to needs: [build, coverage] in the same change rather than in a follow-up. That is the one edit that could regress silently — left as needs: build, a release would publish with the coverage gate red.
  • Job independence is asserted by the diff but proven only by a real run, which is why I did not treat npm run ci as the verification (the diff touches only main.yml and AGENTS.md, neither an input to any gate). This PR's own workflow run is the check; I'm waiting on it and will report both job outcomes here before merge.

…tale

The old name — "Run install, format, lint, build, and test on every push" —
enumerated steps rather than naming the workflow, and was wrong in two
directions: it never mentioned the coverage gate, and "on every push" reads
past the two release-gated publish jobs. `CI` renders as `CI / build` and
`CI / coverage` in the checks list, where the job name already carries the
detail.

Display-only: nothing references the name (no badge, no `workflow_run`, no
sibling workflow), and branch protection keys off JOB names, which are unchanged.

Also: README described the publish jobs as `needs: build`, which the previous
commit falsified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177mPHAdECD18nLLCTwR5rh
Signed-off-by: cliffhall <cliff@futurescale.com>
@cliffhall
cliffhall requested a balanced review from Copilot August 27, 2026 02:49
@cliffhall

Copy link
Copy Markdown
Member Author

Verified on a real push

Run 33033968658 — this is the check the issue asked for before trusting the publish gating.

job started finished duration result
build 02:39:53 02:49:11 9m 18s
coverage 02:39:53 02:46:45 6m 52s
publish / publish-github-container-registry skipped (not a release)

Identical start timestamps, so the two genuinely ran in parallel rather than queueing — and coverage went green on its own runner with no artifact from build, which is the independence claim the whole change rests on.

Wall clock 17m 9s → 9m 18s, a 46% cut — better than the ~10m the issue estimated, because build lost the full 6m50 rather than a net figure.

Second commit: workflow rename

Renamed the workflow Run install, format, lint, build, and test on every pushCI. The old name enumerated steps instead of naming the workflow, and after this change it was wrong twice over: no mention of the coverage gate, and "on every push" reads past the two release-gated publish jobs. It now renders as CI / build and CI / coverage, where the job name already carries the detail.

Safe because nothing consumes it — no status badge, no workflow_run trigger, no sibling workflow file — and branch protection keys off job names, which are unchanged.

⚠️ One caveat worth stating rather than leaving as a surprise: the rename will not show up in the Actions sidebar until this reaches the default branch. GitHub takes a workflow's display name from main, and v2 only reaches main at a milestone merge — the run triggered by the rename commit itself still lists under the old name. That is cosmetic and expected, not a failed rename.

Also fixed a README line that my first commit falsified: it still described the publish jobs as needs: build.

@copilot re-requested for the rename commit, which wasn't in the first review.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@cliffhall

Copy link
Copy Markdown
Member Author

Rename commit's run (33034414631) is green too, and reproduces the split rather than just passing:

job started finished duration
build 02:48:22 02:58:06 9m 44s
coverage 02:48:22 02:54:11 5m 49s

Same start second again, both ✅, publish jobs correctly skipped. Two independent runs now show the same shape, so the ~9-10m wall clock is the steady state rather than one lucky scheduling.

Copilot has reviewed twice (before and after the rename) with no comments either time. Ready for a maintainer look.

@cliffhall
cliffhall merged commit 88ae068 into v2/main Aug 27, 2026
4 checks passed
@cliffhall
cliffhall deleted the v2/chore/2159-parallel-coverage-job branch August 27, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v2 Issues and PRs for v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: run the coverage gate in a parallel job instead of serially after validate

2 participants