ci(test): shard the Vitest suite across 4 runners#2641
Merged
Conversation
The Test job is the whole of CI's critical path: ~8.3 min of a ~9 min wall clock. Its cost is almost entirely fixed per-file overhead rather than the assertions — a recent PR run reported 495s of which only 178s was `tests`, against `setup` 831s, `import` 199s and `environment` 156s (cumulative across workers). Those are paid once per test file because `isolate: true` re-executes the module graph for every file, and isolation has to stay on (vitest.config.mts records the order-dependent failures that turning it off caused). So rather than touch isolation, split the 559 files across 4 runners: - test: a 4-shard matrix, PR-only, running `pnpm test --shard=N/4`. Fixed overhead per job here is only ~30s (checkout + Node + install off a warm pnpm store), so 4 shards trade ~26% more runner minutes for ~3x less wall clock. fail-fast: false so one broken shard doesn't mask the others. - test-coverage: the former push-only coverage path, split into its own unsharded job. Nothing blocks on it, so it keeps feeding Codecov one complete report instead of needing a blob-report merge. Vitest shards by hashing each test file's path, so unit/dom files interleave and the shards self-balance. Verified that `vitest run --shard` really splits and conserves the file/test totals across projects — note `vitest list` ignores --shard, so it cannot be used to check this. Nothing (branch protection, `needs:`, workflow_run) references the old `Test` check name, so the rename is safe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
os-zhuang
added a commit
that referenced
this pull request
Jul 18, 2026
…y setup (#2644) Follow-up to #2641, which sharded the Vitest suite but only parallelized its cost. This reduces the cost itself. `setup` was the largest line item in a CI run — 990s cumulative vs 205s of actual `tests` — because `vitest.setup.dom.tsx` side-effect imports @object-ui/components, fields, plugin-dashboard and plugin-grid and re-registers 9 widgets, and under `isolate: true` that module graph re-executes for every DOM test file. Measured at ~3.3s of pure setup per file. But an empirical run of the whole `dom` project under a trimmed setup showed only ~20 of ~310 files actually render through the ComponentRegistry; the other ~290 paid that 3.3s for nothing. Split the DOM tests into two projects by need: - `dom` (default): the new light `vitest.setup.dom-light.tsx` — base polyfills + jest-dom + RTL cleanup, none of the heavy package graphs. - `dom-heavy`: the ~20 `heavyDomTests` that drive the registry, keeping the full `vitest.setup.dom.tsx` (which apps/console also still uses). `heavyDomTests` was derived empirically, not guessed: every file in it failed under the light setup (e.g. "page:card not registered"). The list is self-correcting — a heavy test left off it lands in the light project and fails loudly in CI, so coverage can't be silently lost, only turned red with a pointer to the fix. Behavior-preserving: full suite is 566 files / 6892 tests (6868 passed, 24 skipped) both before and after, at the same commit. Local cumulative `setup` drops from ~990s to ~167s. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
Testjob 就是 CI 的关键路径:一次 PR 跑 ~8.3 分钟,而整条 CI 的 wall clock 才 ~9 分钟(E2E 只要 1.2 分钟,dev-server 0.4 分钟,docs 在 PR 上直接跳过)。慢的不是断言本身。最近一次 PR 跑的 Vitest summary:
真正执行测试只有 178s;
setup831s /import199s /environment156s 都是按测试文件重复支付的固定开销 —— 因为isolate: true会为每个文件重新执行整张模块图。559 个测试文件里约 305 个走domproject,而vitest.setup.dom.tsx会 side-effect 导入@object-ui/components/fields/plugin-dashboard/plugin-grid并重注册 9 个 widget,平均每个文件白付 ~2.7s。isolate必须保持开启 ——vitest.config.mts里已经记录过关掉它导致上千个顺序依赖失败的历史。做法
不动 isolation,把 559 个文件切到 4 个 runner 上:
test:4 片矩阵,仅 PR,跑pnpm test --shard=N/4。这里每个 job 的固定开销只有 ~30s(checkout + Node + 走热 pnpm store 的 install 只要 6s),所以 4 片是用 ~26% 的额外 runner 分钟换 ~3x 的 wall clock。fail-fast: false,一片挂掉不掩盖其他片的失败。test-coverage:原先 push 才跑的 coverage 路径拆成独立的不分片 job。没有任何东西阻塞它,所以保持给 Codecov 一份完整报告,不必引入 blob report 合并的复杂度和风险。Vitest 按测试文件路径的 hash 分片,
unit/dom文件会自然交错,分片自动均衡,无需手工配平。验证
vitest run --shard确实生效且守恒:同时命中unit和dom两个 project 的过滤器下,2 + 2 = 4文件、9 + 14 = 23测试。vitest list忽略--shard(每片都返回全部 559 个文件),不能用它来验证。needs:/workflow_run引用旧的Testcheck 名(main 未开启分支保护),改名安全。不在本 PR 范围
那 831s 的
setup本身还可以根治 —— 把vitest.setup.dom.tsx拆成轻/重两档,只让真正依赖全量 ComponentRegistry 的测试付重量档的代价。那是结构性改动(需要梳理哪些测试依赖注册副作用),本地开发同样受益,单独开专项。🤖 Generated with Claude Code