Skip to content

ci(nextest): partition unit tests into 4 parallel shards#2019

Merged
bug-ops merged 2 commits intomainfrom
feat/m27/nextest-partition
Mar 19, 2026
Merged

ci(nextest): partition unit tests into 4 parallel shards#2019
bug-ops merged 2 commits intomainfrom
feat/m27/nextest-partition

Conversation

@bug-ops
Copy link
Owner

@bug-ops bug-ops commented Mar 19, 2026

Summary

  • Add ci-partition profile to .github/nextest.toml for hash-based test partitioning
  • Replace sequential test job with 4-way matrix strategy
  • Partition unit tests with --partition hash:K/4 (K=1..4) for parallel execution
  • Expected wall time: ~3 min per shard (vs ~10 min sequential) = ~70% improvement
  • Build job (25 min) remains the pipeline bottleneck

Changes

  • .github/nextest.toml: Add [profile.ci-partition] with test-threads=8
  • .github/workflows/ci.yml: Replace test job with matrix:
    • Strategy: partition: ["1/4", "2/4", "3/4", "4/4"]
    • Each shard downloads the same archive and runs its hash bucket
    • JUnit reports collected from each shard

Test plan

  • Verify CI workflow passes with new 4-shard matrix
  • Confirm each shard runs independently and completes in <5 min
  • Check that total passing test count equals sum across all shards
  • Integration and coverage jobs unchanged

Add ci-partition profile to nextest.toml for hash-based test partitioning.
Replace sequential test job with 4-way matrix, reducing test time by ~70%.

Each shard runs --partition hash:K/4 independently:
- Shard 1/4, 2/4, 3/4, 4/4 run in parallel (~3min each vs 10min total)
- Build job (25min) remains pipeline bottleneck
- No changes to integration tests or coverage jobs
- JUnit reports collected from each shard

Fixes: improves CI pipeline parallelism
@github-actions github-actions bot added ci CI/CD configuration size/S Small PR (11-50 lines) labels Mar 19, 2026
@bug-ops bug-ops merged commit e923c0b into main Mar 19, 2026
25 checks passed
@bug-ops bug-ops deleted the feat/m27/nextest-partition branch March 19, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci CI/CD configuration size/S Small PR (11-50 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant