Skip to content

shreds-e2e: shard tests across 4 runners#3548

Merged
packethog merged 3 commits intomainfrom
ss/dreamy-banach-7481d7
Apr 17, 2026
Merged

shreds-e2e: shard tests across 4 runners#3548
packethog merged 3 commits intomainfrom
ss/dreamy-banach-7481d7

Conversation

@packethog
Copy link
Copy Markdown
Contributor

@packethog packethog commented Apr 17, 2026

Summary of Changes

  • Shard the shreds-e2e workflow across 4 parallel runners
  • setup job discovers all TestE2E_* functions in the checked-out shreds repo, pins the two heaviest tests (TestE2E_MultiUserInstantAllocation ~396s, TestE2E_DeviceScale ~345s) to their own shards, and round-robin distributes the remaining 19 tests across shards 3-4. Matrix is emitted as JSON on the job's outputs.
  • e2e job becomes a matrix job consuming that output, named e2e (shard N), with timeout-minutes: 15 and go test -timeout=12m -run '${{ matrix.run }}'.
  • Reduces wall-clock time of the in-repo shreds e2e run from ~15 min (one monolithic job) to parity with the shreds repo's own CI.

Testing Verification

  • Ran the shard-discovery bash block locally against the current malbeclabs/doublezero-shreds checkout — produced a 4-entry matrix covering all 21 discovered tests with every test appearing in exactly one shard's regex.
  • Parsed the workflow YAML with a yaml loader; confirmed the e2e job's strategy.matrix.include resolves from needs.setup.outputs.matrix.
  • Actual parallel shard behavior will be verified on the PR's first CI run — expect 4 jobs named e2e (shard 1) through e2e (shard 4) running concurrently.

@packethog packethog force-pushed the ss/dreamy-banach-7481d7 branch from 4ea8306 to ce17872 Compare April 17, 2026 20:18
@packethog packethog self-assigned this Apr 17, 2026
@packethog packethog marked this pull request as ready for review April 17, 2026 20:19
@packethog packethog enabled auto-merge (squash) April 17, 2026 20:49
@packethog packethog merged commit a4622f0 into main Apr 17, 2026
36 checks passed
@packethog packethog deleted the ss/dreamy-banach-7481d7 branch April 17, 2026 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants