Skip to content

E2E: trial for more workers + shards#7350

Open
phyllis-sy-wu wants to merge 1 commit intopsyw-0414-E2E-parallel-workersfrom
psyw-0420-E2E-try-more-workers-and-add-shards
Open

E2E: trial for more workers + shards#7350
phyllis-sy-wu wants to merge 1 commit intopsyw-0414-E2E-parallel-workersfrom
psyw-0420-E2E-try-more-workers-and-add-shards

Conversation

@phyllis-sy-wu
Copy link
Copy Markdown
Contributor

@phyllis-sy-wu phyllis-sy-wu commented Apr 20, 2026

WHY are these changes introduced?

Evaluate whether combining more workers (10) with sharding (2 CI machines) gives the best E2E runtime, since each shard gets its own 4 vCPU runner — avoiding the contention that hurt 10 workers on a single machine.

CI runner specs: ubuntu-latest = 4 vCPUs, 16 GB RAM (GitHub docs).

WHAT is this pull request doing?

  • Bumps workers to 10 in playwright.config.ts
  • Adds a matrix strategy with shard: ['1/2', '2/2'] to the E2E CI job
  • Each shard uploads artifacts with unique names to avoid collisions

Benchmark Results:
(under stable conditions, very few teardown failures or flaky retries)

#7309 #7343 #7349 #7350
Workers 5 10 5 10
Shards 1 1 2 2
CI runtime 11m 43s 8m 49s 8m 59s (shard 1/2)
7m 41s (shard 2/2)
7m 44s (shard 1/2)
5m 48s (shard 2/2)

10 workers + 2 shards vs 5 workers + 1 shard (baseline #7309):

Pros:

  • ~4 min faster wall clock (7m 44s vs 11m 43s, 34% reduction), or even faster without teardown failure.
  • Sharding distributes load across machines, reducing per-runner CPU/memory pressure

Cons:

  • 2x CI machine cost (two runners instead of one)
  • 2x setup overhead (build, playwright install, OAuth auth — all duplicated per shard)
  • Split artifacts — two separate Playwright reports to check instead of one
  • 10 workers on 4 vCPU is still tight per machine

Note: Total number of stores/apps created is the same regardless of workers/shards — it's determined by the number of tests. More workers/shards only increases how many run concurrently, not the total count.

How to test your changes?

Compare CI run times across all four trial PRs.

Post-release steps

N/A — trial branch, not intended to merge.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is user-facing, so I've added a changelog entry with pnpm changeset add

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@phyllis-sy-wu phyllis-sy-wu mentioned this pull request Apr 20, 2026
4 tasks
@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/upgrade.d.ts
@@ -1,5 +1,3 @@
-import { getAutoUpgradeEnabled, setAutoUpgradeEnabled } from '../../private/node/conf-store.js';
-export { getAutoUpgradeEnabled, setAutoUpgradeEnabled };
 /**
  * Utility function for generating an install command for the user to run
  * to install an updated version of Shopify CLI.

@phyllis-sy-wu phyllis-sy-wu marked this pull request as ready for review April 20, 2026 15:46
@phyllis-sy-wu phyllis-sy-wu requested a review from a team as a code owner April 20, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant