ci: reduce Cypress parallelism from 6 shards to 2

[rusackas]

SUMMARY

Cypress is the single largest consumer of CI runner-minutes — but the suite it
runs is now tiny. Only 5 spec files / 31 test cases remain under
cypress/e2e; the rest have been migrated to Playwright and React Testing
Library. Yet every matrix job still pays the full per-shard setup cost — Python,
Postgres, test-data import, Node, npm ci, instrumented asset build, Cypress
install (~8–10 min) — before running its slice of those few specs.

On master that's 6 shards × 2 app_roots = 12 Cypress jobs (~143 runner-minutes
for the whole E2E workflow), and with only 5 specs the round-robin split
(scripts/cypress_run.py) leaves a shard empty — so a couple of those jobs spin
up a full environment and run zero specs. The setup overhead, not the tests,
dominates.

This cuts the shard count to 2 (PARALLELISM: 2, kept in sync with the
round-robin split — the two values must match). The push-only /app/prefix
variant is smoke-tested on a single shard via matrix.exclude.

Net job counts:

event	before	after
pull_request	6 Cypress jobs	2
push (master/release)	12 Cypress jobs	3 (2× "" + 1 smoke /app/prefix)

Coverage is unchanged — every spec still runs (round-robin distributes all of
them across the shards), just with far less duplicated setup.

TESTING INSTRUCTIONS

CI-only change. On this PR, confirm exactly 2 cypress-matrix jobs run
(app_root ""), all green. The /app/prefix smoke shard only appears on push
events to master/release branches.

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Includes DB Migration
• Introduces new feature or API
• Removes existing feature or API

🤖 Generated with Claude Code

[bito-code-review]

Bito Automatic Review Skipped - Files Excluded

Bito didn't auto-review this change because all changed files are in the exclusion list for automatic reviews. No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the excluded files settings here, or contact your Bito workspace admin at evan@preset.io.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.07%. Comparing base (cb2a56d) to head (4280e0f).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #40717      +/-   ##
==========================================
- Coverage   64.07%   64.07%   -0.01%     
==========================================
  Files        2663     2663              
  Lines      143505   143505              
  Branches    32993    32993              
==========================================
- Hits        91951    91950       -1     
- Misses      49952    49953       +1     
  Partials     1602     1602              

Flag	Coverage Δ
javascript	67.57% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sadpandajoe]

@rusackas looks like some tests are't running and also we should see if we need to reduce the number of shards for playwright. I think currently playwright only runs on 1 shard. Something else is maybe update the cypress-matrix and run more tests per shard?

[rusackas]

Robot comment incoming, but I think we can cut to 2 shards, since there's a lot less Cypress than there used to be. But it seems we're running all the things, unless you can point to anything missing that the bots aren't spotting. I had it look at how tests are loaded, and the 6-shard CI runs, to make sure we're catching everything here.

[rusackas]

Thanks @sadpandajoe — dug into this and nothing's being dropped, but the numbers are illuminating.

There are only 5 Cypress spec files / 31 test cases left in the repo (the rest migrated to Playwright + RTL — e.g. #39071, #38924, #38368, #35938). scripts/cypress_run.py distributes them round-robin across shards and runs every group, so the union always equals the full set — no specs are skipped.

On master today the matrix is 6 shards × 2 app_roots = 12 Cypress jobs, and with only 5 specs the round-robin leaves a shard empty — so a couple of those 12 jobs spin up the full env (Python + Postgres + Node + asset build, ~8–10 min) and run zero specs, and the rest run ~1 spec each. The whole E2E workflow burns ~143 runner-minutes to execute 31 short tests — almost entirely setup overhead.

Per your "more tests per shard" point — exactly. I've cut this to 2 shards (PARALLELISM: 2): on PRs that's 2 Cypress jobs (3 + 2 specs), on push 2 + 1 smoke /app/prefix. Coverage identical, far less duplicated setup.

reduce the number of shards for playwright. I think currently playwright only runs on 1 shard

Good eye — though Playwright isn't sharded by parallel_id here at all; playwright-tests runs the full required suite once per app_root (the ~13–16 min job). Since Playwright now carries the bulk of E2E, it's the candidate for adding parallelism (its native --shard) if we want it faster — the opposite direction from Cypress.

Bigger picture: with only 5 specs left, it's probably worth finishing their migration to Playwright and retiring the Cypress job entirely. Out of scope here, but flagging it.