Skip to content

perf: messaging E2E suite runs 10x slower than gen shards (23-32min vs 2-4min) #116

@TortoiseWolfe

Description

@TortoiseWolfe

Observation

On PR #114, the three browser `msg` shards each took 5-10x longer than the `gen` shards on the same browser:

Shard Runtime Compared to gen-shard avg
chromium-msg 1/1 28m19s ~10x (gen: ~2m45s)
firefox-msg 1/1 31m46s ~9x (gen: ~3m25s)
webkit-msg 1/1 23m30s ~5x (gen: ~5m)

All passed; not flaky, not broken. But the messaging E2E suite is the long-pole of every PR's CI wall-clock time.

Why this matters

  • A typical PR sits in `UNSTABLE` mergeable state for 25-35 minutes after the gen shards finish, waiting on msg shards.
  • Contributors get the impression CI is stuck (it isn't — just slow).
  • Block on every PR; cannot parallelize further without rearchitecting.

Likely causes (not investigated, just hypotheses worth checking)

  • Supabase realtime subscriptions with `waitFor` timeouts on message arrival.
  • Cross-user message tests serializing on a single Supabase project's rate limits (the `Stagger shards within batch` setup step hints the team has already encountered this).
  • Test setup creating/tearing down many users per test instead of reusing.
  • `encryption-key-derivation` tests that exercise WebCrypto end-to-end per message.

Suggested follow-up

  • Pull a Playwright trace from chromium-msg's slowest tests; identify the top 3-5 tests by duration.
  • See if any can be flattened (single user/multi-conversation instead of multi-user/single-conversation).
  • Consider splitting messaging E2E into 2-3 sub-shards so they parallelize.

Context

Found while monitoring PR #114 (`chore/fork-experience-cleanup`). No regression introduced; the suite was already this slow before the PR. Filing for visibility, not as a blocker.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions