Skip to content

Enable E2E tests on Linux#3295

Merged
ivan-ottinger merged 5 commits intotrunkfrom
enable-linux-e2e-tests
May 4, 2026
Merged

Enable E2E tests on Linux#3295
ivan-ottinger merged 5 commits intotrunkfrom
enable-linux-e2e-tests

Conversation

@ivan-ottinger
Copy link
Copy Markdown
Contributor

@ivan-ottinger ivan-ottinger commented Apr 30, 2026

Related issues

How AI was used in this PR

Drafted with Claude Code under interactive review. The author pushed back on overstated claims (e.g. attributing the OOM to PHP WASM workers when only the symptom was verified) and on a speculative selector change that was reverted after confirming it was unnecessary. Each change here was vetted against the actual test runs and existing codebase conventions.

Proposed Changes

  • Add process.env.E2E guard to autoInstallLinuxCliIfNeeded so a packaged Linux build does not write to the host user's ~/.bashrc or ~/.local/bin/studio during E2E runs.
    • Verified by inspecting the bundle from electron-vite build: vite statically replaces process.env.NODE_ENV with "production", so the existing NODE_ENV !== 'production' guard collapses to a dead false in the bundle (the same substitution applies to the full package flow, which just wraps the vite output). Without the new process.env.E2E term, the function would fire during E2E and call installCli(), which uses os.homedir() directly — and E2E_HOME_PATH does not override that.
    • The Mac/Windows installers use the same NODE_ENV-only guard pattern, but extending the fix to them is out of scope here: the analog has not been verified on those platforms from this Linux box, and shipping a guard on inference alone is the kind of thing this PR is trying to avoid.
  • Reduce concurrent running sites in blueprints.test.ts to one at a time. The shared session previously accumulated 6+ running sites by the end of the file, which crashes memory-constrained hosts (e.g. local Linux VMs). Each blueprint test now runs with only its own site; the default onboarding site is stopped after beforeAll, and each newly created site is stopped after the test completes.
  • Update getStopAllButton in main-sidebar.ts to match both "Stop" (single running site) and "Stop all" (multiple). The button text is conditional in running-sites.tsx; the existing locator only matched "Stop all" exactly, which made the new cleanup logic in blueprints unreliable when one site was running.

Testing Instructions

  1. Build the app: npm run make.
  2. Install Playwright browsers: npx playwright install chromium.
  3. Run the full suite: npm run e2e.
  4. Expected outcome on Linux: 43 pass, 1 skipped (import.test.ts is skipped without the optional e2e/imports/jetpack-backup.tar.gz fixture). A few editor-iframe tests may flake on the first attempt and pass on Playwright's automatic retry.

Verified locally on Linux arm64 (Parallels VM). Mac/Windows behavior should be unchanged: the blueprints cleanup adds ~15-30s to that file's run time, otherwise no functional difference.

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?
  • Mac and Windows E2E jobs still pass on CI.

@ivan-ottinger ivan-ottinger self-assigned this Apr 30, 2026
@ivan-ottinger ivan-ottinger marked this pull request as ready for review April 30, 2026 15:12
@ivan-ottinger ivan-ottinger requested a review from Copilot April 30, 2026 15:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables Linux E2E stability/safety by preventing Linux CLI auto-install side effects during E2E runs and reducing resource usage in the blueprint E2E suite.

Changes:

  • Skip autoInstallLinuxCliIfNeeded() during E2E runs to avoid writing to the host user’s home/profile.
  • Update the main sidebar “Stop” locator to handle both single-site and multi-site labels.
  • Run blueprint E2E tests with a single running site at a time by stopping sites between tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
apps/studio/src/modules/cli/lib/linux-installation-manager.ts Adds an E2E guard to prevent Linux CLI auto-install during E2E.
apps/studio/e2e/page-objects/main-sidebar.ts Updates stop button locator to match both “Stop” and “Stop all”.
apps/studio/e2e/blueprints.test.ts Stops the default onboarding site after setup and stops sites after each blueprint test to reduce peak memory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/studio/e2e/blueprints.test.ts
@ivan-ottinger ivan-ottinger requested review from a team and gavande1 April 30, 2026 15:34
@wpmobilebot
Copy link
Copy Markdown
Collaborator

📊 Performance Test Results

Comparing bcdcd24 vs trunk

app-size

Metric trunk bcdcd24 Diff Change
App Size (Mac) 1507.60 MB 1507.60 MB +0.00 MB ⚪ 0.0%

site-editor

Metric trunk bcdcd24 Diff Change
load 1858 ms 1818 ms 40 ms ⚪ 0.0%

site-startup

Metric trunk bcdcd24 Diff Change
siteCreation 8086 ms 8076 ms 10 ms ⚪ 0.0%
siteStartup 4946 ms 4952 ms +6 ms ⚪ 0.0%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

Copy link
Copy Markdown
Contributor

@gavande1 gavande1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have verified locally on Linux ARM64 — all 43 tests pass, 1 skipped as expected. Thanks for getting E2E green on Linux! 🎉

CleanShot 2026-05-01 at 14 22 25@2x

@ivan-ottinger ivan-ottinger merged commit 828d74b into trunk May 4, 2026
12 checks passed
@ivan-ottinger ivan-ottinger deleted the enable-linux-e2e-tests branch May 4, 2026 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants