E2E: utility - cleanup stores by phyllis-sy-wu · Pull Request #7355 · Shopify/cli

phyllis-sy-wu · 2026-04-21T13:23:04Z

WHY are these changes introduced?

E2E tests create dev stores that can accumulate when tests fail mid-run, CI times out, or teardown fails. This script automates bulk-clean for leftover stores.

WHAT is this pull request doing?

`cleanup-stores.ts`

Standalone cleanup script that finds leftover E2E dev stores, uninstalls their apps, and deletes them via browser automation.

pnpm --filter e2e exec tsx scripts/cleanup-stores.ts              # Full: uninstall apps + delete stores
pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --list        # List stores with app counts
pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --delete      # Delete only stores with 0 apps installed
pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed      # Show browser window
pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --pattern X   # Match stores containing "X" (default: "e2e-w")

Logic

Discovery phase:

Log in via completeLogin helper
Navigate to dev.shopify.com/dashboard/{orgId}/stores
Extract store FQDNs via regex on full page HTML (page.content()) — catches slugs in hrefs/attributes, not just visible text
Filter by name pattern (default: e2e-w)
Paginate via a[href*="next_cursor"] and repeat 3–4 on each page

--list mode (per store):

Navigate to admin.shopify.com/store/{slug}/settings/apps
Wait for either "Add apps to your store" empty state or first button[aria-label="More actions"] (whichever appears first via Promise.race)
Count menu buttons across all pages (paginate via button#nextURL)
Print store name + app count

default mode (per store):

Navigate to admin.shopify.com/store/{slug}/settings/apps
Dismiss Dev Console if visible
Wait for page readiness via Promise.race — either empty state or first app menu button
Check for "Add apps to your store" empty state — only trust positive confirmation as proof of zero apps
If apps present: uninstall all apps on the page (see uninstall logic below)
After uninstall: verify empty state is now visible — if not, skip store deletion and log warning
Navigate to admin.shopify.com/store/{slug}/settings/plan with waitUntil: 'load' + 10s fixed wait for shadow DOM hydration
If URL contains access_account → store already deleted, done
Click delete button (s-internal-button[tone="critical"] → button) — retry click up to 3 times without re-navigation (first click often refreshes the page due to hydration)
Wait for modal scoped by title "Review before deleting store" (.Polaris-Modal-Dialog__Modal:has-text(...))
Check the confirmation checkbox, verify confirm button becomes enabled (retry checkbox up to 3 times)
Click "Delete store" confirm button
Verify: waitForURL for access_account (success) vs store home path (failure → retry from step 7, up to 3 outer attempts)

--delete mode: same as default but skips stores that have apps installed (step 4 → skip if not empty).

Uninstall logic (uninstallAllAppsOnPage):

Find first button[aria-label="More actions"] on the page
Extract app name from div[role="listitem"] → a span
Click menu button → click "Uninstall" → click confirm
If "Uninstall" option not in menu: press Escape, increment consecutiveSkips, try next button
If confirm never appears: increment consecutiveSkips (prevents infinite loop)
Reload page after each uninstall to refresh the list
When no more menu buttons visible: check for button#nextURL pagination → continue on next page

Features:

Empty state safety: never deletes a store unless "Add apps to your store" is positively confirmed
Page readiness wait via Promise.race before checking empty state (avoids false negatives from slow renders)
Modal scoped by title to avoid matching stale/hidden modals
Per-store timing in output
Exports cleanupStores() for use from other scripts

How is this different from per-test teardown?

Per-test teardown (setup/teardown.ts) — knows the specific app name and store FQDN, uses direct URLs, no discovery. Runs automatically in test finally blocks.
cleanup-stores.ts (bulk, manual) — discovers all matching stores via dashboard pagination. Safety net for orphaned stores from failed/interrupted test runs.

How to test your changes?

Create leftover stores by skipping cleanup:

E2E_SKIP_TEARDOWN=1 DEBUG=1 pnpm --filter e2e exec playwright test app

List them:

pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --list

Clean up:

pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed

Example: `pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed`

Direct success

cleanup-store-success.mov

Failure → retry → success

cleanup-store-failure.retry.mov

Expand for complete log

cli % pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed
[cleanup-stores] Mode:    Uninstall apps + Delete stores
[cleanup-stores] Org:     161686155
[cleanup-stores] Pattern: "e2e-w"
[cleanup-stores] Logging in...
[cleanup-stores] Logged in successfully.
[cleanup-stores] Navigating to stores page...
[cleanup-stores] Found 7 store(s)
[cleanup-stores] [1/7] e2e-w1-1776739795521
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-dev-1776739811852
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (61.6s)
[cleanup-stores] [2/7] e2e-w4-1776739795523
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-hot-reload-1776739834188
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (59.0s)
[cleanup-stores] [3/7] e2e-w0-1776739822235
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-hot-delete-1776739877179
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (60.1s)
[cleanup-stores] [4/7] e2e-w3-1776739809449
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-hot-create-1776739877628
  Apps may still be installed (empty state not confirmed) — skipping delete
  (19.3s)
[cleanup-stores] [5/7] e2e-w4-1776739868603
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-multi-cfg-1776739885507
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (60.3s)
[cleanup-stores] [6/7] e2e-w2-1776739904305
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-toml-dev-1776739926063
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (59.9s)
[cleanup-stores] [7/7] e2e-w1-1776739872065
  1 app(s) installed
  Uninstalling apps...
    Uninstalled E2E-mcfg-def-1776739950151
  Apps uninstalled (empty state confirmed)
  Deleting store...
  Deleted
  (36.2s)

Post-release steps

Checklist

I've considered possible cross-platform impacts (Mac, Linux, Windows)
I've considered possible documentation changes
I've considered analytics changes to measure impact
The change is user-facing — I've identified the correct bump type (patch for bug fixes · minor for new features · major for breaking changes) and added a changeset with pnpm changeset add

phyllis-sy-wu · 2026-04-21T13:23:20Z

E2E - QA: init, deploy, versions list, config link, 2nd deploy #7367
E2E: automatic teardown polish #7358
E2E: improve CLI log format #7364
E2E: utility - cleanup all #7357
E2E: utility - cleanup apps #7356
E2E: utility - cleanup stores #7355 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copilot

Pull request overview

Adds a standalone E2E utility script to bulk-discover and clean up leftover dev stores created by failing/aborted E2E runs, using Playwright browser automation against the Dev Dashboard + Store Admin.

Changes:

Introduces packages/e2e/scripts/cleanup-stores.ts with --list, --delete, and default “full” cleanup modes.
Implements store discovery via Dev Dashboard pagination and regex extraction of store FQDNs from page HTML.
Automates app uninstall (with pagination) and store deletion flows with retries and safety checks (empty-state confirmation).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T16:20:07Z

+const isDirectRun = process.argv[1] === fileURLToPath(import.meta.url)
+if (isDirectRun) {
+  main().catch((err) => {
+    console.error('[cleanup-stores] Fatal error:', err)
+    process.exitCode = 1
+  })


isDirectRun compares process.argv[1] to fileURLToPath(import.meta.url) using strict string equality. When running via npx tsx packages/e2e/scripts/cleanup-stores.ts, process.argv[1] is typically a relative path, so this check can be false and main() won’t run at all. Consider normalizing both sides (e.g., path.resolve(process.argv[1])) or comparing URLs via pathToFileURL(path.resolve(process.argv[1])).href === import.meta.url.

Copilot · 2026-04-21T16:20:07Z

+/**
+ * Delete a store via the admin settings plan page.
+ * Caller must ensure all apps are already uninstalled.
+ * Retries the full flow if the page redirects to store home instead of access_account.
+ */
+async function deleteStore(page: Page, storeSlug: string): Promise<void> {
+  const planUrl = `https://admin.shopify.com/store/${storeSlug}/settings/plan`
+
+  for (let attempt = 1; attempt <= 3; attempt++) {
+    try {
+      // Step 1: Navigate to plan page (wait for full hydration before clicking)


This script defines a local deleteStore() helper that largely overlaps with the existing deleteStore browser helper in packages/e2e/setup/store.ts. Having two different implementations with the same name risks divergence and makes future maintenance/debugging harder. Consider reusing the shared helper (and extending it if needed) or renaming this one to avoid confusion (e.g., deleteStoreViaPlanPage).

This was referenced Apr 21, 2026

E2E: automatic teardown polish #7358

Open

E2E: utility - cleanup all #7357

Open

phyllis-sy-wu mentioned this pull request Apr 21, 2026

E2E: utility - cleanup apps #7356

Open

4 tasks

github-actions Bot added the devtools-gardener Post the issue or PR to Slack for the gardener label Apr 21, 2026

phyllis-sy-wu force-pushed the psyw-0420-E2E-utility-cleanup-stores branch 4 times, most recently from 47a7f22 to 1c6838f Compare April 21, 2026 15:57

phyllis-sy-wu marked this pull request as ready for review April 21, 2026 16:15

phyllis-sy-wu requested a review from a team as a code owner April 21, 2026 16:15

Copilot AI review requested due to automatic review settings April 21, 2026 16:15

Copilot started reviewing on behalf of phyllis-sy-wu April 21, 2026 16:15 View session

phyllis-sy-wu mentioned this pull request Apr 21, 2026

E2E: remove dead code #7360

Open

4 tasks

Copilot AI reviewed Apr 21, 2026

View reviewed changes

phyllis-sy-wu force-pushed the psyw-0420-E2E-utility-cleanup-stores branch from 1c6838f to 73d2cae Compare April 21, 2026 18:44

This was referenced Apr 22, 2026

E2E: improve CLI log format #7364

Open

E2E - QA: init, deploy, versions list, config link, 2nd deploy #7367

Open

E2E: utility - cleanup stores

de46b8d

phyllis-sy-wu force-pushed the psyw-0420-E2E-utility-cleanup-stores branch from 73d2cae to de46b8d Compare April 22, 2026 14:48

phyllis-sy-wu linked an issue Apr 22, 2026 that may be closed by this pull request

E2E: Cleanup scripts for leftover test resources #7319

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

E2E: utility - cleanup stores#7355

E2E: utility - cleanup stores#7355
phyllis-sy-wu wants to merge 1 commit intomainfrom
psyw-0420-E2E-utility-cleanup-stores

phyllis-sy-wu commented Apr 21, 2026 •

edited

Loading

Uh oh!

phyllis-sy-wu commented Apr 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

phyllis-sy-wu commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WHY are these changes introduced?

WHAT is this pull request doing?

cleanup-stores.ts

Logic

How is this different from per-test teardown?

How to test your changes?

Example: pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed

Post-release steps

Checklist

Uh oh!

phyllis-sy-wu commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

phyllis-sy-wu commented Apr 21, 2026 •

edited

Loading

`cleanup-stores.ts`

Example: `pnpm --filter e2e exec tsx scripts/cleanup-stores.ts --headed`

phyllis-sy-wu commented Apr 21, 2026 •

edited

Loading