ci(template): upload Playwright artifacts on E2E failure#3
Merged
Conversation
Newly scaffolded packages inherit this template's ci.yml, so without this addition every new package ships without artifact upload — making CI-only E2E flakes hard to diagnose. Without artifacts, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem but evaporate when the runner shuts down. `if: failure()` skips the upload on green runs to save bandwidth + retention. The 4 existing packages generated from older templates without this step (mail, calc, calendar, drive) get the same change via backport PRs in their own repos; text already had it via a hand-applied earlier addition.
This was referenced May 31, 2026
nathanstitt
added a commit
to tinycld/mail
that referenced
this pull request
May 31, 2026
Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it.
nathanstitt
added a commit
to tinycld/calc
that referenced
this pull request
May 31, 2026
Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it.
nathanstitt
added a commit
to tinycld/calendar
that referenced
this pull request
May 31, 2026
Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it.
nathanstitt
added a commit
to tinycld/drive
that referenced
this pull request
May 31, 2026
Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it.
nathanstitt
added a commit
to tinycld/contacts
that referenced
this pull request
May 31, 2026
Mirrors the canonical pattern in the bootstrap template and every other feature repo (mail, calc, calendar, drive, text). contacts was the only feature repo without this step — likely a pre-existing oversight that the recent bootstrap-flag fix swept past. `if: failure()` skips the upload on green runs. 14-day retention. Future packages get this for free via the bootstrap template's ci.yml (updated in tinycld/bootstrap#3, already merged).
nathanstitt
added a commit
to tinycld/calendar
that referenced
this pull request
May 31, 2026
* fix(ci): use bootstrap v2 --assemble-only flag @tinycld/bootstrap v2 (published 2026-05-30 21:52 UTC) removed the --tooling flag in favor of --assemble-only. CI on this repo broke as soon as v2 propagated to `latest`. Same one-line fix already applied to tinycld/app#9; core, drive, and text were migrated alongside the bootstrap v2 release. * ci: upload Playwright artifacts on E2E failure Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it.
nathanstitt
added a commit
to tinycld/mail
that referenced
this pull request
May 31, 2026
* fix(ci): use bootstrap v2 --assemble-only flag @tinycld/bootstrap v2 (published 2026-05-30 21:52 UTC) removed the --tooling flag in favor of --assemble-only. CI on this repo broke as soon as v2 propagated to `latest`. Same one-line fix already applied to tinycld/app#9; core, drive, and text were migrated alongside the bootstrap v2 release. * ci: upload Playwright artifacts on E2E failure Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it. * test(e2e): pass waitFor: package-sidebar-mounted to navigateToPackage After app reverted navigateToPackage's default sidebar wait (#16), every mail spec that called navigateToPackage(page, 'mail') was racing the lazy sidebar chunk. The first clickSidebarItem (or other sidebar- dependent assertion) would fire before the sidebar mounted and time out, even though the test logic was correct. Pass the explicit waitFor Locator the helper now accepts so each spec gates on the canonical "package-sidebar-mounted" testID before proceeding. mail-labels.spec.ts also drops the redundant Compose-button visibility check from its beforeEach — that wait was the previous-generation workaround for the helper not waiting at all, and is now subsumed by the package-sidebar-mounted gate. (The same cleanup was the entire content of mail#6, which becomes a no-op after this PR lands.) Verified locally: 25/27 e2e tests pass in one run; the 2 that failed under parallel pressure both pass when re-run individually. Both failures look like unrelated parallelism flakes, not regressions introduced by this fix. * test(e2e): wait for sidebar mount in navigateToPersonalInbox After page.reload() (used by many specs to surface a newly-inserted inbound row), the lazy sidebar chunk reloads from scratch. The helper was clicking "Inbox" before the chunk re-mounted, racing the click against the sidebar Suspense skeleton. Adds a getByTestId('package-sidebar-mounted').waitFor() up front so the helper is safe to call right after page.reload(). Same package-sidebar-mounted testID the navigateToPackage helper gates on. * test(e2e): bump waitFor timeouts so lazy chunks complete on CI Two timing tweaks for the slowest CI runs: 1. helpers.ts:navigateToPersonalInbox — bump the package-sidebar-mounted waitFor from 15s → 30s. Mail's sidebar chunk takes >15s to compile + render under parallel-worker pressure on the 2-core CI runner. The 15s was my initial guess; 30s matches Playwright's default action timeout and aligns with what other first-mount waits in this repo use. 2. mail-compose.spec.ts — the compose drawer is also a lazy chunk that first-mounts slowly. Bump the 'To' field's first toBeVisible from default 5s → 15s; the subsequent fields stay at default since they're in the same render frame. Both edits are "wait long enough for the cold-cache compile to land", not "paper over a real flake" — once the chunk is up, the rest of the test runs at default speed. * test(e2e): bump per-test timeout to 60s for cold lazy-chunk loads * test(e2e): bump per-test timeout from 60s to 90s for page.reload() chains * test(e2e): wait for sidebar after page.reload() in mail-unified-inbox * ci(mail): set TINYCLD_WARM_PACKAGES=mail on E2E for chunk pre-warming * ci: trigger rerun after app#18 (warmPackageChunks) merged * test(e2e): bump j/k email-row wait + post-reload sidebar wait to 60s
nathanstitt
added a commit
to tinycld/calc
that referenced
this pull request
May 31, 2026
* fix(ci): use bootstrap v2 --assemble-only flag @tinycld/bootstrap v2 (published 2026-05-30 21:52 UTC) removed the --tooling flag in favor of --assemble-only. CI on this repo broke as soon as v2 propagated to `latest`. Same one-line fix already applied to tinycld/app#9; core, drive, and text were migrated alongside the bootstrap v2 release. * ci: upload Playwright artifacts on E2E failure Mirrors the canonical pattern in tinycld/text. Without this, Playwright's trace.zip, screenshot, video, and error-context.md are written to the runner's filesystem on test failure but evaporate when the runner shuts down — making CI-only flakes hard to diagnose without re-running. `if: failure()` skips the upload on green runs. 14-day retention. Bootstrap template gets the same change in tinycld/bootstrap#3 so future packages inherit it. * test(e2e): fix pivot.spec.ts openNewSpreadsheet helper The local helper was using the pre-No-File-panel UI strings ('Calc' heading + 'New spreadsheet' button). Those are gone — the calc index now renders the shared No-File panel with 'A fresh sheet.' headline and 'New sheet' button. Aligns with the equivalent helper in calc.spec.ts (which was updated when the No-File panel landed). All 10 pivot specs pass locally. * test(e2e): use current No-File panel headline + give xlsx parse more time Two unrelated fixes to the only two specs still failing on this branch: 1. calc.spec.ts:1683 (Import CSV) — was waiting for the pre-No-File 'Calc' h2 heading + 'New spreadsheet' button. Aligns with the actual UI: 'A fresh sheet.' h1 (the No-File panel headline). Mirrors the pivot.spec.ts helper rewrite earlier on this branch. 2. calc.spec.ts:12 (opening Team Scorecard) — the first cell header ('Name') was timing out at the default 5s on CI under load. The xlsx parse + grid hydration is the slow path on the cold-cache first navigation; once the grid is up, subsequent header cells appear immediately. Bumps just the first toBeVisible to 15s so the gate matches reality; the next two stay at default. * style(calc): biome format fix from previous commit * test(e2e): update Import CSV to click the unified Upload files card * test(e2e): waitForURL after row click + bump per-test timeout to 60s Two stability fixes to the one spec still failing on calc#8: 1. calc.spec.ts:12 — `getByText('Team Scorecard.xlsx').click()` races the navigation to /a/<org>/calc/<id>; before the calc screen mounts, getByText('Name') was matching the drive recent-view's "Sort by Name" column-header button. Gates on the URL change first so subsequent assertions only fire after we're inside calc. 2. playwright.config.ts — bump per-test timeout to 60s. Calc tests routinely open xlsx files where the grid hydration + xlsx parse runs end-to-end inside the test body; on CI under parallel load the first navigation per worker can take 30-60s. Default 30s doesn't leave room after the navigation overhead. Both spec-level + config-level — neither suppresses flakiness, they just wait for the cold-cache compile to actually finish before asserting on its output. * test(e2e): click by-role for Team Scorecard.xlsx row * test(e2e): use dblclick to open xlsx from drive recent view * ci(calc): set TINYCLD_WARM_PACKAGES=calc on E2E for chunk pre-warming * ci: trigger rerun after app#18 (warmPackageChunks) merged * test(e2e): open xlsx via context menu Open-in-Calc, not row click * test(e2e): assert cell positions via aria-label, not text content
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Playwright artifact upload on E2E failure to the shared bootstrap CI template. Every new package scaffolded by
bootstrap --newinherits this template, so currently they all start without artifact upload — meaning CI-only E2E flakes evaporate with the runner and can't be debugged without re-running.if: failure()keeps green runs free of upload overhead. 14-day retention matches the canonical pattern in tinycld/text.Why this matters
Discovered while investigating a CI-only mail E2E flake (tinycld/mail#3) that couldn't be reproduced locally even at 15 reps across 5 tests at CI's worker count. Without artifacts there was no way to see what the page looked like at the moment of failure.
Backport scope
The 4 existing packages whose CI was generated from earlier template versions and never picked up artifact upload:
tinycld/text already had this step from a hand-applied earlier addition.
Test plan