Skip to content

fix(e2e): add a globalTimeout under the 45m CI cap - #769

Merged
rubenvdlinde merged 2 commits into
developmentfrom
fix/playwright-trace-and-globaltimeout
Aug 8, 2026
Merged

fix(e2e): add a globalTimeout under the 45m CI cap#769
rubenvdlinde merged 2 commits into
developmentfrom
fix/playwright-trace-and-globaltimeout

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Part of the fleet-wide Playwright instrument sweep for ConductionNL/.github#188. Neither change can alter a verdict — both change whether you can see why a verdict happened.

trace

This repo's trace is already retain-on-failure, so only the timeout half applies here.

Reproduced with two minimal Playwright projects differing only in the trace value, same deliberately failing test, retries: 0 in both:

config trace zips written
retries: 0 + on-first-retry 0
retries: 0 + retain-on-failure 1 (7.4 KB)

globalTimeout: 38 * 60_000

The shared quality.yml Playwright job is timeout-minutes: 45. A job cancelled by that cap yields no verdict and no artifacts — the trace upload is if: failure(), the report upload is if: always(), and neither runs on a cancelled job, while gh pr checks still renders it as "fail". Runs cancelled at ~45m16s have been observed in this fleet.

Margin, measured rather than assumed: overhead before Run Playwright tests starts is 2.0-2.4 min (openconnector run 31257480415 = 2m20s; opencatalogi, doriath, openregister in the same band); uploads after it take seconds. 38m + ~2.5m setup + uploads sits ~7 min under the cap. Verified that a fired globalTimeout exits with a tally (3 did not run / 1 passed) plus an HTML report and a trace zip on disk — exactly what a cancelled job does not give you.

Matches the value already landed in nldesign.

Fleet-wide Playwright instrument sweep, ConductionNL/.github#188. Neither
change can alter a verdict; both change whether you can see why a verdict
happened.

This repo's `trace` is already `retain-on-failure`, so only the timeout half applies here.

No repo in the fleet set `globalTimeout`. The shared quality.yml Playwright
job is `timeout-minutes: 45`, and a job cancelled by that cap produces no
verdict and no artifacts: the trace upload is `if: failure()` and the report
upload is `if: always()`, and neither runs on a cancelled job, while
`gh pr checks` still renders it as "fail". Runs cancelled at ~45m16s have
been observed in this fleet. Measured overhead in that job before the
`Run Playwright tests` step starts is 2.0-2.4 min, so 38m leaves ~7 min of
margin while guaranteeing a tally and its artifacts.
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/procest @ 797ab86

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
check-vue3-compile
test-l10n
composer ✅ 100/100
npm ✅ 552/552
PHPUnit
Newman ⏭️
Playwright
Hydra gates

Quality workflow — 2026-08-08 13:41 UTC

Download the full PDF report from the workflow artifacts.

The shared quality.yml resolves its config as
`${playwright-test-path}/playwright.config.ts` and only falls back to the
app-root `playwright.config.ts` when that file is absent (quality.yml
~L2218). This repo ships tests/e2e/playwright.config.ts, so THAT is the
file every CI run has been using — the app-root config fixed in the previous
commit is the one developers load by hand, not the one the gate reads.

Applies the identical `retain-on-failure` + `globalTimeout: 38 * 60_000`
change here. ConductionNL/.github#188.
@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Second commit added: this repo ships a nested Playwright config, and that is the one CI actually loads.

quality.yml (~L2218) resolves CONFIG="${playwright-test-path}/playwright.config.ts" and only falls back to the app-root playwright.config.ts when that file does not exist. Measured across the fleet: 16 of 23 repos with a Playwright config have such a nested file, so the app-root config the original issue tabulated is not the one the gate reads in any of them.

Both files are now fixed — the app-root one (what a developer gets from npx playwright test at the repo root) and the nested one (what CI runs).

@rubenvdlinde
rubenvdlinde merged commit b631126 into development Aug 8, 2026
29 of 30 checks passed
@rubenvdlinde
rubenvdlinde deleted the fix/playwright-trace-and-globaltimeout branch August 8, 2026 14:05
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/procest @ b49b24e

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
check-vue3-compile
test-l10n
composer ✅ 100/100
npm ✅ 552/552
PHPUnit
Newman ⏭️
Playwright
Hydra gates

Quality workflow — 2026-08-08 14:25 UTC

Download the full PDF report from the workflow artifacts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant