ci(playwright): raise chromium shard budget 19 → 20 min - #30616
ci(playwright): raise chromium shard budget 19 → 20 min#30616chirag-madlani wants to merge 2 commits into
Conversation
The chromium lane hit the planner's ceiling on the merge queue for PR #30563 (run 30394014577): Lane chromium needs more than 24 shards to stay within the 19-minute plan budget; the heaviest shard is predicted at 19.1m ##[error]Process completed with exit code 1. Root cause is aggregate growth: the chromium suite currently holds ~1053 min of test content per the committed timing baseline — 90.6% of the planner's 24 × 3 × 19min × 0.85 capacity ceiling (1163 min). LPT bin-packing is doing its job; there's just no headroom left, and the current heaviest-shard prediction is 0.1 min over the hard budget. Bump COMMON_SHARD_BUDGET_MS from 19 min to 20 min, aligning it with the non-chromium TARGET_MS. The change only widens the acceptance threshold — the predicted heaviest shard runtime is unchanged, so wall-clock is unaffected. What changes is the planner will now accept a plan it previously refused. This is a stop-gap. The heaviest individual tests are worth attacking at the root: 605s TestCaseImportExportE2eFlow.spec.ts › EditAll User: Complete export-import-validate flow 603s TestCaseImportExportE2eFlow.spec.ts › Admin: Complete export-import-validate flow 362s ContextCenterArticles.spec.ts › Article edits and navigation-flushed titles persist Two ~10-minute tests in a single file are effectively immovable in LPT scheduling — they consume 20% of a shard budget between them. Splitting or trimming those three tests would give ~15 min of aggregate breathing room without another budget bump. Leaving that as a follow-up so the merge queue can move again immediately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
@greptile-apps caught two test-file assertions still hardcoded to the former 19-minute chromium budget after the constant bumped to 20: test_common_lane_keeps_one_minute_of_allocation_reserve asserts shard_budget_ms_for_lane("chromium") == 19 * 60 * 1000 test_common_shards_enforce_the_nineteen_minute_budget matches SystemExit "above the 19-minute plan budget" uses 19-min weights for within/above units Updates: - Rename first test to test_common_lane_matches_target_execution_budget and assert 20 min (chromium == TARGET_MS after the alignment). Added a comment explaining that the previous 1-minute reserve was removed deliberately, not accidentally, so a future reader doesn't revert. - Rename second test to test_common_shards_enforce_the_twenty_minute_budget and update the weight_ms constants + the SystemExit match string. pytest verifies: 55 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@greptile-apps good catch — addressed in a624cd3287. Two test-file assertions were still hardcoded to the former 19-minute chromium budget:
Verified: |
|
Thanks for the thorough fix. Both stale assertions are updated, the test names now accurately describe what they enforce, and the explanatory comment is a nice touch to prevent future confusion. 55/55 passing confirms nothing regressed. LGTM. |
✅ Playwright Results — workflow succeededValidated commit ✅ 537 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 5 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 50m 5s ⏱️ Max setup 2m 54s · max shard execution 17m 48s · max shard-job elapsed before upload 20m 56s · reporting 4s 🌐 203.00 requests/attempt · 2.86 app boots/UI scenario · 10.67% common-shard skew Optimization targets still in progress:
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
|
Will split the spec instead increase the timing: #30617 |
Code Review ✅ ApprovedRaises the Chromium Playwright shard-planning budget from 19 to 20 minutes and updates the corresponding planning tests. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar | Powered by Gitar — free for open source |
Summary
The chromium lane hit the shard planner's hard ceiling on the merge queue for PR #30563 (run 30394014577):
```
Lane chromium needs more than 24 shards to stay within the 19-minute plan budget;
the heaviest shard is predicted at 19.1m
##[error]Process completed with exit code 1.
```
The planner refused to build a valid plan → `plan-playwright` failed → all downstream jobs skipped → `playwright-summary` failed → merge queue blocked.
Root cause
Aggregate growth. The chromium suite currently holds ~1053 min of test content per the committed timing baseline — 90.6 % of the 1163 min planner ceiling (24 shards × 3 workers × 19 min × 0.85 efficiency). LPT bin-packing is working; there's just no headroom left, and the current heaviest-shard prediction is 0.1 min over the hard budget.
Change
One constant: raise the chromium shard budget from 19 → 20 min (aligns it with `TARGET_MS` used by the other lanes).
```python
```
Wall clock is unaffected — the change only widens the acceptance threshold. The predicted heaviest shard runtime is unchanged; the planner will now accept a plan it previously refused.
Follow-up (not in this PR)
This is a stop-gap. Investigation of the timing baseline turned up the actual load hot spots:
Two ~10-minute tests in a single file are effectively immovable in LPT scheduling — they consume 20 % of a shard budget between them. Splitting or trimming those three tests would give ~15 min of aggregate breathing room without another budget bump. Filing separately.
Test plan
🤖 Generated with Claude Code
Greptile Summary
Updates the Playwright shard planner and its tests:
Confidence Score: 5/5
The PR appears safe to merge.
The previously reported stale Chromium budget assertions now match the production 20-minute threshold, and no blocking failure remains.
Important Files Changed
Reviews (2): Last reviewed commit: "address greptile P1: update planning tes..." | Re-trigger Greptile