ci: key main concurrency group per-sha so main runs stay conclusive (#436)#481
Merged
Conversation
…436) Issue #436: Playwright E2E (and Criterion benchmarks) were red/cancelled across consecutive main merges and nobody noticed, because main never reached a conclusive result. Root cause (verified): the concurrency group fell back to `github.ref` on push, so every main commit shared one group (`<workflow>-refs/heads/main`). With `cancel-in-progress` false the *running* job completes, but GitHub still supersedes *queued* runs in the group — so in a PR-merge train each new main push cancelled the previous queued run, and main rarely produced a conclusive Playwright/benchmark result. A real regression in any covered view would have looked identical to the benign dataset-growth failures (#435). Fix: fall back to `github.sha` (not `github.ref`) on push, so every main commit gets its own concurrency group and is never superseded. PRs are unchanged — `head_ref` still groups per source branch, so cancel-in-progress keeps superseding stale PR runs. Applied to both workflows that shared the pattern (ci.yml, benchmarks.yml). This is the pure-hygiene half of #436; whether to promote Playwright to a required status check remains a separate maintainer decision.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the pure-hygiene half of #436.
Root cause (verified)
#436 reported Playwright E2E red/cancelled across consecutive main merges with nobody noticing. The concurrency block looked correct (
cancel-in-progressis false on push), yetgh run list --branch=mainshowed the last several main runscancelled. Why?On push,
head_refis empty, so every main commit shares one group:<workflow>-refs/heads/main. Withcancel-in-progress: falsethe running job finishes, but GitHub still supersedes queued runs in the group. In a PR-merge train, each new main push cancels the previous queued main run — so main rarely reaches a conclusive Playwright/benchmark result. A genuine regression would look identical to the benign dataset-growth failures (#435).Fix
Fall back to
github.sha(notgithub.ref) on push:Every main commit now gets its own group and is never superseded. PRs are unchanged —
head_refstill groups per source branch, socancel-in-progresskeeps superseding stale PR runs (the desired behavior). Applied to both workflows that shared the pattern:ci.ymlandbenchmarks.yml.No
run:-context change, so no injection surface —head_ref/shaare only used in the concurrency-group key.Out of scope (your call)
Whether to promote Playwright E2E to a required status check (part 2 of #436) is a separate policy decision left to the maintainer.
🤖 Generated with Claude Code