ci: give every validate job a unique check context#2937
Merged
Conversation
The require-ci ruleset requires one check context named "validate" but 13 workflows all emit a job named validate. That makes the required-check match ambiguous: any one of the 13 checks can satisfy the rule, so PRs can merge while most workflows are still failing or not even running. Add a unique workflow-prefixed name: to every validate job so each one becomes its own required context. Harden the five aggregate validators to fail on any non-success result (cancelled and skipped included, not just failure). Add merge_group to visual-test.yml so its validate context can be required safely. Companion ruleset update lives in the PR description.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
require-ciruleset requires one check context namedvalidate, but 13 workflows all emit a job with that name. With duplicate check names, GitHub's required-check matching is ambiguous: any one of the 13 can satisfy the rule, so PRs can merge while most workflows are still red or not running.This rename gives every
validatejob a workflow-prefixed name so each becomes its own required context, and hardens the five aggregate validators to fail on any non-success result (cancelled and skipped included, not just failure).visual-test.ymlgains amerge_grouptrigger so its validate context can be required in the merge queue.Scope is intentionally narrow. Path-filter skips (a workflow that doesn't match a PR's touched paths never produces a check at all) are a separate class of bug and need a gate-job pattern to fix safely. That's the follow-up. Once gates land, the ruleset can require the full set without blocking unrelated PRs.
Rollout note — ruleset must move with the merge. After this merges, no check is named
validateanymore, so the current ruleset's singlevalidaterequirement matches nothing and every PR blocks on a missing check. Admin bypass is also needed to merge this PR itself (only 4 of 13 workflows trigger on.github/workflows/**). Sequence: