Skip to content

Cancel a PR's superseded Build Test Distribute runs - #6742

Merged
Fedr merged 2 commits into
masterfrom
fedr/ci-concurrency
Sep 1, 2026
Merged

Cancel a PR's superseded Build Test Distribute runs#6742
Fedr merged 2 commits into
masterfrom
fedr/ci-concurrency

Conversation

@Fedr

@Fedr Fedr commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Why

build-test-distribute.yml has no concurrency: group, so pushing to a PR branch leaves the previous run alive alongside the new one. On #6738 that turned into a real failure rather than just wasted minutes.

Two runs of fedr/ubuntu26-target were live at once — my commit, then a second push two minutes later — and both reached the point of pushing the brand-new meshlib/meshlib-ubuntu26 image:

winner (run 33490915399) loser (run 33491107890)
push starts 09:24:44 09:25:36
first blob Pushed @ 09:24:46 Retrying in 5 seconds @ 09:25:36
retries / auth errors 0 3 rounds, then unauthorized @ 09:25:51
manifest pushed 09:28:39 never

The loser was denied on its very first blob POST, 52 s after the winner started and while the winner was still uploading. The token is not the problem — the winner created that repository from nothing with zero retries, and as a control both runs pushed the identical tag source-checksum-f4e25eb26edf59ef to the existing meshlib-ubuntu24 repo over overlapping windows (09:22:01→09:24:22 and 09:24:01→09:26:22) and both succeeded. Existing repo plus concurrent same-tag push is fine; a repository that does not exist yet is created as part of the first push, and a second session asking for push scope on that same path while creation is in flight gets unauthorized. docker push retries only ~15 s, so the loser dies.

What

concurrency:
  group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.run_id }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Only pull_request runs share a group, and only they are cancellable. Every other event (push to master, schedule, workflow_dispatch) keys the group on github.run_id, which is unique per run — so a release run is never cancelled by the next push, and equally never sits queued behind the previous one. Keying those on github.ref instead would have serialized consecutive master pushes behind a 1–2 h pipeline, which is worse than the race it fixes.

Cancelling mid-image-push is safe: the manifest is written last, so a cancelled push leaves only orphan blobs and the next run re-pushes the tag.

Verification

I will push a second commit to this branch on purpose and confirm the first run ends up cancelled rather than racing — that is the only way to exercise this end to end, and I will report the result in a comment.

Notes

Labels disable the platforms this cannot affect; no upload-binaries, since nothing here needs a published release.

Two runs of fedr/ubuntu26-target were live at once (my push, then a second
one 2 minutes later) and both reached the point of pushing the brand-new
meshlib/meshlib-ubuntu26 image. The winner created the repository and pushed
with zero retries; the loser, starting 52 s later while that push was still
in flight, was denied on its very first blob POST and died with
`unauthorized: authentication required` after 15 s of retries. Concurrent
pushes of the same tag to an *existing* repo succeeded in the same two runs,
so it is implicit repo creation that is not concurrency-safe, not the token.

Nothing here should serialize a release, so only pull_request runs share a
group and only they are cancellable; every other event keys the group on
run_id, which is unique per run.
@Fedr
Fedr merged commit 16a84db into master Sep 1, 2026
31 checks passed
@Fedr
Fedr deleted the fedr/ci-concurrency branch September 1, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants