Skip to content

Correct the release PR CI note on the Auto Release PR workflow - #4403

Merged
TaprootFreak merged 1 commit into
developfrom
fix/release-pr-comment-accuracy
Jul 27, 2026
Merged

Correct the release PR CI note on the Auto Release PR workflow#4403
TaprootFreak merged 1 commit into
developfrom
fix/release-pr-comment-accuracy

Conversation

@TaprootFreak

Copy link
Copy Markdown
Collaborator

Why

The explanatory comment added in #4397 is wrong. It claimed that a PR opened with GITHUB_TOKEN
starts no pull_request workflow run for its opened activity, so the release PR would open
without CI — and told reviewers that a green release PR does not mean CI ran.

The first half is false and the advice that followed from it was misleading. Comments only; no
behaviour change in this PR.

What actually happens

Verified against run attempt data, which is where the earlier reading went wrong — a run's
top-level conclusion reflects the latest attempt and hides an earlier action_required.

attempt 1 attempt 2
Bot-opened PR #4399 (f6ca9d1) action_required, 0 jobs manual approval → success, 5/1/2 jobs
Bot-opened PR #4379 (0d2b768) action_required, 0 jobs manual approval → success
synchronize after a human merge (a418b0f) success, 5/1/2 jobs
PAT-authored PR #4394 (3595ff1) success, 5/1/2 jobs

Three things follow:

  1. The runs exist but are held. Attempt 1 has zero jobs. A manual approval starts attempt 2,
    which is what actually executes.
  2. The cause is the repository's Actions approval policy (all_external_contributors), under
    which github-actions[bot] is gated — not the token as such. The API cannot attribute a
    historical action_required to a specific policy decision, so the comment names the policy and
    stops there.
  3. Only the opened runs are held. Later synchronize runs, triggered by human pushes to
    develop, execute on attempt 1. So does everything on a PAT-authored PR.

And one detail the original note got right, which the first correction attempt wrongly dropped:
while a release PR sits unapproved, develop push runs for the same head commit do execute and
appear green next to the empty PR run records. On #4399 those push runs finished at 08:08 while the
PR runs stayed jobless until approval at 08:10.

So a freshly opened release PR shows a mix, and the comment now says so, with an actionable
instruction: approve the held runs, then confirm that what you are reading belongs to the
pull_request runs for the current head.

Note on #4380

This is peripheral to the change but worth recording, since it came out of the same investigation:
the release-PR checks that #4380 set out to enforce were fixed by its removal of the
if: github.head_ref != 'develop' job guards. The token switch in the same PR did not contribute
to that — held runs behave the same either way — and it is what put a personal account's name on
every release PR until #4397. No action proposed here; the guards stay removed.

The comment added in #4397 was wrong. It claimed a PR opened with GITHUB_TOKEN
starts no `pull_request` workflow run at all, so the release PR would open with
no CI. Checking the Actions history properly shows something different, and the
practical advice that followed from it was misleading.

What actually happens, verified against run attempt data:

- The runs for the `opened` activity are created, but held. Attempt 1 completes
  as `action_required` with zero jobs, and a manual approval starts attempt 2,
  which is the one that executes. This follows from the repository's Actions
  approval policy (`all_external_contributors`), not from the token as such.
- Only those runs are held. Later `synchronize` runs from human pushes to
  `develop` execute on attempt 1, and the PAT-authored release PRs did too.
- While a release PR sits unapproved, `develop` push runs for the same head
  commit do execute and show up green next to the empty PR run records.

So a freshly opened release PR shows a mix of pending, jobless entries and real
push results. The comment now says that, and tells a reviewer what to do about
it: approve the held runs, then confirm that what they are reading belongs to
the `pull_request` runs for the current head.

Only comments change; the executable configuration is byte-identical.

Two earlier attempts at this note were also wrong, in the opposite direction
each time. The wording is deliberately narrow now: it names the policy rather
than the token, scopes the gate to the `opened` activity, and keeps the
same-head push warning from the original note, which was the one part of it that
was true.
@TaprootFreak

Copy link
Copy Markdown
Collaborator Author

Pre-review completed in 4 rounds before this PR was opened, two independent reviewers per round,
each on its own working copy. Because the author had already been wrong about this behaviour twice,
the reviewers were given the text as a claim to falsify rather than a fact to check, and were asked
to derive the truth from primary Actions data themselves. Every round found a real defect.

Round 1 — 1 LOW / 1 MEDIUM. The replacement text said no check coverage is lost. Both reviewers
established from run attempt data that this is wrong in the other direction: the opened runs are
created but held at action_required with zero jobs until manually approved.

Round 2 — 1 MEDIUM, found independently by both. Two problems. The text attributed the gate to
the token, but the API reports the repository approval policy as all_external_contributors, and
Actions data cannot tie a historical action_required to a specific policy decision. And the claim
that all listed checks are empty is false: develop push runs for the same head commit do execute
while the PR runs are held.

Round 3 — 1 MEDIUM. "pull_request runs on a bot-opened PR are held" was still too broad. Only
the opened runs are; later synchronize runs from human pushes execute on attempt 1, as verified
on the run that followed a merge into an already-open release PR.

Round 4 — 0 findings from both reviewers.

The wording is deliberately narrow as a result: it names the policy rather than the token, scopes
the gate to the opened activity, and keeps the same-head push warning from the original note —
the one part of it that was correct, and which an earlier correction attempt wrongly removed.

Comments only; the executable configuration is byte-identical to develop. CI: 11/11 green.

@TaprootFreak
TaprootFreak marked this pull request as ready for review July 27, 2026 10:13
@TaprootFreak
TaprootFreak merged commit 643ae23 into develop Jul 27, 2026
11 checks passed
@TaprootFreak
TaprootFreak deleted the fix/release-pr-comment-accuracy branch July 27, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant