Skip to content

ci: standardize bot auto-merge - #98

Merged
michen00 merged 4 commits into
mainfrom
ci/standardize-bot-automerge
Aug 5, 2026
Merged

ci: standardize bot auto-merge#98
michen00 merged 4 commits into
mainfrom
ci/standardize-bot-automerge

Conversation

@michen00

@michen00 michen00 commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Summary

Rebuilds bot-automerge.yml on the shape now shared with the sibling repositories, and adds the disarm workflow that makes arming revocable. Two defects motivate it. The check-polling step could not succeed on a healthy pull request — gh pr checks --required --watch races check registration and reports "no required checks reported" before CI has posted — and the do not auto-merge label the workflow tested did not exist in this repository at all, so the kill switch was a condition satisfied by a label nobody could apply. Both are fixed here; the label has been created separately.

The kill switch needed more than a label. Even applied, it only stopped the job from arming again: once auto-merge is enabled it lives server-side, so GitHub goes on merging the moment requirements pass. The new bot-automerge-disarm.yml is the half that actively turns it off, and it fails closed — the tempting shape of "read the label back, disable only when the answer is true" inverts the guard, because a swallowed read reports the label absent and leaves the pull request armed.

dependabot.yml gains a cooldown and the majors policy changes with it, as a matched pair rather than two settings. The actions here are pinned to floating major tags, so Dependabot raises an actions pull request only when a major moves — which made the blanket major hold a permanent hold on every actions pull request there would ever be. Arming those needs a soak period instead, so the cooldown is 7 days on every actions update and 30 on majors. The wider net is earned because this is the one ecosystem whose packages execute with this repository's own token inside jobs holding write scopes; when the risk is a compromised publisher rather than a changed interface, a patch is not safer than a major.

The provenance check is the one addition about safety rather than throughput. .author.login resolves from the commit's author email, so anyone who can push here can set it to Dependabot's noreply address and be read as the bot. Requiring that every commit carry GitHub's own signature — whose payload names committer GitHub <noreply@github.com> — cannot be forged without GitHub's private key, and the payload sits inside the signature so it cannot be edited either. Merges are exempt from the author check but not the signature check, which admits a required Update branch merge without admitting a locally crafted evil merge.

Test plan

  • pre-commit run --files passes on all three changed files — Validate Dependabot Config (v2), Validate GitHub Workflows, Lint GitHub Actions workflow files, yamllint, prettier, typos, codespell
  • actionlint clean across the whole workflows directory, which covers the embedded shellcheck pass on every run: block
  • yamllint caught a 110-character line on the first pass; shortened and re-run clean
  • The do not auto-merge label now exists here — confirmed via repos/michen00/bin/labels/do%20not%20auto-merge
  • All three commit subjects are within this repository's title-max-length=50
  • CI is green — pending on this head
  • A live bot-triggered run. Not verifiable before merge; the next Dependabot or pre-commit-ci pull request is the check to watch.

Reviewer guide

  • Effort: ~15 minutes. The diff is large but mostly comment; three decisions carry it.
  • Read only these: the Decide eligibility step's semver-major branch, the Verify every commit was created by GitHub step, and the fail-closed loop in bot-automerge-disarm.yml.
  • Already covered, please skip: linting, schema validation, the label's existence, and commit-subject lengths are all verified above.
  • The calls only you can make: whether arming github-actions majors on a 7-day cooldown is the trade you want here. The alternative that keeps a hold meaningful is a per-workflow list — hold when the diff reaches a workflow no pull request runs — which the publishing siblings do carry. This repository publishes nothing: every workflow except changelog-autoupdate.yml and pre-commit-autoupdate.yml runs on pull requests, and a break in either of those is a scheduled or dispatched job that fails visibly. So a list here would be maintenance with nothing behind it, and a stale list fails by arming. Worth checking rather than accepting.
  • Known weaknesses: the App path is written but inert, because APP_ID and APP_PRIVATE_KEY are not set in this repository. Nothing breaks — arming falls back to the default token, which is what happens today — but bot merges keep firing no downstream push or pull_request_target: closed events. If those settings are ever added, the private key has to go into both the Actions and Dependabot secret stores, or the App path stays inert on exactly the Dependabot pull requests it was added for. The comment in the file says so; this is the sibling repository's arrangement and the reason both use the same two names.

Notes

main-protect here also carries a Copilot review rule, and require-maintainer-review asks for an approving review on top. Nothing in this workflow waits on either — that is the point of letting native auto-merge do the waiting server-side. The approve step is therefore not optional in this repository, unlike in the siblings that require no review: without it auto-merge would arm and then wait forever on a requirement no bot pull request can satisfy for itself.

The three commits are ordered so each leaves a working tree: the cooldown lands before the policy that depends on it. The workflow rewrite is one commit rather than four because its parts are interdependent — removing the polling step requires native auto-merge to be the gate, and the majors policy lives inside an eligibility step the provenance check gates — so a split would commit a tree that does not work. The commit message enumerates the parts instead.

Files touched

Status legend: + added, ~ modified, - removed, renamed, ~/→ renamed and modified.

status path role
~ .github/dependabot.yml adds the cooldown that replaces the blanket major hold, 7 days on every actions update and 30 on majors
~ .github/workflows/bot-automerge.yml arms actions majors, verifies commit provenance, drops the racing check poll, narrows permissions, re-reads the kill switch before arming
+ .github/workflows/bot-automerge-disarm.yml the missing half: actively disables an armed auto-merge when the label lands, failing closed and verifying rather than trusting

michen00 and others added 3 commits August 4, 2026 23:04
Pairs with the eligibility change that follows. bot-automerge.yml held every
semver-major, and because the actions here are pinned to floating major tags
Dependabot raises a pull request only when a major moves -- so that hold was a
permanent hold on every actions pull request rather than a policy.

Arming those needs a soak period to replace the hold, which is what this is.
default-days covers every update and not only majors, because this is the one
ecosystem whose packages execute with the repository's own token inside jobs
holding contents and pull-requests write. When the risk is a compromised
publisher rather than a changed interface, a patch is not safer than a major.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings this file onto the shape the sibling repositories share. The parts are
interdependent enough that splitting them would leave a tree that does not work,
so they land together and are listed here instead.

Removes the check-polling step. `gh pr checks --required --watch` races check
registration: it reports "no required checks reported" before CI has posted and
fails the job on a pull request that was never unhealthy. Native auto-merge
already gates on the ruleset server-side, and main-protect requires both Run CI
legs, so the wait was redundant as well as wrong.

Narrows permissions to what the calls actually use. packages: read was never read
by anything here -- this repository is shell scripts and BATS. Declaring the
block sets every unlisted scope to none, so a scope kept for symmetry grants real
authority for no reader.

Replaces the github.actor guard with pull_request.user.login. github.actor is
whoever triggered this event, not whoever opened the pull request, so any later
push by another identity made the synchronize event's actor that identity and the
job skipped -- leaving only the opened event as a chance to act, with no retry.

Adds a commit-provenance check, which is the guard the author test cannot be:
.author.login resolves from the commit author email, so anyone who can push here
can set it to the bot's noreply address. Requiring GitHub's own signature, whose
payload names committer GitHub <noreply@github.com>, cannot be forged without
GitHub's key. Merges are exempt from the author check but not the signature one,
which is what admits a required Update branch merge without admitting a local
evil merge.

Arms github-actions majors, per the cooldown added in the previous commit, and
holds every other ecosystem. Nothing here publishes from a workflow no pull
request runs, so there is no per-workflow hold list of the kind the publishing
siblings carry. Re-reads the kill-switch label live before arming, because a
label applied while the run was in flight is absent from the event payload.

Also replaces the ::info:: lines, which are not a workflow command GitHub
recognises and so printed literally rather than annotating anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The kill switch was a condition rather than a control. bot-automerge.yml tested
the label before arming and never again, so a label applied afterwards stopped
the workflow re-running while GitHub went on merging the moment requirements
passed. Once auto-merge is enabled it lives server-side; something has to
actively turn it off, and nothing did.

Fails closed on purpose. The tempting shape -- read the label back, disable only
when the answer is true -- inverts the guard, because a swallowed read then
reports the label absent and leaves the pull request armed. So only a positively
confirmed "off" exits early; anything else attempts the disable, retries, and
verifies. A final read after the loop stops a disarm that worked on the last
attempt from reporting failure.

No branch filter and no fork guard: a suppressor has to be at least as broad as
the thing it suppresses, and disarming a pull request that was never armed is a
no-op, so running when it need not is the cheap failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 5, 2026 06:05
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

CI: standardize bot auto-merge with disarm + provenance guardrails

🐞 Bug fix ✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

Grey Divider

AI Description

• Replace fragile required-check polling with GitHub native auto-merge gating.
• Add a dedicated disarm workflow to reliably revoke bot auto-merge via label.
• Tighten safety: verify bot PR commit provenance and tune Dependabot cooldowns.
Diagram

graph TD
  Dep["dependabot.yml cooldowns"] --> PR(("Pull request events")) --> Arm["bot-automerge.yml (arm)"] --> GHCLI["gh CLI / API calls"] --> Repo[("GitHub PR state")] --> Auto["Native auto-merge"]
  Label(("Label applied")) --> Disarm["bot-automerge-disarm.yml"] --> GHCLI

  subgraph Legend
    direction LR
    _evt(("Event")) ~~~ _wf["Workflow"] ~~~ _api["API/CLI"] ~~~ _db[("GitHub state")]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Use GitHub REST/GraphQL via actions/github-script (no gh CLI)
  • ➕ Removes dependency on gh CLI behavior/availability
  • ➕ More explicit API responses and error handling
  • ➕ Easier to unit-test logic in JS with reusable helpers
  • ➖ More verbose implementation for pagination, approvals, and merge toggling
  • ➖ Requires maintaining JS snippets and API versioning details
2. Adopt a dedicated automerge bot/action (e.g., Mergify/third-party action)
  • ➕ Feature-rich policy engine (labels, rules, retries) out of the box
  • ➕ Often includes robust re-check/disarm semantics
  • ➖ Additional vendor/system to trust for write-scoped operations
  • ➖ More operational overhead and configuration surface area
3. Rely solely on rulesets + manual bot approvals (no auto-arming)
  • ➕ Simplest security posture; no workflow with unconditional merge powers
  • ➕ Avoids concurrency/label race complexity entirely
  • ➖ Loses throughput gains from hands-off bot maintenance
  • ➖ Still needs a process for timely merges once checks pass

Recommendation: Keep the PR’s approach (GitHub native auto-merge + explicit disarm + provenance checks). It minimizes moving parts while directly addressing the real failure modes: unreliable check polling, non-existent kill-switch label, and the server-side persistence of auto-merge once armed. The added provenance gate materially improves safety for write-scoped workflow execution, and the separate disarm workflow correctly makes revocation reliable even under event ordering and transient read failures.

Files changed (3) +401 / -55

Enhancement (1) +98 / -0
bot-automerge-disarm.ymlNew workflow to actively disable auto-merge on kill-switch label +98/-0

New workflow to actively disable auto-merge on kill-switch label

• Adds a dedicated disarm workflow triggered on label events that disables GitHub native auto-merge when 'do not auto-merge' is applied. Implements fail-closed behavior with retries and explicit confirmation reads to avoid leaving PRs armed due to transient API/CLI failures or event races.

.github/workflows/bot-automerge-disarm.yml

Bug fix (1) +288 / -55
bot-automerge.ymlRebuild bot auto-merge: safer eligibility, provenance checks, and label race fixes +288/-55

Rebuild bot auto-merge: safer eligibility, provenance checks, and label race fixes

• Refactors the arming workflow to remove brittle required-check polling and rely on GitHub’s server-side ruleset gating. Adds provenance verification to ensure commits are GitHub-signed and authored by the bot (with a controlled merge exemption), re-reads labels at arm-time to close ordering gaps, and supports optionally arming under a GitHub App token while keeping permissions tight.

.github/workflows/bot-automerge.yml

Other (1) +15 / -0
dependabot.ymlAdd Dependabot cooldown policy for GitHub Actions updates +15/-0

Add Dependabot cooldown policy for GitHub Actions updates

• Introduces cooldown delays for GitHub Actions updates (7 days default, 30 days for semver majors) to replace the previous blanket major-hold behavior. Adds detailed rationale tying cooldowns to the repository’s automerge eligibility policy and supply-chain risk model.

.github/dependabot.yml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR standardizes the bot auto-merge setup to match sibling repositories, fixes the prior “required checks” polling race, makes the do not auto-merge kill switch effective by adding an explicit disarm workflow, and adjusts Dependabot policy to use cooldowns instead of a blanket major hold.

Changes:

  • Rebuilds bot-automerge.yml to rely on native GitHub auto-merge gating (no check polling), add commit provenance verification, and re-check the kill switch right before arming.
  • Adds bot-automerge-disarm.yml to actively disable auto-merge when the kill-switch label is applied (fail-closed loop).
  • Updates .github/dependabot.yml to apply cooldowns for GitHub Actions updates (7 days default; 30 days for semver majors).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
.github/dependabot.yml Adds cooldown settings to replace a blanket major hold and provide a soak period for Actions updates.
.github/workflows/bot-automerge.yml Reworks arming logic (no polling), adds provenance checks, and strengthens kill-switch handling before approval/arming.
.github/workflows/bot-automerge-disarm.yml New workflow to disable auto-merge when the kill-switch label is applied, using a retry/verify loop.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/bot-automerge.yml
Comment thread .github/workflows/bot-automerge-disarm.yml
Comment thread .github/workflows/bot-automerge-disarm.yml
@qodo-code-review

qodo-code-review Bot commented Aug 5, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Provenance check spoofable ✓ Resolved 🐞 Bug ⛨ Security
Description
The provenance gate classifies commits as GitHub-created when they are merely “verified” and their
signed payload contains a committer line, but it never verifies the signature is GitHub’s. Since
committer name/email are user-controlled git metadata, a pusher can create a locally signed commit
with committer GitHub <noreply@github.com> and bypass the guard, allowing auto-merge to proceed on
non-GitHub-created commits.
Code

.github/workflows/bot-automerge.yml[R191-194]

+            (if (.commit.verification.verified == true)
+                and ((.commit.verification.payload // "") | split("\n")
+                     | any(startswith("committer GitHub <noreply@github.com>")))
+             then "github" else "elsewhere" end),
Evidence
The provenance logic only checks for a valid signature and the presence of a committer string in the
signed payload, and then uses that result to accept/reject commits; there is no check tying the
signature to GitHub’s signing identity.

.github/workflows/bot-automerge.yml[190-197]
.github/workflows/bot-automerge.yml[215-224]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The provenance step in `bot-automerge.yml` treats a commit as “GitHub-created” based on `verification.verified == true` plus a committer string found in the signed payload. This does not prove the signer is GitHub; committer metadata can be set arbitrarily before signing.

### Issue Context
The `origin` classification derived from this check is used to decide whether a PR is `trusted` and thus eligible for bot approval and `gh pr merge --auto`.

### Fix Focus Areas
- .github/workflows/bot-automerge.yml[156-230]

### What to change
- Replace the current `payload`-substring heuristic with an API signal that actually binds the signature to GitHub.
 - Prefer GraphQL `Commit.signature.wasSignedByGitHub` (or equivalent) if available via `gh api graphql`.
 - If GraphQL isn’t feasible, augment the REST-based check to validate signer identity/key (e.g., signer/login/fingerprint fields if present) rather than trusting committer metadata.
- Keep the fail-closed behavior (`set -euo pipefail`) and ensure any missing/unknown signature provenance results in `trusted=false` (hold for human).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Kill switch case sensitive ✗ Dismissed 🐞 Bug ☼ Reliability
Description
The kill-switch label is matched case-sensitively in both the disarm workflow and the main job-level
gate, so a case-variant label name can prevent disarming and allow a previously-armed PR to stay
armed. This contradicts the workflow’s own later comment/implementation that treats the label match
as case-insensitive before arming.
Code

.github/workflows/bot-automerge-disarm.yml[R40-42]

+  disarm:
+    if: github.event.label.name == 'do not auto-merge'
+    runs-on: ubuntu-latest
Evidence
The disarm workflow’s job condition and the main workflow’s job gate both rely on case-sensitive
comparisons, while the later arming step explicitly documents and implements a case-insensitive
match—showing intended behavior and current inconsistency.

.github/workflows/bot-automerge-disarm.yml[39-42]
.github/workflows/bot-automerge.yml[59-63]
.github/workflows/bot-automerge.yml[297-302]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`bot-automerge-disarm.yml` only runs when `github.event.label.name == 'do not auto-merge'` (case-sensitive). `bot-automerge.yml` also uses a case-sensitive `contains(..., 'do not auto-merge')` at the job gate, even though the later live re-check uses a case-insensitive match.

### Issue Context
If the configured label’s casing differs (e.g., `Do Not Auto-Merge`), the disarm job can be skipped even though the label was applied, leaving server-side auto-merge enabled.

### Fix Focus Areas
- .github/workflows/bot-automerge-disarm.yml[39-42]
- .github/workflows/bot-automerge.yml[59-63]

### What to change
- Make the disarm condition case-insensitive, e.g.:
 - `if: toLower(github.event.label.name) == 'do not auto-merge'`
- Make the bot-automerge job-level gate case-insensitive as well (to align with the later grep behavior), e.g. normalize the joined label list:
 - `!contains(toLower(join(github.event.pull_request.labels.*.name, '\n')), 'do not auto-merge')`
- Keep the existing live re-check before arming (already case-insensitive).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

Comment thread .github/workflows/bot-automerge.yml
Comment thread .github/workflows/bot-automerge-disarm.yml
A review flagged the provenance gate as spoofable and was half right. It
classified a commit as GitHub-created from `verified == true` plus a payload line
naming `committer GitHub <noreply@github.com>` -- and committer name and email
are git metadata the pusher controls. The claimed bypass, a locally signed commit
carrying that committer line, most likely fails anyway, because GitHub declines
to mark a commit verified when the committer email is not a verified email of the
signing key's owner. But that is a premise the reader cannot check from the file,
and a security guard should not rest on one.

So the gate now requires `.committer.login == "web-flow"`, which GitHub resolves
from the signing key rather than from metadata, plus `reason == "valid"`, and
keeps the payload test as an independent second condition. Measured across four
repositories before changing anything: Dependabot and pre-commit-ci commits both
report web-flow with verified true and reason valid, while a human's own commits
report that human's login -- so this rejects exactly what it should and admits
both bots.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@michen00
michen00 merged commit 9a68375 into main Aug 5, 2026
9 checks passed
@michen00
michen00 deleted the ci/standardize-bot-automerge branch August 5, 2026 06:22
Repository owner deleted a comment from chatgpt-codex-connector Bot Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants