Skip to content

Fix Dependabot CI gate: read checks authoritatively, don't misread API blips as "no CI"#12

Merged
bbkrr merged 1 commit into
mainfrom
fix/gated-merge-authoritative-checks
Jul 15, 2026
Merged

Fix Dependabot CI gate: read checks authoritatively, don't misread API blips as "no CI"#12
bbkrr merged 1 commit into
mainfrom
fix/gated-merge-authoritative-checks

Conversation

@bbkrr

@bbkrr bbkrr commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

What & why

The Dependabot auto-merge CI gate (gated_merge.sh) was parking healthy patch/minor bumps for manual review because it could not tell "this repo has no CI" apart from "I couldn't read the checks right now."

It discovered checks via gh pr checks (the GraphQL status rollup) and mapped empty stdout → "no checks". When several Dependabot PRs open in the same instant, their auto-merge runs poll the API concurrently and the rollup can return nothing for the whole 300s "appear" window — even though the PR's tests check is attached and already green. The gate then fired its fail-safe and labeled needs-manual-review.

Confirmed in realrate/EBA: PRs #99 (pmdarima, minor) and #100 (openpyxl, patch) opened in the same 10-second batch and both got "no CI checks registered within 300s", while the REST check-runs API shows their tests check completing success in 38s — green the whole time the gate saw nothing. Full diagnosis in #11.

The fix

  • Read checks authoritatively from the REST APIs for the PR head SHA: commits/{sha}/check-runs (+ commits/{sha}/status for legacy statuses), excluding this workflow's own run by run-id. These return a well-formed body with a reliable count even while checks are pending, and a real non-zero exit on a genuine error.
  • Distinguish an API failure from a genuine empty result. A failed read now yields a __ERR__ sentinel so the loops keep polling instead of concluding "no CI". Only a successful empty result past APPEAR still routes to a human.
  • Everything else is preserved: exit 0 on every path, merge only when all non-self checks pass, hold red/cancelled for a human, and keep the genuine-no-CI fail-safe.

Tests

test_gated_merge.sh runs the real script against a stubbed gh and pins:

scenario expected
API blips (3×) then green merges (the #11 regression)
green immediately, own in-progress run present merges (self-run excluded)
genuinely no CI routes to human (fail-safe intact)
red CI routes to human

A new tooling-tests.yaml workflow runs shellcheck + these tests on any change under .github/review-tooling/. Both are shellcheck-clean and green locally.

Caveats / scope

  • Only the CI-gate read path changes; classification (patch/minor vs major), approval, and the AI major-bump gate are untouched.
  • The numpy #101 failure from the same batch was a separate issue (AI verdict step raised PermissionDeniedError) and is out of scope here.
  • No untrusted PR code is executed: the query path uses the repo parsed from our own PR URL and a commit SHA from the API; the pull_request_target workflow still never checks out the bump.

Closes #11. Hardens the #2244 CI gate (epic RealRate-Private#2229).

🤖 Generated with Claude Code

gated_merge.sh discovered the PR's checks via `gh pr checks` (the GraphQL
status rollup) and mapped empty stdout to "no checks reported". Under a
simultaneous multi-PR Dependabot batch the rollup can transiently return
nothing, so a check that was attached and already green was misread as
"this repo has no CI" -- and after the 300s APPEAR window the gate parked
healthy patch/minor bumps with `needs-manual-review` (#11).

Read checks authoritatively from the REST check-runs + commit-status APIs
for the PR head SHA instead, and distinguish an API failure from a genuine
empty result: a failed read now yields a "__ERR__" sentinel so the loops
retry rather than concluding "no CI". The genuine-no-CI, red-CI, and
exit-0-on-every-path guarantees are preserved.

Adds test_gated_merge.sh (stubbed `gh`) covering: API blips then green ->
merge, no CI -> human, red CI -> human, self-run exclusion; and a
tooling-tests workflow running shellcheck + the tests on tooling changes.

Closes #11

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bbkrr
bbkrr merged commit b501078 into main Jul 15, 2026
2 checks passed
@bbkrr
bbkrr deleted the fix/gated-merge-authoritative-checks branch July 15, 2026 07:47
bbkrr added a commit that referenced this pull request Jul 15, 2026
The CI gate (gated_merge.sh, #12) now reads checks authoritatively from the
REST Checks API (commits/{sha}/check-runs) and legacy commit statuses
(commits/{sha}/status). The workflow's `permissions:` block did not grant
`checks`/`statuses`, and with a permissions block present any unlisted scope
defaults to `none`. On a PRIVATE repo those reads then 403, which the gate
treats as an unreadable-checks error -- so it fails safe and never merges
(observed on realrate/EBA#100: gate looped on `__ERR__` until timeout).

Granting both read scopes lets the gate see the PR's checks and merge green
patch/minor bumps as intended. Both are read-only.

Refs #11.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dependabot CI gate: transient check-read is misread as "no CI", parking healthy patch/minor bumps

1 participant