Dependabot 0.x-minor hold COMMENTS but does not enforce: #938 kept auto-merge enabled, and the hold label silently failed

[avrabe]

What happened

#938 (object 0.39.1 → 0.40.0) is a 0.x minor = breaking bump. The dependabot-auto-merge.yml hold logic added in #866 fired correctly — the PR carries two 🔒 0.x MINOR bump — held for manual review comments.

But when I checked it today, three days after it opened:

auto-merge: ENABLED (SQUASH)
labels:     dependencies          <- no `major-bump-hold`
checks:     53 pass, 2 FAIL       <- Test and Clippy

So the gate announced a hold it did not enforce. Had the two failing checks been rerun into green — or been flaky — the PR would have squash-merged itself onto main unattended. That is exactly the ordeal 0.9→0.12 shape (auto-merged as "minor", hung Test+Z3 for hours) and #864 (landed with no CI run, left main uncompilable) — the two incidents this hold was written to prevent.

I have disabled auto-merge on #938 manually. The bump is genuinely breaking: Test and Clippy both fail, which is the rule doing its job.

Two distinct defects

1. The hold does not disable an already-enabled auto-merge. The workflow's enable step is correctly skipped when hold == 'true':

- name: Enable auto-merge for patch + true-minor updates
  if: (... patch || ... minor) && steps.zerox.outputs.hold != 'true'

But skipping the enable is not the same as disabling. If auto-merge is enabled on any earlier run — a re-pushed Dependabot branch, a run where fetch-metadata returned an empty previous-version so case "" in 0.*) did not match and hold came out false, or a manual enable — nothing ever turns it back off. The hold branch should call gh pr merge --disable-auto rather than only commenting.

2. The hold label silently fails. gh pr edit "$PR_URL" --add-label "major-bump-hold" 2>/dev/null || true swallows the failure when the label does not exist in the repo, which is why #938 shows only dependencies. The || true was presumably there so a labelling hiccup could not fail the job — but the effect is that the hold's only durable, visible marker is optional. The comment survives; the label, which is what a human or a query would filter on, does not.

Why this is the recurring shape

A check that reports a condition it does not enforce is indistinguishable from a check that works, right up until it matters. Same family as #911 (artifacts citing tests that existed in 0 files) and the v0.56.2 finding that a mutation test which never mutated prints ok. The control has to be sensitive to the failure it controls for — here, the hold's own success criterion should be "auto-merge is off", not "a comment was posted".

Suggested fix

• In the hold branch, gh pr merge --disable-auto "$PR_URL" before commenting.
• Create the major-bump-hold label in the repo and drop the || true, or assert the label landed.
• Red-first: verify on a PR with auto-merge deliberately pre-enabled that the hold turns it off — a hold that only skips its own enable step passes a naive test vacuously.

Found while cutting v0.56.2.