Should a "changes recommended" review verdict fail CI? (three reviewers currently disagree)

[d-morrison]

Should a review verdict of "changes recommended" / "Needs more work" fail the
PR's CI check, or should the check only assert that a review happened?

The three AI reviewers this repo ships currently answer that question three
different ways, and nothing documents the divergence as deliberate. This is a
policy choice with a real do-nothing option, so it belongs here rather than in
the tracker; the issues it produces can be filed once it is settled.

Current state

Reviewer	Does the verdict gate the check?
claude-code-review	No. require-review in .github/workflows/claude-code-review.yml asserts only needs.claude-review.result == 'success' -- a review ran and stated some verdict. The verdict text is never parsed.
jules/review	Yes. The pinned third-party action holds statuses: write and reds its own status on VERDICT: block.
gemini-code-review	No gate job at all -- its concurrency: comment notes the absence explicitly.

.github/workflows/scripts/check-review-execution.sh extracts the
verdict-bearing assistant block but deliberately never classifies it: it only
distinguishes ran / stubbed / errored / quota-skipped.

Issue #157 established that every review must end with an explicit verdict line.
This is the follow-on question that one left open -- now that the verdict is
reliably present, does its content belong in the check's pass/fail?

A recommendation, not a conclusion: no, keep gating on "did a review happen"

Four reasons, in descending weight.

1. Verdict-gating has already cost us once, and the bill is unpaid. A
verdict-gated check can only be cleared by changing code or by convincing the
bot. #373 is the live case: Jules re-issues an identical block at a new head on
a false positive that keys on text we are declining to change, so re-running
cannot clear it. The outcomes are merging past a red required check (which trains
everyone to ignore red) or complying with a finding we believe is wrong.
Extending the same mechanism to claude-code-review -- the reviewer that
actually works -- multiplies that failure mode rather than containing it.

2. The verdict vocabulary is deliberately open.
.github/actions/run-claude-review-attempt/action.yml asks for Ready for
merge or Needs more work "(or another clearly labeled alternative, e.g.
Blocked on human review)". There is no closed enum to key pass/fail on.
Gating would require either constraining that vocabulary -- after which a
one-word drift silently turns a real block green -- or writing a fuzzy string
classifier, which is the LLM-judgment-inside-a-check that algorithmatize-checks
exists to rule out.

3. The verdict carries no severity. "Needs more work" covers a nit and a
data-corruption bug alike, even though the prompt asks the reviewer to
distinguish blocking from optional. And the standing address-every-comment rule
already obliges addressing every finding regardless of label, so a red check buys
no additional compliance from a session running ARDI. The only party it informs
is the human, who can read the verdict.

4. It makes red ambiguous. Today red means "the reviewer crashed or never
finished" -- a mechanical, decidable fact, and the reason #368 and #370 are
diagnosable at all. Folding opinion into the same signal means red no longer
distinguishes a broken reviewer from an opinionated one.

The counter-argument, stated fairly

The case for gating is real: a green check sitting next to a "Needs more work"
verdict reads as approval. CLAUDE.md warns about this repeatedly -- a green
claude-review on a silently-skipped self-mod review (#329's case), and the
standing "green means CI state, not review verdict" caution.

But the cause there is that the check's name does not say what it checks, so
the fix may be naming and surfacing rather than blocking. That is the crux worth
arguing over here.

What the decision would produce

Not a checklist to work now -- these are the issues worth filing once this
settles, listed so the scope of the decision is visible:

• Surface the verdict without gating it. A separately-named, non-required
  check (or the sticky comment) reporting the verdict, warn-and-exit-0. In-repo
  precedent: check-new-line-breaks is advisory by design, and
  fully-clean.md's benchmark case is the pattern for a soft-threshold judgment
  being reported rather than gated.
• Reconcile the three reviewers. If gating is rejected, decide whether Jules'
  verdict-gating should be removed for consistency -- it is upstream behavior,
  so that may mean dropping statuses: write, or accepting the divergence and
  documenting it. Related: jules-review.yml lacks the fork guard and job timeout ai-config's copy has #372, Jules reviewer blocks PRs that configure AI reviewers, as "prompt injection" #373.
• Document the outcome in CLAUDE.md beside the existing require-review
  guidance, so the next session does not re-derive it.
• If a verdict gate ever does ship, its documented override path has to land
  with it, not after -- that is precisely what Jules reviewer blocks PRs that configure AI reviewers, as "prompt injection" #373 lacks.

---

Moved from #377, which was filed in the tracker before we settled that a
policy question of this shape belongs on the discussion board.