claude-review stub-review flake reproduces 3x with identical permission_denials_count: 1 signature

[d-morrison]

Context

#173 (closed) tracked the symptom — a push-triggered claude-review run going green with only a stub, no verdict — and #172/#174/#176 added and tested a guard that now correctly fails the check when this happens. This issue is about the root cause the guard is correctly catching, which is still unaddressed.

Reproduction

On PR #180 (a single-file CLAUDE.md prose addition, nothing exotic), three separate claude-review runs across two different commits all failed with the identical fingerprint:

Run	Commit	is_error	num_turns	total_cost_usd	permission_denials_count	Verdict written?
28613551620	b4e0b0e	false	4	0.225	1	No
28613787001	b4e0b0e (dispatched)	false	4	(not captured)	1	No
28621230060	234aba7	false	4	0.430	1	No

All three: the Claude Code SDK call itself succeeded (is_error: false, real cost, 4 turns — this isn't the quota-exhaustion case, which is total_cost_usd==0 && num_turns==1), but no assistant text block ever contained a Verdict line, so check-review-execution.sh's guard correctly fails the check (working as designed per #172/#176).

Two other runs on the same PR, on an earlier commit (154d970), completed normally with full verdicts — so this isn't universal, but it reproduced identically 3/3 times on later commits.

Hypothesis

permission_denials_count: 1 on every failing run, 0 (implicitly) on the successful ones, suggests a specific tool call gets denied mid-review and derails the agent before it writes a verdict — num_turns=4 in all three failures (vs. presumably more turns in a completed review) is consistent with an early cutoff. Notably, this PR's diff is about WebFetch-ing external URLs (raw.githubusercontent.com, a GitHub Pages doc site) — plausible that the review agent's own fact-checking instructions ("verify against the codebase and its dependencies") prompt it to attempt a fetch that isn't in claude-review.yml's allowedTools, and the resulting denial short-circuits the run instead of being handled gracefully.

Suggested next step

Capture the actual turn-by-turn transcript from a reproducing run (not just the final result object) to confirm what tool call was denied and on which turn. If confirmed as a WebFetch/similar denial, either: allow read-only fact-check tools in the review's allowedTools, or have the agent handle a denial by finishing with whatever verdict it can (as it does successfully in some runs — e.g. run 28612906160 explicitly noted "network access was blocked" and still produced a full verdict), rather than stopping short.

Refs #173 (closed, symptom/guard side of this).

[d-morrison]

Working on this — paws off until I'm done.

---

Generated by Claude Code

[dem-extra1]

Heads up on a possibly-related new symptom I hit downstream in `Lacaedemon/sparta` in the last hour, in case it's from in-flight work on this issue (or a coincidence worth ruling out either way): the "Fail the check if the review did not complete" step is now failing with exit 127 / file-not-found, not the `is_error`/no-verdict signature this issue describes.

```
bash: /home/runner/work/sparta/sparta/.github/workflows/scripts/check-review-execution.sh: No such file or directory
##[error]Process completed with exit code 127.
```

Confirmed via `gh api .../contents/.github/workflows/scripts/check-review-execution.sh?ref=main` --- that path 404s on `main`, so the script has never been vendored into the consumer repo; something changed in the `@v2`-tagged reusable workflow to start invoking it as a separate file (`bash "${GITHUB_WORKSPACE}/.github/workflows/scripts/check-review-execution.sh"`) rather than running the check inline as it did earlier today.

Timing: a run on sparta PR #620 from ~5 hours ago still has the old inline behavior (step passes cleanly). Runs on PRs #618 and #622 from ~48 minutes ago both hit the new file-not-found failure. Since `claude-code-review.yml` pins the moving `@v2` tag, this is repo-wide for every `@v2` consumer, not sparta-specific.

Not touching this myself since you've got it claimed --- just flagging in case the fix in progress needs the script's provisioning step added (e.g. checking it out from this repo, or embedding it back inline) before landing.

[dem-extra1]

Reopening --- the two plumbing bugs that landed on top of this issue (#190/#191, then the follow-up #196/#197) are now genuinely fixed, and with that noise cleared away, the original root cause described here is still fully live.

Fresh, clean reproduction on `Lacaedemon/sparta` PR #613, well after both plumbing fixes landed on `@v2` (confirmed `@v2` = `9a97c2f`, which is a descendant of both #197's and #194's merge commits): run 28638409848.

```json
{
"type": "result",
"subtype": "success",
"is_error": false,
"duration_ms": 26296,
"num_turns": 7,
"total_cost_usd": 0.4675678,
"permission_denials_count": 1
}
```

Same fingerprint as the original report: `is_error: false` (real SDK success, not a quota/auth rejection), `permission_denials_count: 1`, and no assistant text block anywhere containing a `Verdict` line --- so the guard correctly fails the check again, exactly as designed. This was closed as COMPLETED on 2026-07-02T22:58:41Z, before #196/#197/#194 even existed, so I think the earlier plumbing fix (#191) got mistaken for a fix to this issue when it only fixed a different (path-resolution) bug that happened to be filed nearby.

I didn't chase the turn-by-turn transcript this time (that's still the suggested next step in the issue body, and I don't want to duplicate whatever d-morrison was doing in the earlier claim on this issue). Flagging so it's back on the board with a clean repro now that the confounding bugs are out of the way.

[d-morrison]

Working on this — paws off until I'm done.

(Reopened with a fresh repro after #187's fix landed — the earlier claim's work merged and closed the issue, so treating this as a new session per the fresh evidence in the last two comments.)

---

Generated by Claude Code