Seen on UCD-SERG/serocalculator#392, run 30148683481.
The review ran fine and produced a genuine verdict. But the "Claude finished review" comment it posted contained the reviewer's own shell command, verbatim, rather than the review text:
**Claude finished review** — [View run](...)
---
gh pr comment 392 --repo UCD-SERG/serocalculator --body "$(cat <<'EOF'
## Code review
Re-reviewed the current diff (head `17c7496f`). ...
### Verdict
**Needs more work** — one file-naming convention issue to address before merge; everything else checks out.
EOF
)"
The same review body was also posted correctly as a separate claude[bot] comment, so the PR ended up with the review twice — once rendered, once wrapped in an unrendered command. The leading gh pr comment ... "$(cat <<'EOF' and trailing EOF\n)" render as literal text.
Why it matters
It's cosmetic in the sense that no information is lost, but it degrades the two things the guard logic depends on:
- Verdict detection.
check-review-execution.sh looks for a verdict in the posted body. Here the verdict is present, just embedded in a shell command — so this run passed, but a variant that wrapped things slightly differently could plausibly read as a stub review and trigger the gha#185 retry path for no reason, burning a second full review (~$5.55 on this run).
- Reader trust. A comment that opens with a raw
gh invocation looks like a broken run, so a human is likely to discount a review that was actually complete and correct — its one finding was legitimate and I fixed it.
Root cause guess
The reviewer model emitted a tool-invocation-shaped string as its final response text instead of just the message, and the workflow posted that text as-is. That's a model-behavior failure, but the workflow is where it can be defended against.
Suggested fix
Unwrap the pattern before posting: if the body matches ^\s*gh (pr|issue) comment .*<<'?EOF'? … EOF\s*\)?"?\s*$, extract the heredoc contents and post those instead. Cheap, and it also protects against the duplicate-post case (detect that the extracted body was already posted and skip).
Fixture-testable offline alongside the existing cases in .github/workflows/scripts/tests/fixtures/, in the same shape as the stub_review fixtures added for gha#185 — the input is just an execution-output file whose final result text is the wrapped command.
Seen on
UCD-SERG/serocalculator#392, run 30148683481.The review ran fine and produced a genuine verdict. But the "Claude finished review" comment it posted contained the reviewer's own shell command, verbatim, rather than the review text:
The same review body was also posted correctly as a separate
claude[bot]comment, so the PR ended up with the review twice — once rendered, once wrapped in an unrendered command. The leadinggh pr comment ... "$(cat <<'EOF'and trailingEOF\n)"render as literal text.Why it matters
It's cosmetic in the sense that no information is lost, but it degrades the two things the guard logic depends on:
check-review-execution.shlooks for a verdict in the posted body. Here the verdict is present, just embedded in a shell command — so this run passed, but a variant that wrapped things slightly differently could plausibly read as a stub review and trigger the gha#185 retry path for no reason, burning a second full review (~$5.55 on this run).ghinvocation looks like a broken run, so a human is likely to discount a review that was actually complete and correct — its one finding was legitimate and I fixed it.Root cause guess
The reviewer model emitted a tool-invocation-shaped string as its final response text instead of just the message, and the workflow posted that text as-is. That's a model-behavior failure, but the workflow is where it can be defended against.
Suggested fix
Unwrap the pattern before posting: if the body matches
^\s*gh (pr|issue) comment .*<<'?EOF'?…EOF\s*\)?"?\s*$, extract the heredoc contents and post those instead. Cheap, and it also protects against the duplicate-post case (detect that the extracted body was already posted and skip).Fixture-testable offline alongside the existing cases in
.github/workflows/scripts/tests/fixtures/, in the same shape as thestub_reviewfixtures added for gha#185 — the input is just an execution-output file whose finalresulttext is the wrapped command.