Constrain what may follow review, and fix two runner-portability gaps - #348
Conversation
|
The SDK errored at startup before the model produced anything: {"type":"result","subtype":"success","is_error":true,"duration_ms":643,
"num_turns":1,"total_cost_usd":0,"permission_denials_count":0}643 ms, one turn, $0 spent, zero permission denials. Every
This diff cannot reach the review path: it touches The shape -- clean install, immediate Since no automated verdict will be produced while this holds, the substance of the change is covered in the PR description, and the pattern changes are pinned by 38 offline cases with both new guards confirmed to bite when reverted. I will re-check when the workflow recovers. Generated by Claude Code |
Self-reviewThe automated reviewer is down (see the comment above), so this is a self-review, per the fallback in Finding: the prose understated the cost, in all three places it appearedFixed in f3c06e6. The script comment,
Both are pure review requests carrying no instruction to the agent, which is exactly the population the tail constraint was not meant to touch. Citing a weaker example than the ones I had measured is the failure mode This is the one open judgment call in the PR. Checks that came back clean
Considered and deliberately not done
Generated by Claude Code |
|
👀 Picked up by workflow run #30381851377. Setup runs first; Claude itself responds after that. |
|
Correction to my earlier comment on this failure: the cause is known and is not a credentials problem. The bot's account is out until Thursday. Disregard the suggestion to check or rotate Everything else in that comment stands: the immediate Practical consequence for this PR: no automated verdict is obtainable before Thursday, and re-triggering will not produce one. The documented fallback applies -- the self-review above, plus the rest of CI. I have stopped polling hourly and will re-check on Thursday. Generated by Claude Code |
|
@claude review |
|
@claude review |
Post-merge follow-ups to #341, filed as #346. other end open: `@claude can you review this and fix the failing test?` matched, and a match suppresses claude.yml's "Post Claude's response if no code was committed" step. The question went to a read-only reviewer and nobody answered it -- the exact harm the closed-set lead-in was chosen to prevent. `review` may now take an object, but only from a second closed set of deictic references to the PR under discussion, and the request has to end its line. That end-of-line requirement subsumes the old `[^[:alnum:]]|$` whole-word guard against `@claude reviewer`. Anchoring on a bare newline makes CRLF normalization load-bearing, where before it was dead code: `sed 's/\r$//'` only works under GNU sed, so it is now `tr -d '\r'`. The composite probes `base64 -d` vs `-D` for the same reason -- `runs-on` is a consumer-settable input, which is already why the bodies-file encoding avoids `jq --raw-output0`. Also: the late-comment detection step now degrades an unreadable bodies-file to "no late review" instead of reddening the job, matching the policy the collecting step right above it already had, and both steps log their result so a `false` outcome is no longer silent. Tests: 38 cases pass. The four new negatives fail if the tail constraint is reverted, and the CRLF case fails if the normalizer is; the composite's shell logic was simulated against the same payloads the selftest builds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWWei35DwEzyAgzfrH1BfK
Self-review finding. All three places describing the cost of the closed tail set cited `@claude review the test-coverage changes` -- an invented phrasing chosen to sound marginal. The real regressions against #341 are `@claude review the changes I just pushed` and `@claude please review when you get a chance`, both pure review requests carrying no instruction to the agent, and both natural enough to expect in practice. Cite the honest example, and pin both in the test table so widening TAIL_WORD to recover them is a deliberate decision rather than a drift. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BWWei35DwEzyAgzfrH1BfK
f3c06e6 to
58f287f
Compare
|
Rebased onto #351 ( One conflict, in Verified after the rebase: no Two corrections to the PR description above, both now stale:
The failures earlier in this thread were two unrelated outages back to back -- account quota, then the org rename -- neither caused by this diff. #350 tracks the first one, which is a real bug: the quota-exhaustion graceful-skip path cannot be reached, because the step that detects it is gated on the step that quota exhaustion always fails. Generated by Claude Code |
2026-07-28 19:25 UTC |
|
Working on this — paws off until I'm done. Self-review. Posted because What I verified rather than readThe full test table passes: 40 cases, up from 27 on All eight phrasings the docs promise behave as documented.
Eleven adversarial cases beyond the committed table, aimed at the new end-of-line anchor, since that is the part most likely to have collateral effects: bare
Added lines are free of em-dashes, en-dashes, curly quotes, and Findings1. The PR body's test count is wrong (minor, factual). It says "38 cases pass, up from 27." The harness reports 40. The 2. Things I checked and found correct
VerdictReady for merge once finding 1 is corrected — one number in the PR body, no code change. Sequencing note: this PR cannot be validated end-to-end until #351 lands, because Generated by Claude Code |
|
👀 Picked up by workflow run #30387037425. Setup runs first; Claude itself responds after that. |
|
Claude finished review — View run Based on my review, I independently traced the regex logic ( I found no new issues beyond what the PR author's own self-review already surfaced. Review summaryThis PR is already covered by a thorough self-review posted on the thread: comment #issuecomment-5108082215 (2026-07-28 18:21 UTC), posted because the automated That self-review:
I confirmed finding 1 is already resolved: the current PR body's Verification section reads "40 cases pass, up from 27," matching the harness output, and this matches commit I found no new issues in the diff beyond what that self-review already covered and resolved. My own regex trace confirms the VerdictReady for merge — the prior self-review's verdict stands; finding 1 (the only blocking item) has already been corrected, and no new issues surfaced in this pass. |
Closes #346. Post-merge follow-ups to #341.
1. The tail was left open
#341 constrained only what may precede the keyword. The other end stayed
open, so these matched:
All four were
falseunder the old@claude[[:space:]]+reviewpattern.A match suppresses
claude.yml's "Post Claude's response if no code wascommitted" step (
claude.yml:844), so each of these went to a read-onlyreviewer and nobody answered the question -- the exact harm the closed-set
lead-in was chosen to prevent, arriving through the other end of the pattern.
reviewmay now take an object, but only from a second closed set of deicticreferences to the PR under discussion (
this,the latest changes,again,trailing politeness), and the request has to end its line. Same idea as
POLITE, applied to both sides of the keyword.That end-of-line requirement also subsumes the old
[^[:alnum:]]|$whole-wordguard, so
@claude reviewerstill fails without a separate clause.The trade is deliberate and in the cheap direction. An unlisted-but-genuine
object --
@claude review the test-coverage changes-- now self-reviewsinstead of dispatching. By #341's own asymmetry a false negative costs a
self-review; a false positive costs an unanswered question.
2. CRLF normalization was dead code, and is now load-bearing
I verified the
sed 's/\r$//'in #341 changed no outcome on any case:\rwasalready covered by
[[:space:]]in the separator class and[^[:alnum:]]inthe word-boundary class.
Anchoring the tail on a bare newline makes it matter, because GitHub delivers
comment bodies with CRLF. So rather than deleting it as #346 first suggested,
it becomes
tr -d '\r'--\rin a sed BRE is a GNU extension, and BSD/macOSsed reads it as a literal
rand strips trailingrs instead.3.
base64 -dis not portable eitherGNU coreutils spells decode
-d; BSD/macOSbase64spells it-D. Unlikeitem 2 this is on the live path for the late-comment rescan, and per the
composite's deliberate no-
|| truepolicy a decode failure reddens the callingjob. It now probes once and picks the right flag.
Items 2 and 3 are the same standard #341 already applied when it declined
jq --raw-output0becauseruns-onis a consumer-settable input.4. Two halves of the late-dispatch path disagreed
"Collect comments posted after the trigger" ends
|| :+touchso atransient
gh apierror degrades to "no late review". The step right after itdid the opposite by design, under
set -euo pipefail, so an unreadablebodies-file could redden the whole job over what the workflow's own comment
calls an optional nicety. It is now
continue-on-error: true-- still red inthe log, and an empty
matchleaves the dispatch skipped.Both steps also log their result now. A
falseoutcome was previouslyinvisible: the old step's
No late @claude review requests.line went awaywith the refactor, and gated steps just render as skipped.
Verification
constraint: exactly those four fail, nothing else.
no-op: it fails, nothing else does.
simulated locally against the same payloads
_selftest.ymlbuilds, andreproduces both of its existing e2e assertions (
match=trueon thebodies-file,
match=falseon the negative comment), so those steps stillpass unchanged.
check-new-line-breaksclean againstorigin/main; all three touched YAMLfiles parse; added lines are ASCII-only.
Not covered
Same bootstrapping gap as #341:
claude.ymlreaches the composite throughMorrison-Lab/gha/...@v2, which does not resolve until the tag advances pastthis merge.
_selftest.ymlexercises the composite through a local./ref,so nothing goes red in the meantime.
Left for a follow-up
#346's last item -- fenced and inline code are not stripped, though
blockquotes now are, so
comment `@claude review` on the PRstill matches.Not a regression (the pre-#341 pattern matched it too), but it needs its own
decision about what counts as a code span and how fenced blocks nest, rather
than riding along here.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BWWei35DwEzyAgzfrH1BfK
Generated by Claude Code