claude.yml: diagnose a rejected push, preserve the work, and stop narrating it as success - #361
Conversation
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
A rejected post-step push discarded the agent's commits with only a raw git error to explain it, and the "Post Claude's response" step then posted prose describing work the branch did not carry (#360). Add a report-push-failure composite action wrapping a new classify-push-failure.sh: it names the failure kind, emits an ::error:: naming WORKFLOW_TOKEN when that is the cause, and comments the explanation plus a git format-patch of the unpushed commits on the thread. Both of claude.yml's push sites capture their output and call it. Gate the response-post step on the PR push not having failed, so a push that never landed can no longer produce a success-shaped comment. Closes #360 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
|
🤖 Jules is reviewing this PR. Results will appear here shortly (typically 2–5 minutes). |
- CLAUDE.md: nested backticks inside a code span do not parse; state the two credential wordings as plain emphasis instead. - _selftest.yml: mark the synthetic git identity phi-allow, and interpolate the fake push credential so the source line is not itself email-shaped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
See the workflow logs for details. |
Self-review found it. fence_for sizes a fence by counting backtick runs, and grep exits non-zero when the text holds none -- the ordinary shape of a network or auth failure. Under the composite's own `pipefail` that fails the pipeline, which `set -e` turns into an aborted report. It survives today only because every call site wraps it in `x="$(fence_for ...)"`, and bash suppresses errexit inside a command substitution that is part of an assignment. Verified directly: called as a plain statement the old form exits 1 and emits nothing, the fixed form returns a correct three-backtick fence. Tolerate the empty match explicitly, and pin it with a second real `uses:` call in _selftest.yml on a backtick-free log -- the first fixture cannot catch this, since a workflows-permission rejection always carries backticks. Also correct two prose slips found in the same read: the `other` headline said "rejected" while that branch also covers failures the remote never saw, and the non-fast-forward advice named a patch that may not be present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
🤖 Jules ReviewSummaryThis PR gracefully intercepts and diagnoses rejected Strengths
VERDICT: approve Session: |
Self-review (no external verdict available)Neither configured reviewer produced one on this head:
Per the "do the review yourself when the workflow doesn't produce a verdict" rule, here is a self-review rather than a stalled PR. It is not a substitute for an external one -- if either reviewer comes back, I'll disposition its findings normally. Findings I raised against myself, and fixed in
|
Code reviewReviewed the diff ( CLAUDE.md compliance: no violations found. The Bugs found (posted inline):
One thing I checked but did not flag as a bug, since it came up in two of the four parallel passes: the Not flagged inline, worth a look regardless: the PR description says "Both of Everything else — the VerdictNeeds more work — findings 1 and 2 are real correctness gaps in the new feature's core failure-recovery path (the first can silently defeat the whole PR's purpose for large patches); the rest are minor and non-blocking. |
|
Claude finished review — View run Posted the review on PR #361: 5 inline comments plus a summary with the verdict. Findings:
I also verified and rejected one claim two of the four parallel reviewers raised (that a bash- Verdict posted: Needs more work (findings 1 and 2 are genuine correctness gaps; the rest are minor). |
All five confirmed empirically before fixing. 1. HIGH -- the patch truncation ran `printf | head -c | iconv`. head stops reading once it has its bytes, so a patch past the ~64 KiB pipe buffer left printf writing to a closed pipe: SIGPIPE, exit 141, promoted by pipefail and turned into an aborted report by set -e. Reproduced: a 300 KB payload exits 141 and posts nothing, which defeats the whole feature exactly where preserving the work matters most. The cut now reads a file, so there is no producer to kill. 2. MEDIUM -- a missing push log stood down silently, but the calling step can fail before pushing and claude.yml now gates its response-post step off on that same failure, so the thread got no comment at all. Reported as a new `no-push-attempt` kind instead, still carrying the patch. 3. `git format-patch ... 2>&1` folded stderr into the payload on the success path, so a git warning could corrupt the `git am` target. Captured separately and surfaced only as a warning. 4. A comment called GitHub's comment cap 65,536 bytes; CLAUDE.md already records it as characters. 5. Credential redaction missed the userinfo-only `https://token@host` form. One rule now covers both. Also correct the errexit explanation in the fence_for comment: the governing mechanism is that a command substitution's subshell does not inherit errexit by default (shopt -s inherit_errexit flips it), not that the substitution is part of an assignment -- `x="$(false)"` does abort. Selftest gains a third real `uses:` call covering the no-push-attempt path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
See the workflow logs for details. |
Code reviewThis PR has already been through 3 rounds of review; round 3 gave Ready for merge with one remaining minor finding (push-protection missing from 3 enum-documentation sites), which is fixed in the current head commit New findings (posted inline where the diff allowed it):
All 8 fixes from rounds 1-2 were re-verified present and correct in the current diff (SIGPIPE-safe truncation, stderr no longer spliced into the patch, credential redaction broadened, the Verdict: Needs more work |
|
Claude finished review — View run The summary comment posted successfully. All findings have been posted: 5 inline comments plus a summary comment covering the 2 stale-reference locations that GitHub's diff anchor rejected. Review completePR #361 had already been through 3 rounds of review (round 3: "Ready for merge" with one minor fix, confirmed present in the current head
Plus 3 lower-severity doc-accuracy findings (CLAUDE.md misnaming a step, stale references to a step name this PR renamed, and an overclaiming code comment). All 5 posted as inline comments (2 doc findings couldn't be anchored inline since they sit outside the diff's changed-line range, so those went into the summary comment instead). Full summary posted at #361 (comment). VerdictNeeds more work |
Review round 5 on #361 found the patch-suppression gate keyed on the classifier's `kind`, which is a first-match chain. A push that both edits a workflow file and carries a secret matches the workflows-permission clause first, so the gate never fired and the secret-bearing commits were published verbatim -- in the comment AND in the run log, where Actions' masking does not reach commit content. `classify-push-failure.sh` now emits `withhold-patch=` as its own line, computed from the secret-scanning markers before the kind chain runs, and the composite gates publication on that rather than on `kind`. When the markers fire but another kind wins the chain, the classifier appends the no-patch explanation to that kind's advice so the omission is never silent. Also: the no-patch fallback no longer fires when the patch was withheld deliberately -- it appended "see the workflow run for the full log" directly under advice saying the run log omits it for the same reason. The one selftest step covering that case omitted `run-url`, unlike both production call sites, so the contradiction was unreachable there; it now passes it and asserts the absence. Retracts an unverified claim: an earlier revision justified the chain order by asserting GitHub wraps a workflow-permission rejection in the generic `GH013` rule-violation envelope. It does not. That came from a `GH013` line in this repo's own test fixture, added here, not from the run #360 was filed over. The fixture is now verbatim and a separate co-occurrence fixture carries both markers; the design no longer depends on the claim either way. Doc fixes from the same round: CLAUDE.md named the finalize step as the gated one (it is the push step), three references still used the old combined step name, and the response-post comment overstated why the issue path needs no push gate -- the real reason is that the issue push runs several hundred lines later, so its outcome does not exist yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
🤖 Jules ReviewSummaryThis PR expertly resolves an issue where a rejected Strengths
VERDICT: approve Session: |
Round 5 — all five addressed in
|
| assertion | pre-fix | post-fix |
|---|---|---|
| no patch rendered | FAIL | ok |
| credential absent from comment body | FAIL | ok |
| credential absent from run log | FAIL | ok |
| body explains the missing patch | FAIL | ok |
body still names WORKFLOW_TOKEN |
ok | ok |
So pre-fix it reached both destinations, as you said.
2. Self-contradicting fallback — fixed, plus the selftest gap you named
elif [[ "$withhold_patch" != "true" && -n "$RUN_URL" ]], keyed on the withhold flag rather than on kind so it tracks finding 1's fix. And you were right that the selftest could not have caught it: that step omitted run-url unlike both production call sites, so the branch was unreachable. It now passes run-url and asserts the string is absent. That assertion fails against pre-fix code (last row of the same harness).
3. CLAUDE.md "the issue-trigger finalize" — fixed
Now names both steps by title. finalize_issue doesn't call the action at all, as you said.
4. Stale combined-step-name references — fixed, all three
claude.yml header, the extract-run-cost comment, and the CLAUDE.md extract-total-cost entry. Grepped for the old name repo-wide afterward; those three were all of them.
5. Response-post comment overclaims — fixed
Rewritten to say the real reason: push_issue runs several hundred lines below that step, so its outcome does not exist yet and an equivalent condition would always be empty. The uncommitted-changes case is now described accurately rather than folded into the committed one.
Coverage: classifier suite is 24 cases (was 18), including your explicit ask — a log where both patterns co-occur, asserting kind=workflows-permission and withhold-patch=true, plus the withhold value for the three other kinds and the new 5-line output contract. _selftest.yml's four real uses: calls are unchanged in count; the push-protection one gained run-url and the contradiction assertion.
Both new behaviours were checked against pre-fix code and fail there, per the repo's regression-test rule.
Generated by Claude Code
Code reviewThis PR has already been through 5 rounds of review (round 3: "Ready for merge" with one minor fix; round 4: 2 blocking + 3 doc-accuracy findings; round 5/the author's own reply: all five addressed in
All confirmed fixed and consistent with their own test coverage (the classify-push-failure suite's 24 cases, and One new finding, posted inline: VerdictReady for merge |
|
Claude finished review — View run I've completed the review of PR #361 and posted both the inline finding and the summary comment. Summary: This is round 6 of review on a PR that's already been through 5 rounds. I independently re-verified (not just trusted) that all fixes claimed in round 5's reply commit (
All confirmed correct against the actual code and test fixtures, not just the commit message's narrative. One new finding (posted inline on VerdictReady for merge |
Round 6 review finding. The Tests-section paragraph on run-classify-push-failure-tests.sh still described a "three-part output contract" naming only kind, headline, and advice -- stale since round 5 added withhold-patch as its own line in this same PR. The Layout section's report-push-failure paragraph and the script's own header comment were both updated then; this one was missed. Swept the repo for other descriptions of the contract; this was the only other site. Also states why the shape is asserted at all, since the composite reads each field by fixed line offset and a reordering would break it silently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CaeMXRGxi3HrQbxaYTVaRQ
🤖 Jules ReviewSummaryThis PR implements a robust, secure, and incredibly thoughtful mechanism for preserving an agent's commits when a post-run Strengths
VERDICT: approve Session: |
2026-07-29 22:46 UTC |
|
Claude finished review — View run Code Review — #361This PR has already been through 6 rounds of review. I independently re-verified the current state at HEAD ( All 8 findings from the prior round are fixed, confirmed against current code:
The final commit ( I also spot-checked the new This confirms round 6's verdict at Morrison-Lab/gha#361 comment ("Ready for merge," with the one doc-sync gap noted) still stands, and that gap is now closed. No new findings; no inline comments posted ( VerdictReady for merge |
Ready for merge —
|
| reviewer | verdict at a1357c7 |
|---|---|
review / claude-review |
Ready for merge, no new findings — re-verified all 8 prior findings against the code rather than the round summaries |
Jules (jules/review) |
approve — status success, "Review complete (verdict: approve)" |
| Copilot | never weighed in, at any head — quota-refused every request across all seven rounds |
Two independent external approvals at the current head, so this is not resting on self-review. Copilot's absence is stated rather than papered over: it refused on every head, so it was never "reachable" in the sense criterion 2 requires.
CI
All 30 check runs completed, every one success or skipped; the jules/review commit status is success. Nothing queued, nothing in progress.
Worth noting for anyone reading the timeline: mergeable_state read unstable for about a minute after the last check run finished, because jules/review landed at 22:30:21 and the rollup hadn't caught up. That was the rollup lagging, not a failing check.
Threads
All 17 inline threads resolved, swept by thread id rather than from memory of which findings were answered.
Freshness
main unmoved at 267ad3a; git merge-tree reports zero conflicts; local HEAD matches this PR's head.sha; the 24-case classifier suite passes locally at this commit.
What the review rounds actually cost, and what they caught
Two of the findings were bugs that would have shipped the feature broken in precisely the situations it exists for, which is worth recording rather than glossing:
- The SIGPIPE truncation abort killed the whole report for any patch past the pipe buffer — the large-patch case where preserving the work matters most.
- The co-occurring-marker leak published a secret-bearing patch to a public comment whenever a push both edited a workflow file and carried a credential. Confirmed by running the composite against a repo with a planted secret: pre-fix it reached the comment body and the run log.
I also retracted two claims I had published, both in the thread and in the PR body: the GH013-envelope claim (which came from a fixture I wrote in this PR, not from the real #360 log) and the wrong errexit mechanism. The first was load-bearing — the sole justification for the ordering that hid the leak above.
Two follow-ups are tracked rather than folded in: gha#368 for claude-code-review's no-verdict modes, now including the hung-retry variant seen on this PR, and the @v2 bootstrapping gap noted in the PR body, which resolves when the tag advances past this merge.
Not merging — that's a human call.
Generated by Claude Code
Closes #360.
The two defects
A rejected post-step push in
claude.ymldid two bad things at once.The agent's commits existed only on the runner, so they died with the job, with a raw git error in a failed step as the only signal.
And because the PR head SHA never moved, the later "Post Claude's response if no code was committed" step could not tell the rejection apart from a genuinely code-free run, so it posted Claude's prose describing the fixes it had just made onto a branch carrying none of them.
What this does
Diagnose. A new
classify-push-failure.shreads a failed push's output and names the failure kind (workflows-permission,push-protection,non-fast-forward,other, plusno-push-attemptwhich the composite assigns when no log exists) plus advice for it.The
workflows-permissioncase namesWORKFLOW_TOKENand links the README's Permissions section.It keys on the
refusing to allow ... to create or update workflowclause rather than the trailing scope name, because GitHub words that tail differently per credential: a GitHub App is rejected for lacking theworkflowspermission, a PAT for lacking theworkflowscope.Preserve. A new
report-push-failurecomposite action emits the::error::, generates agit format-patchof the commits that could not be pushed, and comments the whole thing on the issue or PR, so the work comes back withgit aminstead of being redone.Both of
claude.yml's push sites capture their output and call it.Withhold when publishing would leak. The classifier emits a second output,
withhold-patch, computed from GitHub's secret-scanning markers independently ofkind, and the composite gates patch publication on that.Those commits carry the secret the push was blocked to contain, and Actions' masking does not cover commit content.
Keeping it separate from
kindis the point:kindis a first-match chain, so a push that both edits a workflow file and carries a secret classifies asworkflows-permission, and a kind-keyed gate would publish the credential anyway.Stop the false claim. The response-post step is now gated on the PR push not having failed.
Only the PR path can have that gate: the issue-trigger push runs several hundred lines further down, so its outcome does not exist yet at that point.
Notes on the implementation
outcome, not on a flag that step wrote.outcomeis set by the runner, so nothing about the report's inputs depends on the step that failed. It is also the wider signal: the push step can fail before reaching the push at all (the auto-commit sweep, the fork lookup), which the action reports asno-push-attemptrather than standing down — since the response-post step is gated off on that same failure, standing down would leave the thread silent, the exact outcome claude.yml: a rejected workflow-file push discards the agent's commit silently, then a post-step claims the fix shipped #360 exists to prevent.strip-non-invoking-markup.shuses. A patch touching a Markdown file carries```lines of its own, which a fixed three-backtick fence would let close the block early.printf ... | head -cleaves printf writing to a closed pipe once head has its bytes, so any patch past the ~64 KiB pipe buffer raised SIGPIPE, whichpipefailpromotes andset -eturns into an aborted report — losing the comment precisely for the large patches that most need preserving.|| push_status=$?rather than a bare call, because the step runs under the defaultbash -e, which would exit on the failing push before the log could be echoed.Verification
run-classify-push-failure-tests.sh: 24 cases. Covers the verbatim claude.yml: a rejected workflow-file push discards the agent's commit silently, then a post-step claims the fix shipped #360 rejection plus its PAT and OAuth App wordings, two non-fast-forward phrasings, the cases that must fall through toother, both secret-scanning wordings, a log where the workflow-permission and secret-scanning markers co-occur (assertingkind=workflows-permissionandwithhold-patch=true), the five-line output contract the composite parses, and that a generic failure's advice does not nameWORKFLOW_TOKEN— naming the secret is only worth anything if it is named solely when it is the cause.action.ymland run against a throwaway git repo whose commit carries a planted credential. Both round-5 fixes were confirmed to fail against pre-fix code: the credential reached the comment body and the run log, and the withheld-patch case appended "see the workflow run for the full log" under advice saying the run log omits it too._selftest.yml'sreview-fail-checkjob runs the offline suite and callsreport-push-failurethrough four realuses:steps withdry-run: true— one per classified kind. Thepush-protectionone asserts no patch is rendered and that no run-log pointer contradicts that.Corrections to earlier revisions of this description
Two claims this body previously made were wrong, and are retracted here rather than only in the review threads:
The
GH013envelope claim. An earlier revision said GitHub wraps a workflow-permission rejection in the genericGH013: Repository rule violationsenvelope, and used that to justify the classifier's chain order. It does not. The real claude.yml: a rejected workflow-file push discards the agent's commit silently, then a post-step claims the fix shipped #360 log has noGH013line; that line came from a fixture in this PR, added under a comment claiming the fixture was verbatim. The design no longer depends on the claim either way, sincewithhold-patchis now computed independently of the chain.The
errexitmechanism. An earlier revision said bash suppresseserrexit"inside a command substitution that is part of an assignment." That is false —x="$(false)"does abort. The actual mechanism is that a command substitution's subshell does not inheriterrexitunlessshopt -s inherit_errexitis set, which makes it a default rather than a guarantee. Thefence_forbug this described was real and is fixed by tolerating the emptygrepmatch where it happens, so the behaviour no longer depends on call context; only the explanation was wrong.What is not covered
claude.yml's own layer above the composite cannot be exercised until@v2advances past this merge — it calls the action viaMorrison-Lab/gha/...@v2— the same bootstrapping gapdetect-review-requestandreport-failurealready carry._selftest.ymlnever invokesclaude.yml, so this does not redden any check in the meantime.Per that same gap, this PR's own review runs the pre-fix
claude.yml@v2.