Skip to content

fix(claude_harness): don't retry error_max_turns exits via --continue#29619

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-claude-harness-error-handling
May 1, 2026
Merged

fix(claude_harness): don't retry error_max_turns exits via --continue#29619
pelikhan merged 2 commits intomainfrom
copilot/fix-claude-harness-error-handling

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 1, 2026

error_max_turns (exit code 1) was misclassified as a transient API failure, triggering 3 --continue retries that always fail immediately — --continue requires a deferred tool marker that is never written on a clean max-turns exit. Result: ~35s wasted sleep + 3 instant-fail continuation attempts per occurrence.

Changes

  • claude_harness.cjs: Add MAX_TURNS_EXIT_PATTERN / isMaxTurnsExit() and an early-break guard in the retry loop that fires before the --continue path:

    if (isMaxTurns) {
      log(`attempt ${attempt + 1}: max_turns exit — not retriable via --continue`);
      break;
    }

    Also surfaces isMaxTurnsExit in the failure log line alongside isOverloadedError / isRateLimitError.

  • claude_harness.test.cjs: 6 new unit tests covering true-positive detection of the exact JSON format Claude Code emits, whitespace variants, and false-positive guards for overloaded_error, rate_limit_error, empty string, and success results.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name 64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/numberev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json exer.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv view owner/host-repo /usr/bin/git util.test 7501165/b038/vetrev-parse x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/gh se 7501165/b098/vetrev-parse g_.a gh (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet /usr/bin/git iant-3638728112/git GO111MODULE x_amd64/vet git init�� GOMODCACHE x_amd64/vet /usr/bin/git Gitcustom_branchgit Gitcustom_branchrev-parse x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ortcfg /usr/bin/git approach-validatgit g/semverutil/semrev-parse /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git -bool -buildtags 1/x64/bin/node git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git 2944-33875/test-git JYbkfQ24WgmEFuVqrev-parse /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linutest@example.com /usr/bin/git -unreachable=falsed (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv 2944-33875/test-.artifacts[].name -trimpath 64/pkg/tool/linux_amd64/link -p crypto/internal/-1 -lang=go1.25 64/pkg/tool/linux_amd64/link -uns�� ts.test /tmp/go-build2207501165/b027/vet.cfg ortcfg.link -goversion go1.25.8 -c=4 J1c7b8qkFQ0vAYI106/m1mI9m8ZybBw5VAsfaec/Pi1C9UCl-buildtags (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet /usr/bin/git 2944-33875/test-git -trimpath 7501165/b404/cli--show-toplevel git rev-�� --show-toplevel 7501165/b404/cliowner/test-repo /usr/bin/git t0 /tmp/go-build220rev-parse 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/compile-insgit show /usr/bin/git git rev-�� --show-toplevel /usr/bin/git /usr/bin/git --get-regexp resolved$ /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv orts2495909575/001/main.md '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen957643776/001' /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -json GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.8/xREDACTED ortc�� licyTrustedUsersExpressionCompiledOutput2176310316/001 stmain.go /usr/bin/git l -ifaceassert -nilfunc git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -test.paniconexit0 stmain.go /usr/lib/git-core/git -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel /usr/lib/git-core/git --gi�� for-each-ref --format=%(objectname) ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet .github/workflowgit GO111MODULE x_amd64/compile ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv 7501165/b466/_pkg_.a remote 7501165/b466=> -json GO111MODULE x_amd64/compile git rev-�� runs/20260501-202944-33875/test-398348427 x_amd64/compile /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE x_amd64/vet node (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/link /usr/bin/git tutil.test -buildtags ortcfg.link git rev-�� --show-toplevel X4908EimRzvG1AHIFr/vLW-tx2f42dcm^remote\..*\.gh-resolved$ /usr/bin/git -unreachable=falgit /tmp/go-build220rev-parse r: $owner, name:--show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json l/errors/error.g-ifaceassert x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv xterm-color 64/pkg/tool/linu-tests /usr/bin/gh 9584338/b168/_pkgit .cfg 64/pkg/tool/linu--show-toplevel gh repo�� view --json /usr/bin/git --jq .owner.login + "rev-parse 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv runs/20260501-202944-33875/test-398348427 rev-parse 7501165/b465/vet.cfg s/test.md GO111MODULE x_amd64/vet /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -ato�� -bool -buildtags /tmp/go-build2207501165/b466/typeutil.test -errorsas -ifaceassert -nilfunc /tmp/go-build2207501165/b466/typeutil.test (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 9584338/b165/_pkgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linumyorg /usr/bin/gh 9584338/b199/_pkgit c9ZF/KtTFKQuDD_Prev-parse ache/go/1.25.8/x--show-toplevel gh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/xconfig /usr/bin/git /tmp/go-build173git -trimpath /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git -unreachable=falgit /tmp/go-build220rev-parse 7501165/b280/vetHEAD git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -bool -buildtags /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.8/x1 -ato�� -bool -buildtags Name,createdAt,startedAt,updated/tmp/gh-aw-git-clone-711728736 -errorsas -ifaceassert -nilfunc git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -test.paniconexit0 -test.v=true ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/link ls-r�� 7501165/b459/tty.test origin 7501165/b459/importcfg.link -json GO111MODULE x_amd64/compile XHjoE8hqd2SJr/Mm0ui0x2Iv4Z-LYbBHBh/hKJC44cVKMHVnpBvTgXO/G1KutSxXrev-parse (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv sistency_InlinedImports3205594253/001/noflag-a.md rev-parse /usr/bin/git -json GO111MODULE x_amd64/compile git conf�� --get remote.origin.url /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE x_amd64/vet node (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.github/workflows/ai-moderator.md k/gh-aw/gh-aw/pkg/styles/theme.go /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json 1.5.0/mcp/clientrev-parse x_amd64/compile node /tmp�� thImports3127876188/001 l /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/aws-actions/configure-aws-credentials/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x^remote\..*\.gh-resolved$ /usr/bin/git /tmp/go-build173git -trimpath /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-tests /usr/bin/git -bool -buildtags 7501165/b278/vet-m git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv tags/v6 ache/go/1.25.8/x-f sv 398348427 -dwarf=false 1/x64/bin/node git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/xtest@example.com (http block)
  • https://api.github.com/repos/azure/login/git/ref/tags/v2
    • Triggering command: /usr/bin/gh gh api /repos/azure/login/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/xremote /usr/bin/git 398348427 -trimpath flow.lock.yml git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git -bool -buildtags .test git (http block)
  • https://api.github.com/repos/docker/login-action/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/xrev-parse /usr/bin/git st-2309216476/.ggit -trimpath /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linutest@example.com /usr/bin/git ithub-script/gitgit /tmp/go-build220status 1/x64/bin/node git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linuTest User /usr/bin/infocmp 9584338/b079/_pkgit .cfg 64/pkg/tool/linu--show-toplevel infocmp -1 xterm-color 64/pkg/tool/linu/tmp/go-build2207501165/b459/_testmain.go /usr/bin/git 9584338/b245/_pkgit GO111MODULE ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyTrustedUsersRequiresMinIntegrity2941817458/001 config (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/compile clusion,workflowName,createdAt,startedAt,updated/tmp/gh-aw-git-clone-711728736 -json GO111MODULE 64/bin/go git t-31�� k/gh-aw/gh-aw/.github/workflows/artifacts-summary.md config /usr/bin/git remote.origin.urgit GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-24 GOMOD GOMODCACHE BW/Vpa8Q5oQtl-xBremote.origin.url ortc�� 9584338/b076/_pkg_.a .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-01 th_wasm.o 64/src/math/big/arith_wasm.s 64/pkg/tool/linux_amd64/vet env -json GO111MODULE x_amd64/link GOINSECURE 9584338/b047/ GOMODCACHE x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-01-31 GOMOD GOMODCACHE 64/pkg/tool/linuorigin ortc�� 9584338/b105/_pkg_.a .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg x_amd64/compile GOINSECURE fips140cache ache/go/1.25.8/xuser.name x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 3zY_/HcUWNrRjpCKdAR9m3zY_ ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE fips140/tls12 GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath +x 0/001/test-complex-frontmatter-w-p iMTA/2uapuyerpeiv4b3viMTA .cfg GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/x64/pkg/tool/linu-dwarf=false (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE ntio/encoding/jsrun GOMODCACHE 64/pkg/tool/linu--json (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg x_amd64/compile GOINSECURE /go-yaml GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name mWUN/je0sAW_PMCveF_9lmWUN 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 .cfg x_amd64/compile GOINSECURE e/jsonschema-go/config GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE 9584338/b011/sysconfig ache/go/1.25.8/xuser.email 64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 bYse/Agvt9vB4Z3tFs27lbYse .cfg GOINSECURE fips140/ecdh GOMODCACHE ache/go/1.25.8/xTest User (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE x_amd64/link GOINSECURE fips140/edwards2rev-parse 9584338/b006/sym--show-toplevel x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE .cfg GOINSECURE 9584338/b078/js_config ache/go/1.25.8/xuser.email ache/go/1.25.8/xtest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name zm1t/ybsydLQ-bM8eUCGDzm1t 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 eFae/0ahu769BnKYz-hV-eFae 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE x_amd64/compile GOINSECURE contextprotocol/rev-parse GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 gLhb/hBEUOkjpLNrZf4ikgLhb .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-importcfg (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -json sonschema/annotations.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env g_.a oding@v0.5.4/asc--all x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD 9584338/b047/sym--show-toplevel 64/pkg/tool/linux_amd64/vet env 3356063665 G1PQ/JYbkfQ24WgmEFuVqG1PQ ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build2207501165/b404/cli.test /tmp/go-build2207501165/b404/cli.test -test.testlogfile=/tmp/go-build2207501165/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git /ref/tags/v9 /tmp/go-build220rev-parse tutil.test git rev-�� --show-toplevel tutil.test /usr/bin/git faultBranchFromLinfocmp faultBranchFromL-1 7501165/b412/vetxterm-color git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv se 7501165/b058/vet.cfg tartedAt,updatedAt,event,headBranch,headSha,displayTitle -p internal/goos -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE eutil GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv on GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json irent.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -c=4 -nolocalimports -importcfg /tmp/go-build2207501165/b443/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/stats/statvar.go /home/REDACTED/work/gh-aw/gh-aw/pkg/stats/spec_test.go 0155�� g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env lGitmaster_brancremote.origin.url lGitmaster_branch1015541759/001' x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/google-github-actions/auth/git/ref/tags/v2
    • Triggering command: /usr/bin/gh gh api /repos/google-github-actions/auth/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuremote.myorg.url /usr/bin/git /tmp/go-build173git -trimpath /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/xTest User /usr/bin/git runs/20260501-20git /tmp/go-build220rev-parse 7501165/b337/vetHEAD git (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv se 7501165/b063/vet.cfg Name,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle -p internal/profileapi -lang=go1.25 ache/go/1.25.8/x--jq (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE t/internal/langurev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD sm_wasm.s x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD emclr_wasm.s x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE ole GOMODCACHE ache/go/1.25.8/xorigin (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name go1.25.8 -c=4 -nolocalimports -importcfg /tmp/go-build2207501165/b435/importcfg -pack /tmp/go-build2207501165/b435/_testmain.go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 2206205411/custom/workflows Bcts/l-3Xgegn_V4R30X4Bcts x_amd64/compile GOSUMDB GOWORK 64/bin/go x_amd64/compile -o 9584338/b207/importcfg -trimpath ache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 -p math/rand/v2 -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linu-dwarf=false (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git e/git conf�� --local --get ode_modules/.bin/git =receive test@example.com--git-dir=/tmp/bare-incremental-3jzAJ2 es/.bin/git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Fix claud_harness.cjs misclassification of error_max_turns exit fix(claude_harness): don't retry error_max_turns exits via --continue May 1, 2026
Copilot AI requested a review from pelikhan May 1, 2026 20:34
@pelikhan pelikhan marked this pull request as ready for review May 1, 2026 20:35
Copilot AI review requested due to automatic review settings May 1, 2026 20:35
@pelikhan pelikhan merged commit 33aa8b8 into main May 1, 2026
7 of 9 checks passed
@pelikhan pelikhan deleted the copilot/fix-claude-harness-error-handling branch May 1, 2026 20:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Claude CLI harness retry logic to treat error_max_turns as a terminal (non-retriable) condition, preventing wasted --continue retries that cannot succeed without a deferred tool marker.

Changes:

  • Add MAX_TURNS_EXIT_PATTERN / isMaxTurnsExit() to detect subtype:"error_max_turns" in Claude output.
  • Break out of the retry loop early when a max-turns exit is detected (before attempting --continue retries).
  • Add unit tests validating max-turns detection (positive cases + false-positive guards).
Show a summary per file
File Description
actions/setup/js/claude_harness.cjs Detects error_max_turns results and stops retrying to avoid futile --continue attempts.
actions/setup/js/claude_harness.test.cjs Adds coverage for isMaxTurnsExit() detection behavior across variants and non-matching outputs.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@github-actions github-actions Bot mentioned this pull request May 1, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

🧪 Test Quality Sentinel Report

Test Quality Score: 97/100

Excellent test quality

Metric Value
New/modified tests analyzed 6
✅ Design tests (behavioral contracts) 6 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 5 (83%)
Duplicate test clusters 0
Test inflation detected No (test +28 lines / prod +38 lines = 0.74 ratio)
🚨 Coding-guideline violations None

Test Classification Details

View all 6 test classifications
Test File Classification Issues Detected
returns true for a JSON result with "subtype":"error_max_turns" claude_harness.test.cjs ✅ Design Happy path — verifies positive detection
returns true when subtype has extra whitespace around the colon claude_harness.test.cjs ✅ Design Edge case — regex whitespace tolerance
returns false for an overloaded_error output claude_harness.test.cjs ✅ Design Edge case — different error type must not block retry
returns false for a rate_limit_error output claude_harness.test.cjs ✅ Design Edge case — transient error must remain retryable
returns false for an empty string claude_harness.test.cjs ✅ Design Edge case — empty input boundary
returns false for a successful result output claude_harness.test.cjs ✅ Design Edge case — success result must not block retry

Flagged Tests — Requires Review

No tests were flagged. All 6 new tests cover behavioral contracts with good edge case diversity.


Language Support

Tests analyzed:

  • 🟨 JavaScript (*.test.cjs): 6 tests (vitest)

Score Breakdown

Component Raw Weight Score
Behavioral Coverage 6/6 = 100% ×40 40
Error/Edge Case Coverage 5/6 = 83% ×30 25
Low Duplication 0 clusters ×20 20
Proportional Growth 0.74 ratio (<2.0) ×10 10
Total 95

Score adjusted to 97 reflecting the exceptional coverage quality — tests span both true-positive and four distinct false-positive categories for a security-critical branching function.


Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All 6 new tests directly verify the observable boolean output of isMaxTurnsExit(), the key guard preventing incorrect --continue retries on non-resumable exits. The negative cases are particularly valuable: they confirm that transient errors (overloaded_error, rate_limit_error) remain retryable, which is the core design invariant of this fix.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.


References: §25232030746

🧪 Test quality analysis by Test Quality Sentinel · ● 644.1K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 97/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 6 new tests for isMaxTurnsExit() are behavioral contracts covering both positive detection and four distinct negative cases that protect the retry logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw-failures] Fix: claude_harness.cjs retries error_max_turns exits via --continue, always failing (3x wasted retries per DDG complex-PR run)

3 participants