fix: retry Strix provider tool protocol failures - #933
Conversation
|
Warning Review limit reached
Next review available in: 6 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@opencode-agent review Evaluate exact current head |
|
@opencode-agent review Evaluate exact current head |
|
@opencode-agent @cwl-noema-review Formal review-only request for exact current head |
|
@coderabbitai review Review exact current head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
head='b4e60dcec7abea2466b5f1d35b7f42e23313ebc8'
base='6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba'
printf '%s\n' '== Local commit availability =='
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
git merge-base --is-ancestor "$base" "$head"
printf 'base is ancestor of head: yes\n'
printf '%s\n' '== Changed files and diff =='
git diff --check "${base}...${head}"
git diff --name-status "${base}...${head}"
git diff --find-renames --find-copies --unified=80 "${base}...${head}" -- \
scripts/ci/strix_quick_gate.sh \
scripts/ci/test_strix_quick_gate.sh
printf '%s\n' '== Affected shell-file structure =='
ast-grep outline scripts/ci/strix_quick_gate.sh --lang bash --items all
ast-grep outline scripts/ci/test_strix_quick_gate.sh --lang bash --items all
printf '%s\n' '== Exact error handling and test assertions =='
rg -n -C 12 -F 'ModelBehaviorError: Tool execute not found in agent strix' \
scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.sh
rg -n -C 8 -e 'fallback|retry|provider|vulnerab|severity|exit|fail' \
scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.shLength of output: 179323 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo='ContextualWisdomLab/.github'
pr='933'
head='b4e60dcec7abea2466b5f1d35b7f42e23313ebc8'
printf '%s\n' '== Pull request identity =='
gh api "repos/${repo}/pulls/${pr}" \
--jq '{number, state, draft, base: .base.sha, head: .head.sha, mergeable_state, review_decision}'
printf '%s\n' '== Check runs for the exact head =='
gh api "repos/${repo}/commits/${head}/check-runs?per_page=100" \
--jq '.check_runs[] | {name, status, conclusion, app: .app.slug, details_url}'
printf '%s\n' '== Commit statuses for the exact head =='
gh api "repos/${repo}/commits/${head}/status" \
--jq '.statuses[]? | {context, state, creator: .creator.login, target_url}'
printf '%s\n' '== Reviews =='
gh api "repos/${repo}/pulls/${pr}/reviews?per_page=100" \
--jq '.[] | {user: .user.login, state, commit_id, submitted_at, body}'
printf '%s\n' '== Unresolved review threads =='
gh api graphql -F owner='ContextualWisdomLab' -F name='.github' -F number="$pr" -f query='
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
pullRequest(number: $number) {
reviewThreads(first: 100) {
nodes {
isResolved
comments(first: 1) { nodes { author { login } body path line } }
}
}
}
}
}' --jq '.data.repository.pullRequest.reviewThreads.nodes[]
| select(.isResolved == false)
| {author: .comments.nodes[0].author.login, path: .comments.nodes[0].path, line: .comments.nodes[0].line, body: .comments.nodes[0].body}'Length of output: 11103 🏁 Script executed: #!/bin/bash
set -euo pipefail
head='b4e60dcec7abea2466b5f1d35b7f42e23313ebc8'
base='6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba'
log='/tmp/coderabbit-shell-logs/shell-output-smoFw4'
printf '%s\n' '== Exact change summary =='
git diff --stat "${base}...${head}"
git diff --numstat "${base}...${head}"
git diff --unified=20 "${base}...${head}" -- scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.sh \
| grep -E -C 25 'ModelBehaviorError|tool-protocol|retryable|fallback|^[-+]{3} ' || true
printf '%s\n' '== Exact match locations in the head and base =='
for ref in "$base" "$head"; do
printf '%s\n' "-- $ref --"
git grep -n -F 'ModelBehaviorError: Tool execute not found in agent strix' "$ref" -- \
scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.sh || true
done
printf '%s\n' '== Focused head excerpts =='
git show "${head}:scripts/ci/strix_quick_gate.sh" > /tmp/strix_quick_gate.head.sh
git show "${head}:scripts/ci/test_strix_quick_gate.sh" > /tmp/test_strix_quick_gate.head.sh
for file in /tmp/strix_quick_gate.head.sh /tmp/test_strix_quick_gate.head.sh; do
printf '%s\n' "-- $file --"
grep -n -E 'ModelBehaviorError: Tool execute not found in agent strix|tool-protocol|is_.*error|retryable|fallback|non-recoverable|threshold vulnerabilities' "$file" \
| head -n 160
done
printf '%s\n' '== Context around every exact signal in the head =='
for file in /tmp/strix_quick_gate.head.sh /tmp/test_strix_quick_gate.head.sh; do
grep -n -F 'ModelBehaviorError: Tool execute not found in agent strix' "$file" \
| cut -d: -f1 \
| while read -r line; do
start=$(( line > 35 ? line - 35 : 1 ))
end=$(( line + 45 ))
printf '\n-- %s:%s-%s --\n' "$file" "$start" "$end"
sed -n "${start},${end}p" "$file" | nl -ba -v "$start"
done
done
printf '%s\n' '== Narrow initial output recovery, if available =='
grep -n -E '^(== Local|base is ancestor|== Changed|[AMDR]\t|diff --git)' "$log" | head -n 100 || trueLength of output: 62263
The existing positive fixture emits both I confirmed that the fallback-success path checks previously reported threshold vulnerabilities before accepting fallback success. The exact head also matches the requested base, and the reported exact-head checks are terminal. No unresolved review threads were returned. 🧠 Learnings usedYou are interacting with an AI system. |
|
Exact-current-head review request for The bounded two-file change classifies only the exact provider-side Review this unchanged head read-only. Do not update the branch, synthesize author approval, merge, release, or weaken protection. @coderabbitai review |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo='ContextualWisdomLab/.github'
pr='933'
head='b4e60dcec7abea2466b5f1d35b7f42e23313ebc8'
base='6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba'
printf '%s\n' '== Commit relation and changed files =='
git cat-file -e "${head}^{commit}"
git cat-file -e "${base}^{commit}"
git merge-base --is-ancestor "$base" "$head"
git diff --check "${base}...${head}"
git diff --name-status "${base}...${head}"
git diff --find-renames --find-copies --unified=60 "${base}...${head}" -- \
scripts/ci/strix_quick_gate.sh \
scripts/ci/test_strix_quick_gate.sh
printf '%s\n' '== Shell file outlines =='
ast-grep outline scripts/ci/strix_quick_gate.sh --lang bash --items all
ast-grep outline scripts/ci/test_strix_quick_gate.sh --lang bash --items all
printf '%s\n' '== Retry predicate and regression coverage =='
rg -n -C 18 -F 'ModelBehaviorError: Tool execute not found in agent strix' \
scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.sh
rg -n -C 10 -e 'is_model_tool_protocol_error|tool-protocol|agent_finish|fallback|retryable|non-recoverable' \
scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.sh
printf '%s\n' '== Pull request and exact-head state =='
gh api "repos/${repo}/pulls/${pr}" \
--jq '{number,state,draft,base:.base.sha,head:.head.sha,mergeable_state,review_decision}'
gh api "repos/${repo}/commits/${head}/check-runs?per_page=100" \
--jq '.check_runs[] | {name,status,conclusion,app:.app.slug}'
gh api "repos/${repo}/pulls/${pr}/reviews?per_page=100" \
--jq '.[] | {user:.user.login,state,commit_id,submitted_at}'
gh api graphql -F owner='ContextualWisdomLab' -F name='.github' -F number="$pr" -f query='
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
pullRequest(number: $number) {
reviewThreads(first: 100) {
nodes {
isResolved
comments(first: 1) { nodes { author { login } body path line } }
}
}
}
}
}' --jq '.data.repository.pullRequest.reviewThreads.nodes[]
| select(.isResolved == false)
| {author:.comments.nodes[0].author.login,path:.comments.nodes[0].path,line:.comments.nodes[0].line,body:.comments.nodes[0].body}'Length of output: 181115 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== Predicate definition and caller =='
rg -n -A35 -B8 '^is_model_tool_protocol_error\(\)' scripts/ci/strix_quick_gate.sh
rg -n -A18 -B12 '\bis_model_tool_protocol_error\b' scripts/ci/strix_quick_gate.sh
printf '%s\n' '== Tool-protocol fake Strix fixture =='
rg -n -A45 -B10 'tool-protocol-primary|tool-protocol-fallback-success|agent_finish' \
scripts/ci/test_strix_quick_gate.sh
printf '%s\n' '== Exact diff only =='
git diff --unified=25 6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba...b4e60dcec7abea2466b5f1d35b7f42e23313ebc8 -- \
scripts/ci/strix_quick_gate.sh \
scripts/ci/test_strix_quick_gate.sh
printf '%s\n' '== Targeted current-head review and unresolved-thread state =='
gh api 'repos/ContextualWisdomLab/.github/pulls/933/reviews?per_page=100' \
--jq '.[] | {user:.user.login,state,commit_id,submitted_at}'
gh api graphql -F owner='ContextualWisdomLab' -F name='.github' -F number=933 -f query='
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
pullRequest(number: $number) {
reviewThreads(first: 100) {
nodes {
isResolved
comments(first: 1) { nodes { author { login } body path line } }
}
}
}
}
}' --jq '[.data.repository.pullRequest.reviewThreads.nodes[]
| select(.isResolved == false)
| {author:.comments.nodes[0].author.login,path:.comments.nodes[0].path,line:.comments.nodes[0].line,body:.comments.nodes[0].body}]'Length of output: 31826
One blocking issue remains.
Restrict the predicate to literal The exact-head check runs are terminal with success, neutral, or expected skipped conclusions. The unresolved-thread query returned no threads. These results do not remove the semantic blocker. I did not approve the pull request. 🧠 Learnings usedYou are interacting with an AI system. |
|
@opencode-agent review Review exact current head |
Rate Limit Exceeded
|
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-current-head source review for b4e60dcec7abea2466b5f1d35b7f42e23313ebc8 against protected main@6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba.
The two-file diff adds one narrowly identified OpenAI Agents SDK ModelBehaviorError family to the existing provider-infrastructure and distinct-model fallback paths. It remains fail-closed: the signal is not treated as a clean scan; threshold vulnerability artifacts produced before fallback still block; fallback exhaustion still fails; and no permission, target scope, severity threshold, credential, or publication boundary changes. The tool token is intentionally bounded to an alphanumeric/underscore name while requiring the full exception class, missing-tool phrase, and target agent strix, covering both observed execute and agent_finish failures without matching a generic target-application error. The permanent regression proves exactly two model attempts and a successful distinct fallback. All nine directly observed exact-head hosted workflows are terminal-success and no inline thread exists.
A qualifying independent reviewer must still submit formal approval on this unchanged head. Do not update the branch, classify provider failure as clean evidence, synthesize author approval, or bypass protection.
1 participant
Summary
Strix can terminate before producing evidence when the selected provider/model emits the exact OpenAI Agents SDK error
ModelBehaviorError: Tool execute not found in agent strix. Classify only this exact provider-side signal as retryable, so configured fallback models can complete the scan; vulnerability evidence remains fail-closed.Validation
bash -n scripts/ci/strix_quick_gate.sh scripts/ci/test_strix_quick_gate.shSTRIX_TEST_CASE_FILTER=tool-protocol-fallback-success bash scripts/ci/test_strix_quick_gate.shgit diff --checkThe change does not alter repository permissions, scan scope, or severity gating.