ci: iterate JSON array root when extracting Claude review text by loothero · Pull Request #146 · Provable-Games/summit

loothero · 2026-05-03T12:36:31Z

Summary

Follow-up to #145. PR #144's CI run (25279155808) showed all four claude-review-* jobs failing in ~35s with no captured comment. Root cause: the action writes execution_file as a single JSON array (JSON.stringify(messages, null, 2) in run-claude-sdk.ts), not as the JSONL stream the previous filter assumed.

The old filter jq -r 'select(.type == "result") | .result' evaluated .type against the array root and hit Cannot index array with string "type", exiting 5. With bash -e that propagated and failed the Extract review output step before the explicit empty-output guard could fire — exactly the silent-pass failure mode #145's Finding A fix was designed to prevent (the empty /tmp/review.txt from the : > /tmp/review.txt line + the downstream [ ! -s ] blocking-check correctly turned an unparseable result into a job failure).

Fix: prepend .[] to both filters (result message + assistant-text fallback) so they iterate over the array root.

Verified locally against a sample matching the action's actual file shape:

old filter:  jq: error (at sample.json:1): Cannot index array with string "type"  (exit=5)
new filter:  FINAL REVIEW BODY  (exit=0)
fallback:    hi  (exit=0)

Test plan

Merge this, then push a sync commit to PR test: trigger Claude reviews across api/client/contracts scopes #144 to retrigger CI
Expect: 4 Claude review comments + 4 Codex comments on PR test: trigger Claude reviews across api/client/contracts scopes #144 (3 scoped + general skipped), pr-ci passes
No *-general jobs run (still gated by the general_review shell step from ci: post Claude reviews from execution_file; fix general_review filter #145)

🤖 Generated with Claude Code

The action writes \`execution_file\` as a single JSON array (\`JSON.stringify(messages, null, 2)\` in run-claude-sdk.ts), not as JSONL. The previous \`jq -r 'select(.type == "result") | .result\`\` filter evaluated \`.type\` against the array root, hit \`Cannot index array with string "type"\`, and exited 5. With \`bash -e\` that propagated and failed the Extract step before the explicit empty- output guard could fire, so all four \`claude-review-*\` jobs failed in ~35s with no captured comment (PR #144 run 25279155808). Add \`.[]\` to iterate over the array root in both the result-message and assistant-text-fallback filters. Verified locally with a sample matching run-claude-sdk.ts's structure: old filter -> exit 5 + parse error; new filter -> review text + exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

gemini-code-assist · 2026-05-03T12:36:35Z

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

vercel · 2026-05-03T12:36:37Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
summit	Ready	Preview, Comment	May 3, 2026 0:46am

coderabbitai · 2026-05-03T12:36:38Z

Warning

Rate limit exceeded

@loothero has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 39 minutes and 11 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ee30d10a-d99e-411f-ae15-f86684a808af

📥 Commits

Reviewing files that changed from the base of the PR and between 293c367 and 4b7c45a.

📒 Files selected for processing (1)

.github/workflows/pr-ci.yml

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch ci/fix-claude-jq-array-filter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Review rate limit: 0/1 reviews remaining, refill in 39 minutes and 11 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

railway-app · 2026-05-03T12:36:39Z

🚅 Deployed to the summit-pr-146 environment in Summit

3 services not affected by this PR

summit-db
summit-indexer
summit-api

Copilot

Pull request overview

This PR fixes the Claude review extraction logic in the PR CI workflow so Claude review jobs can read execution_file correctly after the earlier migration to posting reviews from the action output. It keeps the existing review flow intact while updating the jq filters to match the actual JSON shape produced by the Claude action.

Changes:

Update all four Claude review jobs to iterate over a JSON array root when extracting the final result message.
Apply the same array-root fix to the assistant-text fallback extractor in each Claude review job.
Preserve the existing failure behavior when no extractable review output is found.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ssing output Three additional bugs in the previous PR 146 commit (fc83c6c) that local end-to-end testing surfaced: 1. \`tail -1\` chops the multi-line .result string. Claude's review is a single string with embedded newlines; jq -r emits them as physical line breaks. \`tail -1\` would have kept only the final summary line ("Summary: 0 CRITICAL, 1 HIGH, ..."). The blocking-check greps for \`[(CRITICAL|HIGH)]\` (with brackets) which lives in the body, not the bracketed-summary form — so a real HIGH/CRITICAL finding would have been silently dropped from /tmp/review.txt and the job would have passed. Catastrophic. Fix: \`[.[] | select(.type == "result")] | last | .result // empty\` — collect, take last array element, no tail. 2. jq exits 5 on parse errors of malformed JSON. Under the GitHub Actions default shell (bash --noprofile --norc -eo pipefail {0}), pipefail propagates that through the substitution and set -e kills the step before our explicit empty-output guard fires. Add \`|| true\` to mask jq exit codes; the explicit \`[ -z "$REVIEW" ]\` guard then handles it. 3. SDKResultMessage has two variants: SDKResultSuccess (with .result: string) and SDKResultError (no .result, has .errors[]). When Claude errors mid-task, .result is missing — the assistant-text fallback now joins all text blocks across messages with double-newlines so the comment is informative rather than empty. Also: Extract step now writes a heading-prefixed fallback to /tmp/review.txt on failure paths so Post can post it directly without duplicating the heading (which both Claude's prompt instructions and the previous Post step were prepending). Single canonical heading. Verified end-to-end against the actual Claude Code stream-JSON shape (JSON.stringify(SDKMessage[]) per base-action/src/run-claude-sdk.ts): - multi-line .result: full body preserved ✓ - HIGH-finding multi-line .result: blocking-check still fires ✓ - SDKResultError (no .result): assistant-text fallback ✓ - empty array / malformed JSON / missing file: exit 1 with clear error, fallback comment posted, job fails ✓ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings May 3, 2026 12:36

vercel Bot deployed to Preview May 3, 2026 12:36 View deployment

Copilot started reviewing on behalf of loothero May 3, 2026 12:36 View session

Copilot AI reviewed May 3, 2026

View reviewed changes

vercel Bot deployed to Preview May 3, 2026 12:46 View deployment

loothero merged commit 5f0ae9f into main May 3, 2026
20 checks passed

loothero deleted the ci/fix-claude-jq-array-filter branch May 3, 2026 20:31

railway-app Bot temporarily deployed to Summit / summit-pr-146 May 3, 2026 20:31 Destroyed

loothero mentioned this pull request May 3, 2026

ci: factor Claude review extraction into a locally-testable script #147

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: iterate JSON array root when extracting Claude review text#146

ci: iterate JSON array root when extracting Claude review text#146
loothero merged 2 commits into
mainfrom
ci/fix-claude-jq-array-filter

loothero commented May 3, 2026

Uh oh!

gemini-code-assist Bot commented May 3, 2026

Uh oh!

vercel Bot commented May 3, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented May 3, 2026 •

edited

Loading

Rate limit exceeded

Uh oh!

railway-app Bot commented May 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

loothero commented May 3, 2026

Summary

Test plan

Uh oh!

gemini-code-assist Bot commented May 3, 2026

Uh oh!

vercel Bot commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rate limit exceeded

Uh oh!

railway-app Bot commented May 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented May 3, 2026 •

edited

Loading

coderabbitai Bot commented May 3, 2026 •

edited

Loading