Skip to content

fix(claude-code-review): allow WebFetch/curl for addendum preview checks - #202

Merged
d-morrison merged 3 commits into
mainfrom
claude/survival-analysis-docs-0b39md
Jul 3, 2026
Merged

fix(claude-code-review): allow WebFetch/curl for addendum preview checks#202
d-morrison merged 3 commits into
mainfrom
claude/survival-analysis-docs-0b39md

Conversation

@d-morrison

Copy link
Copy Markdown
Collaborator

Follow-up to #200 (refs #199).

Problem

#200 fixed the CAS-tool denials, but a math review on rme#954 still failed with no verdict — and denials went up (33 → 36) over 81 turns. So the CAS tools weren't the main blocker.

The real remaining gap is the addendum's second instruction: verify computed values and figures against the PR-preview site (rme's addendum: "check any computed value or figure the prose describes against the PR-preview site … rather than trusting the prose's own description"). No fetch tool was granted, so:

  • The code-review plugin fans out to 4 parallel sub-agents.
  • Each tries to fetch the preview for every value/figure it checks.
  • Every attempt hits a permission denial (no WebFetch, no curl).
  • The run finishes is_error:false but never emits a ### Verdict — tripping the same gha#173 / stub-review guard.

Fix

Allowlist WebFetch and Bash(curl:*).

Security: both are read-only GETs. They add no exfiltration capability beyond Bash(python3:*) (already granted), which can already make arbitrary network calls via urllib — this just lets the reviewer fetch directly instead of scripting it.

After merge

Move v2 (auto via slide-major-tag.yml), then re-trigger the review on rme#954.

Testing

  • YAML parses (yaml.safe_load).
  • Tool-policy change only; guard/fixtures unaffected. _selftest.yml exercises the workflow end-to-end on this PR.

🤖 Generated with Claude Code


Generated by Claude Code

A repo's prompt-addendum can tell the reviewer to verify computed
values and figures against the PR-preview site (rme does). No fetch
tool was granted, so each of the code-review plugin's 4 parallel
sub-agents hit a permission denial per fetch attempt — 36 denials on
rme#954 — and the run finished is_error:false but never emitted a
verdict, tripping the gha#173 stub-review guard.

Allowlist WebFetch and Bash(curl:*). Both are read-only GETs and add
no exfiltration capability beyond Bash(python3:*), already granted,
which can make arbitrary network calls via urllib. Refs #199.
Copilot AI review requested due to automatic review settings July 3, 2026 05:25
@github-actions
github-actions Bot removed the request for review from Copilot July 3, 2026 05:25
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Code review: #202

PR: "fix(claude-code-review): allow WebFetch/curl for addendum preview checks" — adds WebFetch and Bash(curl:*) to the --allowedTools list in .github/workflows/claude-code-review.yml's claude_args, plus an explanatory comment and a changelog fragment. Follow-up to #200/#199, meant to fix reviews that stall (no ### Verdict) because the reviewer has no tool to check computed values against a PR-preview site.

No --comment argument was supplied, so per the code-review skill's process these findings are reported here only — nothing was posted to GitHub.

Findings

1. (Blocking) The new tool grant is self-defeating — the untouched system prompt still tells the agent the tools don't exist and forbids using them.
All four independent review passes converged on this, and a dedicated validation pass confirmed it.

  • Grant added by this diff: .github/workflows/claude-code-review.yml#L465-L468WebFetch,Bash(curl:*) inserted into --allowedTools.

  • Contradicting text, left unmodified by this diff, in the same claude_args block: .github/workflows/claude-code-review.yml#L478-L490:

    "This sandboxed environment has no network-fetch tools (WebFetch/WebSearch) enabled. Do not attempt to fetch external URLs to verify a claim..."

    A compliant agent reading its own system prompt will still refuse to fetch the PR-preview site and will still write "could not be independently verified in this environment" — exactly the failure this PR exists to fix. The --allowedTools edit and the --append-system-prompt text now flatly contradict each other. (Confirmed directly: this very review session runs against the pre-merge base workflow, which has neither WebFetch granted nor this text updated, and I did in fact receive that exact "no network-fetch tools enabled" instruction verbatim at session start.) This block needs to be updated to reflect that fetch tools are now available, or the fix does nothing.

2. The new "read-only (GET); no write verbs" security justification is inaccurate for Bash(curl:*).
Found by 3 of 4 review agents, confirmed by an independent validation pass citing Claude Code's own permissions docs.

WebFetch is indeed GET-only, but Bash(curl:*) is a plain prefix/wildcard match on the full command string — it has no awareness of HTTP verbs. It matches curl -X POST ..., curl --data ..., curl -T/--upload-file ... just as readily as a plain GET, so the security rationale ("no write verbs") doesn't hold for the curl half of the grant. This falls under CLAUDE.md's "Fact-check prose against domain knowledge and external sources" review rule — the claim is checkable and wrong as written. (The broader "no new exfiltration capability beyond Bash(python3:*)" argument in the same comment is separately sound and not in question — only the specific "read-only GET; no write verbs" phrasing is inaccurate.)

Non-blocking note (not independently validated): one reviewer flagged that this grant is unscoped WebFetch/Bash(curl:*) (any host), whereas claude.yml's own "Build WebFetch allowlist" step deliberately scopes WebFetch to a curated WebFetch(domain:<host>) allowlist. The reviewer itself rated the incremental risk low (since Bash(python3:*) already permits arbitrary outbound requests via urllib), so I'm not treating this as a blocking finding — just worth a look if tightening this grant to match the repo's existing pattern is easy.

Nothing else stood out: the changelog fragment is correctly named/formatted per changelog.d/README.md, and this PR doesn't touch a workflow_call input, a versioning tag, or a composite/example-stub pairing, so none of CLAUDE.md's doc-sync rules apply here.

Verdict

Needs more work — finding 1 is a genuine, self-confirming bug: the PR grants the fetch tools but leaves the reviewer's own system prompt instructing it not to use them, which defeats the fix's stated purpose. Finding 2 is a smaller factual-accuracy fix to the new comment/changelog text.

Address review of #202:

1. (blocking) The --append-system-prompt still told the agent "no
   network-fetch tools enabled; do not attempt to fetch external URLs",
   which contradicted and defeated the WebFetch/curl grant. Replace that
   text with an instruction that the fetch tools ARE available for
   verifying values/figures against rendered output.

2. Correct the "read-only (GET); no write verbs" claim: WebFetch is
   GET-only, but Bash(curl:*) is a prefix match that also allows write
   verbs (curl -X POST / --data / -T). Reworded in the workflow comment
   and the changelog fragment; the no-new-capability-beyond-python3
   argument is unchanged and still holds.
Copilot AI review requested due to automatic review settings July 3, 2026 05:36
@github-actions
github-actions Bot removed the request for review from Copilot July 3, 2026 05:37

Copy link
Copy Markdown
Collaborator Author

Thanks — finding 1 was a real miss on my part. Addressed both in 24e9927:

1. (Blocking) System prompt contradicted the grant — Addressed. You're right: granting WebFetch/curl did nothing while the --append-system-prompt still said "no network-fetch tools (WebFetch/WebSearch) enabled. Do not attempt to fetch external URLs." I replaced that paragraph with text stating the fetch tools are available for verifying values/figures against rendered output, and kept the "a denied tool call is never a reason to stop early / still emit the ### Verdict" resilience clause. The grant and the prompt now agree.

2. "read-only (GET); no write verbs" inaccurate for curl — Addressed. Correct — Bash(curl:*) is a prefix match that also allows curl -X POST / --data / -T. Reworded both the workflow comment and the changelog fragment: WebFetch is GET-only, curl can issue any verb, and the load-bearing point is unchanged — neither adds capability beyond Bash(python3:*), which already permits arbitrary requests via urllib.

Non-blocking (unscoped WebFetch/Bash(curl:*) vs claude.yml's scoped WebFetch(domain:<host>)) — Rebut, with an offer to defer. claude.yml runs in a single repo with a known preview host, so a curated domain allowlist fits there. This reusable workflow serves consumers with different preview hosts (d-morrison.github.io, ucd-serg.github.io, ucdavis…), and Bash(curl:*) can't be host-scoped through the tool-pattern at all — so a WebFetch(domain:…)-only scope would be a half-measure while curl stays open, and the python3-parity argument already bounds the real risk (arbitrary outbound is already possible). If you'd still like host-scoping (e.g. a preview-fetch-hosts input that builds a WebFetch(domain:…) allowlist and drops the broad curl grant), I'm happy to do it as a follow-up issue rather than expand this PR.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Working on this — paws off until I'm done.

(Rebasing onto main to resolve the merge conflict PR #201 introduced: #201 moved the claude_args block this PR edits into a new run-claude-review-attempt composite action. Re-applying this PR's WebFetch/Bash(curl:*) grant and system-prompt fix to the new location, not touching the review-response work already done in 24e9927.)


Generated by Claude Code

…sis-docs-0b39md

# Conflicts:
#	.github/workflows/claude-code-review.yml
Copilot AI review requested due to automatic review settings July 3, 2026 05:59

Copy link
Copy Markdown
Collaborator Author

Merged main in and resolved the conflict (a3fe1ac): PR #201 (just merged) moved the claude_args block this PR edits into a new .github/actions/run-claude-review-attempt/ composite action, so I took main's version of claude-code-review.yml wholesale and re-applied this PR's actual change — the WebFetch/Bash(curl:*) allowlist addition and the matching system-prompt update from 24e9927 — to that new file instead. Diffed against current main to confirm the composite action carries exactly this PR's intended change and nothing else; claude-code-review.yml itself is now byte-identical to main. Fixture tests and YAML validation still pass.

Base is now 79dd5c4 (post-#201). Should be ready for a fresh review once CI runs on the merge commit.


Generated by Claude Code

@github-actions
github-actions Bot removed the request for review from Copilot July 3, 2026 06:00
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Code Review: #202

PR: "fix(claude-code-review): allow WebFetch/curl for addendum preview checks" — allowlists WebFetch and Bash(curl:*) for the Claude PR-reviewer and updates its system prompt in lockstep, so a repo's prompt-addendum (e.g. rme's) asking the reviewer to check computed values against a PR-preview site no longer causes every one of the 4 parallel sub-agents to hit a permission denial and finish without a ### Verdict.

This PR was already reviewed once by Claude (comment: #202 (comment), verdict Needs more work, two findings). Since then, new commits landed — 24e9927 (fixes) and the merge/rebase a3fe1acd (resolving a conflict with #201, which relocated the edited claude_args block into the new .github/actions/run-claude-review-attempt/action.yml composite action) — so I re-verified the current state against the earlier findings rather than just re-stating the old verdict.

Finding 1 (was blocking) — Fixed, verified. The --append-system-prompt text in .github/actions/run-claude-review-attempt/action.yml (lines 160–166) no longer contradicts the new tool grant. It previously said fetch tools don't exist and told the agent not to try; it now correctly says WebFetch and curl are available and should be used to verify computed values/figures against rendered output. Grant and prompt now agree.

Finding 2 (factual accuracy) — Fixed, verified. The inline comment and changelog fragment no longer claim Bash(curl:*) is "read-only (GET); no write verbs." Current wording (lines 142–143) correctly states WebFetch is GET-only while Bash(curl:*) is a prefix match that also allows write verbs (curl -X POST, --data, -T) — accurate given how Claude Code's Bash(pattern:*) tool-permission matching works (string-prefix match, no verb awareness). The "no exfiltration capability beyond Bash(python3:*)" argument itself remains sound and unchanged.

Non-blocking note (unscoped WebFetch/Bash(curl:*) vs. claude.yml's host-scoped WebFetch(domain:...)) was rebutted by the author with a reasonable explanation (this reusable workflow serves multiple consumer preview hosts, so a single-host allowlist doesn't fit, and curl can't be host-scoped through the tool-permission pattern anyway) and offered as a follow-up issue rather than blocking this PR. That's a reasonable call — not re-raising it.

Rebase/relocation check. Confirmed the merge is clean: diffing against pre-PR main (79dd5c4) shows only the two intended files changed (run-claude-review-attempt/action.yml, changelog.d/review-allow-preview-fetch.fixed.md); .github/workflows/claude-code-review.yml is byte-identical to main (verified directly, not just taking the author's comment at their word). YAML parses cleanly (yaml.safe_load). The changelog fragment's filename/category (*.fixed.md) matches changelog.d/README.md's convention.

No CLAUDE.md doc-sync rules apply here — no new workflow_call input, no versioning-tag change, and no composite/example-stub pairing was touched, matching the prior review's conclusion.

No new issues found beyond what the prior review already caught, and both of those are now correctly resolved in the current code.

Verdict

Ready for merge — the prior review's blocking finding (self-defeating system prompt) and its factual-accuracy finding (curl write-verb claim) are both fixed and verified in the current post-rebase code; the non-blocking scoping note was reasonably deferred by the author. (Note: per this run's configuration, no --comment was requested, so these findings are reported here only and nothing was posted to GitHub.)

@d-morrison
d-morrison merged commit 9b544f8 into main Jul 3, 2026
18 checks passed
@d-morrison
d-morrison deleted the claude/survival-analysis-docs-0b39md branch July 3, 2026 06:05
d-morrison pushed a commit that referenced this pull request Jul 3, 2026
Review finding on #205: the parenthetical implied #202's diff was
fixing the same stub-review bug #201 fixed, and that both PRs' reviews
were touching claude-code-review.yml right up until merge. Neither is
quite right: #202 fixed a different (related) allowlist gap and only
hit #185's stub-review signature as a bystander while it still edited
claude-code-review.yml directly, before a rebase onto #201 moved that
edit into the new run-claude-review-attempt composite action. Verified
the underlying fact via the actual failing run's execution output
(permission_denials_count:1, no verdict) rather than dropping the
citation outright.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B2aHbEXwTUA13vYMuQXqas
d-morrison added a commit that referenced this pull request Jul 3, 2026
…-merge (#205)

* docs: note that fixes to claude-code-review.yml can't self-verify pre-merge

Learned across gha#201/#202: this repo's own claude-review.yml pins
claude-code-review.yml@v2 (the released tag), not a local ref, so a PR
fixing a bug in that file hits the pre-fix version on its own automatic
review until after merge. Both PRs saw their own claude-review check fail
with the exact signature they were fixing, right up until merge.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B2aHbEXwTUA13vYMuQXqas

* fix(docs): correct overstated gha#202 citation in self-verify note

Review finding on #205: the parenthetical implied #202's diff was
fixing the same stub-review bug #201 fixed, and that both PRs' reviews
were touching claude-code-review.yml right up until merge. Neither is
quite right: #202 fixed a different (related) allowlist gap and only
hit #185's stub-review signature as a bystander while it still edited
claude-code-review.yml directly, before a rebase onto #201 moved that
edit into the new run-claude-review-attempt composite action. Verified
the underlying fact via the actual failing run's execution output
(permission_denials_count:1, no verdict) rather than dropping the
citation outright.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B2aHbEXwTUA13vYMuQXqas

---------

Co-authored-by: Claude <noreply@anthropic.com>
d-morrison pushed a commit that referenced this pull request Jul 18, 2026
…loses #266)

Add WebSearch to run-claude-review-attempt's --allowedTools and update
the system prompt in lockstep, so the reviewer can locate authoritative
sources (docs, specs, papers) for the claims, citations, and APIs the
review guidelines tell it to verify, instead of only fetching URLs it
already has (from #202's WebFetch/curl grant). No new egress capability:
python3/curl already permit arbitrary direct requests (see #240), and
WebSearch queries are served through the Anthropic API.

Also reword the three stale 'no network-fetch tools' claims left behind
by #202 (check-latex-macros input description, its prompt block, and the
reference page): the macro checks stay local-checkout-only, now
justified by the pinned submodule version being authoritative rather
than by a tool gap that no longer exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMTBtWqifJeeDRJHVkQZua
d-morrison added a commit that referenced this pull request Jul 18, 2026
)

* start: grant reviewer WebSearch access for fact-checking (closes #266)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMTBtWqifJeeDRJHVkQZua

* claude-code-review: grant the reviewer WebSearch for fact-checking (closes #266)

Add WebSearch to run-claude-review-attempt's --allowedTools and update
the system prompt in lockstep, so the reviewer can locate authoritative
sources (docs, specs, papers) for the claims, citations, and APIs the
review guidelines tell it to verify, instead of only fetching URLs it
already has (from #202's WebFetch/curl grant). No new egress capability:
python3/curl already permit arbitrary direct requests (see #240), and
WebSearch queries are served through the Anthropic API.

Also reword the three stale 'no network-fetch tools' claims left behind
by #202 (check-latex-macros input description, its prompt block, and the
reference page): the macro checks stay local-checkout-only, now
justified by the pinned submodule version being authoritative rather
than by a tool gap that no longer exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMTBtWqifJeeDRJHVkQZua

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants