Skip to content

Optimize security-guard Claude token usage via prompt cache alignment and smaller diff payloads#2085

Merged
lpcox merged 4 commits intomainfrom
copilot/optimize-claude-token-usage-again
Apr 18, 2026
Merged

Optimize security-guard Claude token usage via prompt cache alignment and smaller diff payloads#2085
lpcox merged 4 commits intomainfrom
copilot/optimize-claude-token-usage-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 18, 2026

security-guard was spending avoidable tokens by placing variable PR diff content mid-prompt (breaking prefix cacheability for trailing instructions) and by preloading a larger-than-needed diff payload. This PR restructures prompt composition and tightens input limits to reduce per-run token/cost overhead while preserving review quality.

  • Prompt cache optimization

    • Reordered prompt sections so static instructions (Your Task, Security Checks, Output Format) appear before variable diff content.
    • Moved Changed Files (Pre-fetched) to the end of the prompt to maximize stable Anthropic prefix reuse across turns.
  • PR diff payload reduction

    • Reduced pre-fetched diff cap from 8000 to 5000 bytes.
    • Added explicit truncation notice to steer the agent to get_file_contents when full context is needed.
    • Centralized limit via a single shell variable to keep cap + message consistent.
  • Turn ceiling tightening

    • Reduced Claude max-turns from 12 to 10 to better bound worst-case token spend.
  • Compiled workflow update

    • Updated security-guard.lock.yml to reflect source workflow changes (including updated frontmatter hash and runtime command changes).
engine:
  id: claude
  max-turns: 10
DIFF_LIMIT=5000
head -c "$DIFF_LIMIT" "$DIFF_TMP" || true
if [ "$DIFF_SIZE" -gt "$DIFF_LIMIT" ]; then
  echo -e "\n[DIFF TRUNCATED at ${DIFF_LIMIT} bytes — use get_file_contents for full context]"
fi

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha --verify main p/bin/node (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.68.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha --verify main rgo/bin/node (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 18, 2026 that may be closed by this pull request
8 tasks
Copilot AI changed the title [WIP] Optimize Claude token usage for security guard workflow Optimize security-guard Claude token usage via prompt cache alignment and smaller diff payloads Apr 18, 2026
Copilot AI requested a review from lpcox April 18, 2026 22:55
@lpcox lpcox marked this pull request as ready for review April 18, 2026 22:57
@lpcox lpcox requested a review from Mossaka as a code owner April 18, 2026 22:57
Copilot AI review requested due to automatic review settings April 18, 2026 22:57
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces security-guard Claude token usage by improving prompt prefix cacheability (moving variable PR diff content to the end), shrinking the pre-fetched PR diff payload, and lowering the agent turn ceiling, with corresponding updates to the compiled .lock.yml workflow.

Changes:

  • Reordered the workflow prompt so static instructions come before the variable PR diff, moving “Changed Files (Pre-fetched)” to the end.
  • Reduced the pre-fetched diff cap (8000 → 5000 bytes) and added an explicit truncation notice.
  • Reduced Claude max-turns (12 → 10) and updated the compiled workflow accordingly.
Show a summary per file
File Description
.github/workflows/security-guard.md Lowers max-turns, truncates pre-fetched diffs with a notice, and reorders prompt sections for better caching.
.github/workflows/security-guard.lock.yml Updates the compiled workflow to reflect the new diff truncation logic and --max-turns 10 invocation.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/workflows/security-guard.md:117

  • In the "Your Task" section, step 1 says "Review the pre-fetched diff above", but the "Changed Files (Pre-fetched)" block was moved to the end of the prompt. This instruction is now incorrect and may cause the agent to miss the diff. Update the wording to match the new placement (e.g., "below"), or move the bullet list to follow the pre-fetched diff section.
Analyze PR #${{ github.event.pull_request.number }} in repository ${{ github.repository }}.

1. **Review the pre-fetched diff above** to understand what files changed
2. **Use `get_file_contents`** only if you need full context beyond the diff
  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment on lines 760 to 762
sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --tty --env-all --exclude-env ANTHROPIC_API_KEY --exclude-env GH_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains '*.githubusercontent.com,anthropic.com,api.anthropic.com,api.github.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,cdn.playwright.dev,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,docs.github.com,files.pythonhosted.org,ghcr.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,playwright.download.prss.microsoft.com,ppa.launchpad.net,pypi.org,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,sentry.io,statsig.anthropic.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com' --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --session-state-dir /tmp/gh-aw/sandbox/agent/session-state --enable-host-access --build-local --enable-api-proxy --difc-proxy-host host.docker.internal:18443 --difc-proxy-ca-cert /tmp/gh-aw/difc-proxy-tls/ca.crt \
-- /bin/bash -c 'export PATH="$(find /opt/hostedtoolcache -maxdepth 4 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && claude --print --no-chrome --max-turns 12 --mcp-config "${{ runner.temp }}/gh-aw/mcp-config/mcp-servers.json" --allowed-tools Bash,BashOutput,Edit,ExitPlanMode,Glob,Grep,KillBash,LS,MultiEdit,NotebookEdit,NotebookRead,Read,Task,TodoWrite,Write,mcp__github__download_workflow_run_artifact,mcp__github__get_code_scanning_alert,mcp__github__get_commit,mcp__github__get_dependabot_alert,mcp__github__get_discussion,mcp__github__get_discussion_comments,mcp__github__get_file_contents,mcp__github__get_job_logs,mcp__github__get_label,mcp__github__get_latest_release,mcp__github__get_me,mcp__github__get_notification_details,mcp__github__get_pull_request,mcp__github__get_pull_request_comments,mcp__github__get_pull_request_diff,mcp__github__get_pull_request_files,mcp__github__get_pull_request_review_comments,mcp__github__get_pull_request_reviews,mcp__github__get_pull_request_status,mcp__github__get_release_by_tag,mcp__github__get_secret_scanning_alert,mcp__github__get_tag,mcp__github__get_workflow_run,mcp__github__get_workflow_run_logs,mcp__github__get_workflow_run_usage,mcp__github__issue_read,mcp__github__list_branches,mcp__github__list_code_scanning_alerts,mcp__github__list_commits,mcp__github__list_dependabot_alerts,mcp__github__list_discussion_categories,mcp__github__list_discussions,mcp__github__list_issue_types,mcp__github__list_issues,mcp__github__list_label,mcp__github__list_notifications,mcp__github__list_pull_requests,mcp__github__list_releases,mcp__github__list_secret_scanning_alerts,mcp__github__list_starred_repositories,mcp__github__list_tags,mcp__github__list_workflow_jobs,mcp__github__list_workflow_run_artifacts,mcp__github__list_workflow_runs,mcp__github__list_workflows,mcp__github__pull_request_read,mcp__github__search_code,mcp__github__search_issues,mcp__github__search_orgs,mcp__github__search_pull_requests,mcp__github__search_repositories,mcp__github__search_users --debug-file /tmp/gh-aw/agent-stdio.log --verbose --permission-mode bypassPermissions --output-format stream-json "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"${GH_AW_MODEL_AGENT_CLAUDE:+ --model "$GH_AW_MODEL_AGENT_CLAUDE"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
-- /bin/bash -c 'export PATH="$(find /opt/hostedtoolcache -maxdepth 4 -type d -name bin 2>/dev/null | tr '\''\n'\'' '\'':'\'')$PATH"; [ -n "$GOROOT" ] && export PATH="$GOROOT/bin:$PATH" || true && claude --print --no-chrome --max-turns 10 --mcp-config "${{ runner.temp }}/gh-aw/mcp-config/mcp-servers.json" --allowed-tools Bash,BashOutput,Edit,ExitPlanMode,Glob,Grep,KillBash,LS,MultiEdit,NotebookEdit,NotebookRead,Read,Task,TodoWrite,Write,mcp__github__download_workflow_run_artifact,mcp__github__get_code_scanning_alert,mcp__github__get_commit,mcp__github__get_dependabot_alert,mcp__github__get_discussion,mcp__github__get_discussion_comments,mcp__github__get_file_contents,mcp__github__get_job_logs,mcp__github__get_label,mcp__github__get_latest_release,mcp__github__get_me,mcp__github__get_notification_details,mcp__github__get_pull_request,mcp__github__get_pull_request_comments,mcp__github__get_pull_request_diff,mcp__github__get_pull_request_files,mcp__github__get_pull_request_review_comments,mcp__github__get_pull_request_reviews,mcp__github__get_pull_request_status,mcp__github__get_release_by_tag,mcp__github__get_secret_scanning_alert,mcp__github__get_tag,mcp__github__get_workflow_run,mcp__github__get_workflow_run_logs,mcp__github__get_workflow_run_usage,mcp__github__issue_read,mcp__github__list_branches,mcp__github__list_code_scanning_alerts,mcp__github__list_commits,mcp__github__list_dependabot_alerts,mcp__github__list_discussion_categories,mcp__github__list_discussions,mcp__github__list_issue_types,mcp__github__list_issues,mcp__github__list_label,mcp__github__list_notifications,mcp__github__list_pull_requests,mcp__github__list_releases,mcp__github__list_secret_scanning_alerts,mcp__github__list_starred_repositories,mcp__github__list_tags,mcp__github__list_workflow_jobs,mcp__github__list_workflow_run_artifacts,mcp__github__list_workflow_runs,mcp__github__list_workflows,mcp__github__pull_request_read,mcp__github__search_code,mcp__github__search_issues,mcp__github__search_orgs,mcp__github__search_pull_requests,mcp__github__search_repositories,mcp__github__search_users --debug-file /tmp/gh-aw/agent-stdio.log --verbose --permission-mode bypassPermissions --output-format stream-json "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"${GH_AW_MODEL_AGENT_CLAUDE:+ --model "$GH_AW_MODEL_AGENT_CLAUDE"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
env:
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Claude CLI invocation was updated to --max-turns 10, but the job env still sets GH_AW_MAX_TURNS: 12 later in this step. This leaves the runtime metadata inconsistent with the actual turn limit and with the engine.max-turns value. Update GH_AW_MAX_TURNS in the lock workflow to 10 so all three sources of truth match.

Copilot uses AI. Check for mistakes.
@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented Apr 18, 2026

@copilot address this review feedback #2085 (review)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 18, 2026

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 84.33% 84.41% 📈 +0.08%
Statements 83.56% 83.64% 📈 +0.08%
Functions 87.39% 87.39% ➡️ +0.00%
Branches 74.78% 74.82% 📈 +0.04%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/docker-manager.ts 86.8% → 87.1% (+0.30%) 86.4% → 86.7% (+0.29%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 18, 2026

@copilot address this review feedback #2085 (review)

Addressed in 3cabb26: updated .github/workflows/security-guard.lock.yml to set GH_AW_MAX_TURNS: 10, so it now matches engine.max-turns and the Claude CLI --max-turns 10 value.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color passed ✅ PASS
Go env passed ✅ PASS
Go uuid passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Note: Java required a workaround — ~/.m2 was owned by root, so a custom localRepository path (/tmp/gh-aw/agent/m2repo) was used in settings.xml.

Generated by Build Test Suite for issue #2085 · ● 670.7K ·

@lpcox lpcox merged commit f447948 into main Apr 18, 2026
40 of 43 checks passed
@lpcox lpcox deleted the copilot/optimize-claude-token-usage-again branch April 18, 2026 23:18
@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

✅ GitHub MCP: Optimize security-guard Claude token usage; Handle unversioned OpenAI /responses paths
✅ Playwright: GitHub page title verified
✅ File Writing: Test file created
✅ Bash Tools: File verified

Status: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Offline) Mode

Test Result
GitHub MCP (list merged PRs) ✅ PR #2085 returned
GitHub.com connectivity (HTTP) ✅ 200
File write/read ✅ (pre-step verified)
BYOK inference (this response)

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

Overall: PASS | Author: @app/copilot-swe-agent | Assignees: @lpcox, @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test Results

  • ✅ GitHub MCP — PR listed: "Optimize security-guard Claude token usage via prompt cache alignment and smaller diff payloads"
  • ✅ GitHub.com connectivity — HTTP 200/301
  • ✅ File write/read — /tmp/gh-aw/agent/smoke-test-copilot-24615910440.txt confirmed

Overall: PASS

PR by @app/copilot-swe-agent · Assignees: @lpcox, @Copilot

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results: GitHub Actions Services Connectivity ✅

All checks passed:

Check Result
Redis PING (host.docker.internal:6379) PONG
PostgreSQL pg_isready (host.docker.internal:5432) ✅ accepting connections
PostgreSQL SELECT 1 (smoketest DB, user postgres) ✅ returned 1

🔌 Service connectivity validated by Smoke Services

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test 24615910452

  • Optimize security-guard Claude token usage via prompt cache alignment and smaller diff payloads
  • Handle unversioned OpenAI /responses paths in API proxy sidecar
  • GitHub merged PR review: ✅
  • safeinputs-gh query: ❌
  • Playwright title check: ✅
  • Tavily search: ❌
  • File write/read: ✅
  • Build AWF: ✅
    Overall: FAIL

Warning

⚠️ Firewall blocked 3 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • chatgpt.com
  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "chatgpt.com"
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡ Claude Token Optimization2026-04-18 — Security Guard

3 participants