Skip to content

reduce token usage for daily-syntax-error-quality and dead-code-remover workflows#24914

Merged
pelikhan merged 3 commits intomainfrom
copilot/reduce-daily-syntax-error-tokens
Apr 6, 2026
Merged

reduce token usage for daily-syntax-error-quality and dead-code-remover workflows#24914
pelikhan merged 3 commits intomainfrom
copilot/reduce-daily-syntax-error-tokens

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 6, 2026

These two Copilot workflows consumed ~18.5M tokens in a single combined run (~19% of the monthly budget). Since both use engine: copilot, max-turns is unavailable — optimizations are prompt-level only.

daily-syntax-error-quality.md (9.35M tokens, 129 turns, 0 errors)

  • Reduced test cases from 3 → 2 workflows
  • Added ## Token Budget Guidelines block (≤ 40-turn target)
  • Early-stop noop: when average score ≥ 70 and no individual score < 55, skip Phase 6 entirely — covers virtually all runs given the prior run found nothing
  • Replaced verbose per-test JSON evaluation blobs with a compact one-line scoring format
  • Replaced the 250-line issue template (including inline Go implementation guide) with a 10-line table

dead-code-remover.md (9.1M tokens)

  • Reduced batch from 10 → 5 functions per run
  • Added ## Token Budget Guidelines block (≤ 30-turn target)
  • Early-stop after Phase 2: if no unprocessed functions remain, skip Phases 3–9 immediately
  • Capped grep caller-check output with -m 5
  • Capped build/test output with tail -20

scratchpad/token-budget-guidelines.md

  • Added entries for both workflows with expected impact, budget targets, and alert thresholds
  • Budget targets: daily-syntax-error-quality → 300K–2M tokens/run (typical early-stop vs. issue-creating); dead-code-remover → 4M–5M tokens/run (45–55% reduction from batch halving + output truncation)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 5871526/b092/imprev-parse .cfg git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuupstream /usr/bin/git ry=1 pkg/mod/golang.orev-parse ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name gh-aw.wasm ./cmd/gh-aw-wasm GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json 1.4.1/internal/xcontext/xcontext.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --git-dir x_amd64/vet /usr/bin/git g_.a yVIFwLdjv x_amd64/compile git rev-�� --git-dir x_amd64/compile /usr/bin/git ynMeQl-Yh GO111MODULE 64/pkg/tool/linu--show-toplevel /usr/bin/git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha ErrorFormatting630943636/001 --format=%(objectname) /usr/bin/infocmp -json 4 x_amd64/compile infocmp -1 s/test.md x_amd64/compile ache/node/24.14.1/x64/bin/node -json o x_amd64/compile node (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha se 0226430/b017/vet.cfg ck GOINSECURE 5871526/b132/ari/tmp/test-import-1411827841.js ache/go/1.25.8/x64/src/math/big/-m ache/go/1.25.8/x64/pkg/tool/linuInitial commit sRem�� 8628/001/stability-test.md 0226430/b158/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p github.com/githurev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha xterm-color 64/pkg/tool/linux_amd64/vet /usr/bin/git 5871526/b061/_pkgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 2140291243 xxol/GL-tkTNtkvurev-parse x_amd64/link git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git &#39;/tmp/TestParseDgit &#39;/tmp/TestParseDrev-parse 0226430/b361/vet--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x-test.v=true /usr/bin/infocmp licyMinIntegritygit /tmp/go-build360rev-parse 1/x64/bin/node infocmp (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha master resolved$ /opt/hostedtoolcache/node/24.14.1/x64/bin/node g_.a GO111MODULE x_amd64/vet node /tmp�� /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/agent-performance-analyzer.md x_amd64/vet /usr/bin/git g_.a YGaDW_VvF x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha GOMODCACHE resolved$ /usr/bin/git -json GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git g_.a GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel ache/go/1.25.8/x64/pkg/tool/linutest@example.com /usr/bin/git ithub/workflows -trimpath /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuconfig /usr/bin/git athSetup_GorootOgit faultBranchFromLrev-parse 0226430/b352/vet--show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json ase64.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json ii/equal_fold.go-ifaceassert x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha user.email test@example.com /usr/bin/git g_.a GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /opt/hostedtoolcache/node/24.14.1/x64/bin/node g_.a GO111MODULE x_amd64/link node (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha GOMODCACHE x_amd64/vet /usr/bin/git 346 GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git g_.a LvhFNvMoO x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha 999 -buildtags ache/node/24.14.1/x64/bin/node -errorsas -ifaceassert -nilfunc git-upload-pack t-27�� sistency_GoAndJavaScript3197834977/001/test-inlined-imports-enabled-with-env-template-expressiongit -tests /usr/bin/git -json GO111MODULE x_amd64/compile git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha -b resolved$ /usr/bin/git -json GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git g_.a GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha k/gh-aw/gh-aw -buildtags /bin/sh -errorsas -ifaceassert -nilfunc /bin/sh -c git-upload-pack &#39;/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmain_branch88remote.origin.urgit git-upload-pack &#39;/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmain_branch884881199/001&#39; 1/x64/bin/node -json GO111MODULE x_amd64/compile 1/x64/bin/node (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha k/gh-aw/gh-aw/.github/workflows -buildtags /usr/lib/git-core/git -errorsas -ifaceassert -nilfunc /usr/lib/git-core/git main�� run --auto epo.git --detach GO111MODULE x_amd64/compile git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 5871526/b006/ GOMODCACHE 64/pkg/tool/linuTest User env l.go l_test.go x_amd64/link GOINSECURE GOMOD 5871526/b006/sym--show-toplevel x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/pkg/tool/linu-nolocalimports GOINSECURE 5871526/b078/ GOMODCACHE 64/pkg/tool/linu/tmp/go-build3600226430/b451/_testmain.go env 3939901 GO111MODULE .cfg GOINSECURE g/x/text/unicodeinit ache/go/1.25.8/x64/src/syscall/j/tmp/TestHashStability_SameInputSameOutput3624978628/001/stability-test.md ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ylQP4Z8/vCNYLdc7D8RXanEmFBss env 5871526/b111/_pkg_.a .cfg aw.test GOINSECURE b/gh-aw/pkg/strirev-parse GOMODCACHE aw.test (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b153/_pkg_.a 38wk/F3_s36TZU8RlNGu_38wk 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name g/stringutil/ansi.go 64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/tty GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b121/_pkg_.a .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE /go-yaml GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE g/x/crypto/interrev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b159/_pkg_.a .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 5871526/b011/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env thub/workflows .cfg stants.test GOINSECURE 5871526/b011/memrev-parse ache/go/1.25.8/x--show-toplevel stants.test (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 5871526/b011/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b174/_pkg_.a GO111MODULE x_amd64/compile GOINSECURE g/x/net/dns/dnsm/tmp/js-hash-test-106610975/test-hash.js ache/go/1.25.8/x/home/REDACTED/work/gh-aw/gh-aw/.github/workflows/artifacts-summary.md x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 3939901 go .cfg GOINSECURE g/x/crypto/chachrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE 5871526/b011/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b176/_pkg_.a GO111MODULE x_amd64/compile GOINSECURE light ache/go/1.25.8/x--show-toplevel x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 5871526/b011/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env til.go o x_amd64/vet GOINSECURE 5871526/b011/sysrev-parse ache/go/1.25.8/x--show-toplevel x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet ortc�� 3939901 stmain.go x_amd64/compile GOINSECURE g/x/text/transforev-parse GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json eyset.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD nts.s x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 /strconv GOMODCACHE 64/pkg/tool/linuremote1 ortc�� mpiledOutput1412101613/001 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/consrev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 60236862/001 .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git RxJB/qfqEIUSRa-L/bin/sh GO111MODULE ache/go/1.25.8/xgit-upload-pack &#39;origin&#39; git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha edOutput3601708037/001 .cfg x_amd64/compile GOINSECURE GOMOD 5871526/b036/symabis x_amd64/compile env 5871526/b001/_pkg_.a i3Dw/UIwGdW4V3M93Js0Ri3Dw x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE iE8t3kR/-pSRkXIpremote (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet 8811�� -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha edOutput3601708037/001 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/nistec ache/go/1.25.8/xuser.email 64/pkg/tool/linutest@example.com env 5871526/b001/exe/a.out .cfg 64/pkg/tool/linux_amd64/compile GOINSECURE cfzEF-HBuouHJ/dI-1 GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 5871526/b244/_pkg_.a gNV_/-ERQMY_tDmUJytyNgNV_ ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/strinrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -importcfg /tmp/go-build3600226430/b410/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/tar.go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json age/common.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build3600226430/b396/cli.test /tmp/go-build3600226430/b396/cli.test -test.testlogfile=/tmp/go-build3600226430/b396/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json 1.4.1/internal/m-ifaceassert x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI and others added 2 commits April 6, 2026 15:41
…remover workflows

- daily-syntax-error-quality: reduce test cases from 3→2, add Token Budget
  Guidelines section, compact scoring format, early-stop noop when quality
  is good, remove verbose implementation guide from issue template
- dead-code-remover: reduce batch from 10→5 functions, add Token Budget
  Guidelines section, truncate grep/test output, immediate noop when no
  unprocessed functions found
- Recompile both lock files
- Update scratchpad/token-budget-guidelines.md with optimizations and alert
  thresholds for both workflows

Closes #<issue>"

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/a8890002-99df-44b5-b50b-93c2902a4951

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…ditions, conservative turn target

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/a8890002-99df-44b5-b50b-93c2902a4951

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate and reduce Daily Syntax Error Quality Check token usage reduce token usage for daily-syntax-error-quality and dead-code-remover workflows Apr 6, 2026
Copilot AI requested a review from pelikhan April 6, 2026 15:44
@pelikhan pelikhan marked this pull request as ready for review April 6, 2026 16:09
Copilot AI review requested due to automatic review settings April 6, 2026 16:09
@pelikhan pelikhan merged commit 5cfd3a0 into main Apr 6, 2026
130 of 174 checks passed
@pelikhan pelikhan deleted the copilot/reduce-daily-syntax-error-tokens branch April 6, 2026 16:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates two Copilot-based agent workflows to significantly reduce token consumption via prompt-level constraints (turn targets, early-stop conditions, reduced scope) and documents the new budgets/thresholds in the token budget guidelines.

Changes:

  • Add explicit token-budget guidance (turn targets, early-stop/noop paths) to daily-syntax-error-quality and dead-code-remover.
  • Reduce scope/output volume (2 test cases instead of 3; 5 functions per run instead of 10; compact scoring/report templates; truncated command outputs).
  • Extend scratchpad/token-budget-guidelines.md with new workflow entries, thresholds, and a revision-history note.
Show a summary per file
File Description
.github/workflows/daily-syntax-error-quality.md Adds token budget/early-stop guidance and compresses evaluation + issue template output.
.github/workflows/dead-code-remover.md Reduces batch size, adds token-budget guidelines, truncates grep/test outputs, and adds an early-stop path.
scratchpad/token-budget-guidelines.md Documents expected token impacts, targets/thresholds, and updates the alert thresholds + revision history.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 5

Comment on lines 103 to 107
### 4.1 Caller grep

```bash
grep -rn "FunctionName" --include="*.go" .
grep -rn -m 5 "FunctionName" --include="*.go" .
```
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caller safety check uses grep -m 5, which can miss additional matches after the first 5. That makes it unsafe to conclude “callers only in *_test.go” or “no callers,” and could lead to deleting live code that simply wasn’t shown in the truncated output. Consider collecting the full match set but printing only a bounded summary (e.g., write grep output to a temp file, then print a small head + a list of unique non-test files / counts) so the scan remains complete while the console output stays small.

Copilot uses AI. Check for mistakes.
```bash
go test ./pkg/... 2>&1
go test ./pkg/... 2>&1 | tail -20
echo "Test exit code: $?"
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go test ... | tail -20 changes the pipeline exit status to tail’s status, so echo "Test exit code: $?" will not reflect the real go test result (and the pipeline may succeed even when tests fail). To keep output small without losing correctness, enable pipefail and/or capture the go test exit code explicitly (e.g., via ${PIPESTATUS[0]} or by writing output to a file and tailing it afterwards).

Suggested change
echo "Test exit code: $?"
test_exit_code=${PIPESTATUS[0]}
echo "Test exit code: $test_exit_code"
exit "$test_exit_code"

Copilot uses AI. Check for mistakes.
Comment on lines +68 to +73
**Target**: Complete the full analysis in ≤ 40 turns.

- Test **2 workflows** (not 3) — one simple, one complex.
- One error category per workflow (Category A for workflow 1, Category B for workflow 2).
- **If the average score across both test cases is ≥ 70 and no individual score is < 55**: skip Phase 6 entirely, call `noop` with a one-line summary — do **not** generate the issue or structured report.
- When scores require an issue: use the compact format in Phase 6 — skip verbose per-dimension narratives.
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token-budget early-stop path instructs calling noop, but this workflow’s frontmatter safe-outputs only defines create-issue (no noop). Unless noop is implicitly available, this will cause the run to fail when the early-stop condition triggers. Add noop to safe-outputs (preferred), or adjust the guidance to use an allowed safe-output.

Copilot uses AI. Check for mistakes.
Comment on lines +290 to +295
- Limit output to 20-turn target

**Expected Impact:**
- **Token Reduction**: 85-95% for healthy runs (early-stop fires); 50-70% when an issue is created
- **Quality**: Maintained — compiler error quality still evaluated across all 5 dimensions
- **Runtime**: Reduced from ~129 turns to ≤ 20 turns (most runs end early)
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section says “Limit output to 20-turn target,” but the updated workflow prompt sets a ≤40 turn target. Align the scratchpad guidance with the workflow’s current budget target to avoid confusion/mismatched expectations.

Suggested change
- Limit output to 20-turn target
**Expected Impact:**
- **Token Reduction**: 85-95% for healthy runs (early-stop fires); 50-70% when an issue is created
- **Quality**: Maintained — compiler error quality still evaluated across all 5 dimensions
- **Runtime**: Reduced from ~129 turns to ≤ 20 turns (most runs end early)
- Limit output to ≤40-turn target
**Expected Impact:**
- **Token Reduction**: 85-95% for healthy runs (early-stop fires); 50-70% when an issue is created
- **Quality**: Maintained — compiler error quality still evaluated across all 5 dimensions
- **Runtime**: Reduced from ~129 turns to ≤ 40 turns (most runs end early)

Copilot uses AI. Check for mistakes.
Comment on lines +490 to +491
- Daily Syntax Error Quality Check: reduced to 2 test cases, compact scoring, early-stop noop (target 500K-1M/run)
- Dead Code Remover: reduced batch to 5 functions, truncated grep/test output, early-stop noop (target 1M-2M/run)
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The revision history’s stated token targets don’t match the budget targets defined above and in the alert-thresholds table (e.g., Daily Syntax Error Quality Check is documented as 300K–600K typical / up to 2M, and Dead Code Remover as 4M–5M). Please update these revision-history target ranges to match the current documented budgets so future readers aren’t misled.

Suggested change
- Daily Syntax Error Quality Check: reduced to 2 test cases, compact scoring, early-stop noop (target 500K-1M/run)
- Dead Code Remover: reduced batch to 5 functions, truncated grep/test output, early-stop noop (target 1M-2M/run)
- Daily Syntax Error Quality Check: reduced to 2 test cases, compact scoring, early-stop noop (target 300K-600K typical / up to 2M/run)
- Dead Code Remover: reduced batch to 5 functions, truncated grep/test output, early-stop noop (target 4M-5M/run)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Investigate and reduce Daily Syntax Error Quality Check token usage (9.35M tokens/run)

3 participants