Skip to content

feat(workflow-health-manager): reduce token usage ~42% via pre-agent steps and prompt trimming#28968

Merged
lpcox merged 4 commits intomainfrom
copilot/optimize-workflow-health-manager-tokens
Apr 28, 2026
Merged

feat(workflow-health-manager): reduce token usage ~42% via pre-agent steps and prompt trimming#28968
lpcox merged 4 commits intomainfrom
copilot/optimize-workflow-health-manager-tokens

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 28, 2026

workflow-health-manager was consuming ~1.84M tokens/run (48.2% cache efficiency) because deterministic discovery work and large verbose sections were repeated across all 28 agent turns.

Changes

Pre-agent steps: blocks (eliminates 4–5 agent turns)

  • build-inventory: runs gh aw compile --validate and lists executable workflows into /tmp/gh-aw/agent/ before the agent starts — no more inference-time filesystem scanning
  • load-metrics: pre-filters failing workflows (success_rate < 0.8) from metrics/latest.json via jq, handing the agent a ready-to-use top-20 list instead of requiring repo-memory reads
steps:
  - name: build-inventory
    env:
      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    run: |
      gh aw compile --validate > /tmp/gh-aw/agent/compile-validate.txt 2>&1 || true
      ls .github/workflows/*.md > /tmp/gh-aw/agent/workflow-list.txt || true
  - name: load-metrics
    run: |
      jq '[.workflow_runs | to_entries[] | select(.value.success_rate < 0.8)]
          | sort_by(.value.success_rate) | .[0:20]' \
        "$METRICS_FILE" > /tmp/gh-aw/agent/failing-workflows.json

Prompt trimming (~450K tokens/run across 28 turns)

  • Removed "Important Guidelines" (~30 lines) — restated phase descriptions verbatim
  • Removed "Success Metrics" (~11 lines) — qualitative framing the agent never acts on
  • Replaced 60-line dashboard template with a 15-line skeleton; heading structure is sufficient for format inference

Phase instructions updated

  • Phase 1 now reads from pre-computed files rather than re-running discovery inline
  • Phase 2 starts from failing-workflows.json and includes explicit guidance to batch list_workflow_runs calls

Expected impact: ~775K tokens/run reduction → ~1.07M/run (~42% reduction from 1.84M baseline)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login &#43; &#34;/&#34; &#43; .name ache/go/1.25.8/x64/pkg/tool/linu-buildmode=exe GOINSECURE t/internal/catmsrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-extld=gcc (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo 4162�� ithub/workflows 6237994/b022/vet100 .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuorigin 4861�� /001/noflag-a.md pkg/mod/github.com/google/jsonschema-go@v0.4.3/jsonschema/annotations.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -I /tmp/go-build404config -I ache/go/1.25.8/x^remote\..*\.gh-resolved$ (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json sonschema/annotations.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linuremote.origin.url /usr/bin/git 84/001/test-frongit .cfg ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git 3249325701/custonode 6237994/b053/vet/opt/hostedtoolcache/node/24.14.1/x64/bin/npm ache/go/1.25.8/xinstall git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv kflow.test rev-parse ortcfg.link -json GO111MODULE x_amd64/vet z9V76hYOpkPAX1d4v-/zztPBqUNDi7Vgamhhkrj/pB16H8NKYvjuUWPU88mo conf�� --get remote.origin.url g_.a g_.a GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv g_.a 6237994/b276/vet.cfg 1/x64/bin/node -I /tmp/go-build404-1 -I ache/go/1.25.8/x64/pkg/tool/linu--json t-ha�� vaScript3615964484/001/test-inli--limit /tmp/go-build1416237994/b011/vet100 1/x64/bin/node -o /tmp/go-build404rev-parse /opt/hostedtoolc--show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git g_.a 6237994/b233/vet\n ache/go/1.25.8/x: git rev-�� --show-toplevel ache/go/1.25.8/x^remote\..*\.gh-resolved$ /usr/bin/git ExpressionCompilgit -trimpath /opt/hostedtoolc--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel gh /usr/bin/git list l /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /tmp/TestGuardPogit remote /opt/hostedtoolc--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /v3.0.0 &#39;/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmaster_branch3661795445/001&#39; 1/x64/bin/node -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.8/x1 t-ha�� ithub/workflows/architecture-guatest-logs/run-1 6237994/b420/_testmain.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -errorsas -ifaceassert -nilfunc ache/go/1.25.8/x64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv 3038-33750/test-1869868455 -buildtags bject.type] | @tsv -errorsas -ifaceassert -nilfunc 6237994/b443/importcfg api k/gh-aw/gh-aw/pkg/stats/statvar.go k/gh-aw/gh-aw/pkg/stats/spec_test.go /opt/hostedtoolcache/node/24.14.1/x64/bin/node cli/install.sh..git GO111MODULE x_amd64/compile node (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen971639449/00remote.origin.urgit l e/git -c=4 -nolocalimports -importcfg e/git -C /tmp/TestGuardPolicyMinIntegrityOnlyrepos_only_wnonexistent/repo rev-parse /tmp/go-build1416237994/b456/timeutil.test -json GO111MODULE x_amd64/asm /tmp/go-build1416237994/b456/timeutil.test (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolcconfig /usr/bin/git 3038-33750/test-git -buildtags 64/pkg/tool/linu--oneline git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git -unreachable=falgit .cfg /opt/hostedtoolc--show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm env -json sonschema/annotations.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE h-aw.wasm; \ AF-buildtags (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv GOMODCACHE nF/2rw-RdHCw_apH02zfbN4/KAMAEFRb-buildtags /usr/bin/gh ortcfg .cfg 64/pkg/tool/linu--show-toplevel gh run download 5 /usr/bin/git test-logs/run-5 QuTc/8J1aAAdvjhKrev-parse 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel /home/REDACTED/work/gh-aw/gh-aw/pkg/repoutil/repoutil_test.go /usr/bin/git -json GO111MODULE x_amd64/compile git conf�� waysRecompiles1045344244/001 remote.origin.url /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linu-extld=gcc /usr/bin/git j1KmqPzwP GO111MODULE 64/pkg/tool/linu--show-toplevel /usr/bin/git conf�� --get-regexp ^remote\..*\.gh-resolved$ /usr/bin/git 256956264/.githugit stmain.go 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /v2.0.0 /tmp/go-build1416237994/b086/vet.cfg 1/x64/bin/node (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv IFEu/Jhtymj2WcILd1bgTIFEu -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git 6237�� 2115565622 6237994/b444/_testmain.go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -json GO111MODULE x_amd64/compile /opt/hostedtoolcache/go/1.25.8/x--jq (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv tCGo0QhINLRM5BOySe7e/tCGo0QhINLRM5BOySe7e -goversion 6237994/b461/vet.cfg -c=4 -nolocalimports -importcfg git -C /tmp/TestGuardPolicyMinIntegrityOnlyrepos_only_without_min-integrity3922677420/001 remote clusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260428-193038-33750/test-.artifacts[].name remote /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -json 1.5.0/internal/mrev-parse x_amd64/compile /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuowner/test-repo -ato�� -bool -buildtags mple.com/org/repo.git -errorsas -ifaceassert -nilfunc git (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv HEAD st/dist/workers/forks.js $name) { hasDiscussionsEnabled } } /tmp/bare-increm/opt/hostedtoolcache/node/24.14.1/x64/bin/node gin/full-mode-br--experimental-import-meta-resolve ache/node/24.14.--require git diff�� --binary c1fe3c9f68ab2876--conditions bin/git -m ode_modules/vite-C ache/node/24.14./home/REDACTED/work/gh-aw/gh-aw/.github/workflows git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv . tions/setup/js/node_modules/vite--stdout modules/@npmcli/run-script/lib/node-gyp-bin/go /tmp/bare-incremgit . tnet/tools/git git diff�� --binary (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .default_branch /no-input false r: $owner, name: $name) { hasDiscussionsEnabled } } git init -b maingit 803ba0b9911582eb-C ode-gyp-bin/git 1/x64/bin/node ve g_.a tions/setup/js/node_modules/vitest/suppress-warnings.cjs r: $owner, name: $name) { hasDiscussionsEnabled } } -exist mp ac47942fc15300cc/home/REDACTED/work/gh-aw/gh-aw tions/setup/js/nshow (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .default_branch etup-node/git/reremote.origin.url -m .yml 86_64/git git ode_modules/.bin/home/REDACTED/work/gh-aw/gh-aw/.github/workflows go run k/gh-aw/gh-aw :latest (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv ry=1 64/pkg/tool/linuremote.origin.url 6237994/b469/_pkg_.a ortcfg .cfg 64/pkg/tool/linu--show-toplevel infocmp -1 xterm-color ylQP4Z8/vCNYLdc7D8RXanEmFBss /usr/bin/git 4861282/b191/_pkgit GO111MODULE .test git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260428-193038-33750/test-256956264/.githu@{u} rev-parse /usr/bin/git -json GO111MODULE x_amd64/compile git conf�� --get remote.origin.url om/owner/repo.git -json GO111MODULE x_amd64/vet gh (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260428-193038-33750/test-3821588185/.github/workflows remote /usr/bin/git -json GO111MODULE x_amd64/compile git rev-�� --show-toplevel git /usr/bin/git bytealg/indexbytgit GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-04-21 th_wasm.o 64/src/math/big/arith_wasm.s 64/pkg/tool/linux_amd64/vet env d .cfg x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-03-29 GOMOD GOMODCACHE 64/pkg/tool/linuremote.origin.url ortc�� 4861282/b154/_pkg_.a .cfg tartedAt,updatedAt,event,headBranch,headSha,displayTitle GOINSECURE ntio/asm/internacommit GOMODCACHE 64/pkg/tool/linuInitial commit (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-01-28 GOMOD GOMODCACHE 64/pkg/tool/linuorigin ortc�� (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 4861282/b006/sym--show-toplevel 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 d2UJ/DbmGN00V4XBV3gqgd2UJ x_amd64/link GOINSECURE fips140/ecdsa GOMODCACHE x_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/cons-1 GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg x_amd64/vet GOINSECURE boring/bbig ache/go/1.25.8/xActor: ${{ github.actor }}, Repo: ${{ github.repository }} x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/stri/tmp/test-expr-2592050702.js GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE contextprotocol/config ache/go/1.25.8/x--get-regexp 64/pkg/tool/linu^remote\..*\.gh-resolved$ (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE x_amd64/link GOINSECURE contextprotocol/-d GOMODCACHE x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/link 4861�� 1815287673 zUvD/O0MXJnxKO-Hqrk06zUvD .cfg --format=%H:%ct GOWORK 64/bin/go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE x_amd64/vet GOINSECURE GOMOD 4861282/b006/sym--show-toplevel x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 eFae/0ahu769BnKYz-hV-eFae x_amd64/compile GOINSECURE g/x/crypto/chach/tmp/test-process-3805725736.js GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 gLhb/hBEUOkjpLNrZf4ikgLhb 64/pkg/tool/linux_amd64/compile GOINSECURE fips140/ecdh GOMODCACHE 64/pkg/tool/linux_amd64/compile -c 1815287673 _L0m/ZbUGNwZMKnO7zDW0_L0m .cfg GOSUMDB er 64/bin/go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name xxol/GL-tkTNtkvunLaxVxxol 64/pkg/tool/linux_amd64/compile GOINSECURE fips140/ed25519 GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 QuTc/8J1aAAdvjhK6D-KwQuTc 64/pkg/tool/linux_amd64/vet GOINSECURE g/x/crypto/interconfig GOMODCACHE 64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -json n/codec.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE bug GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 4861282/b027/importcfg .cfg ache/go/1.25.8/x64/pkg/tool/linu-buildmode=exe GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-extld=gcc (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build1416237994/b404/cli.test /tmp/go-build1416237994/b404/cli.test -test.testlogfile=/tmp/go-build1416237994/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x^remote\..*\.gh-resolved$ /usr/bin/git audit-workflows.git -trimpath 6237994/b195/vet--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-buildtags /usr/bin/git 3038-33750/test-du -buildtags 6237994/b334/vet/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv se 6237994/b046/vet.cfg k -I /tmp/go-build404rev-parse -I ache/go/1.25.8/x64/pkg/tool/linuconfig sRem�� se 6237994/b212/vet.cfg .cfg -p internal/filepatrev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linumyorg (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a 0/internal/language/compact/comp-ifaceassert x_amd64/vet GOINSECURE GOMOD abis x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet ch_w�� g_.a GO111MODULE x_amd64/vet GOINSECURE pare_wasm.o 64/src/internal//tmp/gh-aw-test-runs/20260428-193038-33750/test-1815287673 x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv iant-282596544/.--thin GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet phen�� g_.a 0/internal/internal.go x_amd64/vet GOINSECURE 5519 GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv iant-282596544/.--exclude-hidden=receive GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a 0/internal/format/format.go x_amd64/vet GOINSECURE exbyte_wasm.o 64/src/internal/--show-toplevel LW9vmjg/Q2O-E-psHVdv5FZFNYUn (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE Ww7VJguVlRAx env g_.a 0/internal/number/common.go x_amd64/vet GOINSECURE 5519/field 64/src/internal//tmp/go-build1416237994/b456/timeutil.test x_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv 6763/001/stability-test.md 6237994/b003/vet.cfg .cfg -p er -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linushow (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE t/internal/langurev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet estl�� se 6237994/b047/vet.cfg .cfg -I /tmp/go-build404rev-parse -I ache/go/1.25.8/x64/pkg/tool/linu--jq (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD emclr_wasm.s x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD reempt_wasm.s x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 6763/001/stability-test.md 6237994/b099/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet gh-aw ./cmd/gh-aw 64/bin/go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -o DefaultBranchFromLsRemoteWithRealGitmaster_branch3661795445/001&#39; DefaultBranchFromLsRemoteWithRealGitmaster_branch3661795445/001&#39; ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p path -lang=go1.25 ache/go/1.25.8/x1 (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git e/git init�� ndor/bin/git git ode_modules/.bin/git =receive test@example.com--git-dir=/tmp/bare-incremental-aY9ePl /git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI and others added 2 commits April 28, 2026 19:34
- Add pre-agent steps to build workflow inventory and pre-load metrics
  before agent starts, eliminating 4-5 turns of deterministic work
- Replace 60-line verbose dashboard template with concise 15-line skeleton
- Remove redundant "Important Guidelines" section (~30 lines)
- Remove qualitative "Success Metrics" section (~11 lines)
- Update Phase 1 to reference pre-computed /tmp/gh-aw/agent/ files
- Update Phase 2 to use pre-loaded failing-workflows.json and batch API calls
- Recompile lock file to reflect all changes

Addresses token optimization recommendations from issue:
- ~450K tokens/run from prompt trimming
- ~175K tokens/run from pre-agent discovery steps
- ~100K tokens/run from pre-loaded metrics
- ~50K tokens/run from batched API call guidance
Expected reduction: 1.84M → ~1.07M tokens/run (~42%)

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/37bfb199-ab89-4bbf-bdf8-d0a820340df1

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
`2>&1 > file` was wrong (stderr still went to terminal).
Corrected to `> file 2>&1` so both stdout and stderr are captured.

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/37bfb199-ab89-4bbf-bdf8-d0a820340df1

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize token usage in workflow health manager feat(workflow-health-manager): reduce token usage ~42% via pre-agent steps and prompt trimming Apr 28, 2026
Copilot AI requested a review from lpcox April 28, 2026 19:38
@lpcox lpcox marked this pull request as ready for review April 28, 2026 20:08
Copilot AI review requested due to automatic review settings April 28, 2026 20:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces workflow-health-manager token usage by shifting deterministic discovery/metrics work into pre-agent steps and trimming/reshaping the prompt content the agent sees.

Changes:

  • Added workflow steps: to precompute workflow inventory and failing-workflow metrics into /tmp/gh-aw/agent/.
  • Trimmed and refactored the workflow instructions (phases + dashboard template) to rely on precomputed files and reduce repeated verbosity.
  • Updated documentation/status listings and regenerated the compiled .lock.yml.
Show a summary per file
File Description
docs/src/content/docs/agent-factory-status.mdx Removes entries for workflows that no longer appear to exist in .github/workflows/.
.github/workflows/workflow-health-manager.md Adds pre-agent steps + updates phase instructions and dashboard template to cut repeated discovery work and prompt size.
.github/workflows/workflow-health-manager.lock.yml Regenerated lock file reflecting the new steps and prompt structure.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 3

Comment on lines +45 to +59
- name: load-metrics
run: |
set -euo pipefail
mkdir -p /tmp/gh-aw/agent
METRICS_FILE="/tmp/gh-aw/repo-memory/default/metrics/latest.json"
if [ -f "$METRICS_FILE" ]; then
jq '[.workflow_runs | to_entries[]
| select(.value.success_rate < 0.8)]
| sort_by(.value.success_rate) | .[0:20]' \
"$METRICS_FILE" > /tmp/gh-aw/agent/failing-workflows.json 2>/dev/null \
|| echo '[]' > /tmp/gh-aw/agent/failing-workflows.json
else
echo '[]' > /tmp/gh-aw/agent/failing-workflows.json
fi
echo "Metrics loaded: $(jq 'length' /tmp/gh-aw/agent/failing-workflows.json) failing workflows (<80% success)"
Comment on lines +255 to +263
### Phase 1: Discovery (pre-computed)

1. **Scan workflow directory:**
- List all `.md` files in `.github/workflows/` (excluding `shared/` subdirectory)
- Parse frontmatter for each workflow
- Extract key metadata (engine, triggers, tools, permissions)
Pre-computed data is available in `/tmp/gh-aw/agent/`:
- `workflow-list.txt` — all executable `.md` workflow files (one per line, shared/ already excluded)
- `compile-validate.txt` — output from `gh aw compile --validate`

2. **Check compilation status:**
- For each **executable** `.md` file, verify `.lock.yml` exists
- **SKIP** files in `.github/workflows/shared/` directory (reusable imports, not standalone workflows)
- Verify lock freshness with frontmatter hash checks (for example, `gh aw compile --validate`) instead of raw file modification timestamps
- Run `gh aw compile --validate` to check for compilation errors

3. **Build workflow inventory:**
- Create structured data for each workflow
- Categorize by type, engine, and purpose
- Map relationships and dependencies
1. **Read the pre-computed inventory** from the files above; do not re-run discovery from scratch.
2. **Parse frontmatter** for each workflow in `workflow-list.txt` to extract key metadata (engine, triggers, tools, permissions).
3. **Check compilation status** using `compile-validate.txt` — verify each workflow has a `.lock.yml` and note any errors or warnings.
Comment on lines 385 to 389
- name: load-metrics
run: "set -euo pipefail\nmkdir -p /tmp/gh-aw/agent\nMETRICS_FILE=\"/tmp/gh-aw/repo-memory/default/metrics/latest.json\"\nif [ -f \"$METRICS_FILE\" ]; then\n jq '[.workflow_runs | to_entries[]\n | select(.value.success_rate < 0.8)]\n | sort_by(.value.success_rate) | .[0:20]' \\\n \"$METRICS_FILE\" > /tmp/gh-aw/agent/failing-workflows.json 2>/dev/null \\\n || echo '[]' > /tmp/gh-aw/agent/failing-workflows.json\nelse\n echo '[]' > /tmp/gh-aw/agent/failing-workflows.json\nfi\necho \"Metrics loaded: $(jq 'length' /tmp/gh-aw/agent/failing-workflows.json) failing workflows (<80% success)\"\n"

# Repo memory git-based storage configuration from frontmatter processed below
- name: Clone repo-memory branch (default)
@lpcox
Copy link
Copy Markdown
Collaborator

lpcox commented Apr 28, 2026

@copilot address the review feedback of .md files.

The load-metrics step was in `steps:` which runs before the repo-memory
clone. Moving it to `pre-agent-steps:` ensures it runs after the clone,
so metrics/latest.json exists when the step reads it.

Also strengthened the Responsibilities section (sections 1 and 2) to
explicitly reference pre-computed files in /tmp/gh-aw/agent/ instead of
the old contradicting instructions to scan .github/workflows/ or re-run
gh aw compile --validate. Phase 1 now also includes an explicit override
note matching the reviewer suggestion.

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/200dddb8-e218-407a-a162-631497158c01

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 28, 2026

@copilot address the review feedback of .md files.

Addressed all three review points in commit a262bf0:

  1. load-metrics timing bug — moved from steps: to pre-agent-steps:. In the compiled lock file, load-metrics now runs at step 450, after Clone repo-memory branch at step 387, so metrics/latest.json is available when the step executes.

  2. Conflicting discovery instructions — updated the Responsibilities section (1. Workflow Discovery and 2. Health Monitoring) to explicitly reference /tmp/gh-aw/agent/workflow-list.txt and /tmp/gh-aw/agent/compile-validate.txt rather than instructing the agent to scan .github/workflows/ or re-run gh aw compile --validate.

  3. Phase 1 override note — strengthened Phase 1 with an explicit "do not scan .github/workflows/ or re-run discovery" instruction and an override note matching the reviewer suggestion.

@lpcox lpcox merged commit aefea52 into main Apr 28, 2026
26 checks passed
@lpcox lpcox deleted the copilot/optimize-workflow-health-manager-tokens branch April 28, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[copilot-token-optimizer] Token Optimization: Workflow Health Manager - Meta-Orchestrator

3 participants