[agentic-token-optimizer] CLI Consistency Checker: fix broken pre-agent help-collection step wasting ~110 AIC/run

[github-actions]

Target workflow

CLI Consistency Checker (.github/workflows/cli-consistency-checker.md, compiled to cli-consistency-checker.lock.yml)

Why selected: Highest total AIC in the last 7 days among un-recently-optimized, non-"Token" workflows (276.362 AIC in its single run today; last optimized 2026-07-01, outside the 14-day exclusion window).

Analysis period + runs analyzed

5 recent scheduled runs inspected directly via job logs (2026-07-08, 2026-07-16, 2026-07-17, 2026-07-31 success, plus recent run list of last 20 executions).

Cost profile

Metric	Value
Total AIC (7-day window, 1 run)	276.362
Avg AIC/run (7-day)	276.362
Raw tokens (7-day)	67,914
Action minutes (7-day run)	28
Timeout-minutes budget	20
Recent run success rate (last 20)	~70% (6 of 20 recent runs failed/timed out)

Ranked recommendations

1. Fix the broken pre-agent-steps help-collection script (highest priority)

Estimated AIC savings per run: ~60–90

Evidence: In every one of the 5 runs inspected (2026-07-08, 2026-07-16/17, 2026-07-28/29/30, 2026-07-31), the pre-agent step logs the exact same line:

Pre-collected help lines: 0

This step is supposed to make build the CLI and dump all --help output to /tmp/gh-aw/agent/all-help.txt before the agent starts, so the agent can skip rebuilding and re-discovering commands. Instead it silently fails to populate any files, and set -euo pipefail does not catch it because the script still exits 0 (the if [ ${#help_files[@]} -eq 0 ] guard should exit 1, but based on downstream behavior the step is not failing the job — likely because make build succeeds but the loop over top_commands never produces populated files due to the mapfile < <(awk "${extract_commands}" ...) interaction with the double-quoted awk program losing the intended single-quoted regex/field semantics in the compiled shell context).

Because of this bug, the agent must always:

• Detect the empty file (ls -la /tmp/gh-aw/agent/)
• Re-run go build itself (with GOTOOLCHAIN network calls, go env, find-based toolchain discovery)
• Re-implement the same help-collection loop inline
• Only then proceed to actual analysis

This entire detect-and-recover sequence consumes roughly 4–5 minutes and dozens of bash/view calls before any real CLI-consistency analysis begins (confirmed in the 2026-07-31 success run: agent spent 13:44:53–13:49:33, ~4.5 minutes and ~15 tool calls, purely on rebuilding what pre-agent-steps was supposed to have already produced).

Action: Debug and fix the pre-agent-steps script (.github/workflows/cli-consistency-checker.md frontmatter) so /tmp/gh-aw/agent/all-help.txt is reliably non-empty before the agent starts. Likely root cause is in how the extract_commands awk program and mapfile interact when compiled into the final workflow YAML run: block — verify with a direct re-run of the exact compiled script outside the agent sandbox. Also make the guard genuinely fail the job (exit 1 before make build succeeds silently) so failures are visible in CI rather than silently absorbed by the agent.

2. Reduce redundant Go/toolchain discovery when the agent must self-recover

Estimated AIC savings per run: ~15–25 (only relevant while issue #1 persists)

Evidence: When the agent detects the empty help file, it runs ~6 separate bash probes (go version, ls /usr/local/go*/bin/go, ls /opt/hostedtoolcache/go, find / -maxdepth 4 -iname toolchain, go env GOPATH GOMODCACHE GOPROXY GOFLAGS, find / -maxdepth 6 -iname vendor) before finally building. These could be replaced by a single combined bash call, or better, become unnecessary once recommendation #1 is fixed.

Action: If pre-agent-steps continue to be unreliable, add a one-shot fallback bash block to the prompt itself (not discovered ad hoc) that runs make build and the help-collection loop in one combined command, avoiding exploratory find/ls probing.

3. Investigate recurring 20-minute agent timeouts

Estimated AIC savings per run: ~15 (avoided wasted runs, not per-successful-run savings)

Evidence: 3 of the last 20 scheduled runs (§29584842186, §29503252477, and one from 2026-07-08 §28947118997) failed with ##[error]The action 'Execute GitHub Copilot CLI' has timed out after 20 minutes. All three show the agent still using task/read_agent background sub-agents (the Rig-harness pattern) and heavy view/grep/bash volumes (up to 374 view calls in one run) when the timeout hit — consistent with the same pre-agent-step recovery overhead pushing total run time past budget.

Action: Once recommendation #1 is fixed and ~4–5 minutes of setup time is reclaimed, re-evaluate whether 20-minute timeout-minutes is still insufficient before increasing it — increasing timeout without fixing the root cause would just mask the waste.

4. Consolidate Rig-harness input reads

Estimated AIC savings per run: ~5–10

Evidence: The prompt already correctly asks for exactly 3 Rig harness invocations (typo/grammar, flag-consistency, docs-vs-help) with guardrails against retries and full-file dumps — this part is well-designed. However several bash calls in the observed successful run (ls -la .../cli.md, separate python3 -c compaction scripts, separate find for the rig launcher path) could be combined into one preparatory bash block per harness rather than 3–4 separate calls each.

Action: Combine "discover rig launcher path" + "build compact JSON input" into a single bash invocation per harness (currently 2 separate bash calls observed per harness setup).

Caveats

• Only 1 run fell inside the exact 7-day top-workflows.json window; the AIC/turn statistics above are supplemented with 5 additional historical runs read directly via gh api job logs for a larger evidence base.
• total_turns is reported as 0 across all pre-aggregated data sources (a data-collection gap, not specific to this workflow).
• No ## agent: inline sub-agent blocks exist currently and no new ones are recommended — the existing Rig-harness background-task pattern already achieves the same distributed-analysis benefit, and the workflow's remaining major sections (build/config validation, security note, reporting) are tightly coupled to the main agent's synthesis and don't meet the independence/parallelism bar for a new sub-agent.

Raw evidence: repeated "Pre-collected help lines: 0" across runs

run 28947118997 (2026-07-08): Pre-collected help lines: 0
run 29584842186 (2026-07-17): Pre-collected help lines: 0
run 30364730143 (2026-07-28): Pre-collected help lines: 0
run 30457258609 (2026-07-29): Pre-collected help lines: 0
run 30548096876 (2026-07-30): Pre-collected help lines: 0
run 30635503632 (2026-07-31, success): Pre-collected help lines: 0

Every single run checked, across 3+ weeks, shows the same failure. This is a 100% reproduction rate in the sample, indicating a structural bug rather than a flaky/intermittent issue.

References:

• §30635503632 — latest successful run showing recovery overhead
• §29584842186 — 20-minute timeout failure
• §28947118997 — earliest sampled timeout failure with same pre-agent-step bug

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• proxy.golang.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "proxy.golang.org"

See Network Configuration for more information.

Generated by Agentic Workflow AIC Usage Optimizer · auto · 133.1 AIC · ⊞ 10.4K · ◷

• expires on Aug 7, 2026, 7:46 AM UTC-08:00