[audit-workflows] Agentic Workflow Audit — 2026-08-15

[github-actions[bot]]

⚠️ Audit coverage gap

This audit resumed after a 40-day gap (last run: 2026-07-06). No data was collected in between, so any regressions or fixes that occurred during that window are invisible to this report and to the trend charts below. Recommend verifying the audit workflow's own schedule/trigger is healthy so this doesn't recur (see Recommendations).

Headline numbers (24h window, 293 runs)

• Raw success rate: 96.93% (284/293)
• Success rate excluding intentional-failure test workflows: 97.26% (Daily Credit Limit Test, Daily Max AI Credits Test excluded)
• Total AIC cost: $10,072.14 · Total tokens: 11,751,902
• 0 missing-tool, 0 missing-data, 0 MCP-failure reports
• Engines observed: copilot (139), claude (39), pi (103), aider (2), crush (4), codex (5), goose (1)

Both this window's rates are a marked improvement over the last tracked baseline (2026-07-06: 86.25% raw), though the 40-day gap means this can't be attributed to a specific fix with confidence.

Phase 1 — Collection

9 runs did not conclude successfully. After excluding the 2 intentional-failure test workflows (both of which are designed to fail as a credit-limit smoke test), 8 real anomalies remain:

#	Workflow	Signature	Status
1	Avenger	err-config-no-structured-logs (recurrence #19)	Known, but 6/7 runs OK this window — down sharply from 07-06's chronic 100%-fail baseline
2	Linter Miner	copilot-sdk-driver crash after ~12 turns of real work	Known family (copilot-sdk-driver-failures), 1 instance vs. historical highs of ~22/day
3	Daily Ambient Context Optimizer	stuck in_progress step, all downstream steps pending (hang/timeout, not a clean crash)	New
4	(Aider-engine workflow)	0-turn Execute CLI crash	New — first-ever tracked Aider failure
5–6	(Crush-engine workflows ×2)	0-turn Execute CLI crash	New — first-ever tracked Crush failures
7	(workflow using install ripgrep)	transient install flake	New, looks like infra flake not a code defect
8	Daily Container Image Security Scan	failed at "Enforce critical vulnerability and license gates" after a clean 11-write-action agent run	Not a bug — the policy gate correctly caught a real vulnerability/license finding. Excluded from "real failure" framing.

No missing-tool, missing-data, or MCP-failure signals were reported by any run this window.

Token/cost outlier note: the 3 highest-token runs this window (Dev 1.28M tok/$91.57, Daily Evals Feature Report 1.15M tok/$71.46, Daily Cache Strategy Analyzer 951K tok/$18.21) are all Codex-engine and show disproportionately low AIC-per-token vs. other engines (e.g., Daily Container Image Security Scan: fewer tokens but $246.32). This is a cross-engine cost-comparability caveat, not itself a defect — flagging so future trend comparisons don't read it as an anomaly.

Network friction: weekly-editors-health-check had the highest firewall-block rate of any workflow this window — 78/295 requests (26%) blocked — worth checking whether its allowed-domains list is stale.

Phase 2 — Clustering

• Recurring, but improving: avenger-err-config-no-structured-logs and copilot-sdk-driver-failures are both long-tracked issues (open since ~06-08 and ~06-21 respectively) that are still firing but at a fraction of their 07-06 volume (Avenger: 10/10→1/7; copilot-sdk-driver: ~22/day peak→1 instance). Encouraging, but the 40-day blind spot means we can't rule out a relapse having happened and self-resolved in between.
• New engine-crash cluster: the Aider and Crush failures share the exact 0-turn Execute CLI crash after real work signature already tracked for copilot/claude under copilot-sdk-driver-failures. Given the shared signature across 4 different engines now, this looks increasingly like a shared gh-aw CLI-driver/harness issue rather than something provider-specific — worth investigating at the harness level rather than per-engine.
• Infra flakes: the ripgrep install failure and the Ambient Context Optimizer timeout don't match any existing tracked cluster; treated as isolated infra flakes pending a second data point.
• Not a defect: the Container Image Security Scan gate failure is filed separately so it doesn't inflate "real failure" counts in future trend comparisons.
• Unresolved from last audit: Smoke CI (previously a 6-window, 100%-fail-on-push chronic issue) had zero runs in this window, so its status (fixed vs. simply untriggered — no pushes to main observed) cannot be confirmed either way.

Phase 3 — Recommendations

1. Add a health-check on the audit workflow's own schedule/trigger — the 40-day silent gap is the single biggest reliability problem surfaced this run, since it blinds all other tracking.
2. Investigate the new cross-engine 0-turn crash pattern (Aider/Crush) — check whether it shares root cause with the existing copilot-sdk-driver-failures family at the harness/CLI-driver level rather than per-engine.
3. Add retry/backoff for the ripgrep install step — single transient flake this window, cheap to harden.
4. Surface a clearer timeout message for stuck Execute CLI steps — the Ambient Context Optimizer hang was only distinguishable from a clean failure by checking for downstream steps stuck at pending; a driver-level timeout with a clear log line would make this diagnosable without cross-referencing job_details.
5. Continue monitoring Avenger and the copilot-sdk-driver-failures family — both trending down but root causes remain untouched; do not close either issue yet.

Phase 4 — Trend charts (past 30 days)

Workflow health trends

Success/failure counts and the success-rate line both show the fleet recovering from a rough late-June/early-July stretch (lows around 60-69% on partial-window days) back up to the high-80s/96.93% seen in this window. The 40-day data gap between 2026-07-06 and 2026-08-15 is annotated on the chart — treat any apparent "jump" across that gap with caution, since intermediate days aren't observed.

Token usage trends

Daily token usage is highly volatile day-to-day (several days show no data due to reporting gaps noted in prior audits), but the 7-day moving average has trended upward since late June, consistent with more/larger workflows running. Today's 11.75M tokens is among the higher points on the chart; combined with the Codex cost-accounting caveat above, token totals alone are not yet a reliable cross-engine cost proxy.

Repo memory

All 6 memory files under repo-memory/default/ were merged (not overwritten) with today's findings: audit-history.jsonl (new entry), metrics-summary.json (history appended), known-issues.json (19 issues, 4 new), anomalies.json (12 anomalies, 3 new), recommendations.json (18 recs, 4 new), workflow-trends.json (74 tracked workflows, merged with today's snapshot). Storage validated at 80 KB / 7 files, well within limits.

Generated by 🔍 Agentic Workflow Audit Agent · agent · 262.3 AIC · ⌖ 6.4 AIC · ⊞ 8.6K · ◷

• expires on Aug 16, 2026, 1:36 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #53209.