[audit-workflows] 🩺 Agentic Workflow Audit — 2026-07-21 · Healthy 93.4%, Avenger+Smoke CI recovered, 11.5M-token burn anomaly

[github-actions[bot]]

🩺 Agentic Workflow Audit — 2026-07-21

Fleet health: HEALTHY — 93.4% (282/302 runs), the best overall rate since ~06-24. All 20 failures were pre-agent driver failures (Turns=0); zero agent-logic, missing-tool, missing-data, MCP, or noop errors. Two chronic dominant offenders recovered this window.

⚠️ Window note: partial ~16.7h (04:31–21:13Z), 302 populated run dirs of 515 downloaded. There is a 15-day observation gap since the last audit (2026-07-06); days 07-07 → 07-20 were not observed.

---

Key metrics

Metric	Value
Runs	302 (282 ✅ / 20 ❌)
Overall success	93.4%
Prod-main success	93.2% (164/176)
Non-main success	93.7% (118/126)
Fleet tokens	23.47M (⚠️ 49% from one failed run)
Action-minutes	2,954
Missing tools / data / MCP / noop / errors	0 / 0 / 0 / 0 / 0
All failures Turns=0 (pre-agent)	Yes (20/20)

Engine success: copilot 94.0% (174/185) · claude 87.0% (54/62) · pi 97.6% (42/43) · codex 10/10 · gemini 1/1 · antigravity 1/1.

---

🟢 Recoveries (3 chronic issues cleared this window)

• Avenger RECOVERED — 1/1 success. Was chronically 10/10 fail (err-config / no-structured-logs, recurrence 18) and the single largest reliability drain on the fleet. First clean run observed. Watch to confirm.
• Smoke CI clean — zero failures. Was the chronic 100%-fail-on-push offender (recurrence 6, 31% of all 07-06 failures). Watch to confirm — may be fixed or gated.
• Token-reporting-gap RESOLVED — run.TokenUsage now populated for 291/302 runs (was fleet-wide null, recurrence 3).

---

🔴 Top anomaly — 11.49M-token burn on a failed run

Daily Sub-Agent Model Resolution Audit (run 29806493363, copilot/gpt-5.4-mini) consumed 11,490,620 tokens — 49% of the entire fleet's daily spend — then the agent job failed at Turns=0 (agentic_fraction=0) after 591s. No agentic turn was ever produced despite 11.5M tokens, strongly implying a driver-level retry/loop burning context pre-turn. No token/cost guardrail intervened. A single failed run doubled fleet token usage (ex-anomaly fleet = 11.98M).

---

Failure clustering (all 20 fails are Turns=0 driver failures)

Cluster	Count	Known issue	Workflows
copilot-sdk-longrun-0turn	8	copilot-sdk-driver-failures (recur 25)	Breaking Change Checker, CI Optimization Coach, Daily Formal Spec Verifier, Daily Issues Report Generator, PR Code Quality Reviewer, Test Quality Sentinel, Tidy ×2 (push/main)
claude-longrun-0turn	3	claude-agent-job-fail-longrun	Daily VulnHunter Scan ×2, Design Decision Gate 🏗️
smoke-claude-config-fix-branch	6	NEW (active fix)	Smoke Claude ×3, Smoke Claude on Copilot ×2, Smoke Copilot — all workflow_dispatch on copilot/fix-claude-configuration / aw-failures-fix-smoke-claude / update-model-aliases fix branches
pi-gpt54-0tok	1	pi-gpt54-0tok-agentjob-fail (recur 2)	PR Sous Chef
token-burn-anomaly	1	NEW	Daily Sub-Agent Model Resolution Audit
intentional	1	—	Daily Max Ai Credits Test

Cross-engine signature confirmed: the copilot-longrun (8) and claude-longrun (3) failures share the identical 0-turn-after-real-runtime signature — this is a driver-level root cause, not engine-specific. The Smoke-Claude cluster is active debugging on fix branches, not a production regression (main-branch Smoke Claude had only 1 dispatch fail).

---

📊 30-day trends

Health has climbed back to 93.4%, above the 90% line and the highest since the 06-24 peak. The visible line jump reflects the 15-day observation gap (07-07 → 07-20), not a data cliff — no runs were lost, they simply weren't audited.

The 07-21 bar (23.5M) is inflated — a single failed run burned 11.5M of it. Ex-anomaly, the fleet's real spend (~12.0M) is in line with the early-July baseline. Token history remains sparse because TokenUsage was null on most prior days (that gap is now resolved going forward).

---

Recommendations (evidence-linked)

1. [HIGH · NEW] Add a per-run token/cost circuit-breaker for Daily Sub-Agent Model Resolution Audit (copilot/gpt-5.4-mini). Evidence: run 29806493363 — TokenUsage=11,490,620, Turns=0, conclusion=failure, agent dur 591s, agentic_fraction=0. Investigate the gpt-5.4-mini driver path for a pre-turn retry loop and cap token spend so a failing run cannot double fleet usage.
2. [HIGH · recurring] Root-cause the cross-engine Turns=0-after-real-runtime driver failure (copilot-sdk-driver-failures, recur 25). 11 of 20 fails this window (8 copilot + 3 claude) share it. Instrument the driver hand-off; the shared signature across engines points to the AWF/CLI driver layer, not model config.
3. [MEDIUM · new] Investigate Tidy ×2 fail on push/main — a new push-triggered variant of the copilot-longrun cluster; deterministic red on main is worth isolating.
4. [WATCH] Confirm Avenger & Smoke CI recoveries hold over the next 1–2 windows before closing avenger-err-config-no-structured-logs and smoke-ci-copilot-cli-100pct-fail-on-push.

Repo memory updated

audit-history.jsonl (07-21 entry), metrics-summary.json, known-issues.json (recur bumps + 2 new + 3 recoveries), anomalies.json (11.5M token-burn + Avenger recovery), recommendations.json (token-guardrail rec), workflow-trends.json. Memory validated within limits (57 KB / 60 KB).

References: §29806493363 (11.5M-token anomaly)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔍 Agentic Workflow Audit Agent · age00 423 AIC · ⌖ 39.1 AIC · ⊞ 7.6K · ◷

• expires on Jul 22, 2026, 2:01 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #47409.