[audit-workflows] 🩺 Agentic Workflow Audit — 2026-07-06 (86.25%, all 55 fails are 0-turn driver aborts)

[github-actions[bot]]

🩺 Agentic Workflow Audit — 2026-07-06

Window: partial ~21.5h (2026-07-05 23:47Z → 2026-07-06 21:16Z) · 400 runs · 86.25% success · engines via engine_id from aw_info.json.

Metric	Value
Overall success	345 / 400 = 86.25%
Prod-main (main)	178 / 216 = 82.4% (ex-Smoke CI: 89.4%)
Non-main (PR)	167 / 184 = 90.8%
Missing tools / data / MCP failures	0 / 0 / 0
AI credits (AIC)	~29,611 · action-min 4,852 · GH API calls 6,685

Headline: Healthy-on-baseline. Every one of the 55 failures is a 0-turn agent-job driver failure (CLI startup or post-work Execute-CLI abort) — zero agent-logic failures, zero missing tools/data, zero MCP failures. The fleet's agents are working; the harness driver layer is where reliability is lost.

📊 Trend Charts

Overall success has oscillated in the 68–98% band over 30 days with no directional trend; today's 86.25% sits just below the 30-day median. The dips (06-20 Skillet, 06-25 distributed 0-turn, 06-30 Smoke CI onset, 07-03 pi collapse) are each traceable to a specific cluster rather than broad degradation — the baseline is stable, the variance is cluster-driven.

Reported daily tokens collapsed from 20–125M/day (early June) to null/near-zero since ~06-19 — this is a metrics gap (metrics.TokenUsage empty), not a real usage drop, since run volume held at 300–400/day. AIC (AI credits) remains reliable (~29.6K today) and should be the fleet cost metric until token aggregation is restored.

🔴 Failure Taxonomy (55 fails, all 0-turn driver aborts)

Cluster	Count	Engine	Known issue
Smoke CI — 0-turn on push to main	17 (31%)	copilot	smoke-ci-copilot-cli-100pct-fail-on-push (recur 6)
copilot-sdk longrun Execute-CLI 0-turn	~22	copilot	copilot-sdk-driver-failures (recur 24)
claude longrun-0turn (same signature)	4	claude	claude-agent-job-fail-longrun (cross-engine now)
Avenger err-config / no-structured-logs	1	claude	avenger-err-config (recur 18)
chroot node not available (Issues Report Gen)	1	copilot	chroot-node-not-available (recur 11)
Daily Fact 0-turn	1	codex	daily-fact-codex-gpt54-0turn (WATCH, recur 2)
PR firewall-branch smoke cluster	3	copilot/codex/gemini	isolated — see below
misc (Super Linter, credit/limit probes, MCP auth)	~6	mixed	low-signal singletons

Engine success rates & notable failing workflows

By engine: pi 48/48 = 100% · claude 69/74 = 93.2% · copilot 217/262 = 82.8% · codex 10/12 = 83.3% · antigravity 1/1 · gemini 0/1 (firewall-branch smoke).

Top failing / costly workflows this window:

• PR Code Quality Reviewer — 10 fails / 34 runs (29%), highest-cost at AIC ~4,307 — high-frequency PR reviewer, longrun Execute-CLI aborts (18–20m).
• Smoke CI — 17/17 = 100% (0 AIC, 3–5m) — deterministic red on every push to main.
• Impeccable Skills Reviewer 3/31 · Contribution Check 3/7 · Matt Pocock Skills Reviewer 1/33 (AIC ~2,395).
• Daily yamllint Fixer failed on BOTH engines the same day — copilot (36.5m) + claude (27.6m) — cross-engine confirmation the longrun-0turn signature is harness-level, not engine-specific.
• New copilot-longrun names surfaced: Layout Specification Maintainer (23.9m), Copilot Opt (52.8m — longest single failing run), Linter Miner (25m).

Isolated: PR firewall-branch smoke cluster (no fleet action)

On branch copilot/upgrade-gh-aw-firewall-v02724, 3/10 smoke checks failed (Smoke Copilot, Changeset Generator/codex, Smoke Gemini) while 7/10 passed (Smoke Claude, Codex, Pi, Antigravity, AOAI Entra+apikey, Agent Container). Because the majority passed and it is confined to one firewall-change PR, this is a PR-under-review signal (the allowlist change plausibly affects those engines' network paths) — not a fleet-wide infra outage. Flag to the PR author; no audit action.

✅ Recommendations (evidence-linked)

1. [P1 — highest ROI] Fix or mute Smoke CI's 100% push-fail. 17/17 = 100% across 6 observed windows (06-30→07-06); it is 31% of all fleet failures and a permanent red on main. Muting/fixing lifts prod-main 82.4% → 89.4%. Capture the Execute GitHub Copilot CLI stderr/exit-code — 0-turn aborts remain opaque. (rec-2026-07-01-smoke-ci-escalate, rec-smoke-ci-copilot-cli-100pct)
2. [HIGH] Root-cause the copilot Execute-CLI 0-turn family (~71% of fails). Now confirmed cross-engine (4 claude longrun-0turn share the signature). Instrument the Execute-CLI step to emit stderr+exit-code into artifacts and split instant (Smoke CI 3–5m) vs longrun (18–52m) sub-variants — they likely differ (startup vs session-idle/network). (rec-copilot-cli-0turn-rootcause)
3. [HIGH] Avenger err-config still unfixed (recur 18, since ~06-08). Model swaps haven't fixed the no-structured-logs follow-up-invocation path. Fix the follow-up path or keep the schedule paused. (rec-avenger-fix-err-config)
4. [LOW] Close the token-reporting gap. Restore metrics.TokenUsage aggregation (empty since ~06-19) or officially adopt AIC as the fleet token/cost chart metric. (rec-2026-07-06-token-reporting)

References: §28825095587 · §28820262391 · §28821449142

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔍 Agentic Workflow Audit Agent · 395 AIC · ⌖ 41.5 AIC · ⊞ 7.6K · ◷

• expires on Jul 7, 2026, 2:13 PM UTC-08:00