[audit-workflows] 🔍 Agentic Workflow Audit — 2026-07-24 (93.7% · Smoke CI recovered, pi degraded)

[github-actions[bot]]

🔍 Agentic Workflow Audit — 2026-07-24

Full 24h window (2026-07-23T21:59Z → 2026-07-24T21:17Z). 305 runs had complete telemetry (of 565 downloaded run dirs; 260 download-incomplete — the usual bridge/pagination cap). This is the healthiest recorded day to date, with two notable chronic-issue recoveries.

Note: 18-day gap since the prior audit (2026-07-06) — this workflow did not run 07-07 → 07-23.

📊 Headline metrics

Metric	Value
Success rate (raw)	93.4% (285/305)
Success rate (excl. 2 intentional-failure tests)	93.7% (284/303)
Prod-main (main branch)	90.8% (188/207)
Non-main (PR/dispatch)	100% (96/96)
AI Credits (AIC)	16,542
Tokens	14.49M
Action-minutes	3,148
Missing tools / missing data / MCP failures / noops	0 / 0 / 0 / 0

Intentional-failure workflows excluded from rate: Daily Credit Limit Test, Daily Max AI Credits Test.

✅ Two chronic issues RECOVERED

• Smoke CI fixed — 13/13 = 100% success across push, schedule, pull_request, and workflow_dispatch. The chronic "100%-fail-on-push" cluster (red on main since ~06-30, recurrence count 6) appears resolved. Kept on watch for regression.
• Token-reporting gap closed — metrics.TokenUsage / run.TokenUsage is now populated directly fleet-wide (14.49M captured with no token_usage_summary fallback needed). The prior empty-metrics gap (count 3) is resolved.

⚠️ Engine health — copilot up, pi down

Engine	Rate	Note
claude	56/56 = 100%	clean
codex	13/13 = 100%	clean
copilot	165/175 = 94.3%	recovered (Smoke CI fixed); residual = longrun 0-turn singletons
pi	48/57 = 84.2%	degraded — dragged down by PR Sous Chef
gemini / antigravity	1/1 / 1/1	single runs, clean

🔴 Failure taxonomy — all 20 are 0-turn pre-agent driver fails

Every failure was a 0-turn / pre-agent job failure. Zero agent-logic errors, missing tools, missing data, or MCP failures.

Cluster 1 — PR Sous Chef (pi/copilot-gpt-5.4): 8 fails · DOMINANT

• 24/32 = 75% over the window. 8 failures interleaved with successes (not a sustained consecutive-fail incident like the 07-03 collapse).
• All 0-turn. This is the recurring pi-gpt54-0tok-agentjob-fail — pi runs the copilot/gpt-5.4 backend, so it's the same copilot driver family surfacing via pi.
• Solely responsible for pi dropping to 84.2%.

Cluster 2 — copilot claude-sonnet-4.6 longrun 0-turn: 10 singletons

Known issue copilot-sdk-driver-failures (recurrence count 25). One failure per workflow, several after substantial real runtime:

Workflow	Duration	AIC burned
The Daily Repository Chronicle	47.5m ⬅ longest	56.4
Daily Formal Spec Verifier	29.4m	151.1
Daily Security Observability Report	19.8m	—
Daily SPDD Spec Planner	18.8m	—
Daily Agent of the Day Blog Writer	12.8m	—
Daily Issues Report Generator	12.5m	21.3
CI Optimization Coach	13.2m	—
Daily Container Image Security Scan	12.0m	65.9
Daily BYOK Ollama Test	5.5m	—
Designer Drift Audit	4.0m	—

These burn compute for up to 47 minutes before the driver drops the job — the main remaining source of wasted AIC.

Cluster 3 — Agent Persona Explorer (pi): 1 fail

0/1, 23.4m 0-turn. Same pi-gpt54-0tok-copilot-backed family as PR Sous Chef.

📈 Trend charts (30 days)

Success rate closes the 30-day window at its highest point (93.7%), up from the 86–90% baseline of early July. The flat segment 07-06 → 07-24 reflects the audit gap (no data), not a health change. Run volume for this partial-telemetry day (305) is lower than full-fetch days (~400) due to download-incompleteness, not fewer runs.

AIC for the captured window (16.5K) sits mid-range against the 07-04→07-06 ramp (6.3K→29.6K). The 7-day moving average is only indicative given the 18-day gap. Top consumers are all successful claude workflows (Typist 744, Go Logger 626, VulnHunter 613) — cost is concentrated in legitimate heavy analysis, not failures.

🎯 Recommendations

1. [Re-escalated] Stabilize pi/copilot/gpt-5.4 backend — PR Sous Chef (75%) + Agent Persona Explorer are the top prod-main failure source now that Smoke CI is fixed. Pin these pi-backed workflows off the flaky copilot/gpt-5.4 backend, or add a 0-turn driver-failure retry. (rec-pin-autotriage-stable-model)
2. [Ongoing] copilot-SDK longrun 0-turn — 10 workflows fail after up to 47m of runtime. Add a fast pre-flight/liveness check or early-abort so the driver doesn't burn 15–47 min before dropping the job. (copilot-sdk-driver-failures, count 25)
3. [Verify] Confirm Smoke CI fix is durable — 13/13 today; keep the watch one more cycle before closing smoke-ci-copilot-cli-100pct-fail-on-push.

🧾 Repo memory

Updated: audit-history.jsonl, metrics-summary.json, known-issues.json (Smoke CI + token-gap → RECOVERED-WATCH; copilot-sdk count 25; pi-0turn re-escalated count 2), recommendations.json, anomalies.json, workflow-trends.json. Validated at 57 KB.

References:

• §30107063909 — Daily Formal Spec Verifier (29m longrun 0-turn)
• §30106825829 — The Daily Repository Chronicle (47.5m longest fail)
• §30107950973 — PR Sous Chef (pi 0-turn)

Generated by 🔍 Agentic Workflow Audit Agent · age00 · 332.3 AIC · ⌖ 35.3 AIC · ⊞ 8K · ◷

• expires on Jul 25, 2026, 1:53 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-07-25T21:53:39.545Z.

Closed by Workflow