[audit-workflows] 🩺 Agentic Workflow Audit — 2026-07-22 (96.2%, healthiest in weeks; Avenger + token-reporting recovered)

[github-actions[bot]]

🩺 Agentic Workflow Audit — 2026-07-22

Window: 2026-07-22 08:32–21:12 UTC (~12.7h partial) · 314 runs sampled (of ~366 in 24h) · 103 distinct workflows
First audit in 16 days (last: 2026-07-06) — the 07-07...07-21 span has no audit coverage.

Headline: Healthiest window in weeks. 96.2% raw / 96.5% excl. intentional; prod-main 94.2% raw / 94.9% excl. intentional (vs 82.4% on 07-06). Zero missing-tools, missing-data, MCP failures, noops, or agent-logic errors. All 12 failures are 0-turn driver / safe-output job failures. Three chronic offenders recovered.

---

📊 Key metrics

Metric	Value
Total runs (sampled)	314
Success / Failure	302 / 12
Overall success	96.2% raw · 96.5% excl. 1 intentional
Prod-main (main)	130/138 = 94.2% raw · 94.9% excl. intentional
Non-main (PR/dispatch)	172/176 = 97.7%
Missing tools / data / MCP	0 / 0 / 0
Agent-logic errors / noops	0 / 0
Fleet tokens	10.84M (in 8.51M / out 2.33M) — reporting gap resolved
AI credits (AIC)	~14,352 (partial 12.7h)
Action-minutes	~3,331

Per-engine: copilot 198/206 (96.1%) · claude 54/57 (94.7%) · pi 41/42 (97.6%) · codex 7/7 · antigravity 1/1 · gemini 1/1

---

✅ Recoveries (chronic issues that improved)

• Avenger — RECOVERED (watch). Ran 4×, passed 4/4. This was the rejig docs #1 chronic prod-main offender (10/10 fail on 06-27, err-config / no-structured-logs, recurrence 18). First window where it both fired and fully succeeded. Prior "green" days were pauses; this is genuine passing traffic. Watching 2 more audits before closing.
• Token-usage reporting gap — RESOLVED (watch). Fleet token metric flowing again (10.84M via token_usage_summary and per-run run.TokenUsage), after a null-metric gap spanning ~06-19...07-05.
• Daily Cache Strategy Analyzer (codex) — RECOVERED (watch). Ran 1×, passed (was ~100% fail on gh-aw binary-not-found for MCP since 06-15). Single sample.
• Smoke CI — did NOT fire (n=0; push-triggered, no qualifying pushes in window). Its chronic 100%-fail could not be confirmed fixed — absence only. Re-check next audit.

---

🔴 Failure taxonomy (12 = 11 real + 1 intentional)

All failures are 0-turn (agent never completed a turn) — no agent-logic failures.

A. Execute-CLI 0-turn driver failures — 8 (cross-engine, all singletons)

Workflow	Engine	Branch	Action-min
CI Optimization Coach	copilot	main	14
Daily Agent of the Day Blog Writer	copilot	main	26
Daily Issues Report Generator	copilot	main	13
Daily SPDD Spec Planner	copilot	main	25
Smoke Copilot Sub Agents	copilot	main	7
Smoke Claude on Copilot	claude	PR	7
Design Decision Gate 🏗️	claude	PR	12
GitHub MCP Structural Analysis	claude	main	21

Fail step: Execute {GitHub Copilot\|Claude Code} CLI in the agent job. Same signature as the chronic copilot-sdk-driver-failures family — but diffuse and low-volume now (no single-workflow cluster), down from ~71% of all fleet fails during 06-30...07-06.

B. Safe-output partial-intolerance — 3 (agent green, safe_outputs red)

Workflow	Engine	Branch
PR Code Quality Reviewer	copilot	PR (eslint-miner)
PR Code Quality Reviewer	copilot	PR (linter-miner)
PR Sous Chef	pi	main

All three: agent job SUCCESS, safe_outputs job FAILED at Process Safe Outputs. The classic safe-output-partial-failure-intolerance signature — one bad safe-output item reds the whole job and discards good agent work.

Intentional (excluded): Daily Max Ai Credits Test (copilot, main) — credit-guardrail stress test, designed to fail.

---

📈 Trend (last 12 audited windows)

⚠️ Charts unavailable this cycle: matplotlib/seaborn/pandas are not installed in any available interpreter and PyPI install is firewall-blocked (5-min timeout). Trends shown as a table; a missing_tool was filed and a fix recommendation recorded.

Date	Runs	Success	Rate	Tokens	AIC
2026-06-23	174	161	92.5%	—	12,635
2026-06-24	342	317	92.7%	—	26,654
2026-06-25	382	326	85.3%	—	24,173
2026-06-27	320	291	90.9%	—	23,272
2026-06-30	171	139	81.3%	—	—
2026-07-01	84	72	85.7%	—	5,704
2026-07-02	20	12	60.0%	—	1,024
2026-07-03	47	32	68.1%	1.08M	3,539
2026-07-04	101	90	89.1%	4.37M	6,375
2026-07-05	108	97	89.8%	3.36M	6,311
2026-07-06	400	345	86.3%	—	29,611
2026-07-22	314	302	96.2%	10.84M	14,352

Windows are partial (bridge time-caps) and 07-07...07-21 has no coverage; compare rates, not absolute run counts. Success rate is the clear high point of the series, and the token metric recovered.

---

🎯 Recommended actions

1. Charting env gap (MEDIUM, new). Bake matplotlib+seaborn+pandas+numpy into the audit runner image/venv, or allowlist PyPI for the install step, or switch to a pure-Python SVG renderer. Blocks the required PNG trend charts.
2. Safe-output partial-intolerance (MEDIUM, recurring). Make a single failed safe-output item non-fatal to the safe_outputs job (partial-apply + warn) so successful agent work (PR CQR, PR Sous Chef) is not reddened.
3. Execute-CLI 0-turn family (LOW now, watch). Still cross-engine but diffuse; keep the instrumentation ask open (emit CLI stderr/exit-code) but de-prioritized while volume is low.
4. Confirm recoveries hold (watch). Re-verify Avenger, token-reporting, and Cache Strategy stay green over the next 1–2 audits before closing their issues. Confirm Smoke CI status once it next fires.

---

🧠 Repo memory updated

audit-history.jsonl (+1) · metrics-summary.json (07-22 window) · known-issues.json (3 recoveries, 2 recurrences) · anomalies.json (Avenger de-escalated, safe-output anomaly added) · recommendations.json (+2) · workflow-trends.json (snapshot).

References:

• §29930607580 — Blog Writer 0-turn (copilot, 26m)
• §29919301274 — GitHub MCP Structural Analysis 0-turn (claude, 21m)
• §29916003894 — PR Sous Chef safe-output partial fail (pi)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔍 Agentic Workflow Audit Agent · age00 362.8 AIC · ⌖ 35.6 AIC · ⊞ 8K · ◷

• expires on Jul 23, 2026, 2:01 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #47668.