[audit-workflows] 🛡️ Agentic Workflow Audit — 2026-07-23 (healthiest since ~06-24; 97.2%, 0 novel anomalies)

[github-actions[bot]]

🛡️ Agentic Workflow Audit — 2026-07-23

Window: partial day-slice, 251 fully-downloaded runs (logs MCP timed out at 300s before full aggregation; 228 folders empty). First audit since 2026-07-06 (17-day gap).

Headline: Fleet is the healthiest it has looked since ~06-24. Zero novel anomalies, zero missing-tools / missing-data / MCP failures. Every real failure maps to a known family, and several chronic dominant offenders (Smoke CI, Avenger, PR CQR, Cache Strategy) are absent or recovered.

---

📊 Key Metrics

Metric	Value
Raw success	97.2% (244/251)
Ex-intentional	97.6% (244/250)
Prod-main (ex-intentional)	96.3% (129/134)
Non-main	99.1% (115/116)
Missing tools / data / MCP	0 / 0 / 0
Tokens (recovered)	10.97M across 247 runs
AI credits	12,272.9
Intentional failures excluded	1 — Daily Max Ai Credits Test (credit guardrail by design)

Engine health: claude 46/46 (100%) · codex 10/10 (100%) · copilot 149/154 (96.8%) · pi 37/39 (94.9%) · antigravity 1/1 · gemini 1/1.

---

📈 Trend Charts (30d)

Workflow health rebounds to 97.2% — the top of the 30-day band, well clear of the 90% line and a clean recovery from the 82–86% prod-main trough of the 06-30...07-06 Smoke-CI-dominated stretch. Failure bars are near-flat this slice (7 total, 6 real).

Token reporting is back: 10.97M tokens flowed through token_usage_summary after the 06-19→07-06 reporting gap where daily totals were null/tiny. The moving average lifts off the floor — trust the last point more than the mid-window dip, which reflects missing data, not low usage.

---

🔎 Failure Analysis — 6 real fails, all known families, 0 novel

Full failure breakdown

Workflow	Engine	Failing step	Family	Note
PR Sous Chef ×2	pi/gpt-5.4	Process Safe Outputs	safe-output-partial-failure-intolerance	TRUE signature — agent SUCCESS (55K/54K tok) then safe-outputs reddens run
CI Optimization Coach	copilot	Execute Copilot CLI	copilot-sdk-driver	longrun 16m 0-turn (new wf name in family)
PR Code Quality Reviewer	copilot	Execute Copilot CLI	copilot-sdk-driver	10m 0-turn on PR branch — 1/19 only (was 9-10/34)
Daily Issues Report Generator	copilot	Execute Copilot CLI	chroot-node-not-available	0-turn 15m, chronic since 06-12
Daily Container Image Security Scan	copilot	Enforce critical vuln & license gates	security-gate-trip (NEW)	agent ran (22K tok) then post-agent gate blocked — likely working-as-intended finding, not infra

Clustering: copilot Execute-CLI 0-turn family = 3 fails (down from ~71-78% of all fleet fails on 06-30/07-06). safe-output-partial = 2 (now the leading true-signature recurring fail). Security-gate-trip = 1 (probably a real CVE/license finding).

---

✅ Chronic Offenders — Recovered / Absent

• Smoke CI — ABSENT this window. Was the dominant chronic offender (31% of all fails, 100%-fail on every push, 6 windows 06-30...07-06). No copilot-CLI 100%-fail cluster observed at all.
• Avenger — 1/1 SUCCESS. First observed success after err-config/no-structured-logs chronic 100%-fail (recur18, fired ~every 90min).
• PR Code Quality Reviewer — 18/19 (95%), down from a 9-10/34 fail load in the copilot-sdk-driver family.
• Daily Cache Strategy Analyzer — 1/1 success (was ~100% fail on codex binary-not-found).
• Daily Caveman Optimizer — success (was model/param-config 400 fail).
• Auto-Triage Issues — 4/4 on pi/gpt-5.4; recovery holds across 07-04 and 07-23.

⚠️ Caveat: partial window (251/479 folders; several are single-run samples). Single-run successes for Avenger and Cache Strategy are directional evidence, not confirmation of a fix. Reconfirm on the next full window before closing these issues.

---

🎯 Recommendations

1. [MEDIUM · now top-ROI] safe-output-partial non-fatal handling — PR Sous Chef (pi/gpt-5.4) reddens 2/20 at Process Safe Outputs after the agent produced valid 55K-token output. Make the safe_outputs job tolerant of a single failed item (skip-and-warn / per-item status) so one bad item can't red a successful agent run. This is now the highest-value fix since the copilot-CLI-0turn family has declined.
2. [LOW] Classify the container-scan gate-trip — confirm the Enforce critical vulnerability and license gates red corresponds to a real critical CVE/license finding. If intended, treat it as an expected conclusion (like the credit-guardrail tests) so audits don't score it as a regression.
3. [WATCH] Reconfirm chronic recoveries — Smoke CI absence, Avenger success, Cache Strategy success are all single-slice signals. Do not close their issues yet; verify on the next full-coverage window.
4. [LOW · resolved-watch] copilot-CLI-0turn instrumentation still valid but lower priority; token-reporting gap appears resolved.

---

🗂️ Repo Memory

Updated all six files: audit-history.jsonl (+1 entry), metrics-summary.json (latest_window + 07-23 history point), known-issues.json (16 issues — 6 recoveries de-escalated to WATCH, 1 NEW gate-trip), recommendations.json (16 — 2 new), anomalies.json (Smoke CI / Avenger / copilot-0turn de-escalated), workflow-trends.json (29-wf snapshot).

References:

• §29984613158 — container-scan gate-trip
• §30018005233 — PR Sous Chef safe-output-partial
• §30012298950 — CI Optimization Coach 0-turn

Generated by 🔍 Agentic Workflow Audit Agent · age00 · 308.8 AIC · ⌖ 34.8 AIC · ⊞ 8K · ◷

• expires on Jul 24, 2026, 1:54 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #47867.