[audit-workflows] Daily Agentic Workflow Audit — 2026-08-19 (90.9% adjusted, 44-day gap since last audit)

[github-actions[bot]]

Daily Agentic Workflow Audit — 2026-08-19

Window: ~24h, 2026-08-18T21:18:34Z → 2026-08-19T21:00:49Z. This is the first audit since 2026-07-06 — a 44-day gap — so every trend/recurrence comparison below is a coarse before/after, not a continuous daily series. Treat deltas accordingly.

Fleet health: 354 runs, 321 success / 33 failure = 90.7% raw. Excluding the 2 intentional-failure test workflows (Daily Credit Limit Test, Daily Max AI Credits Test) → 352 runs, 320/32, 90.9% adjusted. Scheduled/prod-main runs: 195/218 = 89.4%. PR/issue-triggered: 125/132 = 94.7%.

Key issues this window

1. copilot-sdk-driver-failures (recurring, count 45) — still the rejig docs #1 fail cause: 21 of 32 non-intentional failures (66%) are 0-token/0-turn driver-exit crashes at the Execute-<engine>-CLI step, now cross-engine (13 copilot, 5 claude, 4 codex). Ponytail Reviewer is the new dominant single instance (4/17 = 23.5% of its PR-triggered runs, e.g. §32257020657). Its config loads an external third-party skill (DietrichGebert/ponytail/skills/ponytail-review) — no other workflow in this family is known to load a 3rd-party skill, so it's a plausible (unverified) lead.
2. code-scanning-fixer-0tok-agentjob-fail (new signature, count 4) — a distinct cluster from the driver-exit family: Code Scanning Fixer failed 3/3 with real token usage, single-turn, consistent ~26-minute timeouts (e.g. §32226487341). Looks like a genuine time-budget problem, not a startup crash — recommend raising the per-turn timeout or adding mid-turn checkpointing.
3. Avenger recovering — 8/9 = 88.9% this window vs. a chronic 100%-fail incident recorded pre-gap (2026-06-27). One remaining failure is a 1.7h duration outlier, well outside its normal range — worth isolated review. Given the 44-day gap, can't yet confirm a real fix landed vs. reduced trigger frequency; recommend one more clean window before downgrading severity.
4. Smoke CI absent — zero observed runs this window, despite a long chronic 100%-fail-on-push history (17/17 as of 2026-07-06). Absence isn't evidence of a fix; flagged as unconfirmed rather than resolved.

Charts

Success rate held in the 86–91% band across the 21 historical points before the gap, and today's 90.9% (adjusted) sits comfortably within that prior range rather than showing a regression. The visible gap between the last 2026-07-06 point and today is real missing data, not a flat trend — no daily signal exists for the 44 days in between.

Today's 15.6M-token day is on the high end versus the sparse historical bars, but most historical days have no reported token total at all (see token-usage-reporting-gap, count 4), so the 7-day moving average is built on very few real points and should be read with low confidence until that reporting gap closes.

Phase-by-phase analysis

Collection: Fetched all 354 runs in the 24h window via 4 paginated agenticworkflows logs calls (continuation-chained on count limit reached), merged/deduped by run_id. Engine mix: copilot 169, pi 99, claude 59, codex 20, aider 2, goose 2, crush 1, null 2. Engine success rates: pi 99.0%, aider/goose/crush 100%, claude 91.5%, copilot 87.6%, codex 80.0%.

Clustering: Grouped the 33 raw failures by failure_kind + step signature, not just "agent job failure." Two distinct clusters emerged that superficially look similar but aren't: driver_exit (0 tokens, 0 turns, instant crash at Execute-CLI — 21 runs, cross-engine) vs. agent_logic (real tokens, single-turn, ~26min timeout — Code Scanning Fixer's 3 runs). Deduplicated against known-issues.json's existing 15 entries rather than treating every cluster as new; 6 existing issues matched and were updated in place (recurrence counts incremented, not duplicated).

Recommendation: Every recommendation below cites the specific log evidence it's based on (see recommendations.json): investigate Ponytail Reviewer's external skill load path as a lead for the driver-exit family; raise Code Scanning Fixer's timeout or add checkpointing; hold off closing Smoke CI's known issue on absence alone; hold off downgrading Avenger's severity until a second clean window; and — a process recommendation — confirm this audit workflow's own schedule is firing daily so the 44-day gap doesn't recur.

Synthesis: Net picture: fleet reliability (90.9% adjusted) is in line with the pre-gap historical band, the #1 failure family is unchanged in kind but has rotated its dominant workflow to Ponytail Reviewer, one new distinct failure cluster (Code Scanning Fixer timeouts) needs separate tracking from the driver-exit family, and one chronic offender (Avenger) shows real improvement that isn't yet confirmed durable. Zero missing-tool/missing-data/mcp-failure safe-output events this window.

Full known-issue and recommendation detail

• copilot-sdk-driver-failures — HIGH, recurring, count 45, last seen 2026-08-19
• code-scanning-fixer-0tok-agentjob-fail — signature retitled to reflect the ~26min single-turn timeout, count 4
• avenger-err-config-no-structured-logs — HIGH, count 19, notable improvement this window
• smoke-ci-copilot-cli-100pct-fail-on-push — no observations this window, NOT marked resolved
• token-usage-reporting-gap — count 4, still unresolved, most historical days have null token totals
• 5 new recommendations filed this cycle in recommendations.json, 4 new anomalies filed in anomalies.json (including the audit-coverage gap itself as a process anomaly)

Next actions

• Investigate whether Ponytail Reviewer's external skill dependency is implicated in the driver-exit crash.
• Raise Code Scanning Fixer's timeout or add mid-turn checkpointing.
• Do not close Smoke CI's known issue or downgrade Avenger's severity without another observation window.
• Confirm the audit workflow's own schedule so daily continuity doesn't lapse again.

References:

• §32257020657 — Ponytail Reviewer, driver_exit sample
• §32226487341 — Code Scanning Fixer, agent_logic timeout sample
• §32287796631 — AI Moderator, driver_exit sample

Generated by 🔍 Agentic Workflow Audit Agent · agent · 367.5 AIC · ⌖ 8.82 AIC · ⊞ 8.7K · ◷

• expires on Aug 20, 2026, 1:49 PM UTC-08:00