[audit-workflows] 🔍 Agentic Workflow Audit — 2026-08-13 (38-day gap): 6 chronic issues resolved, new Design Decision Gate hotspot

[github-actions[bot]]

Overview

Daily audit of the last 24h of agentic workflow runs in github/gh-aw (2026-08-12T21:36Z → 2026-08-13T21:09Z, full coverage confirmed). This audit follows a 38-day monitoring gap (prior audit: 2026-07-06) — see the anomaly flagged below.

341 runs · 298 success · 43 failure · 87.39% raw success rate (87.32% excluding 2 intentional-failure guardrail tests)

The headline is good news: six previously-chronic issues appear resolved, and the mix of failures has shifted from a few dominant repeat offenders to a thin long tail plus one new concentrated hotspot.

Key metrics

Metric	Value
Runs / Success / Failure	341 / 298 / 43
Success rate (raw / ex-intentional)	87.39% / 87.32%
AI Credits (AIC) consumed	14,282.41
Token usage (fleet total, 309/341 runs reporting)	19,029,990
Engine mix	copilot 175, pi 83, claude 57, codex 16, gemini 1, aider 2, crush 2, goose 2, opencode 2
Missing tools / missing data / MCP failures	0 / 0 / 0

Resolved this window (all were chronic/recurring as of the last audit):

• avenger-err-config-no-structured-logs — Avenger 6/6 success (was 100%-fail chronic since ~06-08, firing every ~90min)
• chroot-node-not-available — Daily Issues Report Generator ran successfully
• codex-gh-aw-binary-not-found-for-mcp — Daily Cache Strategy Analyzer ran successfully
• pi-gpt54-0tok-agentjob-fail — Auto-Triage Issues 4/4 success on pi/gpt-5.4
• aoai-apikey-persistent-transient-api-error — both AOAI Smoke variants succeeded
• playwright-install-cli-failure — not recurring

Unclear — needs maintainer confirmation: Smoke CI, the prior #1 chronic offender (100% fail on every push as of 07-06, ~31% of all fleet failures), produced zero runs in this 24h window. Logs alone can't distinguish "fixed and quiet," "disabled," "renamed," or "no pushes triggered it" — please confirm workflow status.

New hotspot: Design Decision Gate 🏗️ (claude engine) failed 4/14 = 28.6% of its runs, independently corroborated by the built-in observability insights (29% hotspot flag). All 4 failures show real token consumption (14,955–17,389 tokens) but Turns=0 — the classic longrun-0turn signature previously seen chronically in copilot-sdk-driver-failures, but this is the first time it has concentrated this heavily on one specific (PR-gating) workflow rather than spreading thin. Since this workflow gates PR merges, it's a higher-priority target than the same signature elsewhere.

Still open / recurring:

• copilot-sdk-driver-failures / claude-agent-job-fail-longrun — signature persists but now diffused thin across ~20 different workflows (1 fail each, 10K–82K real tokens, 7–52min durations) rather than concentrated
• code-scanning-fixer-0tok-agentjob-fail — 2/4 fail, but signature evolved: now real token usage (14.8K/10.1K tok, 26min) rather than the original instant 0-tok fail — likely should merge into the longrun family
• daily-fact-codex-gpt54-0turn — 3rd recurrence, low-volume (1 run, 0-tok)
• token-usage-reporting-gap — improved (90.6% of runs now report tokens, up from near-zero) but not fully closed

Trend charts

Daily success/failure counts and success-rate line over the last 30 days plus today. The shaded band marks the 38-day audit gap (2026-07-06 → 2026-08-13) — deliberately left disconnected rather than interpolated, since no data exists for that span. Visible history shows a volatile June (68–98% swinging day to day) settling into today's healthy 87.4%, consistent with the resolved-issues list above.

Daily AI Credits (AIC) consumption, chosen over raw token totals per the known token-usage-reporting-gap issue. The 7-day moving average line stops at the gap boundary rather than bridging it; today's single post-gap reading (14,282 AIC) is shown as an isolated diamond marker since one point can't support a rolling average. Today's AIC is roughly half of the late-June peak (~29.6K), consistent with the lower overall failure/retry volume this window.

Per-run failure sample used for hotspot confirmation

Design Decision Gate 🏗️ run §31729681682 (PR #52541, run_attempt=2): job agent failed at step "Execute Claude Code CLI" with metrics={TokenUsage:15341, Turns:0, ToolCalls:null} — real token consumption, zero recorded turns. Raw CLI stderr was not accessible (permission-denied on the usage/ log subdirectory), so the exact failure reason inside the Claude Code CLI process could not be captured from this audit alone.

Recommendations

1. Confirm Smoke CI status (low effort, resolves ambiguity) — was the rejig docs #1 chronic offender; now silent. Maintainer should confirm fixed/disabled/renamed.
2. Instrument Execute Claude Code CLI stderr/exit-code capture on Design Decision Gate first — it's the most concentrated instance of the still-open longrun-0turn family, and it gates PR merges.
3. Merge code-scanning-fixer-0tok-agentjob-fail into the longrun family — its failure signature has evolved to match (real tokens, long duration) rather than the original instant 0-tok fail.
4. Restore continuous daily auditing — this 38-day gap means several "resolved" issues above can only be dated to "sometime in the gap," and Smoke CI's status is genuinely unrecoverable from logs. Confirm this audit workflow's own schedule is firing reliably going forward.

Full findings, dedup'd issue tracking, and per-workflow trend snapshots have been merged into repo-memory (known-issues.json, recommendations.json, anomalies.json, workflow-trends.json, metrics-summary.json, audit-history.jsonl).

References:

• §31729681682 — Design Decision Gate hotspot sample (claude longrun-0turn)

Generated by 🔍 Agentic Workflow Audit Agent · agent · 280.2 AIC · ⌖ 49.7 AIC · ⊞ 8.3K · ◷

• expires on Aug 14, 2026, 1:50 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #52790.