[audit-workflows] Agentic Workflow Audit — 2026-08-04: 93.2% success, 2 new failure clusters (Goose, Aider)

[github-actions[bot]]

Agentic Workflow Audit — 2026-08-04

Window: full 24h, 2026-08-03T21:42Z → 2026-08-04T21:45Z · 220 runs collected across 3 paginated fetches (no gaps between pages).

⚠️ Audit cadence gap: the previous audit-history entry in repo memory is dated 2026-07-06. This run closes a ~29-day gap in the daily audit's own execution — treat trend comparisons against pre-gap data as directional only, not a continuous trailing-30-day series.

Headline numbers

Metric	Value
Total runs	220
Success / Failure	205 / 15
Raw success rate	93.2%
Success rate (excl. 1 intentional failure)	93.6%
Prod-main (branch=main) rate	116/126 = 92.1% (117/125 ex-intentional = 93.6%)
Non-main (PR) rate	89/94 = 94.7%
Total tokens	6.28M
Total AIC	8,896.98
Action minutes	2,467
Missing tools / missing data / MCP failures	0 / 0 / 0
Engine mix	copilot 137, claude 40, pi 31, codex 6, goose 4, aider 2

All 15 failures are 0-turn/driver-level (14 driver_exit, 1 agent_logic) — zero missing-tool, missing-data, or agent-logic-content failures in the real (non-intentional) set. Daily Max Ai Credits Test failed by design (credit-guardrail trip) and is excluded from rate rollups per policy.

The health chart shows today's 93.2% sitting comfortably inside the pre-gap baseline band (60–98%), with the shaded region marking the 29-day audit blind spot. No sign of fleet-wide degradation resuming after the gap.

Token usage looks like a cliff from ~50M/day (mid-June) to ~6M today, but most of that drop is a reporting-coverage artifact: the long-standing token-usage-reporting-gap known issue means most July days have null fleet-wide token totals rather than genuine near-zero usage. Today's 6.28M is the first fully-populated multi-page collection since the gap and should be treated as the new reliable baseline going forward, not evidence of a 8x usage drop.

Engine reliability breakdown

Engine	Runs	Failures	Rate
claude	40	0	100%
codex	6	0	100%
pi	31	1	96.8%
copilot	137	7 (ex-intentional)	95.6%
goose	4	4	0%
aider	2	2	0%

Failure clusters (evidence-linked)

1. Smoke Goose — 100% failure (4/4 this window, 5/5 in a wider same-engine query) — NEW cluster, no prior record in memory
Two distinct root causes observed across the failing runs, both surfaced in agent-stdio.log after real container start:

• Authentication error: Authentication failed for (172.30.0.30/redacted) Status: 401 Unauthorized. Response: Incorrect API key provided: github_pat_... / ghs_1536... — a GitHub token (PAT/installation token) is being sent to the model proxy where an OpenAI-shaped key is expected.
• A second run instead hit `Request failed: Resource not found (404) at (172.30.0.30/redacted) — a different proxy port with no route.
• Every run also logs Warning: Failed to start extension 'safeoutputs' and 'github' — connection refused to localhost:8080/mcp/safeoutputs and /mcp/github — the MCP gateway is unreachable from the Goose harness at startup on every run.
  Evidence: run_ids 30951782455, 30944874071, 30932534640, 30908234685 (/tmp/gh-aw/aw-mcp/logs/run-(id)/agent-stdio.log).

2. Smoke Aider — 100% failure (2/2) — NEW cluster, no prior record in memory
aider_harness.cjs's pip install step fails before the agent ever starts. Root cause chain: installing aider's dependency fastuuid has no prebuilt wheel for the PyPy 3.11 runner, so pip falls back to a source build via maturin/cargo, which needs index.crates.io. That domain is not in the workflow's allowed-domains list — the CONNECT tunnel returns 403, cargo/maturin fails, and aider_harness.cjs throws Error: Aider installation failed.
Evidence: run_ids 30951915720, 30944867248 — pip: error: subprocess-exited-with-error → Could not connect to server (CONNECT tunnel failed, response 403) → maturin failed → Error: Aider installation failed.

3. copilot-sdk-driver-failures (known issue, recurring since 2026-06-02, recur=24 as of last record) — 4 more instances today
PR Code Quality Reviewer ×3 (run_ids 30943711376, 30943794959, 30935203223 — all 4-turn/1-turn sessions with real tool activity, 0 blocked domains, 0 failed tool executions, then a silent driver exit code 1) and Linter Miner ×1 (real aic=122.4/tokens=68.7K then same silent exit). Both workflow names were already in this issue's affected list from the 07-05/07-06 audits — this is a clean recurrence, not a new signature.

4. Isolated singletons (not deep-dived — single occurrence each, no artifact fetched): CI Optimization Coach, Daily Agent of the Day Blog Writer (both already-known affected name), Slide Deck Maintainer — consistent with historical 0-turn driver-exit noise floor, not escalated.

Recommendations (smallest actionable set)

1. Fix Goose model-proxy auth wiring — the harness is passing a GitHub token where the proxy expects a model API key, and/or routing to the wrong proxy port (10000 vs 10002). Given 0% success across all recent runs, this looks like a config regression rather than flakiness — worth a priority look before the next scheduled Goose run. (Evidence: 4 runs, 2 distinct error signatures, all in agent-stdio.log.)
2. Add index.crates.io to Smoke Aider's allowed-domains, or pin an aider/fastuuid version with a prebuilt PyPy wheel to avoid the cargo/maturin build path entirely. (Evidence: 2/2 runs, identical CONNECT tunnel failed, response 403 → maturin failed chain.)
3. Fix the MCP gateway connection race for Goose — safeoutputs/github extensions fail to connect on 100% of Goose runs (localhost:8080/mcp/... connection refused), independent of the auth issue above. Likely the gateway isn't ready when Goose's harness starts, or Goose isn't using the same gateway wiring as other engines.
4. No new action needed on copilot-sdk-driver-failures — track as-is; today's 4 instances are consistent with the existing 24-recurrence pattern and already-known affected workflow list.
5. Investigate why the audit workflow itself went dark for ~29 days (last entry 2026-07-06 → today 2026-08-04) — this is an operational reliability gap in the auditor, independent of the fleet it's auditing.

Repo memory

known-issues.json, workflow-trends.json, metrics-summary.json, audit-history.jsonl, and anomalies.json updated: copilot-sdk-driver-failures recurrence bumped to 25 (last_seen 2026-08-04); two new OPEN issues added (smoke-goose-proxy-auth-mcp-gateway-failure, smoke-aider-fastuuid-crates-io-blocked); today's metrics appended.

References:

• §30952929613

Generated by 🔍 Agentic Workflow Audit Agent · agent · 309.4 AIC · ⌖ 37.6 AIC · ⊞ 8.4K · ◷

• expires on Aug 5, 2026, 2:05 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #50693.