[audit-workflows] 🔍 Agentic Workflow Audit — 2026-06-22 (94.5% healthy; Avenger & Skillet quiet; copilot-sdk long-run now #1)

[github-actions[bot]]

🔍 Agentic Workflow Audit — 2026-06-22

Window: 2026-06-21T23:23Z → 2026-06-22T21:35Z (~22.2h, 10 paginated batches) · Repo: github/gh-aw

A healthy, on-baseline day. Overall success 94.5% (259/274), prod-main 91.7% (92.9% excluding 2 intentional test workflows), PR/dev branches 98.9%. Zero new failure classes, zero missing-tools / missing-data / MCP failures, and no shared incident — all 31 observability episodes were standalone. Two multi-day chronic offenders went quiet: Avenger (5-day 100%-fail streak) and Skillet (06-20 fleet-wide incident) both produced 0 failures this window.

Metric	Value
Total runs	274 (259 ✅ / 15 ❌)
Overall success	94.5%
Prod-main (main, schedule/push)	170 runs → 91.7% (92.9% ex-intentional)
PR / dev branches	94 runs → 98.9% (1 fail)
Action-minutes	3,159 · GitHub API calls 3,320 · AIC ~3,563
Health	missing-tools 0 · missing-data 0 · MCP-failures 0
Engines	copilot 158 · claude 43 · pi 18 · codex 13 · antigravity 1 · gemini 1

⚠️ Token data unavailable again this window — the token_usage artifact is empty fleet-wide (TokenUsage=0); only the usage artifact was downloaded. Cost/token trend uses historical data through 06-21.

---

📊 Trend Charts (30 days)

Workflow health: Today's 94.5% sits comfortably above the 90% reference line and continues the rebound that began 06-21 (91.7%) after the 06-20 Skillet incident (69.6%). Run volume (274) is back to normal after the inflated 06-20/06-21 counts that were dominated by Skillet's failing push runs. The 30-day trend is stable in the high-80s to mid-90s with two clear dips (06-13 Avenger/PR-Sous-Chef clusters, 06-20 Skillet).

Token usage: The 7-day moving average is trending down toward ~37M/day from the mid-50s, with the 06-12 spike (127M) being a clear outlier. Today's point is absent because the token artifact was empty — a persistent observability gap that has now affected several recent windows and should be fixed so cost trends remain reliable.

---

❌ Failures (15) — all known classes, no cluster

Every one of the 15 failures is a distinct workflow — no single-workflow cluster like Avenger or Skillet on prior days. 2 are intentional self-tests.

Failure breakdown by class

Class	Count	Workflows
copilot-sdk-driver — long-run variant ⬆️	5	Daily Compiler Threat Spec Optimizer (34m), Daily Formal Spec Verifier (29m), Layout Spec Maintainer (22m), Daily Safe Output Integrator (24m), Daily SPDD Spec Planner (24m)
Single-occurrence (watch)	5	Daily Community Attribution Updater (copilot), Daily News (pi), Weekly Workflow Analysis (claude), Workflow Portfolio Analyst (claude), Super Linter Report (copilot)
INTENTIONAL test (by-design)	2	Daily Credit Limit Test, Daily Max AI Credits Test
chroot-node-not-available	1	Daily Issues Report Generator (recur day 2)
copilot-sdk-driver (PR)	1	PR Code Quality Reviewer
Smoke CI transient (1 of 13)	1	Smoke CI (push, main)

Dominant pattern shift: the copilot-sdk-driver long-run variant is now the #1 failure family (5 fails). The agent does real work for 20–34 minutes, then the job fails (session_starts=0 + failed-tool-execution signature) — distinct from the classic 0-token / 25-min permission lockout. This burns ~130 minutes of compute/day with no output and is the highest-ROI fix on the board. It only displaced Avenger because Avenger didn't run today.

---

🟢 Notable resolutions / watch items

Chronic offenders that went quiet (verify root cause)

• Avenger — chronic rejig docs #1 prod-main offender, 100% fail for 5 consecutive days (06-18 x2 → 06-19 x4 → 06-20 x8 → 06-21 x7). Today: 0 runs. The streak is broken by absence, not a confirmed fix — the schedule may have been paused per the standing recommendation. Marked WATCH: confirm fix vs. pause on its next scheduled fire before closing.
• Skillet — the 06-20 fleet-wide incident (103 fails/100%) has fully de-escalated: 103 → 4 → 0. Marked RESOLVED-DEESCALATED; reopen only if push-triggered 0-token failures resurface. Still verify the slash_command(centralized) trigger no longer fires on arbitrary push.

---

🛠️ Top recommendations

1. [HIGH · day20+] Fix copilot-sdk-driver long-run failures — now the dominant fail family (5/day). Flush emitted safe-output on session-idle/terminal instead of reddening; allowlist routine read-only/branch-setup tool calls so the 5-denial abort isn't hit. Highest ROI.
2. [HIGH] chroot-node bind-mount — Daily Issues Report Generator failed 2 days running (node not on PATH in AWF chroot). Bind-mount node + add a node --version pre-flight check.
3. [P1 · WATCH] Avenger — keep paused until the err-config / no-structured-logs root cause is fixed; do not close on absence alone.
4. [WATCH] Token observability gap — the empty token_usage artifact has recurred across multiple windows; restore it so cost trends stay reliable.
5. [WATCH] 5 diverse single-fails + claude long-run pair (Workflow Portfolio Analyst 27m, Weekly Workflow Analysis 14m) — no shared signature yet; flagged for recurrence-watch.

---

Data caveats

• Effective window ~22.2h (quiet gap 21:51–23:23Z, no runs); 10 paginated batches with count≤80 to avoid the logs-tool 120s timeout.
• Tokens unavailable (artifact empty). Engine metadata null for 40 runs (aw_info not fetched); failure engines resolved via the status tool's authoritative engine_id.
• Repo memory updated: audit-history, metrics-summary, known-issues, recommendations, anomalies, workflow-trends (validated 60 KB).

References: §27966232293 · §27970148606 · §27979436784

Generated by 🔍 Agentic Workflow Audit Agent · 520.7 AIC · ⊞ 7.9K · ◷

• expires on Jun 23, 2026, 2:16 PM UTC-08:00

[github-actions[bot]]

Smoke test ping from run 27990513949.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 287.9 AIC · ⌖ 13.9 AIC · ⊞ 19.2K · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #41112.