[audit-workflows] Agentic Workflow Audit — 2026-06-12 (copilot-sdk tool-lockout day 6 + NEW chroot/Node failure)

[github-actions[bot]]

Agentic Workflow Audit — 2026-06-12

Window: 2026-06-12 15:08–21:41Z (~6.5h, partial day — the logs MCP tool timed out at 120s on continuation, so this audit is built from 120 locally-downloaded run directories, richer than the 34-run JSON the tool finished summarizing). Overall ~90.0% success; prod-main 93.2%. Production health is solid — but the same copilot-sdk-driver tool-permission-lockout that has dominated for nearly a week struck 4 prod-main runs again, and a new chroot/Node.js failure appeared.

Summary

Metric	Value
Runs observed	120
Success rate (overall / prod-main)	~90.0% / 93.2%
Total tokens	~126.7M
Total AI credits	~19,992
Engines	copilot 74 · claude 20 · codex 18 · pi 3 · antigravity 3 · gemini 2
Missing tools	2 (web-fetch, close_discussion)
Missing data	1 (cache_memory miss, by-design)
Firewall blocked	241 / 2092 (100% smoke-probe, by-design)

Critical Issues

1. 🔴 copilot-sdk-driver tool-permission-lockout — DOMINANT, day 6 (recurrence 6)
Four prod-main copilot runs hard-failed on guard.tool_denials_exceeded 5/5, classified as "missing tool/permission" and not retried:

Workflow	Run	Denied command
Daily Agent of the Day Blog Writer	27424904625	git checkout -b ... (wasted 6.18M tok / 1061 AIC)
Breaking Change Checker	27427030417	read(.changeset/*.md), git show
Daily Formal Spec Verifier	27428819181	awk ... over *.go
Daily Safe Output Integrator	27437733590	git status, read(workflow *.md)

These are routine read-only / branch-setup commands the workflows legitimately need. Notably, claude runs (DeepReport, Failure Investigator) hit the same denials but tolerated/recovered — the hard-abort is specific to the copilot-sdk-driver path. This is the single highest-value prod-main reliability fix.

2. 🔴 NEW: Node.js not available inside AWF chroot
Daily Issues Report Generator (copilot, prod-main, run 27425935620) aborted at entrypoint:
[entrypoint][ERROR] Copilot CLI requires Node.js, but 'node' is not available inside AWF chroot.
Agent never started. Also [WARN] Failed to transfer .../safeoutputs ownership to chroot user. Picked up by [aw] Failure Investigator. Likely a regression from the active chroot/binaries-source-path refactor (branch copilot/emit-chroot-binaries-source-path was running across the window).

PR / branch failures (all by-design smoke-probe noise)

• Smoke Gemini ×2 — gemini-3.1-flash-tts-preview has no AI-credits pricing → unknown_model_ai_credits. RECUR day 2, gemini engine 0/2 again.
• Smoke Copilot - AOAI (apikey) ×2 — persistent transient API error. Retrying... loop. RECUR day 2.
• Smoke Copilot ×3 — agent succeeded and emitted a valid FAIL-status smoke report (create_issue #38943, set_issue_type, comments all OK), yet the safe_outputs job reddened each time. Smoke emits many output types; one rejected item reds the whole job (safe-output-partial-failure-intolerance). The smoke report itself flags product gaps: GitHub MCP review tool unavailable, mcpscripts-gh unavailable, create_discussion missing label/temp-id support.
• Documentation Unbloat (claude) — agent failed with empty output (items:[]) but downstream skipped and the run was tolerated/green. Low impact.

Trend Charts

Workflow Health (30 days)

Success rate has held in a stable 84–96% band for two weeks; today's ~90% (prod-main 93.2%) is squarely mid-range and healthy. The persistent failure tail is not random — it's the same copilot-sdk-driver lockout class recurring daily, so the band won't tighten until that allow-list/guard fix lands.

Token Usage (30 days)

Daily tokens swing widely (15M–69M) driven by which heavy aggregators run; today's ~127M reflects a busier full-fleet window. The 7-day moving average smooths to a steady ~35–45M baseline — no runaway growth, but note 6.18M of today's tokens were burned then thrown away by the Daily Agent of the Day lockout.

Top token consumers

Tokens	AIC	Engine	Workflow	Result
10.91M	1907	copilot	Daily Ambient Context Optimizer	✅ (under cap — no 429 this window)
9.58M	1719	copilot	UK AI Operational Resilience	✅
6.58M	1132	copilot	Linter Miner	✅
6.18M	1061	copilot	Daily Agent of the Day Blog Writer	❌ wasted (tool-lock)
3.96M	696	copilot	Agentic Workflow AIC Usage Optimizer	✅

Good news: Daily Ambient Context Optimizer stayed under the ~1000 AIC / 25M-token cap this window — no 429 credit-cap abort (recurred 06-09/06-10).

Recommendations

1. [HIGH] Fix copilot-sdk-driver tool-permission lockout (6 days, dominant): broaden the sdk-driver default allow-list to cover git read/branch ops + reads of repo source/markdown, or relax the 5-denial guard for read-only ops; align denial handling to soft-skip-and-continue like the claude engine instead of hard-abort.
2. [HIGH] Restore node inside the AWF chroot: bind-mount the setup-node/nvm path into /host and onto PATH, or bundle node into the chroot image; add a pre-flight node --version check so it fails fast with a clear message. Verify before the chroot/binaries-source-path work merges.
3. [HIGH] Gemini pricing fallback (day 2): add gemini-3.1-flash-tts-preview to the api-proxy AI-credits pricing table or set apiProxy.defaultAiCreditsPricing so unknown gemini models degrade gracefully.
4. [MED] AOAI smoke circuit-breaker (day 2): cap retry budget / max-tokens on the AOAI-apikey probe so a transient-error loop can't burn 2M+ tokens.
5. [MED] Tolerate partial safe-output failure: one rejected item shouldn't redden an otherwise-successful safe_outputs job (recurs on Smoke Copilot).
6. [LOW] close_discussion safe-output: Daily Project Performance Summary Generator legitimately needs a way to close prior daily discussions ([daily performance] Daily Performance Summary - 2026-06-11 #38730, [daily performance] Daily Performance Summary - 2026-06-10 #38441) — currently no such tool.

Context

• Repo memory updated: audit-history.jsonl, metrics-summary.json, known-issues.json (sdk-driver→rec6, +chroot-node NEW), anomalies.json, recommendations.json (validated, 54 KB < 60 KB limit).
• References: §27424904625 · §27425935620 · §27444195593

Generated by 🔍 Agentic Workflow Audit Agent · 461 AIC · ⌖ 32 AIC · ⊞ 8K · ◷

• expires on Jun 13, 2026, 2:06 PM UTC-08:00

[github-actions[bot]]

Smoke bot grunt. Run 27447675349 poke cave wall. Latest talk still alive.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 218.8 AIC · ⌖ 16.6 AIC · ⊞ 20.3K · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #39155.