[audit-workflows] Daily Audit — 2026-05-02 — 31 runs · 94% success · 3 failures

[github-actions[bot]]

Overview

Daily audit of 31 workflow runs on 2026-05-02. Overall health is good with a 94% success rate (26/28 completed runs succeeded), but three failures require attention: two smoke tests blocked by firewall rules and one Design Decision Gate run that hit a security constraint reading files outside the allowed working directory.

Note: This is the first audit run — historical 30-day trend data is not yet available. Charts below reflect today's data only; trends will build up over subsequent daily audits.

---

Summary Metrics

Metric	Value
Total Runs	31 (2 in-progress)
Successful	26
Failed	3
Success Rate	94%
Total Tokens	15.7M
Total Cost	$6.91
Action Minutes	214
Total Turns	340

Engine Distribution: Copilot (16), Claude (10), Codex (2), Crush (1), Gemini (1), OpenCode (1)

---

Workflow Health

Today saw 31 runs across 24 distinct workflows. Design Decision Gate ran 4 times with 1 failure (25%), and 4 Smoke CI runs all succeeded. The 3 failures were isolated to Smoke Crush, Smoke Gemini, and one Design Decision Gate run — all with identifiable root causes.

---

Token Usage & Cost

Token usage is concentrated in a few workflows: Smoke Copilot (2.2M tokens, $0), Daily Project Performance Summary Generator (2.2M tokens, $0), and Smoke Claude (1.6M tokens, $0.95). The highest-cost run was [aw] Failure Investigator at $1.68 (749K tokens) — expected given its deep log analysis task. The Lockfile Statistics Analysis Agent spent $1.03 across 1.4M tokens.

---

Failures

1. Smoke Crush — Firewall Blocks (exit code 1)

• Run: §25260196727
• Root cause: The Crush engine attempted to access catwalk.charm.sh, 169.254.169.254 (AWS metadata), and 172.30.0.30:10003 which are blocked by the firewall. 5 out of 7 requests were blocked (71%).
• Recommendation: Add catwalk.charm.sh to the allowed-domains list for the Smoke Crush workflow, or update the workflow to disable firewall for smoke tests that need broader network access.

2. Smoke Gemini — Firewall Blocks (exit code 1)

• Run: §25260196737
• Root cause: Gemini API endpoint generativelanguage.googleapis.com and internal proxy 172.30.0.30 were blocked. The workflow already suggests using *.googleapis.com wildcard.
• Recommendation: Add generativelanguage.googleapis.com (or *.googleapis.com) to allowed-domains for Smoke Gemini.

3. Design Decision Gate — Security Constraint (file access outside allowed dir)

• Run: §25260257832 | PR: copilot/update-policy-for-missing-data-tools
• Root cause: The Claude agent tried to cat /tmp/gh-aw/agent/adr-prefetch-summary.json — a file outside the allowed working directory /home/runner/work/gh-aw/gh-aw. The workflow was preparing an ADR draft for PR feat: surface missing_tool and missing_data as agent failures in failure issue comments #29804.
• Recommendation: The workflow or the agent spawning sub-tasks should ensure all intermediate files are written to and read from the allowed workspace directory. The pre-fetch summary should be placed in the workspace, not /tmp/gh-aw/agent/.

---

Missing Tools & Data

Issue	Workflow	Details
web-fetch tool missing	Smoke Codex	Codex environment does not have the web-fetch MCP tool; smoke test could not run direct web fetch check
cache_memory miss	Smoke Codex	First run for Codex — no prior cache data available

Both issues are expected for Smoke Codex given it's testing a newer engine without full MCP toolset parity.

---

Observability Insights

View All Insights

Severity	Category	Finding
🔴 High	Network	Smoke Crush: 5/7 requests blocked (71%) — highest firewall friction
🔴 High	Reliability	11 high-anomaly events (score > 0.6) across 31 runs — new log templates in tool_result and error stages
🟡 Medium	Reliability	Design Decision Gate: 25% failure rate across 4 runs
🟡 Medium	Drift	Design Decision Gate: 4–13 turns per run (avg 6.2) — unstable task shape
🟡 Medium	Tooling	Smoke Codex: 1 missing tool, 1 missing data signal
i️ Info	Actuation	4 runs executed write-capable safe outputs; 27 stayed read-only
i️ Info	Execution	29 distinct event templates mined across 6 pipeline stages from 65 events

---

Firewall Summary

Metric	Value
Total Requests	741
Allowed	642 (87%)
Blocked	99 (13%)
Top Blocked Workflow	Daily Project Performance Summary Generator (53 blocked)

Firewall Details by Workflow

Workflow	Blocked	Allowed	Block Rate
Daily Project Performance Summary Generator	53	61	46%
Copilot PR Prompt Pattern Analysis	7	7	50%
Changeset Generator	7	101	6.5%
Agent Container Smoke Test	3	4	43%
Smoke Crush	5	2	71%
Smoke Gemini	~4	~3	~57%

Blocked domains: 169.254.169.254 (AWS metadata), 172.30.0.30:10003 (internal proxy), ab.chatgpt.com, catwalk.charm.sh, chatgpt.com, models.dev, opencode.ai

---

Top Workflows by Cost

Workflow	Engine	Tokens	Cost
[aw] Failure Investigator (6h)	Claude	749K	$1.68
Lockfile Statistics Analysis Agent	Claude	1.4M	$1.03
Daily Caveman Optimizer	Claude	505K	$1.02
Smoke Claude	Claude	1.6M	$0.95
Daily Team Evolution Insights	Claude	823K	$0.78
Design Decision Gate (failed run)	Claude	915K	$0.62

---

Recommendations

1. Fix Smoke Crush firewall: Add catwalk.charm.sh to allowed-domains (currently suggested as catwalk.charm.sh in the warning message from the run).
2. Fix Smoke Gemini firewall: Add generativelanguage.googleapis.com to allowed-domains for Gemini smoke workflows.
3. Design Decision Gate file paths: Ensure agent intermediate files (e.g., prefetch summaries) are written to /home/runner/work/gh-aw/gh-aw/ not /tmp/gh-aw/agent/ to avoid security constraint violations.
4. Design Decision Gate turn drift: Investigate why the same workflow uses 4–13 turns — likely due to varying PR complexity, but worth checking for prompt instability.
5. Daily Project Performance Summary Generator: 53 blocked requests is high (46% block rate) — review which domains are being accessed and consider updating the allowed-domains list or workflow logic.

References:

• §25260196727 Smoke Crush
• §25260196737 Smoke Gemini
• §25260257832 Design Decision Gate (failed)

Generated by Agentic Workflow Audit Agent · ● 479.3K · ◷

• expires on May 3, 2026, 9:43 PM UTC

[github-actions[bot]]

💥 WHOOSH!

The Claude smoke test agent swoops in from the digital ether!

🦸 THE SMOKE TESTER HAS ARRIVED! Run §25263690532 — ALL SYSTEMS GO!

KA-POW! ✅ GitHub MCP online
ZZZAP! ✅ Make build nominal
THWACK! ✅ Serena LSP activated
BIFF! ✅ Playwright navigating
WHAM! ✅ All 19 tests complete!

The agent vanishes in a cloud of compile smoke... until next time! 🌪️

💥 [THE END] — Illustrated by Smoke Claude · ● 312.4K · ◷