Agentic Workflow Audit — 2026-03-31

[github-actions[bot]]

Daily audit of all agentic workflow runs in the past 24 hours for github/gh-aw.

Summary

Metric	Value
Total Runs	38 (22 completed + 4 in progress + 12 skipped/queued)
✅ Successes	27
❌ Failures	11
Success Rate	71%
Total Tokens	11.66M
Total Estimated Cost	$6.25
Missing Tools	1
Engines Active	Claude, Codex, Copilot

---

Workflow Health Chart

AI Moderator dominates failures at 8/12 runs failing (33% success rate). All failures share the same root cause — see Critical Issues below. Other workflows ran cleanly at 100% success.

---

Token & Cost Chart

Daily Documentation Updater consumed the most tokens (1.84M, $1.35) followed by Copilot Agent Prompt Clustering Analysis (1.86M, $1.06) and Sergo (1.61M, $1.07). Smoke Claude spent $0.89 on a test run that ultimately failed due to a push restriction.

---

Critical Issues

❌ AI Moderator — 100% Failure Rate (8/8 runs)

All AI Moderator runs failed because the Codex engine's firewall blocked required domains:

Blocked Domain	Requested By
github.com	git/2.53.0
api.github.com	codex_exec/0.118.0
chatgpt.com	codex_exec/0.118.0

The workflow uses allowed_domains: [defaults], but the "defaults" policy for Codex doesn't include GitHub or ChatGPT domains. The agent needs these to fetch repo data and access the Codex backend.

Recommendation: Add github.com, api.github.com, and chatgpt.com to AI Moderator's allowed-domains configuration.

❌ Smoke Claude — Push Restriction Failure

Smoke Claude (PR feat/service-ports-23756) failed because the patch attempted to write test-smoke-push-23817988152.txt, a file outside the workflow's allowed-files list.

Error:

push_to_pull_request_branch: Cannot push to pull request branch: patch modifies files 
outside the allowed-files list (test-smoke-push-23817988152.txt).

Recommendation: Add test-smoke-push-*.txt to the Smoke Claude workflow's allowed-files config, or verify the smoke test is targeting the correct file pattern.

❌ Smoke Codex — Agent Failure (chatgpt.com blocked)

Smoke Codex failed because chatgpt.com was blocked by the firewall (same pattern as AI Moderator). The Codex agent exited with code 1.

Recommendation: Add chatgpt.com to Smoke Codex allowed-domains.

❌ Changeset Generator — Agent Failure

The Changeset Generator agent (Codex engine) exited with code 1 on PR feat/service-ports-23756. No safe outputs were produced, and no push failure was reported — the agent itself failed to complete. Likely the same domain-blocking issue.

---

Observability Insights

• Reliability: AI Moderator is a failure hotspot — 8 failures across 8 runs (100% failure rate)
• Drift: Issue Monster varied from 3 to 7 turns across runs (avg 4.4) — changing task shape or unstable prompts
• Tooling: Smoke Copilot reports missing Serena MCP tools (activate_project, find_symbol) — expected in this environment, not blocking
• Actuation: 12/38 runs produced write actions (issues, PRs, comments); 17 stayed read-only

---

Performance Highlights

Top Token Consumers

Workflow	Tokens	Cost
Copilot Agent Prompt Clustering Analysis	1,857K	$1.06
Daily Documentation Updater	1,840K	$1.35
Sergo - Serena Go Expert	1,614K	$1.07
Static Analysis Report	1,560K	$1.04
Smoke Claude	1,298K	$0.89
Step Name Alignment	1,128K	$0.84
Auto-Triage Issues	1,093K	$0.00*
Daily Workflow Updater	378K	$0.00*

*Cost shown as $0.00 for some workflows — likely due to engine pricing not captured in episode data.

All Run Results

Workflow	Engine	Conclusion	Event
AI Moderator (×8)	codex	❌ failure	issues/issue_comment
Auto-Triage Issues (×8)	copilot	✅ success	schedule
Issue Monster (×5)	copilot	✅ success	schedule
Daily Workflow Updater	copilot	✅ success	schedule
Daily Documentation Updater	claude	✅ success	schedule
Sergo - Serena Go Expert	claude	✅ success	schedule
Copilot Agent Prompt Clustering Analysis	copilot	✅ success	schedule
Static Analysis Report	claude	✅ success	schedule
Step Name Alignment	claude	✅ success	schedule
Daily DIFC Integrity-Filtered Events Analyzer	claude	✅ success	schedule
Smoke Copilot	copilot	✅ success	pull_request
Agent Container Smoke Test	claude	✅ success	pull_request
Smoke Claude	claude	❌ failure	pull_request
Smoke Codex	codex	❌ failure	pull_request
Changeset Generator	codex	❌ failure	pull_request
Metrics Collector - Infrastructure Agent	—	✅ success	schedule
The Great Escapi	—	✅ success	—

---

Recommendations

1. [High] Fix AI Moderator firewall config — add github.com, api.github.com, chatgpt.com to allowed-domains. This single fix would bring success rate from 71% → ~97%.
2. [Medium] Fix Smoke Claude allowed-files — add smoke test output files to the allowed list.
3. [Medium] Fix Smoke Codex/Changeset Generator — add chatgpt.com to allowed-domains for Codex-engine workflows.
4. [Low] Investigate Issue Monster turn variance (3–7 turns) — may indicate prompt sensitivity to issue content.

---

References:

• §23819196565 — This audit run
• §23817988152 — Smoke Claude failure
• §23818574285 — AI Moderator failure (representative)

AI generated by Agentic Workflow Audit Agent · history

• expires on Apr 1, 2026, 9:34 PM UTC

[github-actions[bot]]

🎉 Beep boop! The smoke test agent was here! 🤖✨

Just swinging by to let you know that all systems are nominal and the automation overlords are pleased with today's audit. The robots have reviewed your findings and officially endorse your recommendations (especially the firewall fix — those poor AI Moderator runs deserve better).

Signed, Smoke Copilot Agent #23820400195 🚀

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

💥 WHOOSH! The Smoke Test Agent swoops in! 🦸

ZAP! Claude engine nominal — 23820400173 reporting for duty!

KA-POW! All systems green! The agentic smoke test has passed through this very dimension and left its mark!

BIFF! Stay vigilant, heroes of the repo! The automated guardian is watching! 🔥

💥 [THE END] — Illustrated by Smoke Claude · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #23954.