[workflow-analysis] Weekly Workflow Analysis — 82.5% success, all 7 failures are copilot-engine

[github-actions[bot]]

Weekly Workflow Analysis — 40 runs (2026-06-29 → 07-06)

Analyzed the 40 most recent agentic-workflow runs from the past week (this is the API window cap; older runs in the range remain via the continuation cursor). Overall health is good, but every failure came from a single engine.

Key Metrics

Metric	Value
Runs analyzed	40
Success rate	82.5% (33 ✅ / 7 ❌)
Total wall-clock	8.1h
Action-minutes	507
Total AIC (est. cost units)	3,446.9
GitHub API calls	662
Write-capable runs	0 (all 40 read-only / safe-output only)

Success rate by engine — 100% of failures are copilot:

Engine	Runs	Fails	Success	Total AIC
copilot	27	7	74%	2,123.3
claude	8	0	100%	1,273.8
pi	4	0	100%	39.3
codex	1	0	100%	10.4

Failure Patterns (the headline)

All 7 failures are copilot-engine runs, and all 7 fail at the exact same step: Execute GitHub Copilot CLI. This is not spread across steps — it is one failure surface. Digging into each run reveals three distinct sub-causes:

View per-failure breakdown

Run	Workflow	Dur	Root cause	Did work complete?
§28784197280	Contribution Check	16.1m	CLI exited non-zero after producing output	✅ 2 safe outputs (issue + comment) emitted
§28783119402	PR Code Quality Reviewer	19.4m	Firewall blocked proxy.golang.org (3 reqs) + non-zero exit	✅ 3 safe outputs, 89/89 tool calls OK
§28782462856	PR Code Quality Reviewer	13.6m	Same workflow, same step	—
§28783119450	Impeccable Skills Reviewer	19.8m	Non-zero exit, high token use	—
§28779209879	Layout Specification Maintainer	23.9m	5 of 68 tool executions failed (genuine tool errors)	partial
§28780544290	Smoke CI	3.3m	Fast startup fail — 0 turns, 0 tokens	❌ never ran a turn
§28778368458	Smoke CI	4.0m	Same fast startup fail (push event)	❌

Three sub-patterns:

1. "False failures" — Contribution Check and PR Code Quality Reviewer both completed their work and emitted valid safe outputs, yet the CLI step reported failure. The Copilot CLI appears to return a non-zero exit code at the end of otherwise-successful runs. These waste action-minutes on reruns and create alert noise.
2. Firewall allowlist gap — PR Code Quality Reviewer needs Go tooling but proxy.golang.org is not in the allowed-domains list, so go-based commands are blocked mid-run.
3. Smoke CI canary — failed twice (scheduled + push) with 0 turns/0 tokens, i.e. the Copilot CLI never started a turn. This is the canary flagging the systemic copilot-CLI issue above.

pr-code-quality-reviewer was independently flagged by the observability layer as a failure hotspot: 2 fails / 4 runs = 50%.

Performance & Cost

• copilot dominates both volume and spend: 27/40 runs (68%) and 2,123 AIC (62% of total).
• Most expensive single runs are claude: Daily Rendering Scripts Verifier (481 AIC), Daily AW Cross-Repo Compile Check (348 AIC / 26.2m), Instructions Janitor (236 AIC).
• Longest runs: Organization Health Report (31.8m), Daily AW Cross-Repo Compile Check (26.2m), Layout Spec Maintainer (23.9m).

Optimization opportunities

• Layout Specification Maintainer — audit flagged "Cheaper Model Available"; 23.9m for its task shape is heavy. Candidate for a smaller model.
• Impeccable Skills Reviewer — consistently makes ~30 GitHub API calls per run vs. ~9–11 for peer reviewers. Worth checking for a redundant fetch/pagination loop.
• PR Code Quality Reviewer & Contribution Check — flagged "Resource Heavy For Domain" (212 / 188 AIC, 93k / 66k tokens). Prompt-tightening could cut cost and reduce the non-zero-exit surface.
• API budget is healthy — 662 calls against a 15k core rate limit; no throttling observed.

Recommended Next Actions

1. Investigate the Copilot CLI non-zero exit (highest priority). Runs that emit valid safe outputs should not be marked failure. Check the Execute GitHub Copilot CLI step's exit-code handling in the copilot engine wrapper — this alone explains ≥3 of 7 failures and all the alert noise.
2. Add proxy.golang.org to the allowed-domains list for Go-tooling workflows (PR Code Quality Reviewer), or vendor/GOFLAGS-cache Go deps so the firewall block can't fail the run.
3. Triage Smoke CI — two 0-turn fast failures in one week means the canary is doing its job; confirm whether it's a transient Copilot CLI availability issue or a config regression on main.
4. Model/prompt tuning for Layout Spec Maintainer (cheaper model) and Impeccable Skills Reviewer (reduce ~30 API calls/run).

References:

• §28783119402 — PR Code Quality Reviewer (firewall block)
• §28784197280 — Contribution Check (false failure)
• §28780544290 — Smoke CI (0-turn fast fail)

Generated by 🔍 Weekly Workflow Analysis · 218.3 AIC · ⌖ 11.1 AIC · ⊞ 4.1K · ◷

• expires on Jul 7, 2026, 3:05 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-07-07T11:05:04.000Z.

Closed by Workflow