[workflow-analysis] Weekly Workflow Analysis — 2026-08-03 sample (269 workflows, 117 runs)

[github-actions[bot]]

Overview

Sampled the most recent 117 runs across the repo's 269 agentic workflows, spanning ~9.5 hours (2026-08-03 00:54–10:24 UTC). Full 7-day pagination hit repeated API timeouts on this run (log noted below); figures past the "extrapolated" line are scaled from this sample, not measured directly. Given every daily workflow fires once/day, a ~9.5h window still captures a representative cross-section of the fleet.

Key metrics

Metric	Sample (9.5h)	Extrapolated / week
Runs	117	~2,070
Failures	18 (15.4%)	~310
Tokens	6.70M	~118.5M
Action-minutes	1,654 (27.6h)	~488h
Engines	Copilot 65 · Claude 29 · Pi 16 · Codex 6 · unknown 1	—

Reliability: 15.4% of sampled runs failed. Two workflows failed twice in this single 9.5h window — Code Scanning Fixer and Discussion Task Miner — the clearest recurring hotspots.

Cost per engine (sample):

Engine	Runs	Avg tokens/run	Avg AIC/run	Fail rate
Codex	6	662,805	15.0	0%
Claude Code	29	35,329	162.0	17.2%
Pi	16	35,618	8.4	6.3%
Copilot CLI	65	17,372	40.1	16.9%

Claude Code runs cost ~4x Copilot's and ~10-19x Pi/Codex's average AIC per run despite moving far fewer tokens than Codex — Codex burns huge token volume (~663K/run) but at very low cost, likely from heavy cache-read reuse.

Actuation: across the full sample, 0 runs executed a write-capable safe output — every run stayed read-only. Likely expected for this mostly analysis/reporting fleet, but worth a sanity check that write-capable workflows aren't silently no-op'ing.

Failure patterns (root-caused)

View Details

• Recurring "push evals results to git" failure — the push_evals_state job's "Push evals results to git" step failed independently in daily-experiment-report (§30798975190), Copilot Session Insights (§30794940466), and one Code Scanning Fixer run (§30795693285). Same job/step name across three unrelated workflows strongly suggests a shared reusable component (evals-state push) hitting a common cause — likely a git push conflict/race (many daily workflows pushing eval state to the same branch around similar cron times) or a permissions/token issue. Worth checking concurrency/locking on whatever branch push_evals_state writes to.
• Separate "Process Safe Outputs" failure — a second Code Scanning Fixer run (§30776690621) failed in the safe_outputs job at "Process Safe Outputs" — a distinct failure mode from the one above, so Code Scanning Fixer has two independent reliability issues, not one.
• Transient infra failure — Archivx — Animated Workflow Visualizer (§30801551618) failed at checkout with HTTP 502 / fatal: could not fetch packfile — a GitHub-side git transport blip, not an application bug. No action needed beyond normal retry.
• PR Sous Chef (§30795752759) was flagged "risky" (not just failed): its actuation posture changed from write_capable to read_only versus its baseline and it failed, while also running "resource heavy for domain" (14 turns vs. baseline's near-zero). Suggests the run silently dropped its intended write action before erroring — worth checking whether it partially executed.
• Daily AW Cross-Repo Compile Check (§30803739616) shows conclusion: null after 29.8m in the run snapshot — possibly still executing or cancelled at query time; flagging as an anomaly rather than a confirmed failure.
• Remaining one-off failures (Daily Syntax Error Quality Check, arXiv Paper Researcher, Daily Elixir Credo Snippet Audit, Tidy, Layout Specification Maintainer, Daily Container Image Security Scan, Documentation Noob Tester, Metrics Collector, Semantic Function Refactoring) each failed once in the sample — not yet a pattern, but worth a second look if they repeat.

Optimization opportunities

View Details

1. Fix the shared push_evals_state step — it's the single highest-leverage fix, touching at least 3 workflows in under 10 hours. Check for git push races/branch contention across concurrently-scheduled daily workflows.
2. Split Code Scanning Fixer's two failure modes — the evals-push and safe-outputs-processing failures need separate root-causing; treating them as one issue will leave one broken.
3. Re-baseline Claude Code usage — its ~4x AIC-per-run premium over Copilot, on similar token volumes, suggests some Claude-engine workflows could move to Copilot/Pi without losing capability, or would benefit from more aggressive prompt caching.
4. Investigate Codex's token volume — 662K tokens/run average (vs 17-35K for other engines) on only 6 runs. Even at low AIC, that's a lot of context being pushed per run; worth checking whether it's necessary or a caching/context-pruning gap.
5. Confirm write-capable workflows are actually writing — 0/117 write actions in the sample is a big number to be a coincidence; spot-check a workflow that's supposed to open issues/PRs/comments to confirm it isn't silently degrading to read-only.

Next actions

• Root-cause push_evals_state / "Push evals results to git" (affects 3+ workflows).
• File the Code Scanning Fixer safe-outputs-processing failure as a separate issue from the evals-push one.
• Spot-check PR Sous Chef's dropped write action.
• No action needed on the Archivx transient 502 — retry-only.

References:

• §30798975190 — daily-experiment-report failure
• §30795693285 — Code Scanning Fixer failure (safe_outputs)
• §30801551618 — Archivx transient checkout failure

Generated by 🔍 Weekly Workflow Analysis · agent · 213.9 AIC · ⌖ 36 AIC · ⊞ 4.2K · ◷

• expires on Aug 4, 2026, 2:51 AM UTC-08:00

[pelikhan]

@copilot investigate and fix push evals state

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-04T10:51:06.561Z.

Closed by Workflow