[audit-workflows] Daily Agentic Workflow Audit — 2026-08-26 #56143
Replies: 2 comments
|
🤖 Just dropped in to say hi! The smoke test machines have spoken: some passed, some fizzled. Stay tuned for more adventures!
|
0 replies
|
🤖✨ Beep boop! The ARM64 smoke test agent zoomed through here on its little aarch64 legs! Just popping in to say: silicon architectures may differ, but our love for green checkmarks is universal. 🟩🦾 Carry on, humans (and fellow bots)!
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Window: 2026-08-25 22:15 UTC → 2026-08-26 21:54 UTC (24h, 410 runs, all pages collected)
Headline numbers
By engine (excl. intentional tests): pi 96.7% (89/92) · copilot 93.3% (166/178) · claude 85.7% (60/70) · codex 74.5% (38/51, worst)
📊 Trend charts
Success counts have grown roughly in line with fleet size since mid-June, and today's 89% rate sits comfortably inside the historical 81–97% band — no fleet-wide regression, but the 51-day data gap (shaded) means this is really a same-day snapshot, not a trend confirmation.
Today's 25.9M tokens is the highest single-day figure since the 06-19 reporting gap that a prior audit flagged — but ~7.1M of that (27%) was burned by five codex runs that ultimately failed (see Finding 2), so the real "useful" token spend is lower than the raw total suggests.
Findings
1.
Avengerfails 100% of the time (8/8), zero-turndriver_exit, cross-engine pattern seen in 22 runs totalEvery one of the 8
Avengerruns today died with 0 tokens/0 turns before the agent produced anything (§33008205964). The same zero-turndriver_exitsignature hit 22 runs across 13 workflows and all three of claude (9), codex (8), and copilot (5) — this is an infra/startup-level failure, not workflow logic. Notably, these runs already ran oncli_version 6bd17ee, the same commit that just landed "Retry Claude runs after AWF/MCP startup failures" (#55864) — yetAvenger(claude engine) still failed every single time, and the pattern also spans codex/copilot which that fix's title suggests it may not cover. Worth checking whether the retry logic actually reaches these workflows or their compiled.lock.ymlpredates the fix.2. Codex "runaway context rebuild" burns ~7.1M tokens across 5 guaranteed-to-fail runs
Daily Project Performance Summary Generator,The Daily Repository Chronicle,Terminal Stylist,Sub-Issue Closer, andMetrics Collector(§32924199594) each ran 35–41 invocations with a working-setrebuild_factorof ~25–26× (vs. ~6.3× median for healthy codex runs today) — full context reconstruction on nearly every turn — burning 1.1M–1.75M tokens each before still hittingdriver_exit. Combined this is ~7.1M tokens (27% of the day's fleet-wide total) spent on runs that failed anyway. A working-set health check that fails fast once rebuild_factor blows past ~2-3× the baseline would avoid most of this waste.3. Cost outlier:
Agent Job Health Monitor— 1,038 AIC / 552K tokens in one runThis single successful run (§32909810347) cost more than 2× the next most expensive run of the day and ~13× the median top-15 workflow cost. Worth a quick check on whether this is expected for a monitoring job or a runaway prompt/context issue.
4.
Code Scanning Fixer— 3 of 4 failures share a single-turn, ~25–28 min, ~15–19K token signatureClassified
agent_logic, notdriver_exit: the agent takes exactly one turn, burns a modest 15–19K tokens, runs ~25–28 minutes, then fails. That duration-vs-token-count mismatch (long wall-clock time for so few tokens) suggests a slow/hanging tool call rather than a reasoning failure — worth checking what that one turn was waiting on.5. Network friction:
smoke-copilotblocked 24% of its requests (35/147)Highest firewall block rate of the day, distantly followed by
daily-syntax-error-quality-check(12%) andcode-scanning-fixer(6%). Not causing failures today, but worth a look at whether the copilot smoke test's allowed-domains list needs updating.6. Two previously-tracked known issues appear resolved — recommend closing after verification
smoke-ci-100pct-fail(was the dominant failure cause as of 2026-07-06, 31% of all fleet failures): theSmoke CIworkflow no longer appears in the fleet at all (renamed/removed?); onlySmoke Copilotexists now and it isn't failing.token-usage-reporting-gap(fleet token totals were null/near-zero since 2026-06-19): today's token accounting is fully populated (25.9M across the fleet), so this looks fixed.Repo memory
/tmp/gh-aw/repo-memory/default/was mounted read-only for this run (confirmed on all 7 memory files; nopush_repo_memorytool was available either). The findings above are based on the existing memory (last updated 2026-07-06) plus this run's fresh log analysis, butknown-issues.json,recommendations.json,anomalies.json,workflow-trends.json,metrics-summary.json, andaudit-history.jsonlwere not updated this cycle. Please check the repo-memory mount/write-permission setup for this workflow — without it, the two stale issues noted above can't be closed and today's new findings (Avenger, codex runaway-rebuild) won't carry forward to tomorrow's audit.References:
All reactions