[audit-workflows] Daily Fleet Audit — 2026-08-14: 89.0% success, Smoke CI zero-runs anomaly, Avenger still chronic

[github-actions[bot]]

Daily Fleet Audit — 2026-08-14

First audit in 39 days (prior audit: 2026-07-06). 300 runs observed in the 24h window (2026-08-13T21:24Z–2026-08-14T21:01Z).

Metric	Value
Runs	300
Success / Failure	267 / 33
Raw success rate	89.0%
Success rate excl. 2 intentional-failure guardrails	89.23% (265/297)
Tokens	19.66M
AI credits (AIC)	13,350.08
Missing tools / data / MCP failures	0 / 0 / 0

Excluded from the adjusted rate: Daily Credit Limit Test (daily-credit-limit-test) and Daily Max Ai Credits Test (daily-max-ai-credits-test) — both intentionally fail as guardrail canaries.

The fleet is healthy overall, but the biggest story this window is an absence, not a failure: Smoke CI recorded zero runs.

Charts

Workflow Health (last 30 days, with the 39-day gap marked)

Success rate has been in the high-80s/low-90s across every observed window since mid-June, with one trough (~60%) on 2026-07-02 tied to a since-resolved pi-engine incident. Today's 89.0% sits right in that historical band, and total run volume (300) is consistent with prior windows (171–400/day) — nothing here suggests fleet-wide drift during the unmonitored gap.

Token Usage (last 30 days, with the 39-day gap marked)

Today's 19.66M tokens is well within the range seen in mid-June (18M–54M/day). More notably, token_usage is now populated for 272/300 (90.7%) runs — the token-usage-reporting-gap issue that left this metric null/near-zero from 2026-06-19 through 2026-07-06 looks resolved; recommend confirming durability over one more window before closing it.

Top finding: Smoke CI went from chronic 100%-fail to zero runs

Smoke CI was the single dominant fleet failure cause for 6 straight audit windows (2026-06-30 → 2026-07-06), responsible for up to 71% of all fleet failures via a 100%-fail-on-push signature at "Execute GitHub Copilot CLI". In this window it had zero runs, despite several qualifying commits landing on main that touch cmd/**, pkg/**, and *.go paths that should trigger it. This is either the best possible outcome (bug fixed, workflow just wasn't exercised) or a regression that silently broke the trigger — a silently-broken canary is worse than a known-red one, since it removes visibility into copilot-engine startup health. Filed as smoke-ci-silent-non-trigger; recommend directly checking .github/workflows/smoke-ci.lock.yml's trigger config and compile status rather than inferring from run absence.

Recurring issues, still open

• Avenger (avenger-err-config-no-structured-logs) — failed 2/2 today (§31777889016, §31797105737), same err-config/no-structured-logs signature first seen 2026-06-13. >60 days chronic, unfixed.
• model-param-config-incompatibility — Daily Caveman Optimizer failed again today (§31839684241), same family unfixed since 2026-06-05.
• [aw] Failure Investigator (6h) (claude-agent-job-fail-longrun) — failed twice today, same longrun-0turn signature.
• Daily Fact (daily-fact-codex-gpt54-0turn) — 0-turn fail again, low-volume but persistent since 06-30.

Candidate-resolved (clean this window, watch one more cycle before closing)

• chroot-node-not-available — Daily Issues Report Generator ran clean.
• codex-gh-aw-binary-not-found-for-mcp — Daily Cache Strategy Analyzer succeeded (§31829332338, 26min, 3.83M tokens) after being ~100%-fail for 2+ weeks.
• pi-gpt54-0tok-agentjob-fail — Auto-Triage Issues 4/4=100% success, 2nd consecutive clean window.
• code-scanning-fixer-0tok-agentjob-fail — Code Scanning Fixer 4/4=100% success.

Other context: audit-process gap and infra migration (click to expand)

Audit persistence gap: this audit's own repo-memory shows no recorded runs between 2026-07-06 and 2026-08-14 (39 days). Either the audit workflow didn't run, or it ran but failed to persist findings, for over a month — a blind spot during which several infra changes landed unmonitored by this process. Filed as audit-workflow-persistence-gap; recommend confirming this workflow's own schedule/repo-memory push path is now reliable.

Infra migration context: the repo is mid-migration from Firecracker microVMs to Cloud Hypervisor for its agent sandbox runtime (3 related commits landed today, culminating in Firecracker removal ~19:13Z). Failures today are spread evenly across the 24h window with no step-change correlation around any of these commits, so this migration is flagged only as something to watch, not asserted as a root cause of anything above.

Recommendations filed this cycle

1. HIGH — Confirm whether Smoke CI's trigger is broken or the underlying bug is fixed (rec-smoke-ci-confirm-trigger-or-fix).
2. P1 (60+ days chronic) — Fix or disable Avenger's err-config/no-structured-logs path (rec-avenger-fix-err-config).
3. MEDIUM — Add a self-check so this audit workflow raises missing_data if it fails to persist repo-memory two days running (rec-audit-persistence-gap-monitor).
4. Close out the 4 candidate-resolved issues above after one more clean window, and close rec-2026-07-06-token-reporting if token_usage reporting stays populated.

Full evidence, recurrence counts, and per-workflow trend data are persisted to repo-memory (known-issues.json, recommendations.json, anomalies.json, workflow-trends.json, metrics-summary.json, audit-history.jsonl).

References:

• §31777889016 — Avenger failure (err-config)
• §31839684241 — Daily Caveman Optimizer failure (model-param-config)
• §31829332338 — Daily Cache Strategy Analyzer success (candidate-resolved)

Generated by 🔍 Agentic Workflow Audit Agent · agent · 406.4 AIC · ⌖ 6.08 AIC · ⊞ 8.3K · ◷

• expires on Aug 15, 2026, 1:46 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #52970.