Repository navigation
[agent-job-health] Agent Job Health: 2026-11-16 (partial coverage): 27 agent failures, 4.0% run-weighted #78480
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary
total_runs= 673 completed run summaries (176 with agent job skipped, 5 with no agent record).distinct_workflows= not enumerated.agent_failures= 27. Agent jobs that executed (success or failure) = 492.Warning
12 of 27 agent failures are one startup-phase incident. Twelve agent jobs failed within about 80 seconds (2026-11-16 03:39–03:40Z), across six PR-triggered workflows, before the agent started. Tracked status is unverified. If it is not already tracked, this cluster meets the issue threshold (at least two workflows).
Failure Rate by Step
Agent-job failures, n = 27.
Tracked Failures
Not verifiable in this run. Every search and list call returned only integrity-filtered results (for example, "10 item(s) removed", and "100 of 100 removed" on the open-issue list). Chronic-offender check, from runs in the window:
Novel Failure Clusters
Treat these as candidates. Tracked status could not be checked.
1. Startup-phase harness failure burst (12 runs, 6 workflows, 3 engines). Created 03:39–03:40Z on 2026-11-16. All are
pull_requestruns. Each of six workflows failed twice: PR Code Quality Reviewer, PR Specification Reviewer, Matt Pocock Skills Reviewer, Test Quality Sentinel (copilot, 8 runs), Design Decision Gate (pi, 2), Ponytail Reviewer (codex, 2).not_started.2. In-run Copilot CLI failures outside the burst (5 runs, 5 workflows). Schedule-triggered: Daily Compiler Threat Spec Optimizer (43792391517), Copilot Opt (43920983850), Contribution Check (43953953360), Daily BYOK Ollama Test (43968997797, model
qwen2.5:0.5b), and Avenger (43907862008, whose startup attempt exited 78 but whose harness retry then succeeded, yet the step still failed). Four had execution statestarted. No error excerpts. This is heterogeneous and is not yet one cluster.3. Single-run failures, one per engine or step.
max_turns. Daily Grader Audit (43827278079), startup exit 124 (timeout). Daily Safe Outputs Git Simulator (43804325532), agent not started, failing at "Isolate simulator memory and read-only ledger signals".invalid_request. Daily Spending Forecast (43842620432), failed at "Install AWF binary", agent not started.View all 27 failing agent runs
invalid_requestmax_turnsSchedule Heartbeat
Partial. No blind spots were determined. This does not mean none exist.
list_workflows; more pages may exist). Eleven workflows aredisabled_manually:auto-close-parent-issues.yml,chroma-issue-indexer.lock.yml,daily-geo-optimizer.lock.yml,daily-hippo-learn.lock.yml,error-message-lint.yml,integration-agentics.yml,label-closed-prs.yml,link-check.yml,pr-description-caveman.lock.yml,security-scan.yml,slide-deck-maintainer.lock.yml. None of them is a schedule-triggered agentic.mdworkflow, so none explains a schedule gap..github/workflows/*.md, not all verified as frontmatter). Per-workflow last-run lookups were not run in this pass. This needs its own pass.Coverage and Data Quality
before_run_id43810512828,end_date2026-11-16T05:28:42Z) ran and failed with "context deadline exceeded". Retrying with the continuation'sprune_older_runs: truewould prune the oldest in-window runs at 1011 of 1024 MB, so I did not retry.summary.jsonstill reports 421 runs. Every figure here comes from per-runrun_summary.json,aw_info.json, andusage/agent/*.json, and the failure clusters in that file are pre-backfill.Recommendations
All reactions