Agent Performance Report - Week of 2026-08-26

[github-actions[bot]]

Agent Performance Report — Week of 2026-08-26

Executive Summary

• Agents analyzed: 20+ (top by run volume, last ~6h window sampled + shared-memory 30-day trend)
• Total runs sampled: 300 across ~35 workflows
• Stale root causes corrected this run: 5 (see below)
• Top performers: Copilot SWE Agent, PR Sous Chef, Issue Monster, CI/CJS core pipelines
• Needs improvement: Q, Agentic Commands (command-gating — expected, not a defect), CGO, CWI (CI approval-pending), AI Moderator (engine failure — persistent)

Root-Cause Hygiene: Stale Alerts Corrected

Re-verified every "pending fix" citation carried over in shared-alerts.md / agent-performance-latest.md against current PR/issue state. Found 5 stale attributions that no longer match reality and updated them:

Cited fix	Status now	Metric moved?	Action
PR #43527 (Q quality gate)	MERGED 2026-07-05	Q workflow still 12/28 (43%) AR this window	Root cause was misdiagnosed as unmerged; real cause is slash_command activation gating — every issue/PR/discussion comment event starts a run, and only bodies starting with the command proceed. Reclassified as command gating (expected), not a defect.
PR #44254 (CLI hang-on-exit fix for Impeccable/Matt Pocock/PR Code Quality/Test Quality Sentinel)	MERGED 2026-07-08	Yes — all 4 agents now show 3/5 success (was 100% AR)	Fix confirmed effective. Removed from active-issue list; agents no longer bottom-tier.
PR #44276 (Smoke CI EACCES fix)	MERGED 2026-07-09	Not independently re-verified this run (no recent Smoke CI runs in sampled window)	Marked resolved pending next Smoke CI run; kept as low-priority watch item only.
Issue #44241 (AI Moderator no safe outputs)	CLOSED 2026-07-08	No — AI Moderator still 16/31 AR + 1 failure this window	Closed issue did not fix the underlying Codex engine 404; recurrence confirms this is a persistent, unresolved issue. Re-flagging with fresh evidence rather than re-filing the same issue.
Issue #38777 (CGO escalation)	CLOSED 2026-07-20	No — CGO still 12/12 (100%) action_required this window	Same pattern: CI workflow gate, not an agent defect. CGO is a CI workflow (action_required = maintainer approval needed for PR-triggered runs), not agentic AR. Reclassified as CI approval-pending.

"Design Decision Gate" cited previously as a deprecation candidate — the workflow no longer resolves via gh workflow list / gh run list, indicating it has already been removed/deprecated. No further action needed; removed from active tracking.

Performance Rankings

Top Performing Agents 🏆

1. Copilot SWE Agent — 11+ PR merges/day historically, ~73% merge rate. Consistently highest quality/effectiveness in the ecosystem.
2. PR Sous Chef — 17/17 (100%) success this window. Stable triage follow-up agent.
3. Issue Monster — 12/12 (100%) success this window.
4. CI — 6/6 (100%) success — core pipeline healthy.

Recovered Agents ✅ (evidence-verified this run)

• Impeccable Skills Reviewer, Matt Pocock Skills Reviewer, PR Code Quality Reviewer, Test Quality Sentinel — all 4 were 100% action_required under the CLI hang-on-exit bug. Fix PR fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254 merged 2026-07-08. Current sample: 3/5 success for all four. Confirmed effective; no longer bottom-tier.

Agents Needing Improvement 📉

1. AI Moderator — 16/31 action_required + 1 failure (52% AR) this window. Root cause: Codex engine 404, still unresolved despite issue [aw] AI Moderator produced no safe outputs #44241 being closed. Recommend: switch engine to copilot per prior recommendation (never implemented) or re-file with fresh evidence since prior issue closed without fix landing.
2. Q workflow, Agentic Commands — Low raw success ratio is expected behavior: both are slash_command-triggered, so every qualifying issue/PR/discussion event starts a run that the pre_activation/activation job stops unless the body starts with the command. Do not treat as activation-refused defects. Recommend computing success rate over activated runs only going forward (data not separable from gh run list alone — would need per-run job-level activation status).
3. Avenger — 4 failures / 2 success (67% failure) this window — new regression not previously flagged; needs investigation.

CI Approval-Pending (not agentic AR) ⏳

• CGO (12/12 AR), CWI (12/12 AR), CJS (3/3 AR), Doc Build - Deploy (4/5 AR) — these are plain CI workflows gated by GitHub Actions' "approve workflow run" permission check for PR-triggered runs, not agent failures. Fix: org/PR-level approval of Copilot-bot's workflow runs, not a workflow-side change.

Deprecated / Removed

• Design Decision Gate — no longer appears in gh workflow list; treat as already deprecated, remove from future tracking.

Behavioral Patterns

• inconsistency: AI Moderator (52% AR despite closed "fix" issue — recurrence indicates root cause was never actually addressed)
• inconsistency: Avenger (new 67% failure rate, no prior flag — needs root-cause investigation next run)
• No over-creation, under-creation, repetition, or scope-creep detected in this sample window.

Recommendations

High Priority

1. AI Moderator Codex 404 — Re-diagnose and switch engine to copilot; do not re-close the tracking issue until the engine change actually lands, since the previous closure without a fix caused immediate recurrence.
2. Avenger failure regression — New pattern (4 failures/2 success). Needs a dedicated investigation next cycle; insufficient log access this run to root-cause further.
3. Stop miscounting command-gated and CI-approval-gated workflows as agent failures — Update the historical scoring methodology in shared memory so Q/Agentic Commands/CGO/CWI/CJS aren't repeatedly re-surfaced as "bottom agents" when the underlying behavior is by design or requires only a maintainer approval click.

Medium Priority

1. Confirm Smoke CI recovery on next run now that EACCES fix (fix: reclaim root-owned sandbox firewall dirs to prevent EACCES on reused runners #44276) is merged.
2. Verify AI Moderator engine switch lands in a durable config change, not just a closed issue.

Trends

• CLI-hang-on-exit cluster: fully recovered (0% → 100% partial-to-full success across all 4 affected agents) — clear win from PR fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254.
• Root-cause hygiene: 5 stale attributions corrected this run, preventing future false "unresolved" reporting.
• Ecosystem health remains held back primarily by (a) unresolved AI Moderator engine issue and (b) unrecognized command/CI-gating patterns inflating apparent failure counts for Q, Agentic Commands, CGO, CWI.

Actions Taken This Run

• Corrected 5 stale root-cause attributions in shared memory (see Root-Cause Hygiene section).
• Verified CLI hang-on-exit fix (fix: reduce post-completion idle watchdog and add cleanup timeouts to prevent Copilot CLI hang on exit #44254) resolved all 4 affected reviewer agents with fresh run data.
• Flagged AI Moderator and Avenger as still-unresolved/newly-regressed via fresh evidence.
• No new improvement issues filed — no new systemic defect found that isn't already tracked or already resolved.

Next Steps

1. Track Avenger failure regression next run with full log access.
2. Confirm AI Moderator engine switch is actually implemented (not just issue-closed).
3. Update scoring methodology to exclude command-gated/CI-approval-gated runs from "agent failure" tallies.

---

Analysis period: 2026-08-26T07:05Z to 2026-08-26T12:59Z (300-run sample) + shared-memory trend since 2026-07-08
Next report: next scheduled run

Generated by ⚡ Agent Performance Analyzer - Meta-Orchestrator · copilot · auto · 61.9 AIC · ⌖ 13.2 AIC · ⊞ 17.6K · ◷

• expires on Aug 27, 2026, 5:05 AM UTC-08:00