Agent Performance Report — Week of 2026-05-28

[github-actions[bot]]

Executive Summary

• Agents analyzed: 16 workflow groups (~236 total workflows)
• Overall quality score: 74/100 (↔ plateau — 5th consecutive flat week)
• Effectiveness score: 72/100 (↔ stable)
• Ecosystem health: 82/100 (↓8 from 90 — more failures today)
• Compilation rate: 100% (236/236) ✅
• Top performers: copilot-swe-agent, spec-enforcer/extractor, Agentic Commands
• Needs immediate attention: safe-outputs validation cluster ([aw-failures] Failure Investigator (6h) — 4 failures, safe_outputs add_comment validation breaks 3 workflows (2026-05-27 19:24 → 2026-05-28 01 [Content truncated due to length] #35351), Copilot CLI cluster ([aw] Copilot CLI Deep Research Agent failed #35388), failure-reporters duplication

Critical Issues

Priority	Issue	Affected Workflows	Status
🔴 P0	safe_outputs add_comment validation (#35351)	PR Sous Chef, Contribution Check, Sub-Issue Closer	Ongoing — 3 workflows fully blocked
🔴 P1	Copilot CLI execution failure (#35388)	Daily News, Deep Research Agent, Issues Report Generator	0% success for 5+ days — platform-level
🔴 P1	failure-reporters 60% duplicate rate	aw-failure-investigator cluster	~20 issues/day, 60% dupes
🟠 P1	LintMonster backlog overwhelm (#35368, #35370)	LintMonster	2417 issue epic, resource/timeout failures
🟡 P2	Silent-skip cluster	Q, Deployment Monitor, CJS, Label Closed PRs	0–33% success, 0 failure logs — NEW: #aw_silentskip
🟡 P2	Daily Safe Output Tool Optimizer runaway (#35316)	Safe Output Tool Optimizer	115 turns / 14.9M tokens

Performance Rankings

Top Performing Agents 🏆

1. copilot-swe-agent (Quality: 88/100, Effectiveness: 84/100)

   • 67% PR merge rate (↑6pp vs prior week); 6 merges May 27
   • Active today: remove-yield-feature PR in progress
   • No behavioral issues detected
   • Reference implementation for the ecosystem

2. spec-enforcer/extractor (Quality: 82/100, Effectiveness: 80/100)

   • 100% success rate, 100% PR merge rate
   • Clean behavioral profile — no patterns detected

3. Agentic Commands (Quality: 78/100, Effectiveness: 76/100)

   • 80% pass rate across 10 runs — most stable PR-triggered workflow
   • No failures, no behavioral issues

4. Content Moderation (Quality: 76/100, Effectiveness: 74/100)

   • 75% success rate across 4 runs
   • Clean behavioral profile

Agents Needing Improvement 📉

1. failure-reporters cluster (Quality: 55/100, Effectiveness: 45/100)

   • Pattern: over-creation + repetition
   • 20 issues/day with 60% duplicate rate = 12 redundant issues/day of noise
   • Pollutes health signals for all other meta-orchestrators
   • Recommendation: Implement deduplication gate (check for existing open issues before filing); throttle to max 5 new issues/category/day

2. LintMonster (Quality: 65/100, Effectiveness: 50/100)

   • Pattern: over-creation + inconsistency
   • Carrying 2417-issue backlog ([lint-monster] [Lint] Break up long functions in pkg/workflow/ (2417 issues) #35368 epic); resource/timeout failures ([aw] LintMonster failed #35370)
   • Recommendation: Shard into bounded batches (max 50 issues per run); add timeout ceiling; pause new issue creation until backlog clears below 500

3. Daily Safe Output Tool Optimizer (Quality: 60/100, Effectiveness: 40/100)

   • Pattern: over-creation + inconsistency
   • 115 turns / 14.9M tokens in one run = runaway reasoning loop ([aw] Daily Safe Output Tool Optimizer failed #35316)
   • Recommendation: Add max-turn budget (30 turns) + early-exit guard on rate-limit detection

4. CGO (Quality: 50/100, Effectiveness: 30/100)

   • Pattern: inconsistency
   • 11% success rate across 9 runs post-regression ([CGO] Workflow failure on main - Run #8016 #35028); stabilizing but not recovered
   • Recommendation: Monitor — add post-build smoke test gate; no campaign assignments until >80% success

Silent-Skip Cluster ⚠️

• Q: 0% success, 8 runs, 0 failure logs — always cancelled/skipped
• Deployment Incident Monitor: 0% success, 8 runs, 0 failure logs
• CJS: 29% success, 7 runs, 0 failures
• Label Closed PRs: 33% success, 6 runs, 0 failures

Tracking issue: #aw_silentskip

Behavioral Pattern Analysis

Pattern Detection Results (via pattern-detector agent)

Agent	Patterns	Risk
safe-outputs cluster	under-creation	🔴 Blocked by P0
Copilot CLI cluster	under-creation	🔴 Blocked by P1
failure-reporters	over-creation repetition	🔴 P1
LintMonster	over-creation inconsistency	🟠 P1
Daily Safe Output Tool Optimizer	over-creation inconsistency	🟡 P2
CGO	inconsistency	🟡 P2
CJS	under-creation inconsistency	🟡 P2
Label Closed PRs	under-creation inconsistency	🟡 P2
Q	under-creation	🟡 P2
Deployment Incident Monitor	under-creation	🟡 P2
PR Description Updater	inconsistency	🟡 P2
Agentic Maintenance	under-creation	🟡 Blocked by P0 — will self-resolve
copilot-swe-agent	(none)	🟢 Healthy
spec-enforcer/extractor	(none)	🟢 Healthy
Agentic Commands	(none)	🟢 Healthy
Content Moderation	(none)	🟢 Healthy

Ecosystem-Level Observations

• 86/92 recent issues created by bots — automation dominates; deduplication is critical infrastructure
• cookie label: 33 issues — high concentration may indicate a single agent creating topically redundant issues; cross-reference with failure-reporters cluster
• P0 [aw-failures] Failure Investigator (6h) — 4 failures, safe_outputs add_comment validation breaks 3 workflows (2026-05-27 19:24 → 2026-05-28 01 [Content truncated due to length] #35351 creates cascade: one systemic failure blocking 3+ independent agents; fixing it unblocks Agentic Maintenance, PR Sous Chef, Contribution Check simultaneously
• Copilot CLI failure is platform-level — no workflow-level fix possible; requires infra escalation

Effectiveness & Resource Analysis

Task Completion Rates

• High (>75%): copilot-swe-agent, spec-enforcer, Agentic Commands, Content Moderation (4 agents)
• Medium (40-75%): PR Description Updater, Agentic Maintenance, LintMonster (3 agents)
• Low (<40%): failure-reporters (effectiveness), CGO, Daily Safe Output Optimizer, Silent-skip cluster (7 agents)

Resource Efficiency Highlights

• Most efficient: Agentic Commands (38s avg), Content Moderation (13s avg)
• Least efficient: Daily Safe Output Tool Optimizer (115 turns / 14.9M tokens — runaway)
• copilot-swe-agent: Good efficiency for PR-quality output delivered

PR Merge Statistics

• High merge rate (>75%): spec-enforcer/extractor (100%), copilot-swe-agent (67%)
• Medium (50-75%): PR Description Updater (est.)
• Low (<50%): No data on other agents creating PRs in this window

Recommendations

High Priority

1. Fix safe_outputs validation P0 ([aw-failures] Failure Investigator (6h) — 4 failures, safe_outputs add_comment validation breaks 3 workflows (2026-05-27 19:24 → 2026-05-28 01 [Content truncated due to length] #35351)

   • Unblocks 3 workflows simultaneously
   • Root cause: agents omitting item_number when target: "*" configured
   • Estimated impact: restore 3+ workflows, improve ecosystem health by ~5-8 points

2. Implement failure-reporters deduplication gate

   • Add check for existing open issues before filing new ones
   • Throttle to max 5 new issues/category/day
   • Estimated impact: reduce issue noise by ~60%, improve health signal quality

3. Resolve Copilot CLI platform failure ([aw] Copilot CLI Deep Research Agent failed #35388)

   • Escalate to infra/platform team — not fixable at workflow level
   • 3 workflows at 0% success for 5+ days

Medium Priority

4. Shard LintMonster into bounded batches (max 50 issues/run)
5. Add max-turn budget to Daily Safe Output Tool Optimizer (30 turns + rate-limit early-exit)
6. Audit silent-skip cluster trigger conditions — see #aw_silentskip

Low Priority

7. Investigate cookie label concentration (33 issues) — possible single-agent over-creation
8. CGO smoke test gate — add post-build validation to catch regressions earlier
9. PR Description Updater variance investigation — determine if tied to safe-outputs P0

Trends

Metric	This Week	Last Week	Direction
Quality score	74/100	74/100	↔ Plateau (5th week)
Effectiveness score	72/100	72/100	↔ Stable
Ecosystem health	82/100	90/100	↓8 (more failures today)
Compilation rate	100%	100%	✅ Stable
copilot-swe-agent merge rate	67%	61%	↑6pp
CGO success rate	~11%	<5%	↑ Stabilizing
Failure-reporter dupe rate	60%	60%	↔ Unresolved

Key trend signal: Quality has plateaued at 74/100 for 5 consecutive weeks. The ceiling is being held by the systemic P0/P1 blockers. Resolving #35351 and the failure-reporter duplication problem are the two highest-ROI interventions to break the plateau.

Actions Taken This Run

• ✅ Filed improvement issue for silent-skip cluster audit (#aw_silentskip)
• ✅ Updated agent-performance-latest.md in shared memory
• ✅ Updated shared-alerts.md with new P2 finding
• ✅ Pattern-detector run: 16 agent profiles classified

Next Steps

1. Fix P0 [aw-failures] Failure Investigator (6h) — 4 failures, safe_outputs add_comment validation breaks 3 workflows (2026-05-27 19:24 → 2026-05-28 01 [Content truncated due to length] #35351 — highest ROI, unblocks 3+ workflows
2. Implement failure-reporter deduplication gate
3. Escalate Copilot CLI failure ([aw] Copilot CLI Deep Research Agent failed #35388) to platform team
4. Investigate silent-skip cluster (#aw_silentskip) trigger conditions
5. Monitor CGO success rate — target >80% before resuming campaign assignments
6. Break quality plateau: once P0/P1 resolved, target 80/100 quality score

---

Analysis period: 2026-05-21 to 2026-05-28
Next report: 2026-06-04
References: §26579184217 | §26557376092 | §26515287616

Generated by ⚡ Agent Performance Analyzer - Meta-Orchestrator · sonnet46 3M · ◷

• expires on May 29, 2026, 2:07 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-29T14:07:56.777Z.

Closed by Workflow