Agent Performance Report - Week of 2026-08-22

[github-actions[bot]]

Executive Summary

• Agents analyzed: 40 (top by run volume from metrics/latest.json, 24h window 2026-08-21→22)
• Total outputs reviewed: 67 safe outputs (52 issues, 15 PRs, 0 comments, 0 discussions) — ecosystem-wide overall_success_rate: 42.3%
• Data caveat: This run's metrics/latest.json was collected via GitHub API fallback (agentic-workflows logs tool timed out on every attempt). Token/cost data unavailable (null). Safe-output attribution only covers items carrying the gh-aw-workflow-id marker.
• Average quality score (sampled): ~66/100 (weighted toward high-volume workflows)
• Average effectiveness score: ~58/100 — dragged down by three high-volume workflows (q, ci, cgo) with executed success rates under 30%
• Top performers: Running Copilot Code Review (98.9%), pr-sous-chef (97.3%), auto-close-parent-issues (100%), Running Copilot cloud agent (94%)
• Needs improvement: q (0.8% success), ci (23.4%), cgo (29.3%), ai-moderator (47.6% of executed runs, still net-negative), agentic_commands (45.5%, but 550 runs = huge blast radius)

Performance Rankings

Top Performing Agents 🏆 (executed ≥10 runs, success rate ranked)

1. auto-close-parent-issues — 64/64 (100%), avg 8.4s runtime. Extremely efficient, zero failures.
2. Running Copilot Code Review — 87/88 (98.9%), avg 424s. High volume, near-perfect reliability.
3. pr-sous-chef — 73/75 (97.3%), avg 501s. Consistent, low overhead.
4. avenger — 23/24 (95.8%), avg 329s. Stable across the week (per prior memory).
5. design-decision-gate / ponytail-reviewer — 42/44 (95.5%) each — note: contradicts prior memory which flagged Design Decision Gate as "100% AR / deprecation candidate." Current data shows a recovered 95.5% success rate. Recommend re-verifying before further deprecation action — do not treat as failing based on stale memory.
6. Running Copilot cloud agent — 78/83 (94%), avg 800s — main Copilot SWE agent PR-creation pipeline, largest source of merged PRs historically.

Agents Needing Improvement 📉

1. q workflow (Success: 2/261 total runs executed = 0.8%, avg runtime 9.9s)

   • Issues: Near-total failure to execute meaningful work; extremely short avg runtime (9.9s) suggests early exit/gating rather than task failure — consistent with prior memory's report of persistent Activation-Refused (AR) gating tied to unmerged quality-gate PR Add shared prompt quality gate for plateaued agent-review workflows #43527.
   • Correction to prior memory: PR Add shared prompt quality gate for plateaued agent-review workflows #43527 ("Add shared prompt quality gate for plateaued agent-review workflows") was merged on 2026-07-05, over 6 weeks ago. The q workflow is still at 0.8% success today, so the merge did not resolve the gating — the root cause tracked in stale alerts is stale/wrong and needs re-diagnosis, not "wait for merge."
   • Recommendation: Re-open investigation into q workflow trigger/condition logic directly (do not rely on Add shared prompt quality gate for plateaued agent-review workflows #43527 lore). Check its on: gating conditions and recent run logs for the actual skip/failure reason.
   • Action: Issue created (see below).

2. ci (CI workflow) (25/107 executed = 23.4% success, avg 214s)

   • This is a plain CI workflow, not agentic — per classification rules, failures here are build/test failures, not agent activation-refusals. Flagging for Workflow Health Manager coordination rather than an agent-prompt fix.
   • Recommendation: Coordinate with WHM; check for a recent regression in build/test stability (77% of executed runs are failing, which is a real reliability problem regardless of CI/agentic distinction).

3. cgo (55/188 executed = 29.3% success, avg 1751s — longest average runtime of any workflow sampled)

   • Prior memory claimed cgo was "stabilizing" (1/5 AR on Jul 8) — current data shows a regression to 70.7% failure rate among executed runs, and very high runtime (~29 min avg). This may indicate a recurrence of the CI regression flagged in July, or a new issue.
   • Recommendation: Escalate to Workflow Health Manager; the July "stabilizing" note is stale and should not gate re-investigation.

4. ai-moderator (10/21 executed = 47.6%, but total volume 279 with only 21 executed — meaning ~92% of triggers never executute, i.e., gated/skipped, not just failing)

   • Prior memory repeatedly flagged Codex-engine 404 failures and recommended switching to the Copilot engine — this recommendation appears to still be unimplemented given the low executed count and sub-50% success among the runs that do execute.
   • Recommendation: Verify whether the copilot-engine switch was ever applied; if not, this is the single highest-leverage fix carried over from prior runs.

5. agentic_commands (250/550 executed... — actually 550 total, 250 successful, 0 recorded failed, success_rate 0.455, meaning ~300 runs are unaccounted for/skipped) — high blast radius given 550 total invocations/day.

   • Recommendation: Because ~55% of invocations produce no completed outcome, audit whether this is intentional gating (slash-command conditions) or silent failures before assuming quality issue.

Zero-Executed / Gated Workflows (not scored as failures per methodology)

• squad: 423 total, 0 executed (100% skipped/gated) — likely slash-command workflow with narrow trigger conditions (squad-implement-worker similarly 106 total / 0 executed).
• deployment-incident-monitor: 93 total, 0 executed.
• workflow-generator: 8 total, 0 executed.
• These are reported separately as trigger/gating patterns per the scoring methodology, not counted as agent failures. Recommend a follow-up check (out of scope for this run) confirming these gates are intentional rather than broken triggers, given the sustained 0% executed rate across the whole window.

Behavioral Patterns

• q: pattern = under-creation (near-total non-execution, 0.8% success) — persists >6 weeks past the fix PR merge date recorded in memory, indicating the prior root-cause attribution was incorrect.
• cgo: pattern = inconsistency (memory shows oscillation between "escalated 100% AR" → "stabilizing 1/5 AR" → now 70.7% failure again over successive runs).
• ai-moderator: pattern = under-creation (92% of triggers never execute) — consistent with previously-reported Codex engine 404s.
• design-decision-gate: prior memory pattern was scope-creep/full-AR deprecation candidate; current data shows no matching pattern (95.5% success) — flag as a stale alert requiring memory correction, not a live pattern.
• No new duplication/scope-creep/repetition patterns detected in this window's high-volume agents beyond what's already tracked in shared-alerts.md.

Coverage / Ecosystem Notes

• 247 of 287 workflows (86%) were active in this window; total safe outputs (67) skew toward create_issue (52) over create_pull_request (15), with zero add_comment/create_discussion outputs recorded — worth checking whether comment/discussion safe-output types are under-utilized or simply unattributed (per the collection note, PRs without the gh-aw-workflow-id marker aren't attributed).

Recommendations

High Priority

1. Re-diagnose q workflow gating (0.8% success, 261 runs) — the previously-cited fix (PR Add shared prompt quality gate for plateaued agent-review workflows #43527) merged 6+ weeks ago without resolving the issue; treat prior root-cause note as stale.
2. Verify ai-moderator engine switch status — confirm whether Codex→Copilot engine change was actually applied; if not, apply it now.
3. Re-investigate cgo regression (29.3% success, longest avg runtime ~29min) — escalate to Workflow Health Manager, do not rely on "stabilizing" note from prior cycle.

Medium Priority

4. Correct stale memory entry for design-decision-gate — current 95.5% success contradicts the "100% AR / deprecation candidate" label carried in shared-alerts.md; update tracking to avoid wrongly deprecating a now-healthy agent.
5. Audit agentic_commands unaccounted-for runs (~300/550, 55%) to distinguish intentional gating from silent failures.

Low Priority

6. Confirm squad, squad-implement-worker, deployment-incident-monitor, workflow-generator zero-executed patterns are intentional trigger gating, not broken conditions.

Trends (vs. daily snapshots)

• Ecosystem overall_success_rate: 95.0% (Aug 17) → 84.4% (Aug 20) → 83.6% (Aug 21) → 42.3% (Aug 22, today) — a sharp decline, though today's figure reflects the API-fallback collection method noted above and may not be directly comparable; flagging for cross-validation once the logs tool is available again.
• active_workflows: 62 (Aug 17) → 66 (Aug 20) → 154 (Aug 21) → 247 (Aug 22) — rising activity volume.
• total_safe_outputs: 368 → 319 → 718 → 67 — large swing, likely reflecting the attribution-coverage caveat in today's collection note rather than a true output collapse.

Actions Taken This Run

• Filed 1 new issue for q workflow re-diagnosis (stale root-cause correction).
• Updated agent-performance-latest.md and shared-alerts.md in shared memory with corrected findings (see Do-Not-Re-File list below — do not re-file ai-moderator, cgo, design-decision-gate per existing IDs, but do note the design-decision-gate status correction).

Next Steps

1. Workflow Health Manager to prioritize cgo/ci reliability investigation.
2. Confirm whether ai-moderator's Codex→Copilot engine switch (recommended repeatedly since July) has shipped; escalate if not.
3. Correct design-decision-gate classification in shared memory to reflect recovery.
4. Next metrics collection run should validate whether today's success-rate drop is a real regression or a collection-method artifact (API fallback vs. logs tool).

---

Analysis period: 2026-08-21T02:51Z to 2026-08-22T02:51Z (24h window) + 30-day trend snapshots (Aug 17-22)
Next report: 2026-08-29

Generated by ⚡ Agent Performance Analyzer - Meta-Orchestrator · auto · 55.8 AIC · ⌖ 18.1 AIC · ⊞ 17K · ◷

• expires on Aug 23, 2026, 5:04 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-23T13:04:31.775Z.

Closed by Workflow