[Outcome Report] Outcome Report — 2026-05-26: 54.5% acceptance rate, 🔴 regression from 100%

[github-actions]

Outcome Scorecard — 2026-05-26

Metric	Value	Status
Acceptance rate	54.5%	🔴 <60%
Zero-touch rate	0%	🔴 0%
Waste rate	20%	🟡 10-25%
Median time to resolution	343 sec (5.7 min)	—
Accepted	6 / 25	—
Rejected	5	—
Pending	14	—
Runs checked	12	—

🔴 Action Items

1. Chaos PR Bundle Fuzzer — Critical — 50% rejection rate (5 rejected, 5 pending). This workflow's output quality needs immediate review. Recommend auditing the prompt and testing with smaller sample before re-enabling at scale.

2. Stuck pending items — 3 items pending >12 hours:

   • PR [instructions] Sync inline skills instructions with v0.74.8 #34905 (Instructions Janitor) — pending 14,512 sec
   • PR [chaos-test] [chaos/r66] mutation-mapper: minor-rename+two-commits #34907-34911 (Chaos PR Bundle Fuzzer) — pending 14,340-14,356 sec
   • Comments on PR feat(setup): add toolcache resolver for Copilot CLI behind feature flag #34918 (Smoke CI) — pending 3,352 sec
   • Add labels on issue (AI Moderator) — pending 13,459 sec (no URL)

   Recommend setting timeout policies for pending outcomes.

3. Missing URLs — 2 pending items with no URL (Contribution Check, AI Moderator). These may be silently failing. Investigate error handling.

4. Zero-touch rate at 0% — All 6 accepted items required human review/edits. This suggests the workflows are not generating production-ready output. Consider whether these should be rejected instead.

Per-Workflow Breakdown

Workflow	Accepted	Rejected	Pending	Acceptance	Waste Rate
Chaos PR Bundle Fuzzer	0	5	5	0%	50%
Contribution Check	0	0	3	0%	0%
Smoke CI	0	0	4	0%	0%
Instructions Janitor	0	0	1	0%	0%
AI Moderator	0	0	1	0%	0%
Agent Performance Analyzer	2	0	0	100%	0%
Claude Code User Documentation Review	1	0	0	100%	0%
Terminal Stylist	1	0	0	100%	0%
Workflow Normalizer	1	0	0	100%	0%
Dev	1	0	0	100%	0%

Trend Analysis

Comparing to 2026-05-25:

Metric	2026-05-25	2026-05-26	Change
Acceptance Rate	100%	54.5%	⬇️ -45.5pp
Zero-touch Rate	0%	0%	➡️ Stable
Waste Rate	0%	20%	⬇️ +20pp
Runs Checked	14	12	-2 runs
Total Outcomes	29	25	-4 outcomes

⬇️ Regressing — Acceptance rate dropped 45.5 percentage points, primarily due to Chaos PR Bundle Fuzzer's 50% rejection rate. This is the single largest factor driving today's degradation.

Rejected Items

All 5 rejections came from Chaos PR Bundle Fuzzer (run #26451165321):

• PR [chaos-test] phase-shifter: amend+two-commits (run 67) #34922 — closed (16 changed, 628 additions, 560 deletions)
• PR [chaos-test] code-cartographer: minor-rename+staged-subset (run 67) #34923 — closed (2 changed, 5 additions)
• PR [chaos-test] eol-sculptor: line-ending-variant+amend (run 67) #34924 — closed (1 changed, 3 additions)
• PR [chaos-test] branch-alchemist: two-commits+staged-subset (run 67) #34925 — closed (2 changed, 5 additions)
• PR [chaos-test] graph-gardener: octopus-merge (run 67) #34926 — closed (16 changed, 628 additions, 560 deletions)

All were auto-closed by the system, suggesting policy violations or test failures.

Reaction Summary & Recommendations

No positive or negative reactions recorded on any outcomes. This may indicate:

• Items not yet reviewed by team
• Reactions not being captured in evaluation
• Low visibility of the outcomes

Recommendations

1. Immediate: Pause or reduce Chaos PR Bundle Fuzzer runs pending prompt audit
2. Short-term: Add URL validation and error handling for missing-URL outcomes (Contribution Check, AI Moderator)
3. Medium-term: Investigate why zero-touch rate is 0% — consider whether accepting-but-edited items should be classified differently
4. Ongoing: Set TTL policies for pending outcomes to flag stuck items automatically

📊 Measured by Outcome Collector · haiku45 39K

• expires on Jun 2, 2026, 1:58 PM UTC