[prompt-analysis] Copilot PR Prompt Analysis - 2026-06-11

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis — 2026-06-11

Analysis Period: Last 30 days | Repository: github/gh-aw

Summary

Metric	Value
Total PRs Analyzed	1,000
✅ Merged	786 (79.2%)
❌ Closed (not merged)	206 (20.8%)
🔄 Open	8
Empty Descriptions	2

Week-over-week trend: Stable (~79-80% success rate across last 4 days)

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate	Trend
⬆️ Upgrade / Dependency	33	29	88%	🟢 Top performer
📄 Documentation	58	49	84%	🟢
🐛 Bug Fix	270	221	83%	🟢
🧪 Testing	170	140	83%	🟢
🔧 Chore / CI	15	12	80%	🟡
♻️ Refactor	56	45	80%	🟡
🆕 Feature	262	206	79%	🟡
🚧 WIP	55	20	37%	🔴 Major outlier

Prompt Length vs Success Rate

Word Count	PRs	Merged	Success Rate
< 50 words	28	21	75%
50 – 99 words	154	118	77%
100 – 199 words	597	482	81% 🎯 Sweet spot
200 – 299 words	163	133	82% 🎯 Sweet spot
300 – 499 words	40	30	75%
500+ words	10	2	⚠️ 20%

Structural Features

Feature	Merged PRs	Closed PRs
Contains code block (```)	85%	76% (+9pp advantage)
References specific files	74%	75% (neutral)
Checklist (WIP-style - [ ])	1%	10% (negative signal)
Average word count	156 words	181 words

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 156 words (concise but complete)
• Include code blocks with inline diffs or examples
• Use conventional commit types in title (fix:, feat:, refactor:)
• Reference specific files/modules with backtick formatting
• Describe the root cause, not just the symptom

Example successful prompts:

1. PR feat(harness): defer initial awf-reflect models probe during OIDC api-proxy startup #38718 — feat(harness): defer initial awf-reflect models probe during OIDC api-proxy startup

   "awf-reflect was probing api-proxy models endpoints immediately at runner startup, which races OIDC token exchange and emits misleading early 503 warnings despite successful runs...`
   → ✅ Merged — Clear root cause, specific component names, actionable fix

2. PR fix: run daily AIC guardrail for label and slash command triggers #38705 — fix: run daily AIC guardrail for label and slash command triggers

   "shouldSkipDailyAICGuardrail unconditionally skipped the guardrail for any workflow_dispatch with a non-empty aw_context...`
   → ✅ Merged — Specific function name, concrete problem statement, inline code

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 181 words (over-explained or unfocused)
• 10x more likely to include checklists (WIP-style task lists)
• WIP/placeholder PRs: only 37% success rate
• Over-broad scope (e.g., multi-system refactors without clear boundaries)

Example prompts that led to closure:

1. PR [WIP] Explore agent persona findings and recommendations #38683 — [WIP] Explore agent persona findings and recommendations

   "- [x] Update .github/aw/test-coverage.md..." — Checklist-only body with no concrete fix
   → ❌ Closed — WIP prefix + checklist body = high closure risk

2. PR Unify Antigravity and Gemini log response parsing #38713 — Unify Antigravity and Gemini log response parsing

   "Typist flagged AntigravityResponse and GeminiResponse as structurally identical..."
   → ❌ Closed — Technically valid but likely superceded or out-of-scope

Key Insights

• 🚨 WIP PRs are a trap: PRs with [WIP] in the title or checklist-only bodies have a 37% success rate — half the overall average. These create queue noise without deliverables.
• 📏 The 100–299 word sweet spot: Prompts in this range have the highest success rates (81-82%). Very short (<50) and very long (500+) descriptions both underperform significantly.
• 💻 Code blocks signal quality: PRs with inline code examples or diffs succeed at 81% vs 71% for those without — a 10 percentage point advantage.

Recommendations

1. ✅ DO: Include a code block (diff, example, or snippet) in every PR description
2. ✅ DO: Target 100–299 words — enough to explain root cause and changes, not more
3. ✅ DO: Use conventional commit types (fix:, feat:, docs:) for faster triage
4. ✅ DO: Name specific files, functions, or components affected
5. 🚫 AVOID: Opening [WIP] or placeholder PRs — use noop/report_incomplete instead
6. 🚫 AVOID: Descriptions over 500 words — they correlate with 20% success rate

Historical Trends (7-day)

Date	PRs	Merged	Closed	Success Rate
2026-06-08	1,000	794	199	80.0%
2026-06-09	1,000	787	203	79.5%
2026-06-10	1,000	793	203	79.6%
2026-06-11	1,000	786	206	79.2%

Trend: Success rate is stable at ~79-80% over the past week. The WIP PR count (55) is a slight concern — prior guardrail work has kept this from growing further.

View Category Keyword Matching Rules

Category	Keywords
Bug Fix	fix, bug, resolve, correct, patch, repair, regress, flak
Feature	add, implement, create, feat, introduce, support
Refactor	refactor, improve, optimize, clean, simplify, consolidate, unify
Documentation	doc, readme, comment, guide, changelog
Testing	test, spec, coverage
Upgrade	upgrade, update, bump, migrate, depend
Chore	chore, ci, lint, format
WIP	wip, work in progress

References: §27378381223

Generated by 🔍 Copilot PR Prompt Pattern Analysis · ⌖ 20 AIC · ⊞ 20K · ◷

• expires on Jun 12, 2026, 1:34 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #38931.