[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-09

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-09

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 805 (80.5%) | Closed: 188 (18.8%) | Open: 7 (0.7%)

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Testing	55	51	92.7%
Documentation	36	33	91.7%
Bug Fix	443	366	82.6%
Refactoring	21	17	81.0%
Other	84	71	84.5%
Security	69	53	76.8%
Feature Addition	292	214	73.3%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 179.9 words (much shorter than closed PRs)
• Most common keywords: github, workflow, actions, copilot, agent, model, engine, fixes
• Categories with highest success: testing (92.7%), documentation (91.7%)

Example successful prompts:

1. PR Add AI Credits pricing for Gemini 3.8 Flash #59748: "Add AI Credits pricing for Gemini 3.8 Flash" → Merged
2. PR Request PR read access for no-fallback branch pushes #59739: "Request PR read access for no-fallback branch pushes" → Merged
3. PR Allow GitHub domains in Daily Go Test Parallelizer #59736: "Allow GitHub domains in Daily Go Test Parallelizer" → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 318.4 words (77% longer than merged PRs)
• Distinctive keywords: operational, value, evidence, study, metric, null, blocked — these terms are strongly overrepresented in closed PRs and rare in merged ones
• Issues identified: verbose/templated prompts, PRs tagged [WIP], and PRs generated from an "operational value study" grading template

Example unsuccessful prompts:

1. PR Block stale workflow lock files in pull requests #59742: "Block stale workflow lock files in pull requests" → Closed
2. PR Add bounded issue-triage workflow #59741: "Add bounded issue-triage workflow" → Closed
3. PR [WIP] Fix CLI version checker configuration issue #59687: "[WIP] Fix CLI version checker configuration issue" → Closed

Special pattern — "operational value study" template: 59 PRs matched an "operational-value"/"evidence + study" prompt template. Only 3 merged (5.1% success rate) vs. 80.5% baseline. This template consistently underperforms across multiple days of historical data (1.8%–5.1% merge rate in the last week alone) and should be avoided or redesigned.

Key Insights

• Pattern 1: Prompts averaging ~180 words succeed far more often than verbose ones averaging ~320 words — conciseness and specificity outperform lengthy, template-heavy descriptions.
• Pattern 2: The "operational value study" / grading-template prompt pattern (keywords: operational, value, evidence, study, metric) has a persistently poor merge rate (~2-5%) across the last several days of data — this is a systemic failure pattern, not noise.
• Pattern 3: Testing and documentation prompts have the highest success rates (>90%), likely because scope is naturally narrow and well-defined, while feature-addition prompts (73.3%) and security prompts (76.8%) lag, suggesting broader or higher-risk scope correlates with more revisions/closures.

Recommendations

1. DO: Keep prompts concise and scoped — target well under 200 words with a specific, narrow ask (mirrors testing/documentation success rates).
2. DO: Reference specific files, error messages, or reproduction steps for bug-fix prompts to keep the 82.6%+ success rate.
3. AVOID: Templated "operational value study"/evidence-grading prompts — this pattern has a consistently low (~2-5%) merge rate and should be discontinued or redesigned before further use.
4. AVOID: [WIP] prefixed or clearly unfinished-scope prompts, which correlate with closures.

Historical Trends

Date	PRs	Merge Rate	Top Category
2026-09-09	1000	80.5%	bug_fix
2026-09-08	1000	81.1%	bug_fix
2026-09-07	1000	80.4%	bug_fix
2026-09-06	1000	81.2%	feature
2026-09-05	1000	80.1%	bug_fix

Trend: Merge rate has been stable in the 80-81% range over the past week. The "operational value study" template pattern first appeared around 2026-09-04 and has consistently shown a very low success rate (1.8%-5.1%) in every subsequent day it appears, confirming it as a persistent anti-pattern rather than a one-off anomaly.

---

Generated by Copilot PR Prompt Analysis (Run: 34402474471)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 25.1 AIC · ⌖ 9.75 AIC · ⊞ 11.2K · ◷

• expires on Sep 10, 2026, 12:49 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #60050.