[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-07

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-07

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 804 (80.4%) | Closed: 189 (18.9%) | Open: 7 (0.7%)

Note: dataset is capped at 1000 PRs (the max returned by the pre-fetch query), so true 30-day volume may be higher; all rates below are relative to this sample.

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	738	618	83.7%
Feature Addition	160	108	67.5%
Security / CVE	66	50	75.8%
Testing	11	9	81.8%
Documentation	4	3	75.0%
Refactoring	1	1	100%
Other	20	15	75.0%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 180 words
• Most common non-stopword terms: github, workflow, sous/chef (feature codenames), actions, copilot, runs, agent, model, engine, fixes
• Prompts tend to open with a concrete, narrow problem statement naming the specific command/feature affected (e.g. "logs --max-storage previously stopped downloads...", "mcpg v0.4.16 rejects the provisional gateway.delegationControllers...")

Example successful prompts:

1. PR Enforce logs cache budgets with selective pruning #59049: "logs --max-storage previously stopped downloads once the cache reached its limit without exposing..." → Merged
2. PR Wire dynamic enclave delegation controller into the workflow runtime #59046: "mcpg v0.4.16 rejects the provisional gateway.delegationControllers stdin config the compiler emitt..." → Merged
3. PR Cache GitHub Actions job metadata in logs output #59039: "The logs command discarded GitHub's jobs API response, limiting cached run data to a reduced project..." → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 309 words (~1.7x longer than merged prompts)
• Most common terms: github, operational, workflow, value, copilot, agent, evidence, study, metric, graders
• A recurring cluster is an "operational value study" template (repeated boilerplate framing about metrics/evidence/graders rather than a concrete code change): 6 PRs this period, only 3 merged (50% rate, well below the 80.4% baseline)
• Broad/aspirational feature requests underperform (67.5%) relative to targeted bug fixes (83.7%)

Example unsuccessful prompts:

1. PR Enable strict mode for all workflow compilations #59246: "Enforce strict validation across all repository gh aw compile invocations. ## Changes - Enable r..." → Closed
2. PR Prune logs cache before storage sizing #59167: "Multi-target logs requests could evaluate --max-storage before date-based cache pruning complete..." → Closed
3. PR Update package guidance for gh-aw v0.88.5 wildcard includes #59124: "Package manifests can use gh-aw v0.88.5's trailing wildcard support to avoid enumerating installable..." → Closed

Key Insights

• Prompt length inversely correlates with success: merged prompts average 180 words vs. 309 for closed prompts — concise, single-issue prompts consistently outperform long multi-part ones.
• Bug-fix framing beats feature framing: narrow "X is broken/inconsistent when Y" prompts hit 83.7% merge rate, while broader "add/implement" feature requests drop to 67.5% — likely because feature scope is more prone to review friction or superseding changes.
• Template-driven "study"/"evidence" prompts underperform sharply: the recurring operational-value-study boilerplate pattern merges at only 50%, half the overall baseline, suggesting generic analytical framing without a concrete code change is a weak prompt strategy.
• Security/CVE-flavored prompts merge at a below-average 75.8%, likely reflecting extra review scrutiny — still notably better than the closed-PR template pattern.

Recommendations

1. DO: Keep prompts short (~150-200 words) and centered on one specific command/module/behavior, ideally citing the exact flag, file, or error observed.
2. DO: Frame requests as concrete bug fixes ("X does Y when it should do Z") rather than broad feature asks — this correlates with materially higher merge rates.
3. AVOID: Repeating the "operational value study" / metrics-evidence-graders boilerplate template without a concrete, actionable code change — this pattern has a 50% merge rate, the weakest of any category tracked.

Historical Trends

Date	PRs	Merge Rate	Top Category (by volume)
2026-09-07	1000	80.4%	bug_fix
2026-09-06	1000	81.2%	feature
2026-09-05	1000	80.1%	bug_fix
2026-09-04	1000	81.3%	bug_fix
2026-09-03	1000	84.4%	bug_fix
2026-09-02	1000	83.9%	bug_fix
2026-09-01	1000	84.1%	feature

Trend: Overall merge rate has slipped from ~84% (Sep 1-3) to ~80-81% over the last several days, a modest but consistent decline. The "operational value study" template pattern first appeared on 2026-09-04 with a very low 1.8-5.1% merge rate in prior days; today it recovered somewhat to 50% but remains well below baseline, and average prompt length for closed PRs has grown steadily (233 → 316 → 309 words over the last three reports), reinforcing the length/specificity insight above.

---

Generated by Copilot PR Prompt Analysis (Run: 34160256699)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 26 AIC · ⌖ 6.9 AIC · ⊞ 11K · ◷

• expires on Sep 8, 2026, 12:46 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #59534.