[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-10

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-10

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 811 (81.1%) | Closed: 188 (18.8%) | Open: 1 (0.1%)

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	789	639	81.0%
Feature Addition	100	86	86.0%
Security	85	67	78.8%
Other	14	11	78.6%
Testing	5	4	80.0%
Operational Value Study Template	6	3	50.0%
Documentation	1	1	100.0%

Note: "Bug Fix" dominates this sample because it's a broad keyword-matched bucket (fix/resolve/error/issue), and most Copilot task prompts in this repo describe fixing a specific reported problem.

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 179.7 words
• Most common keywords: github, workflow, run, actions, copilot, agent
• Merged prompts are noticeably shorter and more targeted than closed ones (179.7 vs 319.6 words)

Example successful prompts:

1. PR Cache workflow run discovery payloads in logs JSONL #59988: "Cache workflow run discovery payloads in logs JSONL" → Merged
2. PR Add frontmatter formatting command #59971: "Add frontmatter formatting command" → Merged
3. PR Preserve frontmatter formatting during workflow upgrades #59969: "Preserve frontmatter formatting during workflow upgrades" → Merged

These share a pattern: a single, concrete, scoped change (one file/feature/command), described in the title itself.

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 319.6 words (78% longer than merged prompts)
• Distinctive vocabulary vs merged: operational, value, evidence, study, metric, only — heavily influenced by a recurring "operational value study template" pattern (6 PRs, only 50% merge rate, well below the 81% average)
• Longer prompts often bundle multiple asks, background justification, and success-metric definitions rather than a single clear task

Example unsuccessful prompts:

1. PR Fix external threat detection on ARC/DinD: stage threat-detect binary into AWF chroot #59746: "Fix external threat detection on ARC/DinD: stage threat-detect binary into AWF chroot" → Closed
2. PR Block stale workflow lock files in pull requests #59742: "Block stale workflow lock files in pull requests" → Closed
3. PR Add bounded issue-triage workflow #59741: "Add bounded issue-triage workflow" → Closed

Notably, these closed examples are not vague — they are specific but touch cross-cutting infra/CI concerns (threat detection, chroot staging, lock-file validation) that likely required more back-and-forth review or design discussion before being mergeable.

Key Insights

• Prompt length correlates inversely with merge success: merged PRs average 179.7 words vs 319.6 for closed PRs — nearly 2x longer. Long prompts often signal broad or under-scoped asks that are harder to land cleanly.
• The "operational value study template" pattern is a clear outlier: only 50% merge rate vs the 81.1% overall average, and its vocabulary (evidence, metric, study) suggests these PRs try to prove/justify a change rather than just make one, adding review friction.
• Single, scoped changes with self-descriptive titles merge more reliably: the top merged examples map to one clear file/feature/command change; closed examples more often touch cross-cutting infrastructure (CI, security chroot staging, lock-file mechanics) that likely needs iterative review.

Recommendations

Based on today's analysis:

1. DO: Keep prompts concise and scoped to a single, well-defined change (~150-200 words) — this matches the profile of the most successful merges.
2. DO: State the task as a clear, actionable title/summary up front (e.g., "Add X command", "Cache Y in Z") rather than burying the ask in background narrative.
3. AVOID: "Operational value study template"–style prompts that emphasize proving value/evidence/metrics over describing the concrete code change — these merge at roughly half the average rate.

Historical Trends

Date	PRs	Success Rate	Top Category
2026-09-10	1000	81.1%	bug_fix
2026-09-07	1000	80.4%	bug_fix
2026-09-06	1000	80.6%	feature
2026-09-05	1000	80.1%	bug_fix
2026-09-04	1000	81.3%	bug_fix
2026-09-03	1000	84.4%	bug_fix
2026-09-02	1000	83.9%	bug_fix
2026-09-01	1000	84.1%	feature

Trend: Merge rate has drifted down slightly over the past week (~84% → ~81%), and closed-PR prompt length has grown (from ~206-315 words range historically to 319.6 words today), reinforcing that overly long/justification-heavy prompts are a persistent risk factor. The "operational value study template" category, first seen around 2026-09-04, continues to underperform at ~50% merge rate across observations.

---

Generated by Copilot PR Prompt Analysis (Run: 34527741038)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 22.5 AIC · ⌖ 7.47 AIC · ⊞ 11.2K · ◷

• expires on Sep 11, 2026, 12:47 PM UTC-08:00