[prompt-analysis] Copilot PR Prompt Analysis - 2026-08-03

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-08-03

Summary

Analysis Period: Last 30 days (all 1000 fetched PRs fall within 2026-07-15 to 2026-08-03)
Total PRs: 1000 | Merged: 752 (75.2%) | Closed: 240 (24.0%) | Open: 8 (0.8%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Testing	181	147	32	82.1%
Other	69	54	15	78.3%
Documentation	101	77	23	77.0%
Bug Fix	466	345	120	74.2%
Feature Addition	134	96	35	73.3%
Refactoring	49	33	15	68.8%

(Rate = merged / (merged + closed), excludes still-open PRs)

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 139.8 words (notably shorter than closed PRs)
• Most common non-generic keywords: test, agent, model, engine, added, safe, cli, changes
• Frequent action verbs: add, fix, refactor, update

Example successful prompts:

1. PR Issue Arborist: trim issue-fetch context to cut single-run token spikes #49974: "Issue Arborist: trim issue-fetch context to cut single-run token spikes" → Merged
2. PR Add unit tests for compiler_yaml_prompt.go prompt-chunking logic #49972: "Add unit tests for compiler_yaml_prompt.go prompt-chunking logic" → Merged
3. PR Document PR ruleset patterns for enforcing agentic workflows #49975: "Document PR ruleset patterns for enforcing agentic workflows" → Merged

These share a pattern: they name a specific file, function, or subsystem and describe a single, bounded change.

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 199.8 words (~43% longer than merged prompts)
• Dominant keywords: cve, redacted, deb, debian, securitytracker, gpl, medium, ubuntu, ghsa, lgpl — almost all from automated container/CVE vulnerability-scan reports pasted into the PR body
• [WIP] prefix: 69 PRs had [WIP] in the title; 55 (79.7%) were closed, only 13 (18.8%) merged — by far the single strongest negative signal found
• Vulnerability/CVE-themed PRs overall: 62 total, split ~evenly (32 merged / 30 closed) — much worse than the 75% baseline merge rate

Example unsuccessful prompts:

1. PR [WIP] Fix vulnerabilities and license violations in container image #49852: "[WIP] Fix vulnerabilities and license violations in container image" → Closed
2. PR [WIP] Update mcp/context7 container image to resolve vulnerabilities #49841: "[WIP] Update mcp/context7 container image to resolve vulnerabilities" → Closed
3. PR Add Cursor engine parity with OpenCode + introduce smoke-cursor workflow #50007: "Add Cursor engine parity with OpenCode + introduce smoke-cursor workflow" → Closed (large multi-part scope)

Key Insights

• [WIP] is a red flag: PRs explicitly marked as work-in-progress close ~80% of the time vs. the 25% baseline close rate — nearly 3x more likely to be abandoned/closed than a typical PR.
• Vulnerability/CVE remediation prompts underperform: These often paste raw scanner output (CVE IDs, package/OS names) rather than a clear task description, and merge at roughly a coin-flip rate (~52%) vs. the ~76% average across all completed PRs.
• Conciseness correlates with success: Merged prompts average ~140 words vs. ~200 words for closed ones — bounded, single-purpose asks with a named target (file/function/subsystem) tend to land better than broad or multi-goal requests.
• Testing-focused prompts are the strongest category (82.1% success), likely because "add tests for X" is inherently narrow and low-risk to review/merge.

Recommendations

Based on today's analysis:

1. DO: Reference a specific file, function, or subsystem and scope the change to one concern (e.g., "Add unit tests for X in file Y").
2. DO: Keep prompts concise (~100-150 words) — describe the problem and expected outcome without excessive background.
3. AVOID: Prefixing PR titles with [WIP] — these are far more likely to be closed than merged; if a task is genuinely incomplete, consider breaking it into a smaller finished increment instead.
4. AVOID: Pasting raw vulnerability-scanner/CVE report output as the entire prompt — instead, summarize the specific fix required (e.g., "upgrade package X to resolve CVE-Y") to give the agent a clear, actionable target.

Notes on Data

• Sample size (1000 PRs) is large and statistically meaningful for the category/keyword breakdowns above.
• All fetched PRs fell within the last ~19 days, comfortably inside the 30-day analysis window — no date filtering was needed.
• No historical trend data was found in cache-memory or repo-memory for this analysis name, so this run establishes the baseline; a history.json-style log should be added in a future run for week-over-week comparison.

---

Generated by Copilot PR Prompt Analysis (Run: 30852483304)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · auto · 21 AIC · ⌖ 5.66 AIC · ⊞ 10.7K · ◷

• expires on Aug 4, 2026, 1:02 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #50400.