[prompt-analysis] Copilot PR Prompt Analysis - 2026-10-22

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-10-22

Summary

Analysis Period: Oct 10–22, 2026 (dataset capped at 200 PRs)
Total PRs: 200 | Merged: 165 (82.5%) | Closed: 29 (14.5%) | Open: 6 (3%)

Merge rate among completed PRs: 85.1%. No PRs had empty bodies.

Prompt Categories and Success Rates

(Categorized by title keywords; completed PRs only)

Category	Total	Merged	Success Rate
Bug Fix	26	22	84.6%
Feature Addition	34	29	85.3%
Refactoring	5	4	80.0%
Documentation	12	9	75.0%
Testing	2	1	50.0% (n too small)
Other	115	100	87.0%

Prompt Analysis

✅ Successful Prompt Patterns

• Average body length: 241 words (merged)
• Common title keywords: model (32), copilot (25), output (13), routing (12), agent (12), smoke (11), engine (9)
• Verbs: stabilize, disable, stage, fix – narrowly scoped, concrete behavior changes

Examples:

1. PR Disable dry-run telemetry using documented environment opt-outs #71032: Disable dry-run telemetry using documented environment opt-outs → Merged
2. PR Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70945: Codex: stage native TOML sub-agents with resolved models and fail closed before startup → Merged
3. PR Stabilize smoke-copilot-sub-agents: mandatory delegation, experiment-independent contract checks #70944: Stabilize smoke-copilot-sub-agents: mandatory delegation... → Merged

❌ Unsuccessful Prompt Patterns

• Average body length: 211 words
• Common title keywords: support, context, native, workflow, daily, artifact, routing, model (each only 2–3 occurrences; no strong signal)
• Tendency toward broad, cross-cutting capability asks

Examples:

1. PR Support team-configurable default models #70349: Support team-configurable default models → Closed
2. PR Generate GitHub-native workflow lockfiles during compilation #70345: Generate GitHub-native workflow lockfiles during compilation → Closed
3. PR Use artifact-backed credit accounting for daily code metrics #69988: Use artifact-backed credit accounting for daily code metrics → Closed

Key Insights

• Scope matters more than category: Category success rates cluster at 80–87%; closed PRs skew toward broad "Support ..."/"Generate ..." capabilities rather than targeted behavior fixes.
• Length buckets: <100 words 80.5% (62/77), 100–200 words 85.2% (52/61), 200–400 words 94.1% (16/17), 400+ words 89.7% (35/39). Very short prompts underperform slightly.
• Documentation is the weakest sizable category (75%, n=12); testing sample too small to conclude.

Recommendations

1. DO: Give 100+ words with concrete acceptance criteria and named components.
2. DO: Scope to one observable behavior (e.g. "disable X using Y").
3. AVOID: Broad "Support ..." asks without defined boundaries; split into smaller PRs.

Historical Trends

Date	PRs (merged+closed)	Success Rate	Note
2026-10-22	194	85.1%	200-PR data cap
2026-10-21	627	83.9%	earlier runs used larger window
2026-10-20	614	83.5%

Trend: Success rate steady at ~84–85%. Note: today's sample is smaller (200 vs ~630), so counts are not directly comparable with prior days.

---

Generated by Copilot PR Prompt Analysis (Run: 39851190356)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · ~15 AIC · ⌖ ~1 AIC · ⊞ 11.6K · ◷

• expires on Oct 23, 2026, 12:48 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #71494.