[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-30

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-30

Summary

Analysis Period: Last 30 days
Total PRs: 560 | Merged: 431 (77.0%) | Closed: 125 (22.3%) | Open: 4 (0.7%)

Merge rate among completed PRs: 77.5%. No PRs had an empty body.

Prompt Categories and Success Rates

Category	Total (completed)	Merged	Success Rate
Bug Fix	221	186	84.2%
Feature Addition	136	87	64.0%
Refactoring	15	8	53.3%
Documentation	35	24	68.6%
Testing	47	39	83.0%
Other	102	87	85.3%

Categories come from keyword matching on the title and the first 300 characters of the body, so they are approximate.

Prompt Analysis

✅ Successful Prompt Patterns

• Average prompt length (merged): 185 words
• Bug-fix, test and small maintenance prompts merge most often (83–85%).
• PR bodies with no file-path references merged 299/345 (86.7%).

Example merged PRs:

1. PR Remove dynamic eval from update-release tests #64502: Remove dynamic eval from update-release tests → Merged
2. PR Recognize consequence-based empty-catch rationale #64501: Recognize consequence-based empty-catch rationale → Merged
3. PR Update stale CLI and scanner image pins #64500: Update stale CLI and scanner image pins → Merged

❌ Unsuccessful Prompt Patterns

• Average prompt length (closed): 364 words, about twice that of merged PRs.
• Feature (36% closed) and refactor (47% closed) work fail most often.
• PR bodies that reference file paths: 132 merged vs 79 closed (62.6% merged).

Example closed PRs:

1. PR Restore PureLock shell utilities to PATH #64497: Restore PureLock shell utilities to PATH → Closed
2. PR Disable credential persistence for safe-output checkouts #64496: Disable credential persistence for safe-output checkouts → Closed
3. PR Support AWF task-level model routing in the Copilot engine #64403: Support AWF task-level model routing in the Copilot engine → Closed

Key Insights

• Length: closed PRs have roughly 2x the prompt length of merged ones, which suggests broad or multi-part tasks.
• Scope: feature and refactor prompts are the weakest categories (64% and 53%). Bug fixes and tests are the strongest.
• File references: prompts that name files have a lower merge rate (62.6%) than those that don't (86.7%). This is likely confounded, because file-heavy prompts tend to be larger, more complex changes. It is not evidence that naming files hurts.

Recommendations

1. DO: Keep each prompt to one narrow, testable change, ideally under about 200 words.
2. DO: Split features and refactors into small incremental PRs with explicit acceptance criteria.
3. AVOID: Bundling several concerns (security, engine and compiler changes) into one task.

Historical Trends

Date	PRs	Merge Rate
2026-09-30	560	77.5%
Earlier (Aug–Sep 2026)	~1000	~84%

Trend: Merge rate is lower than the ~84% seen earlier in the month, and the dataset is smaller (560 vs ~1000 PRs). The drop is driven by feature and refactor prompts.

---

Generated by Copilot PR Prompt Analysis (Run: 36774617409)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 14.7 AIC · ⌖ 7.9 AIC · ⊞ 10.8K · ◷

• expires on Oct 1, 2026, 12:49 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #64842.