[prompt-analysis] Copilot PR Prompt Analysis - 2026-06-12

[github-actions[bot]]

Summary

Analysis Period: Last 30 days
Total PRs: 1,000 | Merged: 786 (79%) | Closed: 206 (21%) | Open: 8

Overall merge success rate (completed PRs): 79.2% — consistent with the rolling weekly average of 79%.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate	Avg Prompt Length
📄 Docs	12	11	1	91%	131 words
🧪 Testing	40	33	6	84%	146 words
✨ Feature	123	99	24	80%	167 words
🔄 Update	15	12	3	80%	146 words
🐛 Bug Fix	781	611	163	78%	205 words
🔧 Refactor	26	20	6	76%	163 words
❓ Uncategorized	3	0	3	0%	119 words

Bug fix prompts dominate at 78% of all PRs. Documentation and testing prompts show highest success rates but represent a small fraction of total volume.

Prompt Length Analysis

Merged PRs average 189 words vs closed PRs at 215 words — longer prompts have a slight negative correlation with success.

View Prompt Length Distribution

Length Bucket	Merged	Closed	Closed%
0–50 words	14	5	26%
51–100 words	75	27	26%
101–200 words	414	100	19% ✅
201–400 words	257	56	18% ✅
400+ words	26	18	41% ⚠️

The 101–400 word range shows the best outcomes. Very long prompts (400+) have a 41% close rate — nearly double the overall average.

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 189 words (moderate, well-scoped)
• Strong use of specific file/function references (e.g., pkg/workflow/, ctxbackground)
• Root cause explanation with supporting evidence (error messages, test failures, static analysis output)
• Clear single-responsibility: one issue → one fix
• Common action verbs: fix, resolve, add, implement, propagate, cap, emit

View Example Successful Prompts

1. PR Add missing pull-requests: read scope to Designer Drift Audit workflow #38855 — "Add missing pull-requests: read scope to Designer Drift Audit workflow"

   Static analysis flagged permissions drift in generated workflows: some jobs use GitHub capabilities requiring token scopes not declared in source fron...
   → Merged ✅ — References specific tool output, names the exact missing scope

2. PR ctxbackground: resolve Background() calls by package identity, not identifier text #38854 — "ctxbackground: resolve Background() calls by package identity, not identifier text"

   `ctxbackground` mixed two detection strategies: `context.Context` parameters were identified by type identity, while `context.Background()` calls were...
   → Merged ✅ — References exact code path, explains the inconsistency precisely

3. PR Cap Code Simplifier runaways with hard per-run budgets and graceful noop exit #38851 — "Cap Code Simplifier runaways with hard per-run budgets and graceful noop exit"

   A scheduled Code Simplifier run spun to 244 turns (12.3M tokens / 4,219 AIC), consumed most of the daily budget, and failed without a structured out...
   → Merged ✅ — Quantified problem (244 turns, 12.3M tokens), clear fix goal

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 215 words (verbose but less focused)
• Multi-instance bulk changes across many files (e.g., "120 instances", "three workflows")
• Vague optimization goals without measurable success criteria
• Common in: refactor PRs (76% success), multi-file chores

View Example Unsuccessful Prompts

1. PR chore: Extract hard-coded file paths to constants (120 instances) #38774 — "chore: Extract hard-coded file paths to constants (120 instances)"

   120+ inline path string literals were scattered across pkg/workflow/, pkg/parser/, and pkg/cli/, flagged by the custom hardcodedfilepath linte...
   → Closed ❌ — Large blast radius (120 instances, 3 packages) increases risk of conflicts/regressions

2. PR fix: ambient context optimization for mattpocock, daily-code-metrics, and test-quality-sentinel workflows #38721 — "fix: ambient context optimization for mattpocock, daily-code-metrics, and test-quality-sentinel workflows"

   Three workflows had measurable token waste identified by the Daily Ambient Context Optimizer: Matt Pocock had ~33 avoidable data-gathering turns...
   → Closed ❌ — Multi-workflow changes in one PR; scope too broad

3. PR Unify Antigravity and Gemini log response parsing #38713 — "Unify Antigravity and Gemini log response parsing"

   Typist flagged `AntigravityResponse` and `GeminiResponse` as structurally identical duplicates...
   → Closed ❌ — Consolidation prompts may encounter resistance when merging sensitive parsing paths

Key Insights

• Scope matters more than length: The 41% close rate for 400+ word prompts likely reflects overly broad changes, not over-explanation. Prompts that scope to a single component succeed at ~80%+.
• Quantify the problem: The most consistently merged prompts include specific metrics (token counts, test failures, specific line numbers), making the fix verifiable.
• Docs and tests are near-certain merges (84–91% success): These categories have the clearest success criteria. Invest in framing other PRs with similar clarity.
• Slight downtrend: Success rate dipped from 81% (mid-May) to 79% over the past two weeks — worth monitoring.

Recommendations

1. DO: Reference specific files, functions, or identifiers in the prompt — e.g., pkg/workflow/compiler.go:142 or the exact token/scope name
2. DO: Include a quantified trigger (test failure message, lint output, metric exceeded) to establish measurable success criteria
3. DO: Scope to a single logical unit — one bug, one file, one workflow; split large refactors into multiple focused PRs
4. AVOID: Prompts that span 3+ files or 100+ instances — high blast-radius changes have double the close rate
5. AVOID: Purely organizational/chore prompts without a functional improvement angle — the "uncategorized" group has 0% success

Historical Trends (Last 7 Runs)

Date	Total PRs	Merged	Success Rate
2026-06-12 (today)	1,000	786	79%
2026-06-10	1,000	793	79%
2026-06-09	1,000	787	79%
2026-06-08	1,000	794	79%
2026-06-07	1,000	791	79%
2026-06-06	1,000	788	79%
2026-06-02	1,000	809	80%

Trend: ⚠️ Slight decline — success rate has stabilized at 79% after a high of 81% in mid-May. The mix shift toward more bug-fix PRs (now 78% of volume) vs. higher-success categories (docs, tests) is likely contributing.

References:

• §27443328679

Generated by 🔍 Copilot PR Prompt Pattern Analysis · 380.8 AIC · ⌖ 55 AIC · ⊞ 20K · ◷

• expires on Jun 13, 2026, 1:23 PM UTC-08:00

[github-actions[bot]]

Smoke bonk. Browser good. Build good. Cave smile.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 350.6 AIC · ⌖ 16.6 AIC · ⊞ 20.3K · ◷

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-06-13T21:23:07.627Z.

Closed by Workflow