[prompt-clustering] Copilot Agent Prompt Clustering — Daily Analysis (2026-06-16)

[github-actions[bot]]

🧩 Copilot Agent Prompt Clustering — Daily Analysis

Period: last 30 days (2026-05-27 → 06-16) · Repo: github/gh-aw · PRs: 1000 (100% full data)

Summary

• Clusters: 8 (K-means / TF-IDF, k by silhouette=0.0365) · Overall merge rate: 79% (791 merged · 205 closed · 4 open)
• Iteration proxy: commits/PR — Copilot-agent PRs don't map 1:1 to gh-aw runs, so turn-counts aren't joined.

Key findings

1. ~62% of work sits in two clusters: generic tests/schema/docs (C2, 39%) and workflow/prompt engineering (C4, 23%). The agent is mostly used for in-repo maintenance and for building more workflows.
2. Merge rate is flat (73–83%) across themes — task type barely predicts outcome; nothing is failing badly.
3. Weakest clusters are the most intricate subsystems: safe-outputs/MCP (C3, 73%) and SDK/harness (C6, 74%). Best: observability (C7, 83%) and AI-credit (C1, 82%).
4. Effort ≠ size. Dependency bumps (C0) touch ~151 files but need the fewest commits (3.1, mechanical); the SDK cluster (C6) has small diffs yet the most iterations (5.3 commits, 5.4 comments) — genuinely hard.

Clusters

C	Theme	PRs	%	Merge%	Commits	Files	Comments	Top terms
C2	Tests, schema & docs	392	39	79	3.8	39	3.7	fix, test, schema, add, docs
C4	Workflow & prompt eng.	230	23	82	3.7	16	2.0	workflow, prompt, experiment, workflows, daily
C7	Observability & metrics	101	10	83	3.5	42	3.8	aic, forecast, job, conclusion, artifact
C1	AI-credit & budget	91	9	79	4.0	55	4.0	ai, credits, ai credits, et, guardrail
C3	Safe-outputs & MCP	74	7	73	4.0	26	3.5	safe, safe output, safe outputs, outputs, output
C6	Copilot SDK & harness	61	6	74	5.3	30	5.4	sdk, copilot, copilot sdk, driver, sdk driver
C0	Dependency bumps	30	3	77	3.1	151	7.6	bump, v0, mcp, mcp server, firewall
C5	WIP: fix failing CI	21	2	76	2.6	17	0.8	fix failing, actions job, failing github, actions, github actions

Representative PRs per cluster

• C2 Tests, schema & docs: #37236 · #36823 · #36664
• C4 Workflow & prompt eng.: #39439 · #36531 · #35596
• C7 Observability & metrics: #39490 · #37463 · #37367
• C1 AI-credit & budget: #39123 · #37589 · #37387
• C3 Safe-outputs & MCP: #39484 · #37229 · #37197
• C6 Copilot SDK & harness: #35936 · #36953 · #37161
• C0 Dependency bumps: #37584 · #35973 · #35782
• C5 WIP: fix failing CI: #38397 · #38265 · #35539

Largest-diff outliers: #36006 (+1.6M, 5272 files) · #36065 · #38369

Recommendations

1. Enrich prompts for C3 (safe-outputs/MCP) & C6 (SDK/harness) — lowest merge rates, most comment-heavy; supply interface contracts & expected MCP tool shapes up front.
2. Keep routing dependency bumps (C0) to the agent — 77% merge at ~3 commits despite 151-file diffs: high-leverage, low-risk.
3. Audit the C2 catch-all (392 PRs, 84 closed) — largest absolute count of closed PRs; tighter task scoping yields the biggest aggregate win.
4. The 21 WIP-CI-fix PRs (C5) get ~0.8 comments — add a lightweight auto-verify gate before they consume merge attention.

Methodology: TF-IDF (1–2 grams, 600 feats, min_df=3) over 3×-weighted title + cleaned body; firewall/code blocks stripped; clusters validated by manual term/example review. Low silhouette is expected for sparse text.

References: §27614318724

Generated by 📊 Copilot Agent Prompt Clustering Analysis · 168.7 AIC · ⌖ 17.9 AIC · ⊞ 13.3K · ◷

• expires on Jun 17, 2026, 4:03 AM UTC-08:00