[prompt-clustering] Prompt Clustering Analysis - 2026-08-14

[github-actions[bot]]

Summary

Analysis Period: Last 30 days (2026-07-15 to 2026-08-14)
Total Tasks Analyzed: 1,041 Copilot-agent PRs
Clusters Identified: 9
Overall Merge Rate: 73.0% (760 / 1,041)

Full Analysis Report

General Insights

• Most common task type: Cluster 1 — ambient/workflow-prompt engineering and agent-behavior stabilization (336 PRs, 32% of all tasks).
• Highest success rate: Cluster 2 — safe-outputs/security-hardening and CI-infra fixes, 91.5% merged.
• Most reviewer-intensive tasks: Cluster 7 (engine harness: watchdogs, versioning) and Cluster 2 average 5.5–5.7 reviews per PR, roughly 2–5x the overall average — these are also two of the highest-merging clusters, suggesting review iteration here converges rather than stalls.
• Outliers: 2 clusters fall ≥10 points below the overall merge rate at ≥15 PRs — Cluster 4 (container/CVE pinning, 67 PRs, 50.7%) and Cluster 0 (docs SEO/llms.txt, 16 PRs, 25.0%).
• Limitation: Workflow-run turn/cost metrics (Phase 2) were not merged in. These 1,041 PRs are authored by the external GitHub Copilot coding agent from issues, not by gh-aw's own agentic workflow runs, so gh-aw's logs command (which reports turns/cost for its own workflow executions) does not map cleanly to per-PR agent effort at this scale. Analysis below is based on PR content, outcome, and reviewer/comment activity only.

Cluster Analysis

Cluster 1: Workflow & agent-prompt engineering

• Size: 336 tasks (32.3% of total)
• Merge rate: 72.3% (near baseline)
• Avg reviews / comments: 1.6 / 3.0 — Avg files changed: 26.4
• Top keywords: md, copilot, github, agent, daily, lock, model, prompt
• Characteristics: The largest bucket — tuning daily/scheduled workflow specs, sub-agent routing, ambient prompt bloat, lock-file regeneration.
• Example PRs: No gh-aw change: Anthropic proxy fix requires gh-aw-firewall #52671, Add gh-aw-detection: true to 8 audit/report/detector workflows #52662, Fix Go Logger safe-output accounting and failure diagnostics #52661, Clarify gh aw init docs for Copilot-only agent/MCP artifacts #52581, Stage behavior-defined engine CLIs into the docker-sbx visible path #52576

Cluster 5: Testing, validation & MCP/safe-outputs infra

• Size: 333 tasks (32.0%)
• Merge rate: 74.2%
• Avg reviews / comments: 2.8 / 4.8 — Avg files changed: 16.1
• Top keywords: cjs, test, error, coverage, tests, path
• Characteristics: Second-largest cluster — Go/JS test coverage, error propagation, MCP script fixes, conformance suites.
• Example PRs: refactor: introduce named type LLMProvider string to replace bare string threading #48485, Restore safe-outputs MCP startup by staging data_schema_normalizer.cjs #48496, Strengthen actionpins spec tests for cache keys, fallback auditing, and container digest validation #48497

Cluster 7: Engine harness stability (Copilot/Codex/Gemini)

• Size: 112 tasks (10.8%)
• Merge rate: 82.1%
• Avg reviews / comments: 5.5 / 15.3 (highest comment volume of any cluster) — Avg files changed: 45.2
• Top keywords: em, sub, aic, run, chef, sous ("sous" = "PR Sous Chef" workflow; "aic" = ambient-context abbreviation)
• Characteristics: Watchdog/hang fixes, engine version pinning, template-injection hardening — high engagement but converges to merge.
• Example PRs: fix: add post-result watchdog to codex_harness to prevent CLI hang-on-exit #48499, Retry Copilot startup no-output watchdog failures in scheduled/push runs #48514, feat: honor engine.version for copilot; add version tests for copilot and gemini #48519

Cluster 3: Go refactors & linters

• Size: 87 tasks (8.4%)
• Merge rate: 73.6% (near baseline)
• Avg reviews / comments: 3.0 / 5.8 — Avg files changed: 17.7
• Top keywords: pkg, pkg cli, cli, helpers, sites, linters
• Characteristics: Mechanical code-quality passes — strings.Builder conversions, context threading, custom linters.
• Example PRs: Reduce EnforceSafeUpdate parameter count with options struct #52614, Guard git command arguments against flag injection (Sighthound findings) #52401, Use static GraphQL query constants in project_command.go #52378

Cluster 4: Container/CVE & digest-pin fixes (outlier)

• Size: 67 tasks (6.4%)
• Merge rate: 50.7% — 22.3 points below overall
• Avg reviews / comments: 1.1 / 2.8 — Avg files changed: 34.4
• Top keywords: image, container, mcp, digest, pin, alpine
• Characteristics: Vulnerability-scan-triggered PRs to repin/rebuild container images (firewall, mcpg, github-mcp-server, misc MCP servers). See root-cause below.
• Example PRs: [WIP] Fix vulnerabilities in container image gh-aw-mcpg #52049, Mitigate github-mcp-server container findings by repinning image and allowlisting Debian base licenses #51980, [WIP] Fix vulnerabilities in api-proxy container image #51333, [WIP] Fix vulnerabilities in container image ghcr.io/github/github-mcp-server:v1.8.0 #51332, [WIP] Update container image to address vulnerabilities #51155

Cluster 6: Squad/persona config exploration

• Size: 27 tasks (2.6%)
• Merge rate: 81.5%
• Avg reviews / comments: 1.8 / 4.0 — Avg files changed: 39.1
• Top keywords: created, pull request, ai, agent
• Characteristics: Scaffolding-style tasks analyzing/configuring squad files, frontmatter, and shared-workflow integration.
• Example PRs: Analyzing agentic_maintenance.yml generation issues #50107, Configuring frontmatter for bundled artifact folders #50108, Checking squad files and skills presence for agent #50124

Cluster 8: CI firefighting ("fix failing job X")

• Size: 16 tasks (1.5%)
• Merge rate: 68.8%
• Avg reviews / comments: 0.8 / 1.0 — Avg files changed: 1.2 (very small, targeted diffs)
• Top keywords: job, actions, thanks, make progress
• Characteristics: Auto-filed "fix failing GitHub Actions job" tasks, one per failing check — small and mechanical.
• Example PRs: [WIP] Fix failing GitHub Actions job lint-go #48975, [WIP] Fix failing GitHub Actions job Integration: CMD Tests #49110, [WIP] Fix failing GitHub Actions job Unit tests (A-C) #49132

Cluster 2: Safe-outputs & security-hardening

• Size: 47 tasks (4.5%)
• Merge rate: 91.5% (best-performing cluster)
• Avg reviews / comments: 5.7 / 11.4 (heaviest review load, but converges best)
• Top keywords: details, threat, detection engine, detection, summary
• Characteristics: Threat-detection banners, egress validation, git-config hardening, regression tests for create_pull_request — high scrutiny, high success.
• Example PRs: Enable structured threat-detection run logs #51091, [review] Migrate compact logs run tables from tabwriter to console.RenderTable #50870, [review] Recognize deferred named recover helpers in goroutinemissingrecover #50867

Cluster 0: Docs SEO / llms.txt & robots.txt (outlier)

• Size: 16 tasks (1.5%)
• Merge rate: 25.0% — 48.0 points below overall, the largest gap of any cluster
• Avg reviews / comments: 0.3 / 0.9 — most PRs got essentially no reviewer engagement
• Top keywords: txt, robots, docs, llms, geo, site
• Characteristics: Repeated, largely duplicate attempts (many tagged [WIP]) to add AI-crawler robots.txt/llms.txt files for GEO (generative-engine-optimization). Multiple near-identical PRs opened across the period.
• Example PRs: [WIP] Add AI-crawler robots.txt allow rules and llms.txt for the docs site #52115, [WIP] Create /llms.txt for the docs site #51616, [WIP] Add robots.txt to docs site allowing AI crawlers #51419, [WIP] Add robots.txt with explicit AI bot allow rules to docs site #51139, Fix docs site domain: github.github.io, not github.github.com #50630

Success Rate by Cluster

Cluster	Tasks	Merge Rate	Avg Reviews	Avg Comments	Top Keywords
2	47	91.5%	5.7	11.4	details, threat, detection engine
7	112	82.1%	5.5	15.3	em, sub, aic, sous chef
6	27	81.5%	1.8	4.0	created, pull request, ai, agent
5	333	74.2%	2.8	4.8	cjs, test, error, coverage
3	87	73.6%	3.0	5.8	pkg, pkg cli, linters
1	336	72.3%	1.6	3.0	md, copilot, agent, daily
8	16	68.8%	0.8	1.0	job, actions
4	67	50.7%	1.1	2.8	image, container, digest, pin
0	16	25.0%	0.3	0.9	txt, robots, llms, geo

Sample of Analyzed PRs

Representative sample (3 per cluster, 27 of 1,041 total — full set not reproduced here)

PR #	Title	Cluster	Outcome	Comments	Reviews
48691	[WIP] Add llms.txt for docs site for improved GEO score	0	Closed	2	0
48697	feat(docs): add robots.txt with AI-bot Allow rules for docs site	0	Closed	1	0
48951	[WIP] Add robots.txt and /llms.txt to the docs site	0	Closed	0	0
48483	Reduce ambient prompt bloat in daily workflow specs	1	Closed	1	3
48502	Strengthen persona workflow guidance for path scoping	1	Merged	1	2
48504	Stabilize Daily Agent of the Day sub-agent model routing	1	Closed	4	0
48905	feat: wrap skill install step summary in progressive disclosure	2	Merged	8	6
48940	fix: use [!WARNING] banner for threat detection engine failures	2	Merged	6	4
49059	fix: clear all git config scopes before overriding extraheader	2	Merged	14	1
48484	refactor: replace string += in loops with strings.Builder (41 findings)	3	Closed	0	0
48486	refactor: replace duplicate experimentStateJSON/experimentRunRecord	3	Merged	5	1
48487	fix: propagate download errors through StartDockerImageDownload	3	Merged	2	1
48498	[WIP] Fix vulnerabilities in ghcr.io/github/gh-aw-firewall/api-proxy	4	Closed	0	0
48521	fix: update brace-expansion to 5.0.8 and upgrade Alpine base image	4	Merged	3	1
48534	Remove vulnerable gh-aw-firewall/squid:0.27.41 pin	4	Closed	0	0
48485	refactor: introduce named type LLMProvider string	5	Merged	5	1
48496	Restore safe-outputs MCP startup by staging data_schema_normalizer.cjs	5	Merged	1	1
48497	Strengthen actionpins spec tests for cache keys, fallback auditing	5	Merged	6	5
50107	Analyzing agentic_maintenance.yml generation issues	6	Merged	15	5
50108	Configuring frontmatter for bundled artifact folders	6	Merged	10	5
50124	Checking squad files and skills presence for agent	6	Closed	0	0
48499	fix: add post-result watchdog to codex_harness to prevent CLI hang-on-exit	7	Merged	10	13
48514	Retry Copilot startup no-output watchdog failures in scheduled/push runs	7	Merged	9	4
48519	feat: honor engine.version for copilot; add version tests	7	Merged	10	7
48975	[WIP] Fix failing GitHub Actions job lint-go	8	Merged	1	1
49110	[WIP] Fix failing GitHub Actions job Integration: CMD Tests	8	Merged	1	1
49132	[WIP] Fix failing GitHub Actions job Unit tests (A-C)	8	Merged	1	1

Full 1,041-row dataset was clustered and used for all statistics above; only this 27-row sample is reproduced in the table for readability.

Key Findings

1. Two-thirds of all tasks are workflow/prompt engineering or test/infra work (clusters 1 + 5 = 669 of 1,041, 64%), both merging near the overall baseline (72–74%) — this is the routine, well-understood workload for the agent.
2. Heavier review load correlates with higher merge rate, not lower. Clusters 2 and 7 have the most reviews/comments per PR (5.5–5.7 reviews, 11–15 comments) yet the two best merge rates (91.5%, 82.1%). Iteration here looks like normal convergence, not friction.
3. The two outlier clusters share a common failure mode: low engagement, not bad code. Cluster 0 averages only 0.3 reviews/PR and Cluster 4 only 1.1 — most unmerged PRs in both clusters were closed with zero comments and zero reviews, indicating they were superseded/abandoned rather than rejected on technical merit.

Lowest-Merge Cluster Root Cause

• Cluster: 0 — Docs SEO / llms.txt & robots.txt
• Merge Rate Gap: 25.0% vs 73.0% overall (gap: 48.0 points) — the single largest gap of any cluster
• Primary Blocker: Scope/complexity mismatch
• Confidence: high
• Evidence PRs: [WIP] Add AI-crawler robots.txt allow rules and llms.txt for the docs site #52115, [WIP] Create /llms.txt for the docs site #51616, [WIP] Add robots.txt to docs site allowing AI crawlers #51419, [WIP] Add robots.txt with explicit AI bot allow rules to docs site #51139, No action needed: robots.txt already resolves GEO audit finding #50336

Blocker Category	Count	Share	Notes
Scope/complexity mismatch	7	58%	Multiple [WIP]-tagged PRs proposing GEO/llms.txt support with no scoped spec; several near-duplicate attempts across the period
Other/unknown	4	33%	Closed with no reviewer engagement, no evidence in comments/reviews
Review friction	1	8%	One PR did receive reviewer pushback

Only 16 PRs fall in this cluster (right at the ≥15 reporting threshold), so treat the 25% figure as a low-sample signal — but the pattern (repeated [WIP] PRs for the same undefined "add GEO files" ask, closed unreviewed) is consistent across all of them.

Secondary outlier worth flagging — Cluster 4 (container/CVE pinning), which is much larger (67 PRs) and thus more statistically robust:

• Merge Rate Gap: 50.7% vs 73.0% (gap: 22.3 points)
• Primary Blocker: Scope/complexity mismatch (76% of unmerged PRs), driven by many closed-with-zero-comments [WIP] "fix vulnerabilities in image X" PRs
• Interestingly, the PRs in this cluster that did merge show heavy "Review friction" signal (34 of 34 merged PRs touched digest-pinning/security-concern language) — merging container-pin fixes requires review, but a large share of the auto-filed attempts never reach a reviewer at all before being closed, most likely superseded by a later scan run or a consolidated fix PR.

Recommendations

1. Consolidate duplicate vulnerability-scan PRs (Cluster 4): Multiple near-identical "fix vulnerabilities in image X" PRs are opened and closed unreviewed across the period. Batching per-scan findings into a single PR per scan run (rather than one per image/finding) would likely raise the merge rate and cut review overhead.
2. Scope the llms.txt/GEO ask before filing (Cluster 0): Five+ [WIP] attempts at the same undefined "add AI-crawler docs support" task suggest the underlying issue lacks an accepted spec. Either close the recurring source issue with a concrete acceptance spec (exact file contents, location, generation step) or stop re-filing until decided.
3. Keep investing review effort in Clusters 2 and 7: these have the heaviest review load per PR but the best outcomes — the pattern (small-diff, high-scrutiny changes to safe-outputs/security and engine-harness code) is working well and shouldn't be trimmed for speed.
4. Add a "no reviewer engagement after N days" signal: a large share of both outlier clusters' failures are PRs closed with zero comments and zero reviews. Surfacing genuinely stale/unengaged PRs earlier (rather than letting them sit until auto-closed) would make the low-merge signal actionable sooner.

---

Generated by Prompt Clustering Analysis (Run: 31790566086)

Generated by 📊 Copilot Agent Prompt Clustering Analysis · agent · 108.1 AIC · ⌖ 50.8 AIC · ⊞ 14.5K · ◷

• expires on Aug 15, 2026, 2:33 AM UTC-08:00