[copilot-cli-research] Copilot CLI Deep Research - 2026-06-16

[github-actions[bot]]

Analysis Date: 2026-06-16 | Scope: 249 workflows, ~136 Copilot (55%), 63 copilot-sdk | Run: §27596173580

---

📊 Executive Summary

Key Findings: max-ai-credits recovering (6→18, +200%); engine.agent still declining (34→21→8); 19+ persistent feature gaps; MCP timeouts never used; blocked-domains barely adopted after launch.
Primary Recommendation: Pair max-tool-denials with every copilot-sdk: true workflow — all 63 SDK workflows run without explicit denial-count protection.

max-ai-credits reversed last week's regression (+12 workflows). min-integrity grew 22→34. A/B testing via experiments: is now tracked at 41 workflows. The engine.agent decline (peak 34 → today 8) suggests a batch removal event. Five agent files remain completely unused.

---

Critical Findings

🔴 High Priority

1. max-tool-denials absent from all 63 SDK workflows
The default is 5 (GH_AW_MAX_TOOL_DENIALS=5) but setting it explicitly communicates intent. With 63 SDK workflows this is the widest single gap.

engine:
  copilot-sdk: true
  max-tool-denials: "5"

2. engine.args / engine.api-target — 19 consecutive runs at zero adoption
engine.args allows injecting raw Copilot CLI flags; engine.api-target enables GHEC/GHES. Persistent non-use across every research run suggests a discoverability problem.

engine:
  id: copilot
  api-target: api.acme.ghe.com
  args: ["--experimental-flag"]

3. engine.agent declining: 34 → 21 → 8
Five agent files exist with no workflows referencing them: grumpy-reviewer, interactive-agent-designer, w3c-specification-writer, create-safe-output-type, custom-engine-implementation. Wire them up or archive them.

🟡 Medium Priority

4. MCP timeouts never configured — engine.mcp.session-timeout + engine.mcp.tool-timeout = 0 usage across all 249 workflows. A hung MCP tool call stalls an agent indefinitely.

engine:
  mcp:
    session-timeout: "4h"
    tool-timeout: "2m"

5. max-continuations at only 7/136 copilot workflows (5%)
Autopilot mode enables self-continuation for long multi-step tasks. Research, refactoring, and multi-file review workflows are prime candidates.

6. blocked-domains barely adopted (1/249)
Only security-review.md uses domain blocklists. Workflows with broad network permissions and sensitive data would benefit.

---

View Feature Usage Matrix

Feature	Available	Used	Rate
engine.args	✅	0	0%
engine.api-target	✅	0	0%
engine.harness	✅	0	0%
engine.token-weights	✅	0	0%
engine.mcp.session-timeout	✅	0	0%
engine.mcp.tool-timeout	✅	0	0%
max-tool-denials	✅	0/63 SDK	0%
network.blocked	✅	1	0.4%
tools.startup-timeout	✅	1	0.4%
engine.auth (WIF)	✅	1	0.4%
mcp-scripts	✅	1	0.4%
max-continuations	✅	7	5%
engine.agent	✅	8	6%
engine.bare	✅	16	12%
max-ai-credits	✅	18	13%
experiments	✅	41	16%
copilot-sdk	✅	63	46%

View Historical Trends (7 runs)

Metric	05-21	06-08	06-10	06-15	06-16
Total workflows	233	340	245	246	249
Copilot %	43%	39%	54%	54%	55%
engine.agent	~25	~13	34	21	8
copilot-sdk	0	63	63	63	63
max-ai-credits	0	~5	14	6	18
min-integrity	-	-	22	22	34
max-continuations	5	6	6	7	7
engine.args	0	0	0	0	0

engine.agent peaked at 34 (06-10) and crashed to 8 by today — likely a batch removal event. max-ai-credits recovering strongly after last week's regression.

---

7️⃣ Action Items

Immediate:

• Add max-tool-denials: "5" to all 63 copilot-sdk: true workflows
• Decide: wire up or archive the 5 unused agent files in .github/agents/
• Add tools.startup-timeout: 120 to high-complexity MCP workflows

Short-term:

• Add max-ai-credits: to the ~118 copilot workflows currently missing it
• Expand network.blocked: to security-sensitive workflows
• Add engine.mcp.session-timeout + engine.mcp.tool-timeout to MCP-heavy workflows
• Expand max-continuations to 10+ more multi-step workflows

Long-term:

• Investigate why engine.args/engine.api-target have zero adoption (docs gap?)
• Audit the engine.agent decline (why did 26 workflows drop agent: between 06-10 and today?)
• Add GHEC/GHES smoke tests to validate engine.api-target

---

Persistent gaps (engine.args, api-target, harness, token-weights, mcp timeouts, max-tool-denials) tracked for 19+ consecutive runs without change.

Generated by 🔬 Copilot CLI Deep Research Agent · 431 AIC · ⌖ 26.8 AIC · ⊞ 19.5K · ◷

• expires on Jun 16, 2026, 9:36 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot CLI Deep Research Agent.

A newer discussion is available at Discussion #39731.