[copilot-cli-research] Copilot CLI Deep Research - 2026-06-15

[github-actions[bot]]

Analysis Date: 2026-06-15 | Run: §27525865107
Scope: 246 total workflows, 133 using Copilot engine (54%)

---

📊 Executive Summary

This is the 7th deep-research run. Three metrics regressed vs the previous run (06-10); persistent feature gaps remain unchanged for 18+ consecutive runs.

⚠️ Regressions this run:

Metric	06-10	06-15	Δ
engine.agent	34	21	-13
max-ai-credits	14	6	-8
sandbox	20	15	-5

---

🔴 High Priority

1. Investigate regressions — engine.agent dropped 34→21, max-ai-credits dropped 14→6, and sandbox dropped 20→15 without a corresponding change in total workflow count (+1 net). Likely caused by recent bulk workflow updates. Audit commits between 06-10 and 06-15.

2. max-tool-denials missing from ALL 63 SDK workflows — Every copilot-sdk: true workflow lacks the SDK-specific guard against infinite tool-denial loops. The default is 5 but must be set explicitly.

engine:
  id: copilot
  copilot-sdk: true
max-tool-denials: 5

3. 66/133 workflows (50%) have no security config — No sandbox, no network. Half of all Copilot workflows allow unrestricted outbound access, enabling prompt injection exfiltration.

network:
  allowed:
    - defaults

4. Budget controls nearly absent — 127/133 (95%) workflows lack max-ai-credits. Runaway agents can exhaust monthly budgets.

---

🟡 Medium Priority

5. engine.args — 18 consecutive zero-usage runs (PERSISTENT). Custom CLI args fully supported since v0.36 but never used in production. ["--share"] would enable session tracking.

6. engine.api-target — 18 consecutive zero-usage runs (PERSISTENT). Required for GHE/GHES/GHEC-with-data-residency deployments but never configured.

7. engine.harness — Never used. Custom startup hooks and retry logic beyond the built-in CAPIError 400 retry have never been exercised.

8. 5 orphaned agent files (unchanged for 7 runs):

• grumpy-reviewer.agent.md
• interactive-agent-designer.agent.md
• w3c-specification-writer.agent.md
• create-safe-output-type.agent.md
• custom-engine-implementation.agent.md

Wire these to workflows or remove them to reduce confusion.

9. min-integrity only 17% — Only 22/133 Copilot workflows require PR approval before acting. PR-triggered workflows without this are vulnerable to prompt injection.

---

🟢 Low Priority

10. engine.token-weights — Never used. BYOK workflows using non-standard models get inaccurate AIC cost accounting.

11. web-fetch only 8% — 10/133 workflows. Research/documentation workflows frequently fetch external content but don't declare the capability.

12. engine.version pin — 0 Copilot CLI pins. The 12 version: fields found are Node/Go/Python runtime versions, not engine pins. Production workflows may break on new Copilot CLI releases.

---

📈 Feature Usage Matrix (133 Copilot workflows)

Feature	Count	%	Status
strict: true	79	59%	✅
network config	67	50%	⚠️ 50% missing
copilot-sdk	63	47%	✅ Growing
engine.agent	21	16%	⬇️ Regression
bare	14	11%	✅
min-integrity	22	17%	⚠️ Low
max-continuations	7	5%	⚠️ Underused
max-ai-credits	6	5%	⬇️ Regression
sandbox	15	11%	⬇️ Regression
web-fetch	10	8%	⚠️ Underused
engine.args	0	0%	🚫 18 runs
engine.api-target	0	0%	🚫 18 runs
engine.harness	0	0%	🚫 Never
engine.token-weights	0	0%	🚫 Never
max-tool-denials	0	0%	🚫 New gap

---

📅 7-Run Historical Trend

Date	Copilot	engine.agent	copilot-sdk	max-cont	max-ai-credits	sandbox
2026-05-21	100	25	—	5	—	16
2026-06-08	132	13	63	6	5	22
2026-06-10	132	34	63	6	14	20
2026-06-15	133	21 ⬇️	63	7	6 ⬇️	15 ⬇️

Persistent zero-usage features: engine.args (18 runs), engine.api-target (18 runs), engine.harness (all runs), engine.token-weights (all runs).

---

✅ Action Items

This week:

• Audit why engine.agent dropped 34→21 (check commits 06-10 to 06-15)
• Audit why max-ai-credits dropped 14→6 (restore where removed inadvertently)
• Add max-tool-denials: 5 to all 63 copilot-sdk: true workflows

This month:

• Add max-ai-credits: 500 to the 127 workflows missing it
• Add network: allowed: [defaults] to the 66 workflows with no security config
• Wire or remove the 5 orphaned .agent.md files

This quarter:

• Document why engine.args has 18 consecutive zero-usage runs
• Test engine.harness with a smoke workflow
• Evaluate engine.api-target for GHE/GHES use cases
• Add engine.version pins to production-critical workflows

---

References: §27525865107 | Research history in memory/copilot-cli-research branch

Generated by 🔬 Copilot CLI Deep Research Agent · 984.5 AIC · ⌖ 42 AIC · ⊞ 19.5K · ◷

• expires on Jun 15, 2026, 9:46 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-06-16T05:46:05.568Z.

Closed by Workflow