[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-07-17

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-07-17

Executive Summary

• Sessions Analyzed: 50 (5 with full conversation transcripts)
• Analysis Period: 2026-07-17 07:04–07:21 UTC (~17-min sampling window)
• Completion Rate: 2.0% (1/50) — floor regime
• Average Duration: 0.40 min overall (sole success ran 15.2 min)
• Experimental Strategy: none this run (standard analysis only; roll=44)

Today is a hard collapse from yesterday's window-high 38% back to the 2% floor — the 4th time completion has hit exactly 2% in the last 8 days (07-10, 07-11, 07-13, 07-17). The single-day CI-gate "conversion" that broke the provenance inversion on 07-16 did not sustain: the classic split re-asserted, with the only success coming from an agentic run. Orphan health remains perfect — 50th+ consecutive healthy day, 0 escalations.

Key Metrics

Metric	Value	Trend vs 07-16
Total Sessions	50	→
Successful Completions	1 (2%)	↓ (was 19 / 38%)
Failed / action_required	44 (88%)	↑
In-progress (unsettled)	5 (10%)	↑
Average Duration	0.40 min	↓ (was 3.28m)
Loop Detection Rate	1 mild (2%)	↑ (was 0)
Context Issues	0	→
Orphan Rate	0% (0/17 PRs)	→ NORMAL
Tool Success Rate	96.9% (287/296)	↓ slightly

📈 Session Trends Analysis

Completion Patterns

The saw-tooth is now unmistakable: both recent peaks (07-14 at 36%, 07-16 at 38%) each collapsed straight back to the 2% floor the very next sampled day. Trailing-7d completion sits at ~15.7%, propped up almost entirely by those two isolated spikes rather than any sustained lift. The floor itself is stable and re-visited roughly every other sampling day.

Duration & Efficiency

Average duration tracks completion tightly: high-completion days carry real coding wall-clock, floor days are dominated by 0-duration CI-gate stubs. Today's 0.40 min average is a single 15.2-min success diluted by 44 instant action_required gate fires. The lone loop bar (07-17) reflects one benign iterative session, not a stuck agent.

Success Factors ✅

1. Agentic provenance: The only success (Running Copilot cloud agent on fix-mapdeletecheck-comment-guard, 15.2 min) is a coding/comment-addressing run, not a CI gate. agentic 1/5 = 20% vs ci_gate 0/45 = 0%.
2. Skill-scaffolded planning: The success invoked developer + agentic-workflows skills early and used report_progress checkpoints — a consistent marker of productive agentic sessions.
3. Bounded turn count: The success completed in 33 turns / 38 tool calls with only 1 tool failure — tight, linear tool use rather than sprawl.

Failure Signals ⚠️

1. CI-gate provenance (the dominant signal): All 45 recurring CI-gate-type sessions scored 0 — Agentic Commands 0/11, Smoke CI 0/7, CGO 0/6, CWI 0/5, Doc Build 0/5, moderators 0/8, CJS 0/2. These are action_required approval-gate churn, not agent failures.
2. CGO zero-anchor persists: CGO held 0/6 today after 0/8 yesterday — a durable single-CI-type dead zone worth a targeted look.
3. gpt-5.4 patch friction: apply_patch failed 3/5 times, all in gpt-5.4 sessions — an engine-specific patch-format issue absent from the claude-sonnet-4.6 runs.

Prompt Quality & Transcript Analysis

Conversation-log metrics (5 agentic transcripts)

Run	Branch	Model	Turns	Tools (ok/fail)	Out tok	Status
29562051117	fix-mapdeletecheck-comment-guard	claude-sonnet-4.6	33	37/1	9.3k	✅ success
29562029008	multi-device-docs-testing-report	claude-sonnet-4.6	96	96/0	31.8k	in_progress
29562115210	fix-integration-cli-add-list	gpt-5.4	68	76/4	13.2k	in_progress
29562041457	cli-version-updates	gpt-5.4	40	59/4	15.3k	in_progress
29562747667	update-generated-code-config (comment)	claude-sonnet-4.6	20	18/0	2.5k	in_progress

• Dual-engine day (down from quad on 07-16): claude-sonnet-4.6 ×3, gpt-5.4 ×2.
• High-quality prompts are large and structured (the success received a 6,134-char task brief; agentic runs get ~2,135-char "continue" re-prompts at turn boundaries). Sessions that open with a skill invocation + code-search subagent correlate with the productive runs.
• Most-used tools: bash (110), view (49), rg (31), read_bash (28), edit (13), report_progress (12).
• Token cost: the 96-turn session accumulated 8.3M cumulative input tokens (~87k context/turn) — the familiar context-accumulation (AWTC) pattern where context growth, not new work, dominates spend.

Orphaned Branch Escalation Alerts 🚨

Branches with ≥5 simultaneous gate firings and no Copilot agent assigned for >2 hours.

Summary

• Orphaned Branches Today: 0 out of 17 open PRs (0%)
• Historical Baseline: ~40% orphaned rate (legacy)
• Status: ✅ NORMAL — 50th+ consecutive healthy day

Escalation Candidate Details

✅ No orphaned branches exceed the escalation threshold today.

• Max active gates on any copilot branch = 1 (well under the ≥5 threshold; only main carried 4 concurrent runs).
• All 15 open copilot/* PRs are Copilot-assigned.
• The 2 unassigned open PRs (fix/codeql-..., signed/jsweep/...) are non-Copilot automation with 0 active gates — not orphan candidates.

CI Waste Estimate

• Orphaned gate-hours today: 0 — no recoverable orphaned CI capacity.
• Today's 0% orphan rate is far below the 50% "elevated" flag line; no waste pattern to escalate.

Notable Observations

Loop Detection, Tool Usage & Strategy Update

Loop Detection

• Sessions with loops: 1 mild (the 96-turn multi-device-docs-testing run issued 10 consecutive python3 -c bash calls for iterative validation).
• Genuine stuck loops / crashes: 0. That session ended cleanly at session.idle.

Tool Usage

• Flaky tool of the day: parallel_validation failed 3/5 = 60% — the 2nd consecutive day it leads failures (80% on 07-16). It has now displaced report_progress as the persistent flaky tool.
• apply_patch 3/5 failures, isolated to gpt-5.4 (see Failure Signals).
• Overall tool success 96.9% across 296 calls.

Strategy Update — WTC (Workflow-Type Completion)

The WTC alert fired correctly: yesterday's inversion break was a 1-day spike, and today the categorical agentic-vs-CI-gate split reverted. CGO remains the zero-anchor CI type across both days. Recorded to strategies.json.

Experimental Analysis

Standard analysis only — no experimental strategy this run (roll=44, threshold 30).

Actionable Recommendations

For Users Writing Task Descriptions

1. Lead with a skill + explicit scope: the sole success opened with skill invocation and a scoped code-search. Frame tasks as "use X skill to do Y in file Z" rather than open-ended asks.
2. Keep coding tasks single-branch and bounded: the 33-turn success contrasts with 68–96-turn in-progress runs; narrower scope converts faster.

For System Improvements

1. Investigate parallel_validation reliability (2 consecutive days ≥60% failure). Potential impact: High — it's now the dominant tool-level failure mode.
2. Investigate gpt-5.4 apply_patch format handling (100% of patch failures were gpt-5.4). Potential impact: Medium — engine-specific and reproducible.
3. Probe CGO's persistent 0-completion across branches (0/6, 0/8). Potential impact: Medium.

For Tool Development

1. No unavailable/missing tools were requested in today's transcripts.

Historical Trends & Statistical Summary

Trends Over Time

• Completion rate: saw-tooth around a stable 2% floor; isolated 36–54% spikes that never persist beyond one sampled day. Trailing-7d ~15.7%.
• Duration: tracks completion — floor days ≈ 0-duration gate stubs; peak days carry multi-minute coding wall-clock.
• Provenance inversion: 6-week agentic-vs-CI-gate split, broke once (07-16), reverted (07-17). CGO durable zero-anchor.

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:      1  (2%)
Failed / action_required:    44 (88%)
In-progress (unsettled):     5  (10%)

Average Session Duration:    0.40 min
Median Session Duration:     0.00 min
Longest Session:             15.2 min (fix-mapdeletecheck-comment-guard)
Shortest Session:            ~0 min (instant gate stubs)

Conversation transcripts:    5  (dual-engine: sonnet-4.6 x3, gpt-5.4 x2)
Loop Detection:              1 mild session (2%)
Context Issues:              0
Tool Calls / Failures:       296 / 9 (96.9% success)
Flaky tool:                  parallel_validation 3/5 fail (60%)

Orphaned Branches:           0 / 17 open PRs (0%) — NORMAL, 50th+ healthy day
Escalations:                 0

Next Steps

• Investigate parallel_validation flakiness (2-day ≥60% failure streak)
• Review gpt-5.4 apply_patch format friction
• Track whether CGO's 0-completion is a fixable single-gate issue
• Confirm whether the provenance inversion stays reverted or re-breaks
• Schedule follow-up analysis next daily run

---

References:

• §29562051117 — sole success (agentic)
• §29562029008 — 96-turn iterative session
• §29562975854 — this analysis run

Generated by 📊 Copilot Session Insights · 312.1 AIC · ⌖ 41.6 AIC · ⊞ 17.2K · ◷

• expires on Jul 18, 2026, 12:04 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #46379.

[github-actions[bot]]

Smoke test discussion ping for run 29638033192.

📰 BREAKING: Report filed by Smoke Copilot · 143.6 AIC · ⌖ 4.55 AIC · ⊞ 19K · ◷
_{Comment /smoke-copilot to run again
Add label smoke to run again}