[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-07-18

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-07-18

Executive Summary

• Sessions Analyzed: 50 (40-min capture window, 06:24–07:04 UTC)
• Analysis Period: Most recent ~50 agent + CI-gate runs on github/gh-aw
• Completion Rate: 8% (4 success / 43 action_required / 2 cancelled / 1 null) — floor regime, flat vs 07-08
• Average Duration: 1.09 min overall (median 0); 7.80 min mean across the 7 executed runs
• Experimental Strategy: None (standard run, roll=76)

Headline: After 40+ consecutive days of empty conversation logs, transcripts were recovered today — 4 real agent logs are available, breaking the longest-standing analysis risk (conversation_log_fetch_failure). This is the first run in over a month with genuine turn-by-turn behavioral data rather than CI/infra metadata only.

Key Metrics

Metric	Value	Trend
Total Sessions	50	→
Successful Completions	4 (8%)	→ (flat vs 07-08)
Failed / Cancelled	2 (4%)	↑ (2 cancelled)
Gate-blocked (action_required)	43 (86%)	↑
Avg Duration (executed runs)	7.80 min	↓ from 8.85
Loop Detection Rate	0 (0%)	→
Tool-call Failure Rate	0 / 98 (0%)	→ (new signal)
Orphan Escalations	0 / 23 open PRs	→ (~52nd healthy day)

📈 Session Trends Analysis

Completion Patterns

Completion sits at the 8% floor for a second consecutive analyzed day, a long way below the 07-04 regime-break spike (54%). The saw-tooth pattern persists: brief spikes decay back to a single-digit floor within one or two days. The 8% today is entirely agentic — no core CI gate executed to green.

Duration & Efficiency

Executed-run mean duration eased to 7.80 min (from 8.85 on 07-08) with a 6.12-min median — real work is concentrated in a handful of longer agentic runs while ~43 gate stubs register 0 duration. Loops remain at zero across every analyzed day, now corroborated for the first time by actual transcripts (turn counts 6–25, all monotonic).

Success Factors ✅

1. Agentic provenance (provenance_inversion holds): All 4 successes were agent-driven runs — Copilot cloud-agent (update-golden-to-latest-version 16.2 min, update-readme-for-eslint-rules 9.28 min), "Addressing comment on PR docs: improve ARC/DinD guide with expanded troubleshooting, log paths, and tool cache guidance #46045" (13.2 min), and Copilot Code Review (5.07 min). 0/4 core CI gates completed. Success continues to come from agentic execution, never from gate sweeps.
2. Clean tool execution: 0 of 98 tool calls failed across all 4 transcripts — no error-recovery churn, no retry loops.
3. Bounded, monotonic turn counts: 6–25 turns per run with steady forward progress; no circular reasoning detected in any log.

Failure Signals ⚠️

1. Gate saturation dominates (86%): 43/50 runs are 0-duration action_required CI-gate stubs — the structural driver of the low headline completion rate, not agent failure.
2. Cancellation on comment-addressing runs: 2 cancelled runs were both "Addressing comment on PR" ([formal-spec] Add PM-10/AppG formal model coverage for security architecture summary #46280 6.1 min, fix(eslint): resolve route/URL/control-flow lint violations in actions/setup/js #46100 4.65 min) — mid-flight cancellation rather than error, worth watching for churn.

Behavioral Analysis from Recovered Conversation Logs

Per-Run Behavioral Profile (4 transcripts)

Run	Model	Turns	Tool Calls	Peak Ctx	Skill	Notes
29633952954	claude-sonnet-4.6	25	34	~93K	—	View-heavy explore: 21 view, 5 edit, parallel_validation, secret_scanning, 2 report_progress
29633959094	claude-sonnet-4.6	19	18	~48K	—	Terse (140 out-tok/turn)
29634108461	gpt-5.3-codex	17	19	~60K	pr-finisher	First non-Claude model observed
29634107366	claude-sonnet-4.6	6	9	~48K	pr-finisher	bash-first PR review (gh pr view, cat, grep) on PR #46280

Cross-Log Findings

• Model diversity (new): 3× claude-sonnet-4.6 + 1× gpt-5.3-codex — first multi-model observation in this analysis series.
• Tool mix: bash (29), view (21), read_bash (8), edit (5), report_progress (4), plus MCP tools (github-mcp-server-*, secret_scanning, vote_memory, reply_to_comment).
• Skill usage: pr-finisher invoked in 2 of 4 runs — the dominant workflow entrypoint.
• Output efficiency: 140–451 output tokens/turn — terse tool-calling with minimal prose, consistent across models.
• No confusion / clarification requests surfaced in any transcript.

Orphaned Branch Escalation Alerts 🚨

Branches with ≥5 simultaneous gate firings and no Copilot agent assigned for >2 hours.

Summary

• Orphaned Branches Today: 0 out of 23 open PRs (0%)
• Historical Baseline: ~40% orphaned rate
• Status: NORMAL — well below the 50% elevated-waste threshold (~52nd consecutive healthy day)

Escalation Candidate Details

✅ No orphaned branches exceed the escalation threshold today.

Max active gate footprint on any single branch is 1 (main = 2); the threshold is ≥5. 5 non-Copilot bot PRs are unassigned (fix/allocation-size-overflow #46375, two signed/jsweep/*, linter-miner/* #46289, eslint-miner/* #46195) but none are gate-saturated, so none qualify as orphaned.

CI Waste Estimate

• Orphaned gate-hours today: 0 — no recoverable orphaned CI capacity.

Actionable Recommendations

For Users Writing Task Descriptions

1. Reference specific files/PRs — the fastest successful runs today (Code Review 5.07 min, PR-comment addressing) operated on concrete PR numbers with pr-finisher; scoped targets keep turn counts low.
2. Prefer scoped edits over broad exploration — the 25-turn/34-call run spent 21 calls on view before 5 edits; front-loading the target file set would compress it.

For System Improvements

1. Keep the conversation-log fetch fixed — Potential impact: High. Logs recovered today after 40+ empty days; whatever changed in the fetch path should be preserved, as behavioral analysis (loops, tool-failure, model mix) is impossible without it.
2. Separate gate-stub runs from agent runs in the session feed — Potential impact: Medium. 86% of the feed is 0-duration action_required stubs, which depresses the headline completion metric and obscures real agent throughput.

For Tool Development

1. Model-attributed metrics — now that gpt-5.3-codex appears alongside claude-sonnet-4.6, per-model success/duration tracking would be valuable. Frequency of need: emerging (1st multi-model day).

Historical Trends & Statistical Summary

Trends Over Time

• Completion rate: 40 → 10 → 2 → 20 → 8 → 4 → 54 → 16 → 8 → 8% (trailing analyzed days); 30-day mean ~13%. Floor regime holds.
• Duration: executed-run mean 7.80 min (↓ from 8.85 on 07-08); overall mean depressed by gate stubs.
• Data quality: major improvement — conversation logs recovered after 40+ empty days.

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:      4 (8%)
Failed/Cancelled:            2 (4%)
Gate-blocked (action_req):   43 (86%)
Null/in-progress:            1 (2%)

Overall Mean Duration:       1.09 min
Overall Median Duration:     0 min (43 gate stubs)
Executed-run Mean:           7.80 min
Executed-run Median:         6.12 min
Longest Session:             16.2 min (Copilot cloud-agent, update-golden)

Loop Detection:              0 sessions (0%) — confirmed via transcripts
Tool-call Failures:          0 / 98 (0%)
Conversation Logs:           4 recovered (sonnet-4.6 ×3, gpt-5.3-codex ×1)

Unique copilot/* branches:   9
Top gate footprint:          deep-report-populate-safeitemscount (13),
                             update-golden-to-latest-version (12),
                             update-readme-for-eslint-rules (8)
Open PRs:                     23 (0 orphaned)

Next Steps

• Preserve the conversation-log fetch fix — it is the single biggest data-quality gain in 40+ days
• Begin per-model (sonnet-4.6 vs gpt-5.3-codex) success/duration tracking now that multi-model runs appear
• Monitor whether the 8% floor persists or saw-tooths upward next cycle
• Schedule follow-up analysis next run

References:

• §29633952954 — 25-turn view-heavy agent run
• §29634108461 — first gpt-5.3-codex run (pr-finisher)
• §29635351746 — this analysis run

Generated by 📊 Copilot Session Insights · 337.5 AIC · ⌖ 43 AIC · ⊞ 17.2K · ◷

• expires on Jul 18, 2026, 11:53 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #46577.