[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-06-08

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-06-08

Executive Summary

Today's sample of 50 sessions fired in a single 45-minute burst (05:53–06:38 UTC) across 6 copilot/* branches. Completion regressed sharply to 4%, a snap-back to the low-productivity floor just one day after the 40% recovery on 06-07. The day was overwhelmingly a CI gate-sweep (90% action_required), and both of the day's failures came from the "Running Copilot cloud agent" workflow — the same reliability soft-spot seen earlier in the window. Orphan-escalation health remains green: zero candidates, 0% orphaned rate vs. the ~40% historical baseline.

• Sessions Analyzed: 50
• Analysis Period: 2026-06-08 (05:53–06:38 UTC, single burst)
• Completion Rate: 4% (2 successes)
• Average Duration: 8.3 min (substantive sessions only); median 10.85 min
• Experimental Strategy: None (standard run; roll 47 ≥ 30)

⚠️ Data quality — metadata only. Agent conversation transcripts were again unavailable (16th+ consecutive day, OAuth). Behavioral analysis (loop detection, internal reasoning, prompt-quality scoring) cannot be performed from the agent's monologue. All findings below are inferred from CI workflow-run metadata and outcome provenance, not from per-session prompts.

Key Metrics

Metric	Value	Trend
Total Sessions	50	→
Successful Completions	2 (4%)	↓ (from 40% on 06-07)
Failed / Cancelled	3 (6%)	↑
action_required (gate sweeps)	45 (90%)	↑
Average Duration (substantive)	8.3 min	↓
Loop Detection Rate	n/a (no transcripts)	—
Orphaned Branches	0 (0%)	→

📈 Session Trends Analysis

Completion Patterns

Completion remains highly volatile: the 40% spike on 06-07 did not hold, snapping back to 4% today — below the 7-day average of 11.7%. The series continues its oscillation between brief recovery days (06-07 at 40%, 05-26 at 46%) and floor days (06-06 at 2%, today at 4%), with no sustained upward trend.

Duration & Efficiency

The bimodal pattern holds: 45 gate sweeps complete in ~0 min while only 5 substantive sessions ran (4.8–18 min range). Average substantive duration (8.3 min) is in the normal band, and the low substantive-session count (grey bars) tracks the low completion rate — fewer real agent runs means fewer completions.

Success Factors ✅

Inferred from workflow provenance of the 2 successful runs:

1. Human-comment follow-up tasks complete reliably: "Addressing comment on PR [awf] Bump firewall images to v0.25.66 and MCPG to v0.3.24 #37708" (14.8 min) succeeded — consistent with the historical success_duration_floor (substantive successes ≥ ~5 min).
2. Lightweight review workflows succeed fast: "Running Copilot Code Review" succeeded in 3.28 min — below the usual ~4.8 min floor, suggesting review-class tasks have a lower success-duration threshold than code-generation tasks.

Failure Signals ⚠️

1. "Running Copilot cloud agent" is the failure hotspot: both of today's failures (feat-shared-workflows-sandbox-field 11 min, fix-environment-variables-expansion 10.85 min) were cloud-agent runs that consumed substantive time before failing. This reliability dip resurfaces after clean cloud-agent days on 06-06/06-07.
2. Heavy gate-sweep dilution: 90% of sessions are action_required CI gates awaiting approval — they inflate the denominator and depress the headline completion rate without representing real agent failures.
3. Branch concentration risk: top-2 branches (fix-environment-variables-expansion, feat-shared-workflows-sandbox-field, 17 runs each) account for 68% of all runs; a stuck branch dominates the sample.

Orphaned Branch Escalation Alerts 🚨

Summary

• Orphaned Branches Today: 0 out of 4 open PRs (0%)
• Historical Baseline: ~40% orphaned rate
• Status: ✅ NORMAL (well below the 50% elevated-waste threshold)

Escalation Candidate Details

Escalation Candidates

✅ No orphaned branches exceed the escalation threshold today.

All 4 in-progress workflow runs are on main (not PR branches), so no PR branch has ≥5 simultaneous gate firings. Of the 4 open PRs, 3 are Copilot-assigned and 1 (#37778, schema-coverage) is unassigned but has 0 active gates and is <1 hour old — below the 1-hour escalation floor.

Branch	PR	Gate Count	Wait Time	Severity	Action
none	—	—	—	—	—

CI Waste Estimate

• Orphaned gate-hours today: 0 — no PR-branch gate fan-out detected.
• Recoverable capacity: none required; capacity is healthy.

Notable Observations

Session Diagnostics & Distribution

Outcome Breakdown

• action_required: 45 (90%) — CI gates awaiting approval
• success: 2 (4%)
• failure: 2 (4%) — both "Running Copilot cloud agent"
• cancelled: 1 (2%) — "Addressing comment on PR Improve lenstringzero precision for len(string) aliases in zero-comparisons #37750", 1.62 min

Branch Distribution (all copilot/*)

Runs	Branch
17	copilot/fix-environment-variables-expansion
17	copilot/feat-shared-workflows-sandbox-field
8	copilot/fix-lenstringzero-precision-issue
4	copilot/awf-bump-firewall-container-images
2	copilot/investigate-effective-tokens
2	copilot/aw-jsweep-fix-tool-denials

Workflow (gate) types fired

Q (12), Agentic Commands (12), Smoke CI (5), CGO (3), Doc Build – Deploy (3), Label Closed PRs (3), PR Description Updater (3), plus AI Moderator / Content Moderation / cloud-agent runs.

Loop & Context Analysis

• Loop detection: unavailable — requires conversation transcripts (missing 16th+ day).
• Context issues: unavailable for the same reason.

Prompt Quality Analysis 📝

Why per-prompt scoring is omitted today

Prompt-quality scoring requires the agent's conversation transcripts to assess how the request was interpreted. Those transcripts have been unavailable for 16+ consecutive days (OAuth limitation on the fetch step). To avoid fabricating percentages, no per-prompt quality distribution is reported. Once transcripts are restored, this section will resume with high/medium/low prompt characteristics correlated to outcomes.

Experimental Analysis

Standard analysis only — no experimental strategy this run (random roll 47 ≥ 30 threshold).

Actionable Recommendations

For System Improvements

1. Investigate "Running Copilot cloud agent" reliability — it produced 100% of today's failures (2/2) and recurs as a failure hotspot across the window. Potential impact: High.
2. Separate gate-sweep runs from agent sessions in the metric — 90% action_required dilution makes the headline completion rate misleading. Reporting a "substantive completion rate" (successes ÷ non-gate runs = 2/5 = 40%) would track agent quality more faithfully. Potential impact: Medium.

For Tool Development

1. Restore conversation-log access (OAuth) — missing for 16+ days. This blocks all behavioral analysis (loop detection, prompt quality, reasoning patterns), reducing this report to infrastructure metadata. Frequency of need: every run. Potential impact: High.

Historical Trends & Statistical Summary

Trends Over Time

• Completion rate: 40% (06-07) → 4% (06-08); 7-day average 12.3% → 11.7%. Oscillating, no durable uptrend.
• Duration: substantive average 8.3 min, in the normal 6–12 min band.
• Orphan health: 0 candidates for the 6th+ consecutive day — sustained healthy regime vs. 40% baseline.

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:       2 (4%)
Failed Sessions:              2 (4%)
Cancelled Sessions:           1 (2%)
Gate Sweeps (action_required):45 (90%)

Avg Duration (substantive):   8.30 min
Median Duration (substantive):10.85 min
Longest Session:             14.80 min
Shortest substantive:         1.62 min

Loop Detection:              n/a (no transcripts)
Context Issues:              n/a (no transcripts)

Orphan Candidates:            0
Orphaned Rate:                0% (baseline ~40%)
Open PRs:                     4 (1 unassigned, 0 gates)
In-progress Runs:             4 (all on main)
7-day Completion Avg:        11.7%

Next Steps

• Triage the two "Running Copilot cloud agent" failures (sandbox-field, env-var-expansion branches).
• Restore conversation-log fetch (OAuth) to re-enable behavioral analysis.
• Monitor whether 06-08's 4% is a one-day snap-back or a return to the sustained floor regime.

---

Analysis generated automatically on 2026-06-08.
Run ID: 27124904766
Workflow: Copilot Session Insights

Generated by 📊 Copilot Session Insights · 197 AIC · ⌖ 9.46 AIC · ⊞ 20.8K · ◷

• expires on Jun 9, 2026, 12:50 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #38072.