[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-09-29

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-09-29

Executive Summary

• Sessions Analyzed: 50
• Analysis Period: 2026-09-29, 06:21:58–07:00:47Z (~39-min fetch window, 6 unique branches)
• Completion Rate: 10.0% (5/50) — 2nd-lowest of the 37-day recorded window (only 08-31 at 4.0% is lower), -40.0pts vs 09-28 (50.0%)
• Average Duration: 0.49 min raw (45 completed sessions; 4.37 min average across the 5 nonzero runs)
• Experimental Strategy: success_channel_isolation

Key Metrics

Metric	Value	Trend
Total Sessions	50	→
Successful Completions	5 (10.0%)	↓ (vs 25, 50.0% on 09-28)
Failed / Action-required	40 (80.0%), 0 outright failures	↑
In-progress (unresolved at snapshot)	5 (10.0%)	↑
Average Duration	0.49 min	↓ (vs 3.84 min on 09-28)
Loop Detection Rate	N/A — no conversation logs	—
Context Issues	N/A — no conversation logs	—

Success Factors ✅

1. Code scanning AI findings (narrow, single-purpose bots): 100% success (5/5) — the only category with any success today, across 4 distinct PRs (Infer Copilot wire API for utility model variants #64177, Allow workflows to narrow add-labels item schemas #64179, Automatically allow OTLP collector endpoints through the firewall #64180 ×2, [WIP] Add distributed ledger capability to repo-memory #64199).
   • Success rate: 100%
   • Example: Code scanning AI findings on PR #64199 completed clean in 320s independent of its branch's broader CI-gate state.
2. Isolation from broad CI-gate bundles: every successful run today was a standalone, narrowly-scoped check untangled from the multi-workflow gate sweep — consistent with the 36-day pattern that narrow bots succeed independently of branch-wide gate health.

Failure Signals ⚠️

1. Review-bot/advisory floor collapse (first full collapse on record): the cohort of 8 normally ~90–100%-reliable advisory bots (Design Decision Gate, Impeccable/Matt Pocock Skills Reviewers, PR Code Quality Reviewer, Ponytail Reviewer, Test Quality Sentinel, PR Data Prefetch, Running Copilot Code Review) went 0/8 (0%) today.
   • Failure rate: 100% (of this cohort)
   • Example: Design Decision Gate 🏗️, PR Data Prefetch, and Ponytail Reviewer all sat at action_required on governance-enforce-self-hosted-runners.
2. Zero true-agentic firings: no Addressing comment on PR or Running Copilot cloud agent runs fired at all — the first zero-volume day in the 37-day recorded window (every prior low-completion day had at least one agentic attempt, even if it failed).
3. governance-enforce-self-hosted-runners branch stuck gate: 10/50 firings (20%), 0% success, no in-flight failures either — just action_required + 1 in_progress, consistent with a branch_level_stuck_gate pattern (queued behind review, not abandoned).
4. Core CI/build gates unchanged at near-zero: Agentic Commands 0/11, Squad 0/10, CWI 0/5, CJS 0/4, CGO 0/4, Doc Build-Deploy 0/2 (0/36 overall) — matches the historical baseline, not a new signal on its own.

Prompt Quality Analysis 📝

Per-Prompt Breakdown

No prompt-level data is available — conversation transcript logs are empty again today (find /tmp/gh-aw/agent/session-data/logs -name "*-conversation.txt" → 0 files), the 37th consecutive recorded day with this gap. Prompt-quality analysis remains blocked pending resolution of the underlying log-extraction issue (see Recommendations and the missing_data signal filed alongside this report).

Orphaned Branch Escalation Alerts 🚨

Branches with ≥5 simultaneous gate firings and no Copilot agent assigned for >2 hours.

Summary

• Orphaned Branches Today: 0 out of 6 active branches (0.0%)
• Historical Baseline: 0.0% orphaned rate (30-day mean over 36 recorded days)
• Status: NORMAL — 37th consecutive healthy day

Escalation Candidate Details

Escalation Candidates

✅ No orphaned branches exceed the escalation threshold today. Max simultaneous in-progress gate count on any single branch was 2 (add-required-labels-support, fix-opentelemetry-firewall-warning, fix-version-comment-for-pinned-actions), well below the ≥5-gate floor. One branch (add-required-labels-support, PR #64174) has no Copilot agent assigned, but its 2-gate footprint doesn't meet the escalation threshold.

CI Waste Estimate

• Orphaned gate-hours today: 0 (no candidates crossed the threshold)
• Recoverable capacity: N/A

Notable Observations

Loop Detection and Session Diagnostics

Loop Detection

• Not measurable — conversation logs unavailable (metadata-only analysis).

Tool Usage

• Not measurable at the turn/tool-call level — no transcript data. At the workflow level, 19 distinct workflow names fired today; Agentic Commands (11) and Squad (10) were the highest-volume, both at 0% success (queued/pending, not failing).

Context Issues

• Not measurable — no transcript data.

Experimental Analysis

This run included experimental strategy: success_channel_isolation (roll=13 < 30 threshold)

Partitioned all 50 firings into four channels — true-agentic, review-bot/advisory, core CI-gate, and code-scanning — and checked whether a day's successes concentrate in exactly one channel while the others sit at literal 0%.

Findings:

• 100% of today's successes (5/5) sit in the code-scanning channel alone.
• The true-agentic channel had zero firings — not just zero successes, zero attempts. This has not happened on any of the prior 36 recorded days.
• The review-bot/advisory channel went 0/8 — its first full collapse from the historical ~90–100% reliable floor.
• The core CI-gate channel held at 0/36, matching its usual near-zero baseline (so this channel is not new signal — the novelty is the other two).

Effectiveness: High — this is a cleaner, more granular lens than provenance_inversion (which assumes at least two channels still contribute to the success count) and immediately flags today as qualitatively different from a typical low-completion day.
Recommendation: Refine — promote channel_isolation_day to a standing tracked pattern (now logged in patterns.json) and watch for recurrence to distinguish one-off infra blips from a structural shift.

Actionable Recommendations

For Users Writing Task Descriptions

• No prompt-level signal available today (zero agentic firings, no transcripts) — nothing to recommend from this run's data specifically.

For System Improvements

1. Investigate the review-bot/advisory 0/8 collapse: this cohort has been near-100% reliable for 36+ days; a same-day full collapse across 8 independently-named workflows suggests a shared dependency (e.g., a common prefetch step, rate limit, or infra hiccup) rather than 8 unrelated failures. Worth a manual look at one of the action_required runs (e.g. Design Decision Gate 🏗️ on governance-enforce-self-hosted-runners) to confirm root cause.
   • Potential impact: High (this cohort is usually the most reliable signal in the whole dataset)
2. Check whether the zero true-agentic-firing gap reflects a genuine lull or a triggering issue: 37 days of history never showed zero attempts before.
   • Potential impact: Medium

For Tool Development

1. Conversation transcript log extraction: still 0 files after 37 consecutive recorded days. This remains the single largest standing gap — it blocks all turn-by-turn behavioral analysis (tool usage, loop detection, prompt quality, error recovery). Flagged via missing_data alongside this report.
   • Frequency of need: every one of the last 37 daily runs
   • Use case: turn-level model/token/tool-call analysis as originally scoped for this workflow

Historical Trends and Statistical Summary

Trends Over Time

• Completion rate trend: sharp reversal — 09-27 (48%) → 09-28 (50%) → 09-29 (10%), breaking a 2-day uptrend; 36-day mean sits at ~31.7% (range 4–78%), so today is well below average but within the historical floor territory already seen on 08-25/08-31/09-04.
• Average duration trend: 09-28 (3.84min) → 09-29 (0.49min), among the lowest on record — consistent with the absence of any (typically longer-running) true-agentic firings today.
• Orphan rate: flat at 0.0% for the 37th consecutive day — no change.

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:      5   (10.0%)
Action-required (pending):   40  (80.0%)
Failed Sessions:              0  (0.0%)
Cancelled Sessions:           0  (0.0%)
In-Progress Sessions:         5  (10.0%)

Average Session Duration:    0.49 min (raw, 45 completed) / 4.37 min (5 nonzero runs only)
Median Session Duration:     0.0 min
Longest completed run:       320s (5.3 min, Code scanning AI findings PR #64199)
Shortest nonzero run:        217s (3.6 min, Code scanning AI findings PR #64179)

Loop Detection:              N/A (no transcript logs)
Context Issues:              N/A (no transcript logs)
Tool Failures:                0 recorded (no transcript logs; workflow-level only)

High-Quality Prompts:         N/A (no transcript logs)
Medium-Quality Prompts:       N/A (no transcript logs)
Low-Quality Prompts:          N/A (no transcript logs)

📈 Session Trends Analysis

Completion Patterns

Completion rate has been saw-toothing between roughly 4% and 78% for over a month with no stable regime; today's 10.0% snaps a 2-day uptrend (48% → 50% → 10%) and lands in the historical floor band alongside 08-25, 08-31, and 09-04. The failed/action-required count (red) spiked back up to 45 today, its highest point since early September, mirroring the drop in successful completions (green).

Duration & Efficiency

Average duration has been highly volatile, with pronounced spikes on 09-01, 09-16, 09-20, and 09-23 (each driven by a handful of long-running genuine task runs or status-resync artifacts) against a floor of near-zero on pure gate-stub days. Today's 0.49-minute average sits at that floor, consistent with zero true-agentic firings — the metric with the most explanatory power for duration spikes across the whole window.

Next Steps

• Review recommendations with team, especially the review-bot 0/8 collapse
• Manually inspect one action_required review-bot run on governance-enforce-self-hosted-runners to confirm root cause
• Continue tracking channel_isolation_day as a standing pattern
• Prioritize conversation-transcript-log extraction fix (37-day standing gap)
• Schedule follow-up analysis tomorrow (2026-09-30)

---

Analysis generated automatically on 2026-09-29
Run: §36533981502
Workflow: Copilot Session Insights

References:

• §36533981502 — this analysis run
• §36534143059 — Design Decision Gate 🏗️ (action_required, review-bot collapse example)
• §36531802868 — Code scanning AI findings success example

Generated by 📊 Copilot Session Insights · claude · agent · 211.9 AIC · ⌖ 10 AIC · ⊞ 6.1K · ◷

• expires on Sep 29, 2026, 11:27 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #64437.