[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-05-22

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-05-22

Executive Summary

• Sessions Analyzed: 50
• Analysis Period: 2026-05-22 06:02:08Z → 06:21:14Z (sampling window: 19 min, narrowest in the 14-day series)
• Completion Rate: 2.0% (1 success, 49 action_required, 0 failure)
• Copilot Agent Run: 1 cloud agent — 18.22 min, success on copilot/refactor-semantic-function-clustering-please-work
• Spec Orphans: 0 — 15th consecutive day at zero orphan threshold
• Open PRs: 12 (up +5 from 7 on 05-21 due to a synchronized burst of 5 unassigned chaos/* PRs)
• Experimental Strategy: none — standard analysis only this run
• Data Quality: infrastructure-only (14th consecutive run; conversation transcripts unavailable)

Key Metrics

Metric	Value	Trend vs 05-21
Total Sessions	50	→
Successful Completions	1 (2.0%)	↓ (was 6 / 12.0%)
Failed / action_required	49 (98.0%)	↑ (was 43 / 86.0%)
Conclusive Failures	0 (0.0%)	↓ (was 1 / 2.0%)
Avg Copilot Agent Duration	18.22 min	↑ (was 12.3 min avg over 4 runs)
Top Branch Concentration	16/50 (32%)	↓ (was 25/50 / 50%)
Unique Branches	6	↑ (was 5; 14-day max)
Agentic Commands + Q Share	26/50 (52%)	↑ (was 13/50 / 26%)
Spec Orphans	0	→
Open PRs	12	↑ (was 7)

📈 Session Trends Analysis

Completion Patterns

Completion rate dropped from 12% (05-21) to 2% (05-22), mirroring the 05-18 → 05-19 reversal exactly (22% → 2%). The 14-day average remains ~7%, with 05-18 (22%) the lone outlier; the dominant steady state is 86–98% action_required with sporadic 2–8% conclusive wins. The metric is misleading on its own — open-PR backlog tracking shows real merge throughput is meaningfully higher than the daily-success ratio suggests.

Duration & Efficiency

Today's single Copilot cloud agent run (18.22 min) sits firmly in the >15-min "high-success" band per the historical_trend_regression strategy — consistent with the day's only agent succeeding. Max gates per branch (16) is down from 25 on 05-21 and well below the 35 peak on 05-12, but still elevated. Duration is trending upward across the window (10.15 min on 05-11 → 18.22 min on 05-22) — tasks are getting more complex even as success rates hold steady.

Success Factors ✅

Patterns associated with the successful completion today:

1. >15-minute agent duration: The 18.22-min cloud agent run on the dominant branch (32% share) was the sole success — strongly consistent with the historical band "Duration >15min = 100% success" (see historical_trend_regression strategy).
2. Dedicated cloud agent run: The single Running Copilot cloud agent workflow firing was the only non-action_required outcome in 50 sessions. When a real agent runs in this window, it succeeds.
3. High-dominance branch with agent intervention: The branch absorbing 32% of the queue is the one that got the cloud agent — matches the inverse gate-count-to-conclusiveness pattern: an agent run cleared the only conclusive event on the highest-gate branch.

Failure Signals ⚠️

1. 98% action_required share (49/50) — same shape as 05-19. When no agents are firing, the queue is dominated by gate sweeps. The Agentic Commands + Q pair alone produced 26 action_required events (52% of total).
2. Gate-sweep workflow dominance: Agentic Commands (13) + Q (13) account for the inactive bulk of the queue. Both fire on every push without conclusive output in this window.
3. 5 unassigned chaos/ PRs* created in a 21-second window (06:00:40Z → 06:01:17Z) — none have agents yet. They sit at the edge of the 2h-warning band as of sampling time. No gate activity yet, but they're a future orphan risk.

Prompt Quality Analysis 📝

Conversation transcripts remain unavailable for the 14th consecutive run (gh auth login OAuth blocker). Direct prompt-quality scoring is not possible from infrastructure data alone. The signals below are inferred from session shape, not from agent reasoning.

Inferred High-Quality Signals (from successful 05-22 run):

• Single, well-scoped task — one cloud agent run, no retry loops visible
• Branch is dedicated to one refactor goal (refactor-semantic-function-clustering) — narrow scope
• 18.22 min completion — in the duration band where success is empirically near-certain

Inferred Low-Quality Signals (from action_required dominance):

• Multiple gate-sweep workflows (Agentic Commands, Q, Smoke CI) firing without agent intervention — suggests pushes without an agent assignment hook, or agents declining to engage

Orphaned Branch Escalation Alerts 🚨

Branches with ≥5 simultaneous gate firings and no Copilot agent assigned for >2 hours.

Summary

• Orphaned Branches Today: 0 out of 12 open PRs (0%)
• Historical Baseline: ~40% orphaned rate
• Status: ✅ NORMAL (well below baseline; 15th consecutive day at zero)

Escalation Candidates

✅ No orphaned branches exceed the escalation threshold today.

The 5 unassigned chaos/* PRs (created 06:00-06:01Z) sit just at the edge of the 2h-warning band but have zero in-progress gates in the 6-hour lookback, so the orphan filter correctly excludes them. The 6 copilot/* branches that absorbed all 50 sweep sessions all have a Copilot assignee on their PRs.

View open PR roster & assignment status

PR	Branch	Assignees	Gate Count	Wait Class
#33954	copilot/add-request-review-mode	pelikhan, Copilot	0 active	n/a
#33953	chaos/line-ending-normalizer-r48	(none)	0 active	edge of warning
#33952	chaos/minor-renamer-r48-rename	(none)	0 active	edge of warning
#33951	chaos/selective-stager-r48-staged-subset	(none)	0 active	edge of warning
#33950	chaos/code-archaeologist-r48-two-commits	(none)	0 active	edge of warning
#33949	chaos/paranoid-reviewer-r48-amend	(none)	0 active	edge of warning
#33947	copilot/refactor-semantic-function-clustering-please-work	pelikhan, Copilot	0 active	n/a
#33946	copilot/lint-monster-migrate-logging	pelikhan, Copilot	0 active	n/a
#33945	copilot/lint-monster-fix-resource-leaks	pelikhan, Copilot	0 active	n/a
#33944	copilot/aw-step-name-alignment-fix	pelikhan, Copilot	0 active	n/a
#33852	copilot/add-create-check-run-safe-output	pelikhan, Copilot	0 active	n/a
#33219	copilot/deep-report-bind-mount-nodejs	Copilot, gh-aw-bot	0 active	n/a

Gate counts use the in-progress-runs feed (6h lookback). Sweep activity above happens on completed runs, captured in the sessions-list.

CI Waste Estimate

• Orphaned gate-hours today: 0 — no candidates triggered
• Recoverable capacity: n/a

Notable Observations

Loop Detection

• Sessions with loops: 0 conclusive cycles; agent did not iterate (single 18.22-min run on the dominant branch)
• Max gates per branch: 16 (top branch) — well below the 14-day high of 35 (05-12)
• Per-branch conclusiveness rates (consistent with Inverse Gate-Count to Conclusiveness strategy):
  • add-create-check-run-safe-output: 4 sessions, 0 conclusive (0%)
  • aw-step-name-alignment-fix: 6 sessions, 0 conclusive (0%)
  • lint-monster-fix-resource-leaks: 7 sessions, 0 conclusive (0%)
  • lint-monster-migrate-logging: 7 sessions, 0 conclusive (0%)
  • add-request-review-mode: 10 sessions, 0 conclusive (0%)
  • refactor-semantic-function-clustering-please-work: 16 sessions, 1 conclusive (6.25%)

Tool Usage

• Most used workflows: Agentic Commands (13), Q (13), Smoke CI (7), CGO (6), Doc Build - Deploy (6)
• Agent-bearing workflow: Running Copilot cloud agent — 1 firing, 1 success
• Burst pattern: Largest single burst was 8 fires at 06:11:32Z — biggest since 05-12's 14. Coincides with PR Add request_review protected-files mode for create_pull_request #33954 creation. Second-largest: 5 at 06:02:11Z (the 5 chaos PR creations within seconds).

View full workflow firing distribution

Workflow	Total	Success	action_required
Agentic Commands	13	0	13
Q	13	0	13
Smoke CI	7	0	7
CGO	6	0	6
Doc Build - Deploy	6	0	6
CJS	2	0	2
AI Moderator	1	0	1
Content Moderation	1	0	1
Running Copilot cloud agent	1	1	0

Context Issues

• Sessions with confusion: not directly observable without transcripts
• Clarification requests: not observable from infrastructure data

Experimental Analysis

This run did NOT include an experimental strategy. Standard analysis only.

The previously-tested Inverse Gate-Count to Conclusiveness strategy (added 05-21, marked High effectiveness) holds again today: the 16-session top branch is at 6.25% conclusive while every smaller branch is at 0%. The hypothesis (high gate count → branch waiting on agent action, not on CI) is now corroborated across two consecutive days with very different completion profiles (12% vs 2%).

Actionable Recommendations

For Users Writing Task Descriptions

1. Use dedicated copilot/* branches with one clear refactor goal — today's success branch (refactor-semantic-function-clustering-please-work) had a precise, scoped name and absorbed an 18-min agent run cleanly.
2. Avoid pushing without an agent hook — the 5 unassigned chaos/* PRs created today are now sitting at the orphan-warning edge. Either assign Copilot at PR-open time or expect gate sweeps without resolution.
3. Expect 15+ min for substantive refactors — the >15-min duration band correlates with 100% success in the historical data; under-budgeting agent time triggers retry loops.

For System Improvements

1. Replace completion_rate_pct as the headline metric — Impact: High. Today's 2% reads as a bad day, but it's operationally identical to 05-19 and the cloud agent did its job. Net PR throughput (e.g., open_prs delta minus new PRs) is more informative.
2. Add an "approval bottleneck" severity tier to the orphan filter — Impact: Medium. The strict orphan filter has flagged 0 for 15 consecutive days; meanwhile, gate sweeps on agent-assigned branches dominate (49/50 today). The current filter is calibrated for a failure mode that hasn't been seen.
3. Watch chaos/ PR cohorts* — Impact: Low-Medium. The 5-PR synchronized chaos burst on 05-22 is the largest unassigned-PR creation event in the 14-day window. If they ever attract gate activity, they will be the first true orphans in 15+ days.

For Tool Development

1. Fix conversation transcript export — Frequency of need: every run. The gh auth login OAuth blocker has produced data_quality: infrastructure-only for 14 consecutive runs (since 05-06). All behavioral analysis is currently inferred from session shape.
2. Surface "sweep-after-success" pattern — Frequency of need: 5+ days. Same-second gate bursts immediately after agent runs (seen on 05-16, 05-17, 05-19, 05-21) suggest the post-success retry isn't aware that the agent already cleared the work.

Trends Over Time

• Completion rate trend: Bounces between 2–22% with no clear linear drift; 05-21 (12%) → 05-22 (2%) is a one-day reversal, same pattern as 05-18 → 05-19 four days earlier
• Average duration trend: Increasing — 10.15 min (05-11) → 18.22 min (05-22), with 22.27 min outlier on 05-19. Tasks getting more substantive
• Orphan rate trend: Flat at 0% for 15 consecutive days — far below the historical 40% baseline
• Open PR backlog: 22 (05-18) → 13 (05-19) → 11 (05-20) → 7 (05-21) → 12 (05-22) — the +5 today is entirely chaos/* fixture PRs; real backlog is unchanged

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:      1   (2.0%)
Failed Sessions:             0   (0.0%)
Action-Required Sessions:    49  (98.0%)
In-Progress Sessions:        0   (0.0%)

Sampling Window:             19.1 min (06:02:08Z → 06:21:14Z)

Copilot Agent Runs:          1
  - Successful:              1   (100%)
  - Avg Duration:            18.22 min
  - Duration Band:           >15 min (high-success)

Unique Branches:             6
Top Branch Share:            16/50 (32%)
Largest Burst Size:          8 fires (06:11:32Z)

Workflow Concentration:
  - Agentic Commands + Q:    26/50 (52%)
  - CI workflows total:      19/50 (38%)

Open PR Backlog:             12 (delta from 05-21: +5)
Spec Orphans:                0 (15th consecutive day)
In-Progress Runs (all):      3 (all on main)

Next Steps

• Re-test orphan filter calibration — 15 consecutive days at zero suggests it's too strict for the dominant failure mode
• Decide on a replacement headline metric for completion_rate_pct (proposal: net PR backlog change + agent-bearing-success count)
• Track chaos/* PR cohort behavior: do they ever attract gate firings, or are they cleanup fixtures?
• Investigate why session 26271338057's updated_at is the only one differing from created_at — confirm the upstream module is filtering completed runs correctly

---

References:

• §26275531347 — this analysis run
• §26271338057 — the only successful agent run today (18.22 min)
• §26213113053 — yesterday's analysis (05-21) for trend comparison

Analysis generated automatically on 2026-05-22.

Generated by 📊 Copilot Session Insights · ● 18.1M · ◷

• expires on May 23, 2026, 8:15 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Session Insights.

A newer discussion is available at Discussion #34188.