[copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-07-25

[github-actions[bot]]

🤖 Copilot Agent Session Analysis — 2026-07-25

Executive Summary

• Sessions Analyzed: 50
• Analysis Period: 2026-07-25 (06:15–07:23 UTC) · trends over trailing 30 days
• Completion Rate: 12% (6/50)
• Average Duration: ~2.0 min (median 0 min; longest 20.9 min)
• Experimental Strategy: MEDR — Model Efficiency Divergence Ratio (roll=0, experimental)

Today's 12% completion sits below the 30-day mean of 17.6% and marks a pullback from yesterday's 36%, snapping a 4-day above-mean run (07-20 → 07-24). The important nuance: unlike recent high days that were single-branch artifacts, today's 6 successes are spread across 6 distinct branches (SCC = 0.17, the lowest concentration in the window) — this is genuine, distributed fleet health rather than one branch clearing its full gate suite. Fleet stayed clean: 0 loops, 0.69% tool-failure rate (all recovered), 0 orphaned branches.

Key Metrics

Metric	Value	Trend
Total Sessions	50	→
Successful Completions	6 (12%)	↓ (from 36%)
Failed/Abandoned	0 failed / 44 gate-blocked (action_required)	→
Average Duration	~2.0 min	↓
Loop Detection Rate	0 (0%)	→
Context Issues	0 (0%)	→
Tool-Failure Rate (agentic)	2/288 (0.69%)	→
Success Concentration (SCC)	0.17	↓ (healthier vs 0.83 on 07-24)

Success Factors ✅

1. Agentic (non-CI-gate) provenance: Every one of today's 6 successes came from an agentic workflow — 3× Running Copilot cloud agent and 3× Addressing comment on PR (Add post-update SHA integrity validation for actions-lock entries #47907, fix(lint): resolve 4 targeted golint-custom findings in pkg/cli, cmd/gh-aw, and pkg/linters #47910, refactor(linters): consolidate duplicated import-editing and file-lookup helpers into internal/astutil #47912).

   • Success rate: agentic 6/6 = 100%; CI-gate 0/44 = 0%.
   • The 6-week provenance split (WTC) holds clean, mirroring 07-17.

2. Distributed completion, not a single-branch artifact: The 6 successes landed on 6 different branches.

   • SCC = 0.17 (max 1 win per branch) vs 07-24's 0.83.
   • Reading: today's headline % reflects broad fleet progress, safe to interpret as health.

3. Zero loops, high tool reliability: Across 6 rich conversation logs, 288 tool calls ran with only 2 recoverable failures and no repetitive/circular patterns.

   • Tool success rate: 99.3%.

Failure Signals ⚠️

1. CI-gate approval saturation dominates: 44/50 sessions are action_required — awaiting approval gates, not agent failures.

   • "Failure" rate: 88% of sampled runs are gate-blocked churn (median duration 0 min = re-fired gate stubs).

2. CGO remains the zero-anchor CI type: CGO completed 0/5 again today, consistent with its multi-week 0-line.

   • This is the single most persistent CI-gate failure category in the window and the best candidate for a targeted fix.

3. gpt-5.3-codex carried all recoverable errors (see Experimental Analysis): the lone codex run held 2/2 of the day's tool failures while all sonnet runs were clean.

Prompt Quality Analysis 📝

Conversation logs show agentic sessions opened with substantial, well-scoped user prompts (8k–16k chars) and immediately invoked a skill or targeted search — a high-quality pattern correlated with the 100% agentic success rate today.

High-Quality Prompt Characteristics (from today's 6 agentic logs)

• Skill/search-first turn 1: 5/6 sessions began with a skill or search_code_subagent call rather than blind editing.
• Concrete PR/issue anchor: the 3 Addressing comment sessions referenced a specific PR — all 3 succeeded.
• Large, context-rich opening prompt (8k–16k chars): correlated with completion.

Low-Quality Signal

• No low-quality agentic prompts observed today; the 44 non-completions are CI-gate approval stubs (no free-text task prompt), not prompt-quality failures.

Orphaned Branch Escalation Alerts 🚨

Branches with ≥5 simultaneous gate firings and no Copilot agent assigned for >2 hours.

Summary

• Orphaned Branches Today: 0 out of 11 open PR branches (0%)
• Historical Baseline: ~40% orphaned rate
• Status: NORMAL ✅ (well below the 50% elevated-waste threshold)

Escalation Candidate Details

Escalation Candidates

✅ No orphaned branches exceed the escalation threshold today.

• Only 2 in-progress workflow runs in the last 6 hours, both on main (no PR branch reached the ≥5-gate footprint required to orphan).
• 9 of 11 open PRs are Copilot-assigned; 2 unassigned PRs (fix/code-scanning-651-652-graphql-injection, signed/jsweep/issue-title-dedup) have 0 active gate runs, so neither qualifies for escalation.

CI Waste Estimate

• Orphaned gate-hours today: 0 gate × 0h ≈ 0 CI-minutes wasted.
• Recoverable capacity: none needed — fleet is at baseline-healthy for the ~59th consecutive day of 0% orphan rate.

Notable Observations — Loop Detection & Diagnostics

Loop Detection

• Sessions with loops: 0 (0%)
• Turn counts ranged 31–73 (mean 40.8); no repetitive/circular tool sequences.

Tool Usage (across 6 agentic conversation logs)

• Most used tools: bash (125), read_bash (31), github (29), view (25), edit (16), report_progress (14), rg (13).
• Tool success rate: 286/288 (99.3%). Failures: parallel_validation ×1, rg ×1 — both recovered.
• Missing tools: none requested.

Context Issues

• Sessions with confusion: 0. No clarification requests observed.

Experimental Analysis

This run included experimental strategy: MEDR — Model Efficiency Divergence Ratio

Today's 6 agentic logs spanned two backend models (claude-sonnet-4.6 ×5, gpt-5.3-codex ×1), enabling a per-model efficiency comparison. MEDR = mean turns of the higher-turn model ÷ mean turns of the lower-turn model, with tool-failure share attributed by model.

Findings:

• MEDR = 2.12 — the lone gpt-5.3-codex run used 73 turns / 76 tool calls vs claude-sonnet-4.6's mean 34.4 turns / 42 calls (n=5).
• Codex held 100% of the day's tool failures (2/2 = 2.6% fail rate) while all 5 sonnet runs were 0-failure.
• Both models reached completion — codex simply consumed ~2× the turn budget and produced the only recoverable errors.

Effectiveness: Medium (numerator rests on n=1 codex run — directional only).
Recommendation: Refine. Re-run on the next mixed-model day; if codex consistently exceeds ~1.8× sonnet turns with a failure monopoly, MEDR becomes a per-model efficiency early-warning. Confound: task difficulty and prompt size differ per run and are not controlled.

📈 Session Trends Analysis

Completion Patterns

Completion rate oscillates in a sawtooth between ~2% trough days and ~40–54% peak days; today's 12% is a mid-low point that ends a 4-day above-mean streak. Successful completions (green) and gate-blocked counts (red) remain near-perfect mirror images, confirming that the headline number is driven almost entirely by whether agentic branches clear their gate suites on a given day.

Duration & Efficiency

Average duration stays low (today ~2.0 min) because the median is pinned at 0 by the mass of gate-stub sessions; the visible spikes (07-22 ≈15m, 07-20 ≈9.9m) are days with long agentic runs. Loop counts remain at zero across the entire window (single bar on 07-17), so longer averages reflect genuine work, not looping.

Actionable Recommendations

For Users Writing Task Descriptions

1. Anchor to a concrete PR/issue: all 3 Addressing comment sessions referenced a specific PR and all succeeded. Prefer "address the review comment on PR #NNNN about X" over "fix the review feedback".
2. Lead with scope + expected outcome: today's successful agentic prompts were 8k–16k chars with clear acceptance context, enabling a skill/search-first turn 1.

For System Improvements

1. Separate "gate-blocked" from "failed" in reporting (High impact): 88% of action_required sessions are approval-gate stubs, not agent failures — surfacing them separately would make the true agent success rate legible.
2. Investigate CGO's persistent 0-line (Medium impact): CGO has completed 0/N for weeks; a targeted look at its build gate could unblock a whole workflow category.

For Tool Development

1. Per-model turn/cost telemetry (needed in 6/6 logged sessions): MEDR had to be reconstructed from raw turn= log lines. A first-class per-model turns/tokens/failure summary would make efficiency divergence trackable without log scraping.

Historical Trends & Statistical Summary

Trends Over Time

• Completion-rate trend: sawtooth, 30-day mean 17.6%; today 12% is below mean and ends a 4-day above-mean run.
• Duration trend: stable-low; median pinned at 0 by gate stubs, occasional agentic spikes (07-22 ≈15m).
• Quality trend: 8th consecutive day of rich conversation logs; loops = 0 for the full window; tool reliability ≥99%.
• Concentration trend: SCC dropped from 0.83 (07-24, single-branch) to 0.17 (07-25, distributed) — a genuinely healthier completion profile.

Statistical Summary

Total Sessions Analyzed:     50
Successful Completions:      6 (12%)
Failed Sessions:             0 (0%)
Gate-blocked (action_required): 44 (88%)
In-Progress Sessions:        0 (0%)

Average Session Duration:    ~2.0 min
Median Session Duration:     0 min (gate stubs)
Longest Session:             20.9 min
Non-zero-duration sessions:  6 (the agentic runs)

Loop Detection:              0 sessions (0%)
Context Issues:              0 sessions (0%)
Tool Failures:               2 of 288 calls (0.69%, all recovered)

Models (agentic logs):       claude-sonnet-4.6 ×5, gpt-5.3-codex ×1
Mean turns / agentic run:    40.8 (range 31–73)
Success Concentration (SCC): 0.17
Orphaned branches:           0 / 11 (0%)

Next Steps

• Review recommendations with team
• Split "gate-blocked" from "failed" in the completion metric
• Investigate CGO's persistent 0-completion line
• Re-run MEDR on the next mixed-model day to test the codex-efficiency signal
• Schedule follow-up analysis tomorrow

---

Analysis generated automatically on 2026-07-25.
Run ID: 30149242598
Workflow: Copilot Session Insights

References:

• §30149219073
• §30148595563
• §30147191594

Generated by 📊 Copilot Session Insights · age00 · 279.6 AIC · ⌖ 37.1 AIC · ⊞ 17.2K · ◷

• expires on Jul 25, 2026, 11:59 PM UTC-08:00

[github-actions[bot]]

Stone bot watch. Latest talk found. Run 30151690498 stomp here.

📰 BREAKING: Report filed by Smoke Copilot · gpt54 · 75.7 AIC · ⌖ 8.29 AIC · ⊞ 21K · ◷
_{Comment /smoke-copilot to run again
Add label smoke to run again}