Skip to content

[deep-report] Distinguish merge-time invalidation from true failure in CI/session conclusion data #58986

Description

@github-actions

Description

Copilot Session Insights (discussion #58953, 2026-09-06) discovered and verified that CI/session reporting cannot distinguish a PR-merge-time branch invalidation from a genuine task failure. When PR #58909 merged at 2026-09-06T04:00:28Z, every gate/review workflow still in flight on that branch (PR Code Quality Reviewer, Design Decision Gate, Test Quality Sentinel, Ponytail Reviewer, Matt Pocock Skills Reviewer, Impeccable Skills Reviewer, PR Data Prefetch, Stale Lock Files, and 5× Agentic Commands) recorded conclusion=failure at the identical timestamp, rather than cancelled/invalidated. This single merge event produced 17 of that day's 50 session "failures" (34% of the fleet) with zero actual regressions behind it — a false-alarm rate large enough to look like a fleet-wide incident if not manually cross-checked against gh pr view <PR> merge timestamps.

This was verified directly: the failure timestamps (04:00:28–30Z) were cross-referenced against PR #58909's actual mergedAt (2026-09-06T04:00:28Z) and matched exactly.

Expected Impact

Downstream reporting (Copilot Session Insights and any other workflow-health rollup that reads raw conclusion fields) would stop misclassifying merge-time cascade invalidations as real regressions, removing a recurring source of false-alarm noise in daily fleet-health metrics. Concretely: tag same-second, same-branch failure clusters that coincide with a PR merge event as invalidated (or similar) distinct from failure in the underlying run-conclusion data, or expose the base-branch merge timestamp alongside run conclusions so downstream analyzers can make this distinction themselves without needing a manual gh pr view cross-check each time.

Suggested Agent

New Agent (or an enhancement to the agentic-workflows log/session-data pipeline that Copilot Session Insights and similar reports consume) — this is a data-pipeline/classification fix, not something the reporting workflow itself can resolve on its own.

Estimated Effort

Medium (1-4 hours) — requires correlating run conclusions with PR merge events at the pipeline level and adding a new classification value.

Data Source

DeepReport Intelligence analysis, 2026-09-06 (incremental cycle), sourced from discussion #58953 ([copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-09-06).

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔬 Deep Report · claude · agent · 200.7 AIC · ⌖ 9.99 AIC · ⊞ 12.8K ·

  • expires on Sep 8, 2026, 4:40 AM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions