Skip to content

[copilot-opt] Add pre-PR validation gate to eliminate 34+ -again retry branch cycles #27323

@github-actions

Description

@github-actions

Problem

Copilot repeatedly creates new PR branches when tests or CI fails rather than fixing the issue on the existing branch. The pattern fix-testsfix-tests-againfix-tests-yet-again shows the agent opens 3+ separate PRs for the same problem, each triggering a full reviewer fan-out cycle, before converging on a solution. Similar multi-attempt patterns appear across at least 34 PR branches containing again or yet-again suffixes.

Evidence

  • Analysis window: 2026-04-06 to 2026-04-20 (14 days)
  • Sessions analyzed: 50 workflow runs, 1000 PRs cross-referenced
  • Key metrics and examples:
    • copilot/fix-tests (8 runs) + copilot/fix-tests-again (6 runs) + copilot/fix-tests-yet-again (5 runs) = 19 PR runs for one failing test task
    • copilot/refactor-semantic-function-clustering (8) + -again (2) + -another-one (1) = 11 runs for one refactor
    • 34 total PR branches contain -again or -yet-again, indicating at least 34 cases of failed-first-attempt retry cycles
    • 199/1000 PRs (20%) ended as CLOSED (abandoned) rather than merged — high abandon rate correlates with retry churn
    • Each retry branch triggers a new full fan-out (~22 workflow runs), multiplying wasted CI minutes

Proposed Change

  • Add a pre-PR smoke test gate in the Copilot coding agent workflow: run make test-unit (or go test ./... selective) before opening any PR; only proceed if tests pass
  • Enable the push-to-pull-request-branch flow for follow-up fixes — push corrections to the existing PR branch instead of opening a new branch (the fix-tests-again pattern suggests this flow is not being used)
  • Add a maximum retry count (max_retries: 3) to agentic fix workflows; escalate to human after limit rather than indefinitely reopening

Expected Impact

  • Eliminate the -again / -yet-again branch proliferation, reducing wasted PR create/abandon cycles estimated at 34+ cases
  • Reduce total CI minutes for fix-test tasks by ~65% (19 runs → ~3 targeted attempts before escalation)
  • Cut the 20% PR abandonment rate by preventing premature PR creation before validation passes

Notes

  • Distinct root cause category: late / missing pre-PR validation gate
  • Data quality caveats: No events.jsonl or per-session conversation logs available to confirm exact sequence of agent actions. Retry patterns are inferred from branch naming and PR counts in the cross-referenced PR dataset.

Generated by Copilot Opt · ● 1.3M ·

Metadata

Metadata

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions