Skip to content

[copilot-opt] Reduce lint-fix PR churn: enforce make fmt && make lint in pre-submission agent validation #31560

@github-actions

Description

@github-actions

Problem

A recurring pattern of Copilot PRs exclusively fixes lint failures introduced by prior Copilot PRs. This creates a secondary wave of fix PRs that consume additional agent sessions without adding features or fixing user-facing bugs. Each lint-fix session is wasted throughput that could be directed at productive work. The 20.7% closed-without-merge rate (207 of 1,000 PRs) is partially attributable to this churn.

Evidence

  • Analysis window: 2026-04-23 to 2026-05-11
  • Sessions analyzed: 1,000 Copilot PRs
  • Key metrics and examples:
    • 12 PRs with titles explicitly mentioning lint failures (e.g., "Fix lint-go CI failure by formatting label names test file", "Fix lint-go failure by replacing static fmt.Errorf usages flagged by perfsprint", "Fix lint-go CI failure by modernizing loops and string handling")
    • 3 active branches on 2026-05-11 specifically for test/CI remediation: copilot/fix-test, copilot/fix-tests, copilot/fix-tests-again — all showing CI failures in the session window
    • 6 of 50 workflow runs (12%) on 2026-05-11 concluded as failure, all on fix-related branches
    • AGENTS.md mandates make fmt after Go changes and make lint before every PR, with explicit warnings that unformatted code causes CI failures — yet the pattern persists
    • PR cross-analysis shows lint and test remediation is a repeated category across the 14-day window, with distinct PRs fixing identical categories of linter violations (perfsprint, unused, testifylint)

Proposed Change

  1. Strengthen the AGENTS.md two-checkpoint validation to make the pre-submission checkpoint (make agent-report-progress) mandatory before the agent can call report_progress, with an explicit self-check step in the agent instruction that verifies zero lint errors.
  2. Add a lightweight early linting gate in CI that runs make fmt check and make lint in under 60 seconds, returning results before the heavyweight test suite, so failures are surfaced earlier in the run and agents receive clear fix signals faster.
  3. Track a "lint-fix PR ratio" metric (lint-fix PRs ÷ total PRs) weekly; target < 2% (currently ~1.2% but clustering suggests bursts after certain change categories).
  4. For the perfsprint and testifylint linters specifically: add pre-run .golangci.yml baseline rules so the agent can catch these locally with make lint before pushing.

Expected Impact

  • Eliminates the secondary lint-fix PR wave, saving 1 full agent session per lint-fix PR (estimated 10–15 sessions/month based on current rate)
  • Reduces CI noise on Copilot branches and decreases the number of failure conclusions
  • Increases the merge rate by ensuring PRs that reach review are already lint-clean

Notes

  • Distinct root cause category: late validation failure / insufficient pre-submission linting
  • Data quality: conversation-level events.jsonl was not available for this analysis window; lint attribution is inferred from PR titles and CI failure conclusions. Direct tool-latency measurement was not possible without event logs.

Generated by Copilot Opt · ● 24.7M ·

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions