Skip to content

Add fast-fail Copilot CLI startup diagnostic step#43906

Closed
pelikhan with Copilot wants to merge 2 commits into
mainfrom
copilot/deep-report-add-fast-fail-diagnostic
Closed

Add fast-fail Copilot CLI startup diagnostic step#43906
pelikhan with Copilot wants to merge 2 commits into
mainfrom
copilot/deep-report-add-fast-fail-diagnostic

Conversation

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Copilot CLI 0-turn startup failures are the dominant production failure family — workflows burn 10-24 minutes of agent work before the failure surfaces. This adds a lightweight preflight step that probes CLI readiness upfront and fails in seconds with an actionable diagnosis.

New step: Probe Copilot CLI readiness

Inserted before Execute GitHub Copilot CLI in standard mode (skipped for engine.command and engine.copilot-sdk: true). Checks:

  1. Binary availabilitycommand -v copilot
  2. Version healthcopilot --version
  3. Auth credentialsCOPILOT_GITHUB_TOKEN non-empty (skipped in BYOK mode when COPILOT_PROVIDER_BASE_URL is set, or copilot-requests mode when S2STOKENS=true)

On failure, exits non-zero and writes a categorized error (binary / version / auth) to $GITHUB_STEP_SUMMARY.

- name: Probe Copilot CLI readiness
  id: copilot_startup_check
  env:
    COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
    S2STOKENS: "..."
    COPILOT_PROVIDER_BASE_URL: "..."
  run: |
    # checks binary, version, and auth — emits diagnosis to $GITHUB_STEP_SUMMARY on failure

Changes

  • copilot_engine_execution.go — added copilotStartupDiagnosticScript, buildCopilotStartupDiagnosticStep(), and updated GetExecutionSteps() to prepend the diagnostic for standard mode
  • Test files — updated 9 test files to expect 2 steps (diagnostic + execution) for standard-mode Copilot; SDK and custom-command tests unchanged (still 1 step)
  • threat_detection_test.go — tightened wantNotContains check to use 6-space-indented permission form, since the diagnostic script now embeds the string copilot-requests: write in a comment
  • Golden files + lock files — regenerated via make update-wasm-golden and make recompile

Adds a new "Probe Copilot CLI readiness" step that runs before
"Execute GitHub Copilot CLI" in standard (non-SDK, non-custom) mode.
The diagnostic checks:
- Binary availability (copilot in PATH)
- Version probe (copilot --version)
- Auth credentials (COPILOT_GITHUB_TOKEN non-empty, unless BYOK or S2STOKENS mode)

On failure it writes an actionable summary to $GITHUB_STEP_SUMMARY and
exits with a clear error category (binary, version, auth), stopping the
workflow in seconds instead of after 10-24 minutes of agent work.

Updates all affected tests, golden files, and recompiles all lock files.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add fast-fail diagnostic for copilot-CLI 0-turn startup failures Add fast-fail Copilot CLI startup diagnostic step Jul 7, 2026
Copilot AI requested a review from pelikhan July 7, 2026 02:34
@pelikhan pelikhan closed this Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Add fast-fail diagnostic for copilot-CLI 0-turn startup failures

2 participants