Skip to content

Record upstream tracking and the concrete follow-up for the Copilot CLI install failure - #10282

Merged
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/investigate-aw-detection-runs
Jul 28, 2026
Merged

Record upstream tracking and the concrete follow-up for the Copilot CLI install failure#10282
Evangelink merged 2 commits into
mainfrom
dev/amauryleve/investigate-aw-detection-runs

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Refs #10203.

What this is (and isn't)

#10203 is the bot-managed [aw] Detection Runs tracker. Its single recorded entry (run 30106781448, Grade Tests on PR, parse_error) was investigated in the issue comments and documented in #10267: the detection job died at Install GitHub Copilot CLI on a transient curl: (22) … 504, so safe_outputs was skipped and the grading comment never posted.

That investigation concluded there is no repo-side fix, and I re-verified that against upstream today — it still holds. So this PR does not eliminate the failure; it can't be eliminated from this repo. What it does is close the loop the previous write-up left open: it records where the fix lives, what unblocks it, and what we do here when it ships.

Docs-only change to .github/workflows/README.md. No workflow source or .lock.yml is touched, so no gh aw compile is required.

Upstream state, verified 2026-07-28

Latest gh-aw release v0.83.4 (2026-07-27) — this repo's locks are compiled on v0.83.1
DefaultCopilotVersion 1.0.73 on v0.83.1, 1.0.75 on v0.83.4 and main
gh-aw-actions compat.json max-agent still 1.0.56 for gh-aw ≥ 0.72.0
install_copilot_cli.sh byte-for-byte unchanged since v0.83.1 (both curls already carry --retry 3 --retry-delay 5 — the budget the failing run exhausted)

Nothing in v0.83.2 – v0.83.4 touches the install path, so the drift has only widened.

The two upstream items now recorded

  • github/gh-aw#48358 (open) — "DefaultCopilotVersion drifts past compat.json max-agent, forcing a network install on every job and disabling the toolcache path". This is the tracking issue promised in the [aw] Detection Runs #10203 investigation comment.
  • github/gh-aw#48519 (open) — "honor engine.version for copilot". This is the unblocking change.

Why #48519 matters more than it looks

Its own description notes it does not restore compat-matrix resolution — an explicit version still skips it. That reads like it doesn't help, but it does, because find_cached_copilot_bin already short-circuits on an exact match:

Found candidate: /opt/hostedtoolcache/copilot-cli/1.0.56/x64/bin/copilot (version: 1.0.56, arch: x64)
Exact version match found: /opt/hostedtoolcache/copilot-cli/1.0.56/x64/bin/copilot

That branch returns before the cache-TTL check. So once engine.version is actually honored, pinning it to 1.0.56 — the version the hosted runner image caches, which is also the compat.json ceiling — turns today's range: none..none miss into a hit and removes both downloads (agent + detection), not just makes them rarer.

The README now spells out that follow-up: bump + recompile, add the engine: block (noting no workflow declares one today, so check whether a shared/ import can carry it before touching ~30 files), and verify the log flips from No compatible toolcache entry found / -> Downloading ... to Exact version match found:.

Not closing #10203

Refs, not Fixes — the tracker is bot-managed, says "Do not close this issue manually", and auto-expires via its gh-aw-expires marker (2026-08-23).

Validation

  • .github/workflows/README.md is inside markdownlint's ignores (.github/workflows/**/*.md) and outside the link-checker's scope (docs/ + root README.md), so neither gates this — checked anyway: no trailing whitespace, single trailing newline, both new anchors (#tracking-the-upstream-fix, #compile-on-the-pinned-toolchain-and-check-the-pins-afterwards) resolve to real headings, and the new [gh-aw-48358] / [gh-aw-48519] reference definitions sit with the existing [gh-aw] one.
  • Every upstream claim above was read from source at the pinned tag (v0.83.4) rather than inferred: pkg/constants/version_constants.go, pkg/workflow/copilot_engine_installation.go, pkg/workflow/copilot_installer.go, actions/setup/sh/install_copilot_cli.sh, and gh-aw-actions' compat.json.

…LI install failure

The "Known transient failures" section documented the `detection` job's
`Install GitHub Copilot CLI` failure and correctly concluded there is no
repo-side fix, but it left the reader without a way to know when that stops
being true.

Add a "Tracking the upstream fix" subsection recording the two upstream items:
github/gh-aw#48358 (the DefaultCopilotVersion / compat.json drift) and
github/gh-aw#48519 (honor `engine.version` for the copilot engine).

#48519 is the unblocking change. It does not restore compat-matrix resolution
(an explicit version still skips it), but `find_cached_copilot_bin` already
short-circuits on an exact toolcache match before the cache-TTL check, so once
`engine.version` is honored, pinning it to the version the hosted runner image
caches removes both CLI downloads. The follow-up steps and the verification
signal to look for in the run log are spelled out.

Also refresh the version facts: gh-aw v0.83.4 is out and bumps
DefaultCopilotVersion to 1.0.75, while this repo's lock files are compiled on
v0.83.1 / 1.0.73; install_copilot_cli.sh itself is unchanged since v0.83.1.

Refs #10203

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 47ebbbd3-1ddc-4dc2-8f1d-51f8d0460eac
Copilot AI review requested due to automatic review settings July 28, 2026 07:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documents the upstream path for resolving recurring Copilot CLI installation failures.

Changes:

  • Records relevant upstream issue and PR.
  • Documents the future version-pinning and verification steps.
  • Updates current gh-aw version details.
Show a summary per file
File Description
.github/workflows/README.md Adds upstream tracking and remediation guidance.

Review details

  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread .github/workflows/README.md Outdated
@Evangelink
Evangelink enabled auto-merge (squash) July 28, 2026 08:07
…he hit

The follow-up guidance told the reader to "follow the toolcache, not the
matrix" when the cached CLI and the compat window diverge. That is wrong and
unsafe: `max-agent` is the supported compatibility ceiling, so pinning to a
cached version outside `min-agent`..`max-agent` would run every agentic
workflow on an unsupported Copilot CLI just to save a download.

Require the pin to satisfy both constraints — inside the compat window *and*
present in the toolcache — and keep paying for the download when they diverge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 47ebbbd3-1ddc-4dc2-8f1d-51f8d0460eac
Copilot AI review requested due to automatic review settings July 28, 2026 08:08
@Evangelink Evangelink added the state/needs-review Awaiting review from the team. label Jul 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 1/1 changed files
  • Comments generated: 0 new
  • Review effort level: Medium

@Evangelink
Evangelink merged commit 694e7e6 into main Jul 28, 2026
22 checks passed
@Evangelink
Evangelink deleted the dev/amauryleve/investigate-aw-detection-runs branch July 28, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-review Awaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants