Skip to content

Copilot engine unusable on plans limited to automatic model selection; pinning a model does not help #46531

Description

@oscarvalenzuelab

Problem

Related to #26223, but a distinct case that its workaround does not cover: on a Copilot Free account (automatic model selection only), the copilot engine fails with 400 The requested model is not supported in every configuration — including the #26223 workaround of pinning model: gpt-5-mini.

Environment

  • gh-aw v0.81.6, gh-aw-firewall 0.27.11
  • Copilot CLI 1.0.65 (bundled; also reproduced after bumping the install to 1.0.71)
  • Account: Copilot Free (free_limited_copilot), CLI enabled
  • COPILOT_GITHUB_TOKEN: fine-grained PAT with the Copilot Requests permission, passes the activation validation step

Evidence

  1. Inside the workflow, the CLI lists models through the api-proxy and receives gpt-5-mini as available, fetches its model info successfully (264k context) — then POST /responses returns 400. otel span: api_proxy.copilot.request, url.path=/responses, http.response.status_code=400.
  2. The identical CLI version with the identical PAT works outside Actions: with no --model flag it auto-routes to gpt-5-mini and completes normally (0.33 AI credits).
  3. Frontmatter model: gpt-5-mini (a valid alias in the awf models table) → same 400 in ~4s.
  4. Frontmatter model: auto → rejected pre-flight by the proxy: model 'auto' is retired or unsupported. Did you mean 'gpt-4'?
  5. No model in frontmatter → engine default (premium) → 400, matching Feature: Auto-detect available models or gracefully fallback on 400 errors (Copilot Pro/Education) #26223.
  6. Proxy startup logs API proxy enabled: Copilot=true (github-token) even with the secret configured; the agent container starts with --exclude-env COPILOT_GITHUB_TOKEN.

Analysis

Copilot Free permits automatic model selection only — the CLI rejects explicit --model values for this plan even locally, while auto-routing works. The workflow path always ends up sending an explicitly pinned model (the alias table has no pass-through equivalent of the CLI's automatic routing), so every request is rejected for auto-only plans. The models catalog is not entitlement-filtered, which is why listing succeeds while completions 400 — that combination makes the failure look like a proxy bug when it is a plan/request-mode mismatch.

Proposed solution

Support pass-through of the Copilot CLI's automatic model selection — e.g., let model: auto (or an explicit model: none) mean "do not pin a model on the upstream request" instead of resolving through the alias table. That would make the copilot engine usable on plans limited to automatic selection.

Reproduction

https://github.com/oscarvalenzuelab/agentic-ai-github-actions.github/workflows/dependency-remediation.md; failing runs 29671785973, 29671898890, 29673674928, 29674165470, 29674408291.

Metadata

Metadata

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions