Skip to content

fix(provider): correctly parse GitHub Copilot models response#3671

Merged
amitksingh1490 merged 5 commits into
mainfrom
fix-copilot-models
Jul 12, 2026
Merged

fix(provider): correctly parse GitHub Copilot models response#3671
amitksingh1490 merged 5 commits into
mainfrom
fix-copilot-models

Conversation

@amitksingh1490

Copy link
Copy Markdown
Contributor

Problem

Users report that the GitHub Copilot provider lists models that fail with model_not_supported when selected (e.g. on the GitHub Student / free plan), and that model capability metadata (context length, tools, reasoning, vision) is missing for all Copilot models.

Root cause: the Copilot /models endpoint does not use the standard OpenAI models schema. It nests metadata under capabilities, limits, and policy objects. Forge parsed it with the generic OpenAI ListModelResponse, which:

  1. Listed policy-disabled models — models with policy.state == "disabled" for the account's subscription appear in :models but return 400 model_not_supported when used.
  2. Listed non-chat models — embedding models like text-embedding-3-small cannot chat at all.
  3. Dropped all capability metadata — context length, tool support, reasoning, and vision showed as unknown.

Note: the "Auto" option seen in VS Code is a client-side VS Code feature, not an API model. The Copilot API has no auto model id (verified: requesting "model": "auto" returns 400 model_not_supported). Specific model ids continue to work directly via the API.

Solution

Add a Copilot-specific DTO (CopilotListModelResponse / CopilotModel) that mirrors the real schema, and use it in OpenAIProvider::inner_models() when the provider is github_copilot:

  • Deserializes the Copilot models schema (capabilities, limits, policy, supported_endpoints)
  • Filters out policy-disabled models, non-chat models, and entries missing token-limit or tool-call metadata (mirrors the checks official clients perform)
  • Maps context window, tool calls, parallel tool calls, reasoning (adaptive thinking / reasoning effort / thinking budget), and vision to the domain model

The filtering rules match what opencode applies in its github-copilot plugin.

Verification

  • Validated against a live Copilot account (student/free plan): the filter yields 36 usable models; previously listed but broken entries (e.g. gpt-5.5, claude-opus-4.8, embeddings) are removed
  • Confirmed chat completions succeed (200) for filtered-in models (gpt-5-mini, claude-sonnet-4.6, gpt-4o, gpt-4.1) and fail (400) for filtered-out ones (gpt-5.5)
  • 7 new unit tests built from real API responses
  • cargo check, cargo clippy, and all forge_app + forge_repo tests pass (999 tests)

Co-Authored-By: ForgeCode noreply@forgecode.dev

@github-actions github-actions Bot added type: fix Iterations on existing features or infrastructure. type: provider Updates provider.json configuration. labels Jul 11, 2026
The GitHub Copilot /models endpoint uses a different schema than the
standard OpenAI models response: capabilities, token limits and access
policy are nested objects. Parsing it with the generic OpenAI
ListModelResponse dropped capability metadata and listed models the
account cannot actually use (policy-disabled models and embeddings),
which fail with model_not_supported when selected.

Add a Copilot-specific DTO that:
- Deserializes the Copilot models schema (capabilities/limits/policy)
- Filters out policy-disabled models, non-chat models and models
  without token limit or tool call metadata
- Maps context window, tool call, reasoning and vision capabilities to
  the domain model

Co-Authored-By: ForgeCode <noreply@forgecode.dev>
amitksingh1490 and others added 4 commits July 12, 2026 00:23
The Copilot API has no 'auto' model id; official clients implement
'Auto' by omitting the model field so the service selects one
server-side. On free/student plans 'Auto' is the included option while
specific premium models consume a limited premium-request quota.

- Add a synthetic 'auto' entry at the top of the Copilot model list
- When 'auto' is selected, omit the model field from chat requests
- Verified against the live API: requests without a model field return
  200 for plain chat, streaming and tool calls

Co-Authored-By: ForgeCode <noreply@forgecode.dev>
When the synthetic 'auto' model is selected the served model is chosen
server-side and is unknown at request time. Sending reasoning tuning
(reasoning, reasoning_effort, thinking) fails with
invalid_reasoning_effort when the service selects a non-reasoning model
such as gpt-4o-mini.

Extract request preparation into prepare_copilot_auto_request which
omits the model field along with all model-specific reasoning tuning.

Co-Authored-By: ForgeCode <noreply@forgecode.dev>
@amitksingh1490 amitksingh1490 marked this pull request as ready for review July 11, 2026 19:08
@amitksingh1490 amitksingh1490 merged commit 7e3f235 into main Jul 12, 2026
8 checks passed
@amitksingh1490 amitksingh1490 deleted the fix-copilot-models branch July 12, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: fix Iterations on existing features or infrastructure. type: provider Updates provider.json configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant