Test: BYOK Custom Endpoint provider

[vijayupadya]

Refs #316915

• Windows Karthik Nadig (@karthiknadig)
• macOS Paul (@pwang347)

Complexity: 4

Create Issue

---

Introduces a new BYOK provider — Custom Endpoint (customendpoint) — that lets you plug any OpenAI-compatible, Responses-API, or Anthropic Messages-compatible endpoint into Copilot Chat. It replaces the legacy OpenAI Compatible (customoai) provider, which only ever supported Chat Completions and is now marked deprecated (Insiders only).

Test1: Steps

• Open the language model picker by clicking on the gear button in the model selection
• Click on 'Add Models' and select 'Custom Endpoint'

• Enter name, key
• Select 'Chat Completions' api type

• In the opened chatLanguageModels.json fill in the model info like below and save

[
{
  "vendor": "customendpoint",
  "apiKey": "YOUR_API_KEY",
  "apiType": "chat-completions",
  "models": [
    {
      "id": "gpt-4o",
      "name": "GPT-4o (chat completion)",
      "url": "https://api.openai.com/v1/chat/completions",
      "toolCalling": true,
      "vision": true,
      "maxInputTokens": 128000,
      "maxOutputTokens": 16000
    }  
  ]
}
]

• From model picker select the model you added above
  NOTE: Some times the new custom model may not show up in the model picker immediately, for those cases please restart the VSCode after adding the custom endpoint. This is being investigated separately.
• Send few prompts including something that triggers tool calls (e.g. analyze file foo.ts)
• In chat debug logs, validate that the custom endpoint url and model was correctly choosen and response is accurate

Test2: Response API - Steps

• Repeat the above steps by choosing apiType 'responses'
• You can use below model setting

[
{
  "vendor": "customendpoint",
  "apiKey": "YOUR_API_KEY",
  "apiType": "responses",
  "models": [
    {
      "id": "o4-mini",
      "name": "o4-mini (responses)",
      "url": "https://api.openai.com/v1/responses",
      "toolCalling": true,
      "vision": true,
      "maxInputTokens": 128000,
      "maxOutputTokens": 16000
    }  
  ]
}
]

Test3: Messages API - Steps

• Repeat the above steps by choosing apiType 'messages'
• You can use below model setting

[
{
  "vendor": "customendpoint",
  "apiKey": "YOUR_API_KEY",
  "apiType": "responses",
  "models": [
    {
      "id": "claude-sonnet-4-6",
      "name": "Claude Sonnet 4.6 (Messages)",
      "url": "https://api.anthropic.com/v1/messages",
      "toolCalling": true,
      "vision": true,
      "maxInputTokens": 200000,
      "maxOutputTokens": 64000
    }  
  ]
}
]

Test4: Bonus Steps

If you have a custom model or custom endpoint , you can try using that .

[vs-code-engineering]

updated

[N0SAFE]

I have tried this new option and it seems that the API key is not being forwarded correctly to the provider. With the exact same configuration as the customoai one, it now errors with "Invalid API key," even though I have made sure to set the correct API key.

{
"name": "opencode",
"vendor": "customendpoint",
"apiKey": "${input:chat.lm.secret.-4ab72f15}",
"apiType": "messages",
"models": [
{
"id": "glm-5",
"name": "glm-5",
"url": "https://opencode.ai/zen/go/v1/chat/completions",
"toolCalling": true,
"vision": true,
"maxInputTokens": 128000,
"maxOutputTokens": 16000
}
]
},
{
"name": "OpenAI Compatible (Deprecated)",
"vendor": "customoai",
"apiKey": "${input:chat.lm.secret.-4ab72f15}",
"models": [
{
"id": "glm-5",
"name": "glm-5 (Deprecated)",
"url": "https://opencode.ai/zen/go/v1/chat/completions",
"toolCalling": true,
"vision": true,
"maxInputTokens": 128000,
"maxOutputTokens": 16000
}
]
}

[aiday-mar]

I am getting the same error as the user above and I am taking the API keys from https://platform.openai.com/api-keys

[smerschjohann]

at least in the message from N0SAFE (@N0SAFE), I can see that the wrong apiType is selected for that endpoint