[Test] BYOK improvements

[vritant24]

Refs: #318026, #317882, #317881, #317595, #317419, #317382, #317308, #314825

• Any OS @federicobrancasi

Complexity: 3

Authors: @vritant24

Create Issue

---

Setting up BYOK providers

For adding Built-in BYOK providers:

• get a secret (apices) from this key vault (I recommend anthropic or open_ai)
• Using Chat: Manage Language Models > Add Models, add the corresponding provider (Anthropic or Open AI)
  • the UI should take you through the configuration

For adding 3rd party extension based providers, there can be many ways, but I have tried LM Studio:

1. LM Studio. You will need to install https://lmstudio.ai on your device and download a model on it. This extension will help you connect to the locally running model and show it in the Copilot Model Picker

There are other ways, but I haven't tried them:
AWS Bedrock https://marketplace.visualstudio.com/items?itemName=arifum.bedrock-vscode-chat (you will need to create an account and then connect with that
AI Toolkit for VS Code

Testing

Toggling visibility for models

• Open Chat, open the model picker, and select Manage Models....
• Toggling the visibility should hide/show the model from the chat window model picker

Manage configured providers

• Configure a Custom Endpoint in Manage Language Models view.
• The configure icon on the group added should allow you to
  • rename the group
  • update the API key
  • add a model
  • delete the group

Configuration file Intellisense

• Use Go to Settings for a configured provider from the main VS Code window.
• Confirm chatLanguageModels.json opens and offers JSON completions and validation for model/provider configuration.

Agents window

• In a local session in the Agents window, open the model picker and confirm Manage Models... is available and allows you to select the same models available in the editor window.

Utility model settings

• Open Settings and locate chat.utilityModel and chat.utilitySmallModel.
• Inspect each setting's available model choices.
• Confirm GitHub Copilot-provided models are not listed as explicit choices, while Default and available non-Copilot/BYOK models remain available.

[federicobrancasi]

Tested on macOS.
No issues found.