How to configure an OpenAI Compatibe Model Provider which supports dynamic model discovery?

[singhal-shagun]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

VS Code

Body

Hi! I'm trying to set up a custom chat model provider in VS Code, and I'm running into a limitation.

The current situation:

• When I use built-in providers (Google, Ollama, etc.), VS Code automatically shows all available models in the Chat: Manage Language Models modal
• When I install a language provider extension from the Marketplace, then too, available models appear automatically in the Chat: Manage Language Models
• But when I configure a custom endpoint via the "Manage Language Models" option, I'm required to manually enter each model's configuration as a JSON object in chatLanguageModels.json

What I'm trying to do:
I want to configure an OpenAI-compatible API provider where:

1. I only need to specify the base URL and the API key
2. VS Code automatically discovers and displays available models (without me manually listing them)

Is there a way to set up such a dynamic model discovery for custom endpoints without building a full VS Code extension? Or is writing an extension the only way to achieve this?

[rdpsawada]

Right now, VS Code doesn’t expose automatic model discovery for custom endpoints configured through chatLanguageModels.json the same way it does for built-in providers or provider extensions. That JSON-based setup is static, you must define models manually. Dynamic discovery (for example, calling an OpenAI-compatible /v1/models endpoint and populating the model list automatically) is handled by provider implementations, not by the generic custom endpoint configuration. So if you want users to enter only a base URL + API key and have models appear automatically, the supported path today is to build a language model provider extension that implements discovery logic and surfaces the models to VS Code. Without an extension, there isn’t currently a built-in mechanism for auto-discovering models from arbitrary custom endpoints.

[kartik99917]

You're not missing anything — this is a real gap, not user error. The built-in "Custom OpenAI Compatible" path through chatLanguageModels.json is in the manual-entry bucket of BYOK providers: some providers auto-discover models with just a base URL + key, others require you to hand-list each model. Custom endpoints currently fall in the second group, and there's already an open bug (microsoft/vscode#289015/#289003) about custom OpenAI-compatible configs not even reliably appearing in the picker, so it's a rough edge right now, not just a missing nice-to-have.
The dynamic-discovery hook you want does exist — it's the languageModelChatProviders contribution point, which lets an extension implement provideLanguageModelChatInformation to fetch and return the model list at runtime (that's exactly how Google/Ollama auto-populate). So yes, an extension is the mechanism for what you want. The good news is you likely don't need to write one yourself — there are marketplace extensions (e.g. vscode-custom-llm-provider) that already wrap arbitrary OpenAI-compatible endpoints with exactly this auto-discovery behavior. Worth trying one of those before building your own.

[rehuux]

The short answer is that you are encountering a hard limitation of VS Code's current architecture for custom endpoints. You are required to build an extension or use a third-party extension to achieve automatic model discovery. The manual configuration is the only method available for a "vanilla" custom endpoint setup .

Why Manual Configuration is Required

When you use the built-in Chat: Manage Language Models option to add a Custom Endpoint, you are directly writing to the chatLanguageModels.json configuration file . This file defines a static list of models. Because it is a static configuration, there is no built-in mechanism for dynamic model discovery for these specific endpoints . This is intended to provide users with fine-grained control over which models appear .

How to Achieve Dynamic Model Discovery

To get the automatic model listing you want, you have a few options that go beyond the manual JSON configuration:

1. Use a VS Code Extension: Extensions that use the Language Model Chat Provider API can register and manage model providers programmatically . This is what allows built-in providers like Google and Ollama to show dynamic lists.
2. Third-Party Extensions: Several extensions have already solved this problem for custom endpoints. For example, the "Addi" extension allows you to connect to any OpenAI-compatible API and enables dynamic model discovery, with models appearing directly in Copilot's model picker . Another option is "LM Manager," which provides a graphical interface for managing custom configurations in chatLanguageModels.json and may help simplify the management process .
3. Use VS Code Insiders: Some experimental features are only available in the Insiders build. For instance, the "CustomOAI" provider, which supports automatic model discovery from a /models endpoint, is currently gated behind a feature flag and only appears in VS Code Insiders . A minimal extension exists to unlock this provider in the Stable version .

[singhal-shagun]

Thanks for responding!

BTW, by "Addi", are you referring to "AddiEx"?
And I couldn't find "LM Manager" in the marketplace.

[khazar08]

As of today, if you're using VS Code's built-in custom model endpoint support via chatLanguageModels.json, there is no built-in mechanism for dynamic model discovery. You must explicitly define each model in the JSON configuration.

The automatic discovery you're seeing with providers like Google, Ollama, or Marketplace extensions happens because those providers implement the necessary logic themselves. They can query an endpoint (for example, /models on an OpenAI-compatible API), retrieve the available models, and register them with VS Code programmatically.

A plain custom endpoint configuration is essentially static metadata. VS Code knows the base URL and credentials, but it does not automatically call the provider's model-listing API and populate the model picker.

Therefore, if your goal is to enter only a base URL and API key and have models appear automatically, you'll need some code that performs discovery and registers the models. In practice, that means writing a VS Code extension (or modifying an existing one) that acts as a language model provider. The extension can query the OpenAI-compatible /models endpoint and dynamically expose whatever models are available.

So the answer is: with the built-in custom endpoint configuration alone, no; for dynamic discovery, a provider extension is currently required.

[kartik-012]

Hey @singhal-shagun....... I went through the discussion and here's the simple version.

From what I understood, what you want isn't fully supported in VS Code yet.

Right now, you have 3 options:

1. Use the Addi extension – This is probably the easiest way. Just add your server URL and API key, and it can handle the models for you. No need to build your own extension.

2. Try VS Code Insiders – It has a feature called CustomOAI that can automatically detect models from your server. It's still experimental, but it's worth trying if you're okay with using Insiders.

3. Write a small extension – Only if you need full control. It doesn't have to be a big project, but it's more work than the other two options.

The main reason this doesn't work today is because chatLanguageModels.json only supports manually adding models. It wasn't designed to automatically discover them from a custom server.

So I'd try Addi first, then VS Code Insiders, and only consider writing an extension if those don't meet your needs.

Hope that helps... i tried my best to tell u