Export functions to add/remove custom models to other VS Code extensions

[lupreCSC]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

We are trying to use continue with some locally hosted models were happy to see that there is way to provide a custom model querying code via the config.ts. Sadly, this is somewhat limited because it does not allow access to the vscode API object, which we would to interact with VSCode's authentication providers. It gets even more difficult if we want to use some features provided by another extension inside the config.ts code. A more flexible programmatic way to add models to the continue extension would be appreciated.

Solution

VS Code allows extensions to export an API object with functions for other extensions to invoke from the activate function. It would be great if continue could export a function to add a custom model.

Would that be a feature you would be willing to consider?

(Unfortunately unable to join discord).

[sestinj]

@lupreCSC thanks for making the detailed PR! I apologize for taking so long to get back with you.

I feel this adds a good amount of complexity, so I'd want to understand the reasons really well. I think I understand the need to use authentication for custom LLMs at a high level, but what is the exact mechanism you'll want to use for this? And when you say "we want to use some features provided by another extension inside the config.ts code", what sorts of features are you interested in using?

Depending on what you're looking for, I think that a possible simpler option would to be to pass the vscode object to the modifyConfig function

[lupreCSC]

Hi @sestinj and please excuse the delay in my response.

Our setting is that we are running a model that is not using the OpenAI API and requires OAuth / OpenID Connect based authentication from our authorization servers. We have now set up a VSCode extension that implements model API requests as well as the authentication and are looking for a way to register that into the model list of the Continue extension.

The following are the problems with modifyConfig as we currently see them:

• it forces suppliers of additional models to "outsource" the model implementation to the user in the sense that the set up instruction would always be something like "please copy the following into the config.ts in this particular location" - and will likely become especially complex for users that would want to use multiple custom models; by contrast, the ability to register models programmatically would be as simple as installing another extension for the user
• config.ts modifyConfig becomes hard to work with for any but the most trivial custom models due to lack of proper code completion and syntax highlighting when editing in VSCode (since it isn't aware of existing imports and other context because the file is dynamically loaded/parsed by the Continue extension instead of being part of a regular import tree)
• implementations in modifyConfig cannot rely on functionality provided by other, existing extensions (e.g., authentication logic) even if the vscode object would be passed in. Importing functionality from another extension requires that extension to be listed as in the extensionDependencies of the package.json of the importing extension (i.e., Continue in this case).

Having Continue export an API endpoint that allows other extension to register models solves all these issues:

• for the user, adding complex custom models becomes as easy as installing another extension
• implementors of custom models enjoy full VSCode functionality since they are simply implementing an extension like any other
• extensions providing additional custom models can properly depend on Continue and any other extension providing required additional functionalities