Support dynamic apiKey

[AlexanderYastrebov]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

In our environment we'd like to use custom model served via proxy.
This proxy requires API key that user may obtain via pre-installed authentication tool (get_token).
The proposal is to support dynamic API key by using standard output of the tool.

Solution

Possible solution could be to modify config.ts such that it executes the tool:

const { execSync } = require("child_process");

export function modifyConfig(config: Config): Config {
  config.models
    .filter(model => model.apiKey && model.apiKey.startsWith("$"))
    .forEach((model) => {
      const cmd = model.apiKey.substr(1).trim();
      Object.defineProperty(model, "apiKey", {
        get() {
          return execSync(cmd, { encoding: "utf8" }).trim();
        },
      });
    });
  return config;
}

and config.json

{
  "models": [
    {
      "title": "myLLM",
      "provider": "openai",
      "model": "bar/baz",
      "apiBase": "https://example.com/v1",
      "apiKey": "$ get_token"
    }
  ],
...

Another option is to allow apiKey to be a function:

const { execSync } = require("child_process");

export function modifyConfig(config: Config): Config {
  config.models
    .filter(model => model.apiKey && model.apiKey.startsWith("$"))
    .forEach((model) => {
      const cmd = model.apiKey.substr(1).trim();
      model.apiKey = () => {
        return execSync(cmd, { encoding: "utf8" }).trim();
      };
    });
  return config;
}

The challenge is to ensure that apiKey property getter is invoked each time its needed to refresh the value.

[AlexanderYastrebov]

A possible workaround for now could be to update config.json via cron. VSCode detects config update and picks up new key.

[AlexanderYastrebov]

A possible workaround for now could be to update config.json via cron. VSCode detects config update and picks up new key.

IntelliJ plugin does not reload config.json, only when saved from the IDE itself due to this related code.

[AlexanderYastrebov]

IntelliJ plugin does not reload config.json

Looks like its not easy to implement external updates, see super old ticket https://youtrack.jetbrains.com/issue/IJPL-2188/Background-changes-by-external-tool-not-being-picked-up-until-VFS-refresh-happens

For externally changed files they suggest File -> Reload All from Disk 🤣

[ViZiD]

It would be great if there was a feature like that 👍👍👍

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[AlexanderYastrebov]

A possible workaround for now could be to update config.json via cron. VSCode detects config update and picks up new key.

Here is the script that works for me:

#!/bin/bash -l
# Note -l above that is needed to have proper environment variables

set -eou pipefail

jq --arg token "$(get_token)" \
    '.models[0].apiKey = $token' \
    ~/.continue/config.json > ~/.continue/config.json.tmp \
    && mv ~/.continue/config.json.tmp ~/.continue/config.json

and then add this script to the crontab.

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!