Adding an LLM Provider

Adding a New LLM Provider

This guide walks through adding a new LLM provider to CortexPrism.

Steps

1. Create the Provider File

Create src/llm/your-provider.ts. Most providers extend OpenAICompatibleProvider:

import { OpenAICompatibleProvider } from "./openai-compatible.ts";

export class YourProvider extends OpenAICompatibleProvider {
  readonly name = "your-provider";
  readonly defaultModel = "your-default-model";

  constructor(apiKey?: string, baseUrl?: string) {
    super(
      apiKey ?? "",
      baseUrl ?? "https://api.your-provider.com/v1",
      "your-provider",
      "your-default-model",
    );
  }
}

For non-OpenAI-compatible providers, implement the LLMProvider interface directly:

import type { LLMProvider, CompletionOptions, CompletionResult, CompletionChunk } from "./types.ts";

export class YourProvider implements LLMProvider {
  readonly name = "your-provider";
  readonly defaultModel = "your-default-model";

  constructor(private apiKey: string) {}

  async complete(options: CompletionOptions): Promise<CompletionResult> {
    // Implement complete API call
  }

  async *stream(options: CompletionOptions): AsyncIterable<CompletionChunk> {
    // Implement streaming API call
  }
}

2. Register in Factory

Add to src/llm/factory.ts in the provider creation function.

3. Add Config Schema

Add the provider to the ProviderConfig type union in src/config/config.ts.

4. Add to Setup Wizard

Add the provider option to the interactive setup in src/cli/setup.ts.

5. Add Model Listing (if applicable)

Create a yourProviderModels() function and register it in src/server/models.ts.

6. Add Provider-Specific Settings (optional)

If the provider has unique parameters, add them to CompletionOptions in src/llm/types.ts and wire them through in the provider adapter.

7. Document

• Add to the README provider list
• Add to the LLM Providers wiki page
• Add a CHANGELOG entry

LLMProvider Interface

interface LLMProvider {
  readonly name: string;
  readonly defaultModel: string;
  complete(options: CompletionOptions): Promise<CompletionResult>;
  stream(options: CompletionOptions): AsyncIterable<CompletionChunk>;
}

CompletionOptions

interface CompletionOptions {
  messages: Message[];
  systemPrompt?: string;
  temperature?: number;
  maxTokens?: number;
  topP?: number;
  model?: string;
  stream?: boolean;
  signal?: AbortSignal;
  // Provider-specific:
  reasoningEffort?: "low" | "medium" | "high";
  repetitionPenalty?: number;
  numCtx?: number;
  keepAlive?: string;
  // ... and more
}

See Also

• LLM Providers — All 24 supported providers
• Model Routing — Router integration
• Contributing — General contribution guidelines