Support Custom Models and Providers via Configuration File

[badlogic]

Support Custom Models and Providers via Configuration File

Problem

Currently, pi (coding-agent):

• Hardcodes claude-sonnet-4-5 as the default model for new sessions
• Fails if no Anthropic API key is set, even if other providers are available
• Cannot add custom providers or local models (Ollama, vLLM, LM Studio, etc.)

Solution

Add support for custom models and providers via ~/.pi/agent/models.json configuration file.

Configuration File Structure

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "apiKey": "OLLAMA_API_KEY",
      "api": "openai-completions",
      "models": [
        {
          "id": "llama-3.1-8b",
          "name": "Llama 3.1 8B (Local)",
          "reasoning": false,
          "input": ["text"],
          "cost": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0},
          "contextWindow": 128000,
          "maxTokens": 32000
        }
      ]
    },
    "custom-provider": {
      "baseUrl": "https://api.custom.com/v1",
      "apiKey": "CUSTOM_API_KEY",
      "api": "openai-completions",
      "models": [
        {
          "id": "legacy-model",
          "name": "Legacy Model",
          "reasoning": false,
          "input": ["text"],
          "cost": {"input": 1.0, "output": 2.0, "cacheRead": 0, "cacheWrite": 0},
          "contextWindow": 8192,
          "maxTokens": 4096
        },
        {
          "id": "new-model",
          "name": "New Model",
          "api": "openai-responses",
          "reasoning": true,
          "input": ["text", "image"],
          "cost": {"input": 0.5, "output": 1.0, "cacheRead": 0.1, "cacheWrite": 0.2},
          "contextWindow": 128000,
          "maxTokens": 32000
        }
      ]
    }
  }
}

Key Features

API Key Resolution:

• Check if apiKey value exists as environment variable → use env var value
• Otherwise treat as literal API key
• Example: "apiKey": "OLLAMA_API_KEY" checks process.env.OLLAMA_API_KEY first, then treats as literal

API Override:

• Provider-level api sets default for all models
• Model-level api overrides provider default
• Allows mixing APIs through same baseUrl

Model Priority (No Hardcoded Defaults):

1. CLI args (--provider, --model)
2. Restored from session (if --continue or --resume)
3. Saved default from settings.json
4. First available model with valid API key
5. null (allowed in interactive mode)

Error Handling:

• Interactive mode: Allow startup with no model, show error in TUI on message submission
• Non-interactive mode: Fail early if no model or API key available
• Model selector: Reload models.json on every open (allows live editing)
• Invalid models.json: Silent failure, return empty model list (user sees "no models" when using /model)

Implementation Tasks

Code Changes

New Files:

• packages/coding-agent/src/model-config.ts
  • loadAndMergeModels() - Load built-in + custom models
  • loadCustomModels() - Parse ~/.pi/agent/models.json
  • resolveApiKey(keyConfig) - Resolve env var or literal
  • getAvailableModels() - Filter models with valid API keys
  • validateConfig(config) - Schema validation with TypeBox
  • parseModels(config) - Convert config to Model[]

Modified Files:

• packages/coding-agent/src/settings-manager.ts

  • Add defaultProvider?: string to Settings
  • Add defaultModel?: string to Settings
  • Add getDefaultProvider(), setDefaultProvider()
  • Add getDefaultModel(), setDefaultModel()

• packages/coding-agent/src/main.ts

  • Remove hardcoded "anthropic" and "claude-sonnet-4-5"
  • Implement model selection priority (CLI → session → settings → first available → null)
  • Add isInteractive check: parsed.mode === undefined && parsed.messages.length === 0
  • Non-interactive: fail early if no model/API key
  • Interactive: allow null model, validate on submission
  • Update getApiKeyForProvider() to check custom providers first

• packages/coding-agent/src/tui/tui-renderer.ts

  • Add error display in TUI (no console.error)
  • Check model + API key on message submission
  • Show helpful error messages with next steps
  • Clear errors after user fixes issue

• packages/coding-agent/src/tui/model-selector.ts

  • Call getAvailableModels() fresh on every open
  • Show custom models alongside built-in ones
  • Save selected model as default via settingsManager
  • Show error in TUI if no models available (with example config)

Documentation Updates

packages/coding-agent/README.md:

• Add new section: "Custom Models and Providers"
• Document models.json structure with examples
• Explain API key resolution
• Explain API override behavior
• Add example configs for Ollama, vLLM, LM Studio
• Update "Planned Features" to remove "Custom/local models" (now implemented)

CHANGELOG.md:

• Add entry under "Unreleased" section
• Title: "Custom Models and Providers via Configuration File"
• List key features:
  • Support for custom providers via ~/.pi/agent/models.json
  • No longer hardcodes Anthropic/Claude as default
  • Smart model selection: CLI args → session → settings → first available
  • Live reload of models.json (no restart needed)
  • Per-model API override support

Testing Checklist

• Fresh install with no API keys → interactive mode shows helpful error
• Fresh install with no API keys → non-interactive mode fails with helpful error
• Fresh install with ANTHROPIC_API_KEY → uses Claude
• Create models.json → models appear in /model selector
• Edit models.json during session → /model shows updated list
• Agent writes models.json → /model picks it up immediately
• Invalid models.json → graceful handling, shows "no models"
• Model with API override → uses correct API
• Model selection → saves to settings.json as default
• Restart → uses saved default from settings.json

Example User Flows

No API keys, interactive mode:

$ pi
[TUI starts]
You: hello
[Error in TUI: "No model selected. Set an API key or create ~/.pi/agent/models.json"]
[User sets ANTHROPIC_API_KEY in another terminal]
You: /model
[Shows Anthropic models, user selects one]
You: hello
[Works!]

Add Ollama during session:

You: I want to use my local Ollama server
Assistant: I'll help set that up...
[Agent writes ~/.pi/agent/models.json]
You: /model
[Shows Ollama models]
[User selects llama-3.1-8b, continues with local model]

Non-interactive without API key:

$ pi "write a haiku"
Error: No models available.

Set an API key environment variable:
  ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY, etc.

Or create ~/.pi/agent/models.json
[exits with code 1]