Connect OpenAI and Other Remote APIs

How-to. Let agents use the OpenAI API, any OpenAI-compatible server, or the Anthropic API. Agents stay network-less; requests go through llm-bridge. Remote models and llm-bridge explains how this works.

Remote means your code leaves the machine and tokens are billed. Each task is capped by token_budget (3M by default for remote models).

1. Add the key to secrets.env

# secrets.env  (mode 600, read only by the llm-bridge container)
OPENAI_API_KEY=sk-...

2. Add the model to config/models.yaml

OpenAI:

  gpt-5-mini:
    backend: openai
    endpoint: https://api.openai.com/v1
    model_name: gpt-5-mini
    api_key_env: OPENAI_API_KEY
    token_param: max_completion_tokens   # newer OpenAI models reject max_tokens
    drop_params: [temperature, top_p]    # reasoning models reject custom sampling
    max_agents: 4
    token_budget: 2000000

Any OpenAI-compatible server (llama.cpp, LM Studio, SGLang, TGI, vLLM elsewhere, OpenRouter, Groq, …):

  llamacpp-local:
    backend: openai
    endpoint: http://127.0.0.1:8080/v1   # llm-bridge uses host networking, so 127.0.0.1 works
    model_name: qwen3-coder
    max_output_tokens: 8192              # clamp for servers with small output limits
    # api_key_env: OPENROUTER_API_KEY    # if the server needs a key
    # extra_body: {provider: {sort: throughput}}   # provider-specific fields

Anthropic API (pass-through):

  claude-haiku:
    backend: anthropic
    endpoint: https://api.anthropic.com
    model_name: claude-haiku-5-5
    api_key_env: ANTHROPIC_API_KEY

3. Apply and test

docker compose up -d                       # recreates llm-bridge with the new secrets.env
docker compose restart agentbox agentbox-launcher
scripts/smoke-test.sh gpt-5-mini

delegate_task checks the key and the endpoint through llm-bridge before starting the agent. A missing key or a rejected key is reported as an error, without running anything.

Requirements for the upstream model:

• it supports function or tool calling;
• it has a context of ≥ 64k tokens;
• it streams (stream: true). Servers that ignore streaming still work.