Join our Discord for discussions, feature requests, and community support.

The smartest, fastest, and completely free local router for your AI coding needs.

modelrelay is an OpenAI-compatible local router that benchmarks free coding models across top providers and automatically forwards your requests to the best available model.

• 💸 Completely Free: Stop paying for API usage. We seamlessly provide access to robust free models.
• 🧠 State-of-the-Art (SOTA) Models: Out-of-the-box availability for top-tier models including Kimi K2.5, Minimax M2.5, GLM 5, Deepseek V3.2, and more.
• 🏢 Reliable Providers: We route requests securely through trusted, high-performance platforms like NVIDIA, Groq, OpenRouter, OpenCode Zen, Ollama, Kiro, and Google.
• ⚡ Lightning Fast: The built-in benchmark continually evaluates metrics to pick the fastest and most capable LLM for your request.
• 🔄 OpenAI-Compatible: A perfect drop-in replacement that works seamlessly with your existing tools, scripts, and workflows.

npm install -g modelrelay

# Start it
modelrelay

Once started, modelrelay is accessible at http://localhost:7352/.

Router endpoint:

• Base URL: http://127.0.0.1:7352/v1
• API key: any string
• Model: auto-fastest (router picks actual backend)

• Docker Engine
• Docker Compose (the docker compose command)

mkdir modelrelay

cd modelrelay

curl -fsSL -o Dockerfile https://raw.githubusercontent.com/ellipticmarketing/modelrelay/master/Dockerfile
curl -fsSL -o docker-compose.yml https://raw.githubusercontent.com/ellipticmarketing/modelrelay/master/docker-compose.yml

docker compose up -d --build

Once running, modelrelay is accessible at http://localhost:7352/.

Use modelrelay onboard to save provider keys and auto-configure integrations for OpenClaw or OpenCode.

modelrelay onboard

If you prefer manual setup, use the examples below.

modelrelay onboard can auto-configure OpenCode.

If you want manual setup, put this in ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "router": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "modelrelay",
      "options": {
        "baseURL": "http://127.0.0.1:7352/v1",
        "apiKey": "dummy-key"
      },
      "models": {
        "auto-fastest": {
          "name": "Auto Fastest"
        }
      }
    }
  },
  "model": "router/auto-fastest"
}

modelrelay onboard can auto-configure OpenClaw.

If you want manual setup, merge this into ~/.openclaw/openclaw.json:

{
  "models": {
    "providers": {
      "modelrelay": {
        "baseUrl": "http://127.0.0.1:7352/v1",
        "api": "openai-completions",
        "apiKey": "no-key",
        "models": [
          { "id": "auto-fastest", "name": "Auto Fastest" }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "modelrelay/auto-fastest"
      },
      "models": {
        "modelrelay/auto-fastest": {}
      }
    }
  }
}

modelrelay [--port <number>] [--log] [--ban <model1,model2>]
modelrelay onboard [--port <number>]
modelrelay install --autostart
modelrelay start --autostart
modelrelay uninstall --autostart
modelrelay status --autostart
modelrelay update
modelrelay autoupdate [--enable|--disable|--status] [--interval <hours>]
modelrelay autostart [--install|--start|--uninstall|--status]
modelrelay config export
modelrelay config import <token>

Request terminal logging is disabled by default. Use --log to enable it.

modelrelay install --autostart also triggers an immediate start attempt so you do not need a separate command after install.

During modelrelay onboard, you will also be prompted to enable auto-start on login.

modelrelay update upgrades the global npm package and, when autostart is configured, stops the background service first and starts it again after the update.

Auto-update is enabled by default. While the router is running, modelrelay checks npm periodically (default: every 24 hours) and applies updates automatically.

Use modelrelay autoupdate --status to inspect state, modelrelay autoupdate --disable to turn it off, and modelrelay autoupdate --enable --interval 12 to re-enable with a custom interval.

Use modelrelay config export to print a transferable config token (base64url-encoded JSON), and modelrelay config import <token> to load it on another machine. You can also import by stdin:

modelrelay config export | modelrelay config import

POST /v1/chat/completions is an OpenAI-compatible chat completions endpoint.

• Use model: "auto-fastest" to route to the best model overall
• Use a grouped model ID such as minimax-m2.5, kimi-k2.5, or glm4.7 to route within that model group
• Use model: "tag:<name>" to route to the best available model carrying a user-defined tag
• For grouped IDs, modelrelay selects the provider with the best current QoS for that group
• Use model: "tag:<name>" (e.g. tag:coding) to route to the best currently available model carrying that capability tag, regardless of which specific model that turns out to be. This is useful because the free models behind modelrelay come and go as availability changes — routing by tag survives a given model disappearing, where routing by a specific model/group ID does not.
• In the Web UI, pinned models can now use either Canonical Group mode (default, pins the same model across providers) or Exact Provider Row mode from Settings
• Streaming and non-streaming requests are both supported

GET /v1/models returns the models exposed by the router.

• Model IDs are grouped slugs such as minimax-m2.5, kimi-k2.5, and glm4.7
• Each grouped ID can represent the same model across multiple providers
• When you select one of these IDs in /v1/chat/completions, modelrelay routes the request to the provider with the best current QoS for that model group
• auto-fastest is also exposed and routes to the best model overall
• Each entry includes a tags array combining curated capability tags with any user-defined tags.
• Add custom tags in the Web UI by opening a model row and editing Custom Routing Tags. Assignments are shared across providers for the canonical model and stored in ~/.modelrelay.json.

For example, assign coding to several models in the UI, then request model: "tag:coding". Normal QoS ranking, availability filtering, and retry behavior choose the best currently eligible tagged model.

Example:

{
  "object": "list",
  "data": [
    { "id": "auto-fastest", "object": "model", "owned_by": "router" },
    { "id": "minimax-m2.5", "object": "model", "owned_by": "relay", "tags": ["agentic", "general", "coding"] },
    { "id": "kimi-k2.5", "object": "model", "owned_by": "relay", "tags": ["agentic", "coding", "general"] },
    { "id": "glm4.7", "object": "model", "owned_by": "relay", "tags": ["agentic", "coding", "general"] }
  ]
}

Every model is assigned one or more capability tags from a fixed vocabulary: coding, reasoning, general, fast, agentic. Tags are curated by maintainers (see tags.js) rather than freeform, so tag:<name> routing always resolves to a known, checkable capability.

Use model: "tag:<name>" in /v1/chat/completions to route to the best currently available model with that tag, instead of naming a specific model. This is the recommended way to request "a good coding model" or "a fast model" without hardcoding a model ID that might stop being served.

• Router config file: ~/.modelrelay.json
• API key env overrides:
  • NVIDIA_API_KEY
  • GROQ_API_KEY
  • CEREBRAS_API_KEY
  • SAMBANOVA_API_KEY
• OPENROUTER_API_KEY
• OPENCODE_API_KEY
• OLLAMA_API_KEY
• OLLAMA_BASE_URL
• OLLAMA_MODEL
  • CODESTRAL_API_KEY
  • HYPERBOLIC_API_KEY
  • SCALEWAY_API_KEY
  • KIRO_REFRESH_TOKEN
  • KIRO_OAUTH_CLIENT_ID (optional, for AWS Builder/IDC refresh flow)
  • KIRO_OAUTH_CLIENT_SECRET (optional, for AWS Builder/IDC refresh flow)
  • GOOGLE_API_KEY

Kiro OAuth notes:

• Base endpoint is preconfigured to https://codewhisperer.us-east-1.amazonaws.com/generateAssistantResponse
• Current Kiro model IDs include claude-sonnet-4.5 and claude-haiku-4.5
• Authentication uses OAuth access tokens refreshed from:
  • KIRO_REFRESH_TOKEN, or
  • ~/.aws/sso/cache (auto-detected refresh token), following OmniRoute’s approach.

For hosted Ollama, set OLLAMA_API_KEY and optionally override OLLAMA_BASE_URL / OLLAMA_MODEL. If you leave the Ollama base URL blank in the UI, modelrelay defaults to https://ollama.com/v1. With a valid Ollama API key, modelrelay will discover available Ollama models automatically. If you point Ollama at a local host such as http://127.0.0.1:11434, modelrelay will also auto-discover models and does not require an API key.

modelrelay supports configuring multiple OpenAI-compatible upstream endpoints (vLLM, llama.cpp, custom relays, etc.). Each endpoint exposes a single model id and is routed independently.

• In the Web UI, click + Add Endpoint under the OpenAI-Compatible endpoints group, supply a name, base URL, model id, and optional API key. Each endpoint then gets its own provider row with status, ping, and rate-limit information.
• modelrelay automatically probes /v1/models on each endpoint and exposes every returned model as a routable row. The manually configured model id (if any) is merged in as a fallback. Discovery is on by default and can be toggled per-endpoint with the "Discover models from /v1/models" checkbox.
• Endpoints are stored in ~/.modelrelay.json under composite keys like openai-compatible:my-vllm:

  {
    "apiKeys": {
      "openai-compatible:my-vllm": "sk-…",
      "openai-compatible:groq-clone": "sk-…"
    },
    "providers": {
      "openai-compatible:my-vllm":    { "enabled": true, "name": "Local vLLM", "baseUrl": "http://localhost:8000/v1", "modelId": "qwen-coder" },
      "openai-compatible:groq-clone": { "enabled": true, "name": "Groq Clone", "baseUrl": "https://example/v1",        "modelId": "llama-3.3-70b" }
    }
  }

• Legacy single-endpoint configs (a bare openai-compatible entry without an instance suffix) are migrated automatically to openai-compatible:default on first run.
• The legacy env vars OPENAI_COMPATIBLE_API_KEY / OPENAI_COMPATIBLE_BASE_URL / OPENAI_COMPATIBLE_MODEL continue to work and apply to the :default instance.
• Endpoints can also be managed via the API: POST /api/openai-compatible/endpoints (body: {name, baseUrl, modelId, apiKey?}) and DELETE /api/openai-compatible/endpoints/<id>.

• In the Web UI, open Settings -> Configuration Transfer to export/copy/import a token.
• The token includes your full config (including API keys, provider toggles, pinning mode, bans, filter rules, and auto-update settings).
• Treat tokens as secrets. Anyone with the token can import your keys/settings.
• Alternative: copy the config file directly from ~/.modelrelay.json to the other machine at the same path (~/.modelrelay.json).

To trigger a manual npm update and restart the service, run:

npm i -g modelrelay@latest
modelrelay autostart --start

You can point the updater at a local tarball instead of the npm registry:

npm pack
MODELRELAY_UPDATE_TARBALL=./modelrelay-1.8.3.tgz pnpm start

If you want the Web UI to always show an update while testing, set a higher forced version:

MODELRELAY_FORCE_UPDATE_VERSION=9.9.9

If the tarball filename does not contain a semantic version, also set:

MODELRELAY_UPDATE_VERSION=1.8.3

When MODELRELAY_UPDATE_TARBALL is set, the Web UI update flow and modelrelay update install from that tarball and bypass the normal Git checkout update block. This is for local testing only. MODELRELAY_FORCE_UPDATE_VERSION only affects version detection; the actual install still comes from the tarball path.

⭐️ If you find modelrelay useful, please consider starring the repo!