Repository navigation
Releases: exitCodeNihil/llm-router
Releases · exitCodeNihil/llm-router
Release list
v0.2.0
0.2.0 (2026-10-13)
⚠ BREAKING CHANGES
- remove edge nodes and the --mode flag
Features
- add a Rust data-plane gateway that joins gateway groups (17ab06d)
- add gateway groups and a sync protocol for data-plane gateways (3eee415)
Performance Improvements
- encode gateway usage batches off the request threads (a9f1472)
Code Refactoring
- remove edge nodes and the --mode flag (a902c36)
v0.1.2
v0.1.1
v0.1.0
Initial release.
- OpenAI-compatible gateway (
/v1/chat/completions,/v1/completions,/v1/embeddings,
/v1/models) with streaming passthrough and exact token accounting - Providers: Azure OpenAI / AI Foundry (Entra ID or api-key outbound auth, native price
catalog), Google Vertex AI (Application Default Credentials or a service-account JSON key;
Gemini and every Model Garden MaaS model through the
OpenAI-compatible endpoint, Claude through rawPredict; publisher-catalogue discovery and
gcp/catalog prices), OpenRouter (discovery fills in its published prices from the
catalog) and any OpenAI-compatible backend (vLLM, LM Studio, TGI, Ollama)
with custom per-model pricing - Pass-through auth mode (
oauth_passthrough): forwards the caller's own upstream
credential andanthropic-betaheader instead of storing one, so Claude Code can run
on a Claude subscription through the gateway (clients authenticate withX-Llmr-Key);
that traffic is always recorded unpriced.LLMR_SEED_CLAUDE_SUBSCRIPTION=1seeds it with
oneclaude-*model, so new Claude models need no gateway update - Wildcard model names: a model name ending in
*serves every name with that prefix
(exact names first, then the longest prefix), with the requested name sent upstream when
the upstream name ends in*too - Virtual API keys, users, teams; budgets (daily/monthly/total) and rpm/tpm rate limits
per key/user/team; model policy on teams, users and keys (a person's default is the union of
their teams' policies; keys only narrow), per-member budget shares within a team,
member-minted team keys, and a Models page for every signed-in user - Inbound Azure Entra ID / GCP identity-token authentication with claim mapping and
optional auto-provisioning - Web console: dashboards, key management, teams, providers/deployments, pricing,
OIDC SSO, RBAC (admin / team admin / member) - Hardening from the pre-release review:
LLMR_ENCRYPTION_KEYis required and
placeholder values are refused (it signs console sessions); the workspace IDE is served
from its own origin (LLMR_IDE_LISTEN/LLMR_IDE_ORIGIN) and the proxy strips the
caller's credentials before the container; console mutations are checked against the
browser'sOrigin; keys of disabled users stop working; deleting a user removes their
workspace containers and volumes; deleting a team asks before revoking its keys; the
last enabled admin cannot be disabled or demoted and first-run setup only opens on an
empty database; members cannot set budgets or rate limits on their own keys; database
errors are no longer echoed to clients; streamed Claude usage counts cache writes - Edge gateway mode: config long-poll sync, sealed disk cache for cold starts and
control-plane outages, batched usage shipping - Workspaces: per-user containers with VS Code (code-server), a terminal and a pi-powered
chat; admin-managed templates (base, python, node, go, java, infra) with in-console
Dockerfile editing and image builds; per-user files injected into every workspace;
$HOMEon the volume so toolchain caches persist; Docker/podman and Kubernetes runtimes - Deploy: single static binary, Docker image, docker-compose, Helm chart (both modes)