Release Olla v0.0.29 · thushan/olla

What's in this release

Olla is a high-performance proxy and load balancer for LLM infrastructure.

# Docker
docker pull ghcr.io/thushan/olla:v0.0.29

# Binary (see assets below)
./olla --config config.yaml

New embedded, read-only admin dashboard at /internal/ui/ - Overview (fleet status, success rate, latency, live requests-per-second sparkline), Endpoints (per-endpoint health, priority, latency, model count) and Models (inventory grouped by family, with hosting endpoints). Built in Svelte 5 + TailwindCSS, polled (not pushed) against the existing /internal/status* JSON, and served from the same listener as the proxy - no second port (#205).
The dashboard has no authentication. Access is controlled by a new dashboard.access_policy config block (allowed_cidrs + allowed_hosts), loopback-only by default. The published Docker image ships with allowed_cidrs pre-widened to the RFC1918 ranges so docker run -p 40114:40114 ghcr.io/thushan/olla:latest gives you a working dashboard with no config mount - which also means the container is reachable from anyone else on the same LAN through that published port (#213). See the Admin Dashboard docs for the full security model.
A binary built without make build-web (e.g. plain go build or go install) now logs a clear startup warning and serves 503 at /internal/ui/ instead of a silent placeholder (#213).

Native GET /internal/metrics in Prometheus text format, built from the same data as /internal/status and /internal/stats/models - no external exporter needed for core proxy monitoring. Thanks to @Puupuls for the contribution (#188).
/internal/status, /internal/status/endpoints and /internal/status/models now emit a weak ETag, so polling clients (including the new dashboard) can use If-None-Match and get 304s instead of re-fetching the full payload (#205, #213).
Sticky-session and routing decisions are now in the structured "Request completed" log line (sticky_outcome, routing strategy/action/reason, provider_model, translator fallback reason), not just response headers. Addresses issue #178 (#182).
Successful proxy requests log at Info again in the access log; a prior regression had silently demoted them to Debug (#211).

A request landing on an endpoint whose circuit breaker is open now fails over to the next available endpoint instead of failing outright. The endpoint is dropped from that request's candidate list only - its persisted health is left alone, since an open breaker already reflects accumulated failure state and demoting health on top of it isn't the retry path's job (#213).
Strict routing no longer lets an unroutable model_aliases entry silently proxy to the wrong backend: a request for an alias whose target model exists on no endpoint now fails fast with 404/503 and routing_action: rejected, matching how unknown models are already handled. Reported by @skaravos (#197, issue #191).
config/models.yaml now actually parses. It used \d inside double-quoted YAML scalars, which is invalid YAML, so the file has never loaded on any install - the failure was swallowed and Olla silently fell back to embedded defaults, discarding any customisation with no diagnostic. Reported by @billford (#206, issue #204).
New --validate-config flag checks configuration and provider profiles without starting the server, with a pass/warn/fail report and exit codes (#210).
logging.level in config is now actually applied to the runtime logger, instead of being parsed and ignored (#209).

Refreshed the built-in model catalog for 2026: recognises more families and variants (GPT-OSS, GLM, Kimi, Granite, Nemotron, EXAONE, Hunyuan, MiniMax, OLMo, InternLM, SmolLM, Command-R, Forge) and MXFP4/NVFP4 quantisation (#207).
Wired model_extraction.family_aliases and special_rules.preserve_family into the family-extraction pipeline - both were parsed but previously had no effect. Fixes Kimi-K2 being misclassified under deepseek (shared GGUF architecture lineage) (#208).

An endpoint URL with embedded user:pass@host credentials now fails startup instead of loading silently and leaking the credentials into every status/dashboard JSON response. The boot error rewrites the URL into a ready-to-paste auth: block with placeholders, so the fix doesn't require re-typing the real credentials (#213).
The id and url fields on /internal/status and /internal/status/endpoints are now derived from a sanitised URL (userinfo, query and fragment stripped) rather than echoing the raw configured URL (#213).
/internal/status no longer reports "status": "critical" on a healthy fresh boot with no traffic yet. A new system.has_traffic boolean lets clients branch on the no-traffic state directly instead of parsing the success_rate string (#213).

Streamlined the container image and switched to .dockerignore for a smaller, more predictable build context (#213).
Added a release dry-run validation step to the release pipeline (#213).

Userinfo URLs: - url: "http://user:pass@host:8080" now fails startup. Move credentials into an auth: block:

- url: "http://host:8080"
  auth:
    type: basic
    username: user
    password: pass

Status id/url fields: now derived from a sanitised URL rather than the raw config value. Bookmarked dashboard deep-links to a specific endpoint will change once after upgrading. Clients comparing url for identity should switch to id.
Zero-traffic status: a fresh boot with all endpoints healthy no longer reports "critical". Check system.has_traffic if you branch on traffic state.
Strict routing on unroutable aliases: /olla/proxy/ with zero healthy endpoints now returns 503 instead of 502. Update any monitor keyed on that status code.
owned_by in converted model listings: the shared organisation-extraction logic used by the vLLM, vLLM-MLX, SGLang, llama.cpp, LMDeploy and Lemonade converters now also matches a hyphenated leading segment (e.g. Qwen2.5-7B → owned_by: "Qwen2.5") rather than falling through to a generic default. Treat owned_by as a best-effort label, not a canonical identifier.

Thanks to everyone who contributed to this release:

@Puupuls - contributed the native GET /internal/metrics Prometheus endpoint (#188).
@skaravos - reported the strict-routing alias bypass (issue #191).
@billford - reported the config/models.yaml parse failure (issue #204).
@sg-shag - reported the missing sticky-session/routing decision logging (issue #178).

Full Changelog: v0.0.28...v0.0.29

Documentation: thushan.github.io/olla | Issues: github.com/thushan/olla/issues