v0.0.25

VoidLLM 0.0.25

Two production defects, provider-correct health checks, and a UI that can be used with a keyboard.

Fixes

• A rate limit from an upstream now fails over. A 429 was returned straight to the client even when another deployment had capacity, and it was recorded as a circuit-breaker success, which reset that deployment's accumulated failure count. Spreading load across several provider accounts is the usual way to stay under per-account limits, and that setup did nothing for the most common upstream failure. The deployment now enters a short cooldown derived from Retry-After, and a rate limit is also a reason to fall back to a configured fallback model.

• Anthropic token counts were too low whenever prompt caching was used. Cache-read and cache-write counts are reported separately from the input count and were being discarded, so prompt and total tokens were under-reported. That skewed cost estimates and under-consumed token budgets, which read the total.

• Health checks talk to each provider the way it expects. Probes were OpenAI-shaped for every upstream, so checks against Anthropic, Gemini, Azure and Vertex failed permanently: a wrong "degraded" status and a billable request wasted on every interval. The same assumption sat in the admin connection test, where an operator who had configured a model correctly was told it was broken. Probes with no equivalent for a provider, such as a model list on Azure, now report as not applicable instead of as a failure.

• Dropdowns opened from a table are no longer cut off by the table's scroll area, and they follow the field as the page scrolls.

• Dropdowns can be operated by keyboard. Arrow keys, Home, End and Enter did nothing unless the dropdown had a search box - which is nearly all of them. Opening one now starts on the current value rather than the top of the list.

Features

• Cached prompt tokens are tracked and priced. Every major provider bills tokens served from a prompt cache differently from fresh input, and writes into the cache differently again, so a single input rate drifted from the real invoice - and drifted further the better the cache hit rate. Models take optional cached-input and cache-write prices; leave them unset and the cost is calculated exactly as before.

Security

• Dependency update addressing an advisory in the frontend build toolchain.

Upgrading

Nothing to do. Schema migrations are applied automatically and skipping releases is supported.

If you use prompt caching with Anthropic, expect reported token counts and costs to rise after the upgrade. That is the correction described above, not new consumption - the previous numbers were too low.

Docker: ghcr.io/voidmind-io/voidllm:0.0.25