v0.5.1 — Ollama 0.33.x prompt-cache metrics

Tracks Ollama 0.33.x, and fixes a prefill-speed regression that arrived with it.

Fixed

Prompt tokens/second over-reported whenever the prompt cache was warm.

Ollama 0.33.3 redefined prompt_eval_duration to time only the uncached prompt tokens, while prompt_eval_count still counts the whole prompt. Dividing one by the other inflated the figure by the cache hit ratio — measured against a live 0.33.3 daemon, a prompt with 37 of its 38 tokens served from cache read 4375 tok/s instead of 115. The rate is now taken over the tokens actually evaluated.

Component tests mounted without the i18n plugin, so the SlashCommandDropdown suite failed outright. The test harness now installs the real English catalog, and those tests assert against the shipped strings.

Added

• Prompt cache reuse is visible. Session metrics show a Cached Prompt card and split the token distribution bar into its cached and evaluated halves. Benchmark question results record the cached share too.
• The Protocol Observatory names the divergence. The same fact arrives under three different fields depending on the protocol — prompt_eval_cached_count (native NDJSON), usage.prompt_tokens_details.cached_tokens (OpenAI-compatible) and usage.cache_read_input_tokens (Anthropic-compatible). The envelope diff now shows all three side by side with this run's values.
• audio capability, which Ollama 0.33.3 began reporting for models carrying an audio tower.

Changed

• Cache metrics stay undefined rather than zero when a daemon never reports them — "not reported" and "nothing was cached" are different claims, and only the second is worth showing as 0%.
• Anthropic-compatible input_tokens now means the prompt total minus cached tokens on Ollama 0.33.3. The Observatory reconciles it back to the prompt size so the three protocols stay comparable.
• READMEs (EN/FR/ZH/AR/SR) updated for Ollama 0.33.x; new Observatory strings across all six locales.

Install

npx llmxray

docker pull djovaneli/llmxray:0.5.1