Repository navigation
v1.4.0
Catches up with what the runners changed over the summer.
Fixed
- vLLM 0.12 renamed its kv-cache metric, so the pane sat at 0%. Both names are read now.
- Anything answering on a probed port (a dev server on :8000, say) no longer shows up as a phantom vllm or llama.cpp model.
- Streamed OpenAI-style requests without a usage block now get recorded. llama.cpp's own timings are used when the stream carries them, otherwise tok/s is estimated from the chunk count and marked as such. The bytes your client gets are still untouched.
- nvidia-smi
[N/A]fields show asn/ainstead of a fake 0, and stay out of the energy total. - A rocm-smi response mtop can't read surfaces as an error instead of a blank GPU pane.
New
- The proxy records
/v1/responsestraffic, streamed or not. - llama.cpp router mode (llama-server started without
-m) lists every loaded model. - LM Studio 0.4's v1 REST API, with a fallback for older installs.
- A CHANGELOG, backfilled to 1.0.
brew and scoop pick this up as usual; winget follows once Microsoft merges the bump.