Repository navigation
mlxtop v3.1.2
Running requests now show their own generation speeds, llama.cpp reports a live
generation rate, and Linux detects AMD GPUs without extra tools.
Dashboards
- Each running request shows its own speed. While the runtime reports a
decode speed per request, the Overview queue lists its running requests in
its own rows, each with the tokens it has generated and its speed, such as
8f31a2 1.7k · 41.0 tok/s, so a client's own tokens/s can be checked
against it while the generation panel reports the whole server. oMLX
reports these for every generating request; mlx-serve now does too, from
each session'sgenerated_tokens, read through the same stepping window as
the server's decode speed; Splash has one while a request runs alone. A
request without one shows—and keeps its row. The panel keeps its size,
and the history returns when nothing is running. - Prompt load shows the tokens each request generated. On a panel tall
enough to keep four rows of bars, a row under the size labels shows each
request's generated tokens. Whether the row is there depends on the panel's
height alone, so the bars never shrink partway through a session. - llama.cpp reports a live generation rate. Active decoding slots supply
the current output rate, while slots still prefilling do not count toward it.
The adapter accepts both object- and list-shaped/slotsresponses. - Linux reads AMD GPU counters from amdgpu sysfs. Overview shows utilization,
VRAM and temperature without a ROCm install. A runtime-suspended card stays
listed without polling counters that could wake it on older kernels. - mlx-serve session rates reset after a server restart. Reused request IDs
cannot inherit a previous process's per-request speed.