Repository navigation
mlxtop now covers local LLMs beyond Apple Silicon: NVIDIA GPUs and Windows,
plus the mlx-serve and Splash runtimes.
- Windows (x86_64) support: memory, commit charge, processes and runtime
detection through the Windows APIs, NVIDIA GPUs throughnvidia-smi, logs in
%LOCALAPPDATA%\mlxtop, a PowerShell installer (scripts/install.ps1) and a
release zip. Windows exposes no paging counters through those APIs, so paging
rates show as unavailable rather than zero. - NVIDIA findings: the assessment names a model partly or fully on CPU
(Ollama placement), a card with full VRAM (97%, held until below 94%), and
thermal, power-brake or hardware clock slowdowns on a busy card. The GPU
panel's STATE column shows the clock-limit reason, and full VRAM turns the
card's memory reading yellow. - Add Splash (
MLXTOP_PROVIDER=splash, default port 8000), detected by its
Metal engine process. It polls/statusfor the model, each running request's
prompt and output counts with prefill progress, waiting requests, a generation
rate from the engine's decode counter, the prefill speed of its prefill
commands, prefix-cache reuse, Metal memory against its limit, DFlash draft
acceptance, server TTFT p50, and released weights, thermal state and memory
pressure. An API key goes inMLXTOP_PROVIDER_API_KEY. - Add mlx-serve (
MLXTOP_PROVIDER=mlx-serve, default port 11234) with process
detection. It polls/metricsfor queues, a live generation rate that counts
tokens of requests still running, a prefill rate that follows the prompt being
processed now, interval and cumulative cache reuse, and MLX active memory. It reads
/metrics.jsonfor the model and the running requests' prompt, cached and
output counts; sessions that only hold a prefix cache are not listed. The
server needs--metrics, and an API key goes inMLXTOP_PROVIDER_API_KEY.