Skip to content

mlxtop v3.0.0

Latest

Choose a tag to compare

@github-actions github-actions released this 17 Oct 15:48
· 2 commits to main since this release
bff0179

mlxtop now covers local LLMs beyond Apple Silicon: NVIDIA GPUs and Windows,
plus the mlx-serve and Splash runtimes.

  • Windows (x86_64) support: memory, commit charge, processes and runtime
    detection through the Windows APIs, NVIDIA GPUs through nvidia-smi, logs in
    %LOCALAPPDATA%\mlxtop, a PowerShell installer (scripts/install.ps1) and a
    release zip. Windows exposes no paging counters through those APIs, so paging
    rates show as unavailable rather than zero.
  • NVIDIA findings: the assessment names a model partly or fully on CPU
    (Ollama placement), a card with full VRAM (97%, held until below 94%), and
    thermal, power-brake or hardware clock slowdowns on a busy card. The GPU
    panel's STATE column shows the clock-limit reason, and full VRAM turns the
    card's memory reading yellow.
  • Add Splash (MLXTOP_PROVIDER=splash, default port 8000), detected by its
    Metal engine process. It polls /status for the model, each running request's
    prompt and output counts with prefill progress, waiting requests, a generation
    rate from the engine's decode counter, the prefill speed of its prefill
    commands, prefix-cache reuse, Metal memory against its limit, DFlash draft
    acceptance, server TTFT p50, and released weights, thermal state and memory
    pressure. An API key goes in MLXTOP_PROVIDER_API_KEY.
  • Add mlx-serve (MLXTOP_PROVIDER=mlx-serve, default port 11234) with process
    detection. It polls /metrics for queues, a live generation rate that counts
    tokens of requests still running, a prefill rate that follows the prompt being
    processed now, interval and cumulative cache reuse, and MLX active memory. It reads
    /metrics.json for the model and the running requests' prompt, cached and
    output counts; sessions that only hold a prefix cache are not listed. The
    server needs --metrics, and an API key goes in MLXTOP_PROVIDER_API_KEY.