Skip to content

mlxtop v3.2.0

Choose a tag to compare

@github-actions github-actions released this 03 Nov 04:05
· 17 commits to main since this release
0adf2f5

Journal

  • GPU rows mark real load changes. GPU load wobbling around a line during
    processing, such as 84% → 79% → 81% or 99% → 94%, no longer writes a row
    each sample. The busy band turns on at gpu_warn_load and holds until load
    drops below gpu_warn_exit, and saturation clears 10 points below
    gpu_critical_load. A GPU BUSY assessment row now files under the GPU
    filter and replaces the duplicate "busy burst started" row.
  • "GPU busy" in Settings now sets GPU BUSY. The assessment used a fixed
    80%; it now reads gpu_warn_load (default 75%), so the Settings value and
    the state agree.
  • Telemetry online is recorded once. An idle server whose rate comes and
    goes between polls no longer logs "live API telemetry online" every few
    seconds. It logs again for a new provider or source, or after the server
    went away and came back.
  • Choose what the Journal records. journal.prompts, journal.llm,
    journal.queue, journal.memory, journal.paging, journal.gpu and
    journal.thermal turn each kind of row off, from config.json or the new
    Journal section in Settings. System rows, assessment findings such as
    HEAVY PAGING or GPU THROTTLED, and a move to critical memory pressure are
    always recorded.

Dashboards

  • Strata servers are supported. Detects the strata process or accepts
    MLXTOP_PROVIDER=strata, using port 8080 by default. Strata serves vLLM's
    gauge names on /metrics/prometheus beside its own strata: live-rate
    gauges, so mlxtop shows live decode and prefill speeds, active and waiting
    queues, KV occupancy, prefix-cache reuse, mean first-token time and engine
    GPU memory. Model names come from the metric labels and /v1/models;
    individual request history uses the client usage recorder.

  • TensorFold is now supported. Detects the tensorfold or
    tensorfold-native process or accepts
    MLXTOP_PROVIDER=tensorfold, using port 8080 by default. Reads model names,
    active and queued requests, prefix-cache reuse, KV occupancy and device memory
    through the existing authenticated provider connection. Throughput is sampled
    from completed-request counters and excluded from live throughput charts and
    slowdown alarms; individual request history uses the client usage recorder.

  • Sushi servers report their own generation and prefill rates. The
    mlx-serve adapter now reads the sushi: gauge names when a server
    publishes them instead of mlx_serve: (sushi serve does), beside the
    shared vLLM-shaped queues. Decode, prefill, per-request speeds and prefill
    progress work unchanged; MLXTOP_PROVIDER=mlx-serve stays the selection,
    and a running sushi process is detected as mlx-serve.

  • Every Overview panel scales with the terminal, in balance. The
    throughput row held seven rows on any terminal tall enough for the full
    layout, and the request row held six until the first request arrived, so
    on a small window — a laptop beside a server, an SSH session from another
    machine — idle panels kept their space while the memory, compression and
    paging charts absorbed the whole change and grew to half the screen. All
    three rows now follow the same floor-share-ceiling shape the Journal
    already had, so the host charts keep near two fifths of the height at
    every size instead of dwarfing the rest, and each window resizes with the
    screen dimensions between compact and tall.

  • The prefill rate stays after the prefill finishes. A finished
    request's prefill speed was held for three seconds and then replaced by a
    live zero, so a request that decoded for longer than that showed
    PREFILL 0.0 for the rest of its run. The last measured speed now stays,
    labeled LAST PREFILL, until a newer prefill replaces it. A fresh
    dashboard seeds its first reading from the server's cumulative finished
    prefill counters instead of showing nothing. The chart does not redraw
    that held reading on every poll, and it is never labeled LIVE.

  • The cache panel keeps a reading through a long decode. Prefix-cache
    counters only move when a request completes, so the interval rate went
    blank while a request decoded. When no completed request supplied an
    interval rate, mlxtop now uses the prefix reuse of the requests still
    running.