v0.108.0: GPU utilization on Linux, coherent Mamba3 fallback flags

GPU utilization telemetry on Linux

gpu_util was macOS-only, so every CUDA run reported n/a — exactly where
"is the GPU actually busy?" tends to get asked. Linux now samples
nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader,nounits,
reporting the maximum across visible devices to match the macOS ioreg
behavior.

Sampling runs only where the macOS sampler already did: the opt-in
MIXLAB_MLX_MEM_LOG_EVERY cadence and live /debug/mixlab/telemetry
requests. It is never per-step by default, and carries a 750 ms timeout so a
hung driver cannot stall the caller. A missing binary, driver error,
unsupported [N/A] reading, or timeout leaves gpu_util=n/a and omits
gpu_util_percent from JSON — none of them fail training. A measured idle
device reports 0%, which stays distinguishable from having no sample.

What this number is not: it is device-wide, not process-scoped and not
CUDA_VISIBLE_DEVICES-filtered. Other processes contribute to it, and
sampling between steps can miss bursts. It complements kernel-dispatch
diagnostics and cannot by itself establish that a given Mixlab operation ran
on the GPU.

Mamba3 scan-fallback flags now agree with the guard

The runtime notice advertised MIXLAB_MAMBA3_DISABLE_CUDA_PRIMITIVE=1 as the
way to get a debug fallback, but for fused canonical-block training the guard
rejects that flag on its own — it also requires
MIXLAB_ALLOW_MAMBA3_MLX_SCAN_FALLBACK=1. Following the advertised command
produced an error telling you to do something else.

Both now read a single shared header, so the advertised command is the
accepted one by construction. The messages additionally separate two controls
that were easy to conflate: MIXLAB_DISABLE_MAMBA3_COMPILED_STEP selects the
uncompiled low-memory training step and does not disable the scan
primitive or authorize the MLX scan fallback. The rejection now also points
Metal users at MIXLAB_MAMBA3_DISABLE_METAL_PRIMITIVE and notes that
standalone scan and eval programs never needed the training opt-in.

docs/canonical_mamba3.md gains the two-flag debug recipe, with an explicit
warning that the opt-in is an escape hatch rather than a promise that a large
comparison is memory-safe.

Install

brew tap mrothroc/mixlab https://github.com/mrothroc/mixlab
brew install mixlab

Or build from source:

CGO_ENABLED=1 go build -tags mlx -o mixlab ./cmd/mixlab/