v0.6.1 — MLX supervisor hardening, speculative decoding, vision embedding honesty

Reliability + observability hardening on top of 0.6.0's multi-MLX foundation. The big wins: an MLX supervisor that can no longer get stuck restarting an orphan-port forever (orphan reap + crash-window quarantine), vision embedding chips that stop lying when the backend isn't installed, a brew install ollama-herd path that actually works, a tunable FLEET_MLX_MAX_INFLIGHT_PER_MODEL for operators willing to trade memory for batched throughput, dashboard color semantics that finally agree with the product's "idle hardware is waste" thesis, and speculative decoding live on the dedicated 30B compactor for an immediate Claude-Code-summarization-pass speedup. Source-read research in docs/research/mlx-lm-stability-and-concurrency.md confirmed v0.31.3 is the right pin; v0.31.2 would have traded our quarantine-able bug for two uncontainable ones.

Added

• Speculative decoding enabled on the dedicated context compactor (port 11441, Qwen3-Coder-30B-A3B-Instruct-4bit). Re-tested on our pinned mlx-lm==0.31.3 and confirmed it works on standard transformer MoEs with a Qwen3-1.7B-4bit draft (≈94 tok/s on M3 Ultra, no ArraysCache error). Live config in ~/.fleet-manager/env adds "draft_model":"mlx-community/Qwen3-1.7B-4bit","num_draft_tokens":4 to the 30B-A3B server entry only — every Claude Code request's pre-summarization pass benefits, which is the hot path. Not enabled on port 11440 (Qwen3-Coder-Next-4bit, the main coding model): the Qwen3-Next architecture uses Mamba/SSM-style linear attention layers that mlx-lm represents with a non-trimmable ArraysCache, so spec decoding still hits ml-explore/mlx-lm#1081 at speculative_generate_step:531. The earlier "always blocked on ArraysCache" framing was wrong — the bug is architecture-specific, not version-specific. Updated post-mortem in docs/issues/mlx-speculative-decoding-blocked.md. Per-spec draft_model + num_draft_tokens were already plumbed through MlxServerSpec.from_dict in 0.6.0 — this release is config-only on the production fleet.

Fixed

• FLEET_MLX_MAX_INFLIGHT_PER_MODEL env var — tunable per-model concurrent-request cap on the MLX proxy. Default 1 (strict serialization, matching historical behavior). Bump to 2 or 3 to let mlx_lm.server's BatchGenerator process multiple requests in one inference pass — empirically validated 2026-04-27 on the local fleet (3 concurrent requests took 0.55s wall vs 1.10s sum, confirming real parallelism). The default stays conservative because each in-flight request carries its own KV cache state, so 2× concurrent 100K-token prefills is 2× memory pressure; concurrent paths in mlx_lm.server have historically been bug magnets (#965, #1166 — both fixed in v0.31.3, but the pattern is real). Operator opt-in only. Full source-read + live test in docs/research/mlx-lm-stability-and-concurrency.md. 5 new tests cover defaults, clamping (negative/zero values floor to 1), the 2-concurrent-acquires-then-third-blocks behavior, and per-model semaphore independence. Confirmed in the same research that mlx-lm v0.31.3 is the right pin even though it's where we hit #1208 — downgrading to v0.31.2 would trade our quarantine-able bug for two uncontainable ones (#1166 Qwen3-Next concurrent crash + #1181 thread-local stream crash).

• MLX supervisor now detects and SIGKILLs orphan mlx_lm.server processes on the configured port before spawning its own. A previous herd-node session killed via pkill -9 -f "bin/herd-node" (without also killing mlx_lm.server) leaves the MLX subprocesses alive — Popen(start_new_session=True) makes them survive their parent's death; they get reparented to launchd and keep holding ports 11440 / 11441. The next supervisor startup couldn't bind, exited rc=1 in ~2 seconds, and the quarantine guard kicked in against a process that didn't exist while the orphan kept serving requests. Result was 17 hours of mlx_server_quarantined warnings against a fleet that was actually serving traffic, just not under any supervisor's control. New find_orphan_mlx_pids_on_port() (psutil-based, identity-strict — only kills processes whose cmdline mentions mlx_lm.server AND whose net_connections show binding to our port). MlxSupervisor.start() calls it before its own Popen and SIGKILLs anything found, with a loud WARNING explaining what was reaped. 8 new tests for the strict identity check (right cmdline + right port = kill; wrong port = leave alone; non-mlx process on right port = leave alone; permission errors handled). Operator restart recipe in CLAUDE.md updated to pkill -9 -f "bin/herd|mlx_lm.server" so the manual path doesn't reproduce the trap. See observation in docs/observations.md 2026-04-27.

• MLX supervisor now quarantines a crash-looping subprocess instead of restarting it forever. On 2026-04-26 a stuck-state in mlx_lm.server v0.31.3 (latest) triggered a 420-restart, 2.5-hour crash loop on the local fleet — the supervisor's _monitor correctly detected each crash and restarted at the (capped) 60s exponential-backoff cadence, but had no upper bound on how many times it would keep doing that. Now: ≥5 crashes within a 5-minute rolling window switches the supervisor into "quarantined" state with a 10-minute restart interval, and surfaces a CRITICAL mlx_server_quarantined health-check recommendation pointing at logs + likely causes. Quarantine clears automatically once a restart stays up for the full window — so transient bursts still recover gracefully; only persistent failures get throttled. Tunable via _QUARANTINE_FAILURE_COUNT / _QUARANTINE_WINDOW_S / _QUARANTINE_RESTART_INTERVAL in mlx_supervisor.py. 8 new tests for the threshold + windowing logic. Underlying mlx-lm bug filed upstream as ml-explore/mlx-lm#1208. See observation in docs/observations.md 2026-04-26.

• Vision embedding chips no longer lie about availability when onnxruntime is missing. The dashboard previously rendered DINOv2 / SigLIP / CLIP chips on a node card whenever the model weights were cached on disk, regardless of whether the embedding backend (onnxruntime) could actually load them. Operators saw "available" chips and assumed the service worked; the first real /embed call returned HTTP 500. The collector now probes for onnxruntime import on every heartbeat and refuses to advertise vision-embedding models when the backend isn't loadable, so the chips disappear (honest) instead of misleading. Paired with a new vision_backend_missing health check (WARNING) that fires when weights ARE cached but the backend is missing — operators see "Vision embedding backend not installed on <node>. Run uv sync --extra embedding ..." in the Recommendations panel instead of silently-disappearing chips. Also closes the recurring root cause: the local-deploy snippet in CLAUDE.md was uv sync (without --extra embedding), which is destructive — every restart stripped the embedding deps. Updated to uv sync --all-extras so optional capabilities stay resident across restarts. New vision_embedding_status: dict field on heartbeat + NodeState carries {backend_available, cached_model_count} so future health checks have a clean signal to read. 9 new tests in tests/test_server/test_health_vision_backend.py covering both fires-on-missing and silent-when-fine paths plus the collector probe directly.

• brew install ollama-herd now actually works. The Homebrew formula at geeks-accelerator/homebrew-ollama-herd had been broken throughout 0.5.x — Homebrew's pip install --no-binary :all: policy forced source builds for pydantic-core, which required Rust to bootstrap maturin, which the formula didn't depend on; six pyproject.toml deps (cryptography, cffi, pycparser, tiktoken, regex, websockets) were also missing from the formula's resource blocks; and pydantic-core was version-mismatched against the bundled pydantic (2.45.0 vs the required 2.41.5). Fix shipped to the tap as geeks-accelerator/homebrew-ollama-herd@71856f3 (no PyPI republish needed). Verified end-to-end on macOS Apple Silicon: clean fresh-user install in ~5 minutes, all critical imports clean, both herd and herd-node CLIs functional. The release checklist in CLAUDE.md was updated to make the brew end-to-end install test a non-negotiable gate so this class of failure can't recur. Background and post-mortem in docs/observations.md (entry: 2026-04-25).

Added

• Platform-aware thermal signal — new ThermalMetrics on the heartbeat with state (nominal / warning / unknown), temperature_c, and source fields. Linux nodes now report real peak temps from psutil.sensors_temperatures() (scanning coretemp / k10temp / zenpower / cpu_thermal drivers) and flag warning above 85°C. macOS and Windows honestly report unknown — Apple Silicon's machdep.xcpm is Intel-only, powermetrics requires sudo, and pmset -g therm only reports past events; psutil.sensors_temperatures() isn't implemented on macOS at all. The dashboard's .bar-thermal overlay now uses the reported signal when available and falls back to the CPU≥95% proxy only when state is unknown, so Linux operators get first-class thermal detection and macOS operators keep the existing behavior with a clean seam for future upgrades. See src/fleet_manager/common/system_metrics.py::get_thermal_metrics.
• /dashboard/color-states dev route — renders the utilization bar in every Axis B warning state side-by-side (normal / memory warning / memory critical / CPU thermal) plus gradient sweeps for CPU (cyan→purple) and memory (soft-blue→deep-purple). Uses the live dashboard CSS + JS so any change to production styling reflects here automatically. Linked from docs/guides/dashboard-color-reference.md and used by the marketing-site agent for consistent screenshot captures.
• docs/handoffs/ollamaherd-com-color-refresh.md — self-contained brief for the agent maintaining the marketing site. Documents the palette change, explains why the old landing-page screenshots now contradict the product's own messaging, and provides the live reference URL for capturing replacements.

Changed

• Dashboard color semantics: utilization is no longer a warning. CPU, memory, and per-node RAM bars now render in a blue→purple gradient (utilizationColor(pct, metric)) instead of the old green-to-red busy-is-bad scale. A Mac Studio at 95% memory is the product working as designed, not a problem — the previous coloring contradicted the product's own thesis that idle hardware is waste. Warning state moved to a separate visual axis: .bar-warning / .bar-critical outlines fire on OS-reported memory pressure (psutil.virtual_memory().pressure), and .bar-thermal fires on sustained ≥95% CPU as a throttling proxy. Preserved: disk bar still uses the busy-is-bad scale (disk full genuinely breaks things), and the capacity-score bar keeps its green-at-high-availability semantic. Also added docs/guides/dashboard-color-reference.md as a one-page cheat sheet for future UI additions so the semantic doesn't drift back into server-ops defaults. See docs/plans/dashboard-color-semantics.md for the full rationale and audit.