Turn starts are now instant — at any think-time
TTFT after idle: 4.5–5s → ~350ms flat (measured at 6s, 12s, and 25s think-time gaps in a resident session).
Root cause
Apple's RDMA provider puts the Thunderbolt data-plane queue-pairs to sleep after ~1–2 seconds of pipeline inactivity, and they take ~3 seconds to wake — which landed exactly on interactive think-time between chat turns (prefill measured 2.5× slower on the first post-idle request). No GPU/matmul keepwarm can reach those QPs; only real pipeline traffic does.
Fix
The 1 Hz coordinated keepwarm tick now carries a tiny symmetric send/recv pair through the actual pipeline queue-pairs (rank0 sends first, rank1 receives first — deadlock-free by construction). The flag rides the op payload, so both ranks always agree regardless of env state. Failure policy is die-fast into the proven guard → teardown → auto-relaunch path rather than limping into a desynced collective. Kill switch: MLX_M3_KEEPWARM_DATAPLANE_PING=0.
Also in this release
- Keepwarm cadence retuned for the measured 1–2s decay window (1s interval, 0.5s idle gate, large-cache throttle removed) + request-start warmup enabled
- Gateway: sticky-oMLX grace window (interactive oMLX sessions defer M3 auto-start; capped at 30s by default) with M3-wakeup attribution logging to identify background resurrection culprits
- Launcher refuses mixed-code launches (self-syncs rank1) — prevents a data-plane message off-by-one from version skew
- Keepwarm transaction releases the generation lock only while still the recorded owner (stale-steal safety)
- Dashboard: live prefill tok/s during the prefill phase (was 0 until completion)