v0.24.2
What's Changed
fix(connector): keep full-block save intents inside the resumed block table (#449)
Production K3 (TP8, MTP) hit RuntimeError: save block/hash count mismatch ... blocks=1 hashes=10 on every attention layer once KV pressure started preempting requests, and the failure took the whole engine down with it.
- Save window after preemption: the full-block save window was bounded by
base_block_idx + len(table), counting the externally loaded prefix twice. After a preemption vLLM rebuilds the block table from the resume chunk while the per-request accumulator kept the first life's offset, so the window ran past the table and block IDs were silently truncated against the full hash list. The window is now bounded by the table itself, and a resumed request restarts its bookkeeping from the refreshed external hit (_rebase_resumed_request) while keeping already-stored progress. Save-only mode no longer saves a partially recomputed block under a valid hash. - Save worker resilience (
worker.py): the raise used to kill the save thread, after which every later request stayed held on a save that never completed and the engine leaked KV cache until restart (2 running / 18 waiting / 2 tok/s in the incident). The save worker now survives a failing batch, a malformed intent drops only its own request's save, and completion is always reported so blocks get released. - Hybrid load under a shrinking hit (
_load_block_ids_by_group): when the hybrid reconcile fell back to an earlier recurrent checkpoint, the load carried one target fewer than the query lease and the engine rejected it (query lease block count 5 does not match destination block count 4), escalating to "Service unavailable". The load now covers every leased block and pads unwanted ones withNonetargets.
Verified on Kimi-Linear-48B-A3B (single GPU, 400-block GPU KV budget to force preemption): 3 rounds / 294 requests / 3908 preemptions with 0 save mismatches and 0 load failures, where 0.24.1 died on the lease/destination mismatch. Multi-turn vllm-bench load (128 conversations, 3 turns, 2000+ preemptions) is clean.
Full Changelog: v0.24.1...v0.24.2