Skip to content

v0.23.0

Choose a tag to compare

@xiaguan xiaguan released this 29 Jun 07:23
55d72a7

What's Changed in v0.23.0

✨ Features

  • MLA layer-split KV cache — page-first per-shard slots (#372)
    Page-first storage now covers MLA layer-split, where each tensor-parallel rank registers a disjoint subset of layers. Registrations are grouped by layer-set into shards (total_slots = num_shards): a layer-split instance has one shard per rank (each rank writes its own sub-page), while full-replica keeps a single shard with block-striped writes. Pipeline parallelism and DCP remain excluded. Validated end-to-end on an 8-GPU H200 node with GLM-5.1-FP8 (byte-identical warm-hit, per-shard load with zero failures, gsm8k 5-shot 0.95, warm-vs-cold bench A/B of 1.73x throughput / -47% P99 TTFT).
  • Vendored vLLM NIXL connector (#366)
    Imports the vLLM NIXL KV connector into the tree under pegaflow/nixl_connector/ (push/pull schedulers and workers, TP mapping, metadata, stats), so the connector version is owned by PegaFlow instead of pulled from an external import.

🐛 Fixes

  • core: preserve per-slot NUMA on block rebuild to prevent 0-slot re-serve (#370)
  • nixl: use vendored connector imports (#371)
  • metaserver: size the removal pipeline for eviction bursts (#369)
  • connector: include pp_size and mla_layer_split in the namespace so layouts differing only in those dimensions no longer collide (#367)

🧹 Chores

  • downgrade tokio to 1.48.0 (#364)

⚠️ Compatibility

Strict version handshake — client and server must both be 0.23.0 (exact CARGO_PKG_VERSION match at registration). SSD cache is ephemeral and wiped on restart.

Full diff: v0.22.10...v0.23.0