Skip to content

v0.3.14

Latest

Choose a tag to compare

@ShangmingCai ShangmingCai released this 03 Nov 11:15
e1ea5d3

This release brings major upgrades across Mooncake Store, Transfer Engine, TENT, Expert Parallelism (EP), Process Group (PG), ecosystem integrations, and hardware support, with 689 commits and 99 new contributors

Highlights

Transfer Engine & TENT

  • Added POSIX shared-memory transport for efficient intra-node DRAM transfers.
  • Improved transfer efficiency with pipelined operations, selective signaling, and native RDMA notifications, alongside optional adaptive RDMA congestion control.
  • Enhanced TENT with asynchronous TCP transfers, multi-rail routing, large-read striping, load-aware lane selection, and idle connection reclamation.
  • Expanded Prometheus metrics and improved RDMA connection lifecycle management, memory registration, and data integrity diagnostics.

Mooncake Store

  • Added NVLink-based Extended GPU Memory (EGM) pooling on Grace-Blackwell and Vera Rubin platforms, enabling direct GPU access to shared host memory.
  • Improved tenant-aware metadata management, quota accounting, and eviction with finer-grained locking and proactive reclamation.
  • Enhanced client resilience with recoverable heartbeat states, asynchronous cleanup, and improved RPC isolation.
  • Expanded external storage capabilities with DFS shard scaling, OSS/S3 backends, and optional SSD-to-DRAM prefetching.

Mooncake EP & Process Group (PG)

  • Added an NCCL Device API backend for EP ElasticBuffer, supporting elastic peer management and rank replacement recovery.
  • Introduced a new device collective framework in PG with IBGDA support.
  • Added JIT compilation for EP CUDA backends and PG PyTorch adapters, removing PG's dependency on specific PyTorch versions.

Integration

  • vLLM added cross-encoder embedding reuse through Mooncake Store for multimodal disaggregated serving.
  • TokenSpeed integrated Mooncake Store as an L3 KV cache backend, alongside Transfer Engine support for PD/EPD disaggregation.
  • TensorRT-LLM integrated Mooncake Store into KVCMv2 for content-addressed KV cache pooling and cross-instance prefix reuse.
  • LMDeploy integrated Mooncake Store for shared KV cache pooling and cross-instance prefix reuse.

Platforms & Distribution

  • Expanded hardware support across Hygon DCU, Cambricon MLU, MetaX MACA, Moore Threads MUSA, Biren SUPA, NVIDIA Data Direct, and AWS Trainium/EFA.
  • Added Intel XPU support across Transfer Engine, Store, and EP, including PCIe P2P, GPUDirect RDMA, GDS, and host staging.
  • Enhanced Ascend NPU support with the new MemFabric transport and mooncake-transfer-engine-memfabric wheel package, alongside native UB Transport, HCCS, and A5 RoCE capabilities.
  • Expanded PyPI wheel support to Python 3.14.
  • Added build support for macOS, Kylin OS, and RISC-V.

What's Changed

New Contributors

Full Changelog: v0.3.13...v0.3.14