Skip to content

v0.3.11

Choose a tag to compare

View all tags
@ShangmingCai ShangmingCai released this 21 May 09:32
· 865 commits to main since this release
v0.3.11
b4ccdc3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • Fix/tent batch transfer merge boundary by @Primary33 in #1704
  • docs: add TorchSpec in Mooncake README by @zhyncs in #1709
  • [Store] Add eviction policy for BucketStorageBackend with batch master notification by @zhangzuo21 in #1646
  • Support duration units for master TTL flags by @Primary33 in #1684
  • E2E storage backend test by @maheshrbapatu in #1660
  • [CI] Add hixl roce samples on ASCEND platforms. by @VNightMare in #1697
  • Remove non-portable GCC-internal headers from Transfer Engine by @Copilot in #1716
  • [STORE] introduce HA backend abstraction by @YiXR in #1678
  • [TENT] fix: avoid resetting RDMA endpoint on duplicate concurrent bootstrap by @00fish0 in #1705
  • [Bugfix] Fix tent_metrics build errors with TENT_METRICS_ENABLED=ON by @staryxchen in #1712
  • [PG] Fix group size expansion by @caozhanhao in #1706
  • build: add memory-aware compile/link parallelism by @staryxchen in #1718
  • [Skill] feat: add trouble shooting skill by @stmatengss in #1724
  • [EP] make num_ranks more flexible by @ympcMark in #1725
  • [Misc] Improve developers' experience for EP & PG by @UNIDY2002 in #1708
  • [TE] refactor ascend direct transport & adapt to dummy real mode of store by @ascend-direct-dev in #1720
  • [STORE] add Redis leadership backend and HA regression coverage by @YiXR in #1722
  • store: split client HA/control-plane threads and suppress zero-seg he… by @XucSh in #1736
  • [Transfer Engine] add initial MACA build path and CUDA-like adapter by @Dayuxiaoshui in #1731
  • add required library when build by @xleoken in #1674
  • [PG] optimize p2p-proxy buffer size by @JunlinW113 in #1735
  • [Store] Add hard pin mechanism for eviction-protected objects by @he-yufeng in #1728
  • [PG] Increase kP2PBufferSize to unlock full performance potential by @UNIDY2002 in #1740
  • Fix/tent store metadata override by @Primary33 in #1743
  • [bug fix]Inconsistent parameter of NVMeoFTransport::submitTransferTask by @yz53665 in #1748
  • add script for metadata management by @whybeyoung in #1746
  • [Store] adapt to dummy real mode for ascend by @ascend-direct-dev in #1723
  • [STORE] abstract snapshot catalog and add Redis snapshot backend by @YiXR in #1739
  • Fix/tent slice queue race by @Primary33 in #1737
  • [PG] Add GPU barrier support for mooncake-pg by @LuLuji04 in #1751
  • [STORE] support Redis ACL username authentication and reorganize HA by @YiXR in #1757
  • [TENT] fix tebench RPATH to find libasio.so at runtime by @alogfans in #1750
  • add missing steps to efa transport readme by @bob-bins in #1759
  • [PG] Enable asynchronous recovered-rank initialization with deferred join by @UNIDY2002 in #1744
  • [DOC] Add troubleshooting for RDMA MKEY resource exhaustion by @00fish0 in #1765
  • [TE] Fixed an issue where start_timestamp is retrieved but batch_desc has already been freed by @hjchen2 in #1760
  • [TE] Fix simultaneous open handshake in RdmaEndpoint by @caozhanhao in #1733
  • [tent] Set Auto_Connect as default by @Cheng-China in #1758
  • [PG][TENT] Fix first-collective hangs on NVLink/MNNVL bootstrap by @KMSorSMS in #1755
  • fix(tent): address build issues and enable CI coverage by @Primary33 in #1768
  • [Skill] run mooncake unit tests locally by @stmatengss in #1774
  • [Store] batch remove by @LujhCoconut in #1756
  • fix: correct TP zero-copy put semantics by @zxpdemonio in #1685
  • [Store] Fix SSD offload failure in Metadata Server mode (#1729) by @zhangzuo21 in #1771
  • Fix TENT Python binding initialization and build output by @XucSh in #1778
  • Bump version to 0.3.10.post1 in pyproject.toml by @ShangmingCai in #1788
  • [Store] support resolving master RPC address from interface by @YiXR in #1784
  • Bump google.golang.org/grpc from 1.59.0 to 1.79.3 in /mooncake-common/etcd by @dependabot[bot] in #1785
  • [Transfer Engine] Round-robin slice batch across QPs in RdmaEndPoint::submitPostSend by @usernamehaha2022 in #1721
  • [CI] Optimize CI/CD workflow execution order to implement a fail-fast mechanism by @LujhCoconut in #1782
  • [STORE] split HA runtime and unify standby lifecycle by @YiXR in #1777
  • [Build] add yalantinglibs submodule by @stmatengss in #1781
  • [Store] Add C API for Mooncake Store by @jiangyukunok in #1763
  • [P2P] fix: cannot disable NV_PEERMEM and enable CUDA at the same time by @stmatengss in #1797
  • [Bugfix][Build] Fix S3SnapshotObjectStore Pimpl error & improve build reliability by @timzhang0727 in #1796
  • fix: fix eviction notification unit test to ensure deterministic FIFO order by @00fish0 in #1800
  • [TENT] Fix duplicate notify recv WR posting and PLOG misuse in RDMA endpoint by @dtcccc in #1803
  • [CI] fix bugs with CI pr1782: fail-fast on format check and restore Ascend/Integration as PR gates by @LujhCoconut in #1806
  • [Store] Add Upsert API for in-place object updates by @00fish0 in #1662
  • [TENT] Disconnect before registering memory by @Cheng-China in #1807
  • Update codeowners by @ykwd in #1819
  • [Store] Add Go language bindings for Mooncake Store by @jiangyukunok in #1764
  • [Misc] Fix silent failure in code_format.sh when clang-format is missing by @caozhanhao in #1824
  • [PG] Implement graceful shutdown and reland CPU-only tests to CI by @caozhanhao in #1795
  • [PG]: fix barrier imple problem by @KMSorSMS in #1792
  • [TE] Enabling UB Transport on the Kunpeng SuperNode Phase 1 by @zchuango in #1805
  • [PG] Introduce comprehensive test suite by @yuechen-sys in #1790
  • [STORE] tighten snapshot correctness and reload snapshot-only standby from catalog by @YiXR in #1801
  • Add native Rust bindings for Mooncake Store with usage example and CI integration by @Copilot in #1810
  • [TENT] Fix NVLink IPC address for sub-allocated GPU tensors by @he-yufeng in #1831
  • [PG][TE][TENT] Add dedicated peer liveness probe for recovery and enable elastic GPU test by @KMSorSMS in #1808
  • [TE] feat: setup the RDMA for mlu device. by @phantomlei3 in #1799
  • Optimize ci fail-fast scheme by @LujhCoconut in #1813
  • fix: increase ParallelAllocation test pool to 32MB to avoid flaky slab race by @00fish0 in #1841
  • [Bug fix] Fix get tcp port collision by @zhangzuo21 in #1816
  • [TENT] Register base address of a buffer instead of its sub-allocated address into BufferDesc by @shuoerw in #1837
  • [TE] Add Multi-Protocol Support for DRAM-CXL-SSD tiered storage by @hemist in #1832
  • [TENT] Fix stale segment cache via withCachedSegment and async invalidation by @caozhanhao in #1826
  • [Store] Introduce HA OpLog abstraction and LocalFS oplog store by @duhaode520 in #1804
  • [TRANSFER_ENGINE] align USE_MACA with MUSA GPU paths and docs by @Dayuxiaoshui in #1814
  • feat(store): expose drain job control via master HTTP API by @XucSh in #1815
  • [TENT] Fix potential deadlock and UAF in synchronizeLocal in #1826 by @caozhanhao in #1849
  • fix(docker): respect PYTHON_VERSION build-arg when building wheel by @staryxchen in #1745
  • [CI] Harden CI pipeline: path filtering, concurrency, on-demand E2E, and security fixes by @00fish0 in #1846
  • [store] Add get_into_ranges to support Grouped Scatter RDMA Reads by @zxpdemonio in #1717
  • [CI] fix: slash command /run-e2e-ci fails for fork PRs by @00fish0 in #1859
  • [Store] Fix with_hard_pin failure in python API by @0oshowero0 in #1873
  • [Bug fix] Prevent redundant replica pinning during offloading by @ertcmm in #1853
  • [TransferEngine] Add retry, async execution, and graceful shutdown for TENT TCP transport by @staryxchen in #1866
  • [TransferEngine][ROCm] Add ROCm HIP support to the Mooncake Python package by @knitcapcat-amd in #1742
  • [Docs] Add SSD offload benchmark results by @zhangzuo21 in #1835
  • [Store] Support SSD offload via Python setup() interface by @zhangzuo21 in #1857
  • [MISC] Add CODEOWNERS for efa_transport directory by @stmatengss in #1885
  • [TE] Enabling UB Transport on the Kunpeng SuperNode Phase 2 by @zchuango in #1855
  • [TransferEngine][MACA] Align MACA build paths with CMake options by @Dayuxiaoshui in #1888
  • [TENT] Wire up cross-transport failover with safety limits and observability by @staryxchen in #1878
  • [TENT] Enhance memory registration with transport type support by @staryxchen in #1877
  • [Store] Support SSD Metrics by @LujhCoconut in #1879
  • [TENT] Fix crashes caused by negative numa_node and incorrect config type inference by @dtcccc in #1894
  • [Store] Enable NUMA-segmented allocation for RDMA RealClient-only mode by @00fish0 in #1838
  • [CI] fix: health_check_test flaky due to fixed sleep, use polling for master down detection by @herbertskyper in #1868
  • [CI] add configurable GitHub mirror fallback for Ascend checkout by @staryxchen in #1896
  • feat(tent): replace raw RdmaEndPoint* with weak_ptr for endpoint lifecycle safety by @staryxchen in #1897
  • [Store] Fix hardcoded 127.0.0.1 bind address in standalone client RPC… by @zhangzuo21 in #1900
  • [TENT] Fix static library link group for final targets and reformat related CMake files by @staryxchen in #1893
  • [PG][TENT] Fix CUDA collective wait semantics and NVLink small-transfer completion by @KMSorSMS in #1863
  • [Doc] Update Client Explanation by @ykwd in #1905
  • [TE] Add fi_read support, endpoint LRU eviction, and multi-NIC striping for EFA transport by @whn09 in #1821
  • [Store] Enabling setting SSD offload path using python interface by @zhangzuo21 in #1884
  • [Store] Expose batch_replica_clear in Python binding by @hnts03-moreh in #1848
  • [CI] add format hook by @stmatengss in #1904
  • [TENT] Batch transfer requests using cudaMemcpyBatchAsync by @shuoerw in #1890
  • Minor bug fixes and improvments for Mooncake Store and Transfer Engine by @nickyc975 in #1895
  • [Store] Fix segfault in disk-replica/offload paths when handling GPU VRAM pointers by @LujhCoconut in #1892
  • fix(ci): retry ascend submodule update via GitHub mirrors by @staryxchen in #1924
  • [TENT] add FaultProxyTransport for fault injection testing by @staryxchen in #1907
  • [TE] PTE-aware auto-split large MR registration for EFA transport by @whn09 in #1912
  • [Store] Add client bandwidth metrics for real and dummy clients by @stmatengss in #1874
  • [store] Bug Fix: Local Disk Replica Metadata Not Cleaned Up After Store Node Offline by @Colors-111 in #1914
  • [Store] unify file storage backend env vars under MOONCAKE_OFFLOAD_ p… by @LujhCoconut in #1929
  • [Integration] connector_v1: subclass SupportsHMA so PD-disagg works for hybrid models by @HarshavardhanK in #1931
  • [Store] auto-enable MC_STORE_MEMCPY in TCP-only environments by @dtcccc in #1936
  • [TE] Fix a 1-second stall in RDMA WorkerPool due to store-buffer reordering by @chestnut-Q in #1932
  • [CI] Restore auto-triggered ascend-test and integration-test in ci.yml by @LujhCoconut in #1943
  • [PG] Fix wait() hang during CUDA Graph capture by @caozhanhao in #1933
  • [Store]: Wait for all tasks to complete before completing a batch by @nickyc975 in #1906
  • Pin a patched Go toolchain and track etcd go.sum for wrapper builds by @Copilot in #1937
  • Refactor ASIO shared target into mooncake-common by @Copilot in #1926
  • Bump version to 0.3.10.post2 in pyproject.toml by @ShangmingCai in #1949
  • Update build_wheel.sh package installation logic by @ShangmingCai in #1950
  • [Store] setup logs dir for realclient_main by @mzygQAQ in #1925
  • [Store] Support storage hierarchy with offload-on-evict mode by @hnts03-moreh in #1899
  • [Store][K8s-Native][1/N] Go-library for k8s leader election by @vladnosiv in #1910
  • [TE] EFA SRD shared-endpoint refactor: drop per-peer fid_ep model by @whn09 in #1944
  • [Build] Upgrade yalantinglibs to 6a0e067d by @stmatengss in #1961
  • [TE] reduce reg overhead by @stmatengss in #1964
  • [Docs] Update Mooncake Conductor design and API docs by @yejj710 in #1977
  • [docs] Document TCP port exhaustion and pool limitations by @zhangzuo21 in #1954
  • [transfer_engine] fix: drain endpoint waiting list via periodic reclaim by @ccs1112 in #1952
  • [transfer_engine] feat: make RDMA QP pkey_index configurable via MC_PKEY_INDEX by @yan-lgtm in #1985
  • [bugfix] Enable Mermaid rendering in Sphinx docs by @yejj710 in #1981
  • [Store] Fix DSO missing error when building with USE_3FS=ON by @yz53665 in #1983
  • [CI] route fork PR to pull_request_target for ascend/integration tests by @staryxchen in #1989
  • [CI] optimize build-flags job and add TENT compilation test by @00fish0 in #1775
  • [Store] fix(test): avoid in-process master port collisions by @Lin-z-w in #1987
  • [Store] Unified parallel tensor IO by @zxpdemonio in #1389
  • [TENT][Sunrise] Add sunrise_link transport, platform support, and UT … by @HomeDish in #1915
  • fix: hardcode Ascend mirror URL and remove pull_request_target routing by @staryxchen in #1992
  • fix (CI): remove pull_request_target trigger completely by @staryxchen in #1994
  • [Store] support host mem in ascend dummy-real mode by @ascend-direct-dev in #1917
  • [Store] Add standalone HTTP mount_shm/unmount_shm API with reconfiguration support by @Lin-z-w in #1928
  • [Store] Add Rust smoke test, benchmark, and CI coverage by @stmatengss in #1927
  • [Bug fix]: async offload RPC to prevent client expiration and shm_not_mappedFix/issue 1711 async offload rpc by @zhangzuo21 in #1734
  • [Store] Fix: KEYS_ULTRA_LIMIT error in eviction-enabled BucketStorageBackend by @Colors-111 in #1941
  • [CI] fix: temporarily exclude ub_transport_test from CI by @00fish0 in #2007
  • [Store] fix invalid acquire with release for atomic store by @NUABO in #1996
  • [Store] Fix LOCAL_MEMCPY segfault for multi-process-per-node deployments (e.g. vLLM data-parallel) by @LCAIZJ in #1995
  • [TENT] Recover cooled-down RDMA rails and add failover e2e tests by @staryxchen in #1984
  • [TENT] Fix RPC server IPv6 binding on IPv6-only hosts by @Nishikant-K-P in #2014
  • [TE] Fix DMA-BUF validation using wrong CUDA device index by @Nishikant-K-P in #2015
  • [TE] To avoid inconvenience during debugging, remove redundant checks. by @Cheng-China in #2020
  • [Store][Rust] Add batch_put_from, batch_get_into, batch_is_exist wrappers by @Dayuxiaoshui in #1993
  • Save the active device with hipGetDevice on entry and restore it before by @Lzy17 in #2018
  • [TE] Fix EFA segfault race and DP>1 peer_map_ thrashing by @whn09 in #2023
  • [Store] Add lock-free MmapArena allocator for buffer mmap path by @Venkat2811 in #1820
  • Allow customizing client port range by @riZZZhik in #2008
  • [Build] Define BUILD_BENCHMARK option and gate benchmark subdirectories by @tmchow in #2033
  • Updating news section in README by @Copilot in #2038
  • [TransferEngine] Init CUDA primary context before dmabuf-based mem registration by @ivanium in #2034
  • [TE][FIX]: Typo: PRC -> RPC in min/max ports by @riZZZhik in #2037
  • [TransferEngine] Unify fabric allocator plumbing by @stmatengss in #2028
  • [TE]: Fix possible dead lock in RDMA transport connection setup by @nickyc975 in #1959
  • [TransferEngine] Use allocation base addr for dmabuf-based mem registration by @ivanium in #2035
  • [PG] Inherit ProcessGroup to fix dynamic getSize() after extend_group_size_to by @UNIDY2002 in #2040
  • [TE] fix(efa): request libfabric API 1.18 so device RDMA is the default on all EFA generations by @whn09 in #2041
  • [TE] rdma: fix use-after-free crash in ibv_post_send caused by concurrent QP destruction by @dtcccc in #1903
  • [doc] vllm MooncakeStoreConnector by @LujhCoconut in #2042
  • [PG] Fix P2P regression caused by ProcessGroup inheritance (#2040) by @UNIDY2002 in #2043
  • [PG] Refactor P2PProxy with shared chunk pools and receiver-driven credit-based flow control by @caozhanhao in #1971
  • [EP] Add a comprehensive test for MooncakeEP by @caozhanhao in #1695
  • fix(transfer-engine): add missing empty checks for batch methods by @SYaoJun in #2046
  • docs: add vLLM Mooncake Store blog post to README updates by @Copilot in #2052
  • [Tent][AMD] Add AMD CDNA4 (ROCm/HIP) platform support with benchmark tooling by @zhangnju in #2021
  • [Store] Add DSA-like allocator benchmark workload by @yokinoshitayoki in #2051
  • [Store] fix(master-metrics): track local SSD storage in Master metrics by @LujhCoconut in #1969
  • [Store] feat(store): add allocate-backed segment mount and unmount APIs by @Lin-z-w in #2006
  • [Doc] feat: readme hardware by @stmatengss in #2053
  • [TENT] Add reference counting to RdmaTask to prevent UAF by @alogfans in #2047
  • [Store]Fix SSD offload in Metadata Server mode: sync ClientBuffer to metadata server by @Colors-111 in #1939
  • [engram] support engram by @zxpdemonio in #1483
  • fix(rust): add missing empty check for batch_is_exist by @SYaoJun in #2045
  • [Store] Fix disk replica read paths for GPU KV cache (LOCAL_DISK zero-copy, DISK temp-buf scatter) by @LujhCoconut in #2004
  • [TransferEngine] Add maca_transport for Metax MACA C500 intra-node P2P by @Dayuxiaoshui in #2059
  • [tebench] support nvlink xport and register buffers per-allocation by @jinke446 in #2073
  • [TENT] fallback to per-task cudaMemcpyAsync when driver lacks batch s… by @jinke446 in #2072
  • [TransferEngine] Fix GPU dependency in transfer_engine_bench by @xiejibing in #2068
  • [TransferEngine][Integration] feat: add MACA/MetaX GPU support and fix RDMA dmabuf registration by @JoeZhang-0x000 in #2019
  • [devcontainer] use build.dockerFile instead of dockerfile and add libxxhash-dev by @vie-serendipity in #2067
  • [Store] Optimize SpinLock via adding a fast-path check by @LujhCoconut in #2076
  • [Store][Fix]: enable local memcpy for metadata local replicas by @fang-tech in #2029
  • [Store] Add DSA-like workload allocation strategy and update allocator benchmarks by @yokinoshitayoki in #2080
  • [Store] Expose is_local_disk_replica() to Python + enable offload RPC in standalone mooncake_client by @zhewenl in #2083
  • [Store] add SSD offload support for ascend platform by @ascend-direct-dev in #2005
  • [Store] Add ObjectDataType enum for type-aware metadata by @he-yufeng in #1719
  • [Docs] Tag 3fs Feature as Experimental by @ykwd in #2062
  • [Doc] Align SSD offload docs with implementation by @stmatengss in #2027
  • [Doc] Fix the inconsistent param description by @ykwd in #2103
  • [Store] Add comprehensive test suite for Python binding error handling by @yurekami in #2097
  • [CI] pin torch version to 2.11.0 by @UNIDY2002 in #2105
  • feat(ci): update EP/PG torch versions — drop 2.9.0, add 2.12.0; switch CUDA 12.8→12.6 by @UNIDY2002 in #2101
  • [Store][K8s-Native][2/N] K8s leader election by @vladnosiv in #1956
  • [Store] Fix Rust store build path by @ZhenyuePan in #2114
  • [TE] Expose sendProbe via Python binding by @kflansburg in #2088
  • [Store] fix: prevent cross-process memcpy segfault when MC_STORE_MEMCPY auto-enables by @Yeuvoir in #2001
  • [Store] feat: add graceful segment unmount APIs by @Lin-z-w in #2065
  • [TENT] Fix batch getTransferStatus premature FAILED aggregation by @staryxchen in #2055
  • fix: mooncake_master -version prints release version and git commit hash by @Copilot in #2110
  • Fix scale-up semantics with two-phase extension protocol by @UNIDY2002 in #1968
  • feat: add Hygon DCU/DTK and Iluvatar CoreX platform support by @UNIDY2002 in #2118
  • [Doc] Add missing space by @ykwd in #2104
  • [TENT] Apply TE's changes of RDMA transport by @alogfans in #2102
  • [CI] change dependency libboost-all-dev to libboost-dev by @UNIDY2002 in #2129
  • [Docs] clarify Redis build options by @Lin-z-w in #2136
  • [Store] Report master metrics as per-second rates over time window by @Lin-z-w in #2082
  • [Doc] fix ssd offload deployment doc by @Baichuan7 in #2139
  • [CI/Build] Support configurable build dir in build_wheel.sh by @stmatengss in #2133
  • [TE] Update IntraNode NVLink transfer method cuMemcpy -> cuMemcpyAsync by @TTThanos in #2012
  • CI: refactor TENT build and add test step by @staryxchen in #2142
  • [PG] support external TE by @UNIDY2002 in #2149
  • [TransferEngine] Gate auto failover on status polling by @ZhenyuePan in #2122
  • [CI/Build] Switch WITH_NVIDIA_PEERMEM to env variable by @stmatengss in #2066
  • [TENT] Proxy manager bugfixes by @alogfans in #2091
  • [Store] Fix std::terminate crash on mooncake client shutdown (Issue#2115) by @LujhCoconut in #2125
  • [Doc] Update EFA transport doc: SGLang section + vllm-router + p5.48xlarge bench by @whn09 in #2154
  • feat(store): add NoF segment metadata management by @zwtao40 in #2143
  • [Store] correct GC interval default fallback in FromEnvironment by @LujhCoconut in #2126
  • [TE] Fix pytorch precision problem when using IntraNode NVLINK by @TTThanos in #2163
  • [Store] Fix Ascend dummy reconnect shm replay by @ascend-direct-dev in #2158
  • Bump version to 0.3.11 in pyproject.toml by @ShangmingCai in #2166
  • [Store] L2->L1 promotion-on-hit by @yzhan1 in #2071
  • Use sudo -E for make install in release workflow by @ShangmingCai in #2169

New Contributors

Full Changelog: v0.3.10...v0.3.11

Contributors

tmchow, whn09, and 81 other contributors
Assets 14
Loading