Repository navigation
MoRI v1.2.5
This release covers the 105 commits since v1.2.4.
The published wheels are now built on manylinux_2_28. They install on any distro with glibc 2.28 or newer, and they carry their own gRPC, so no part of mori needs gRPC on the host.
The other major themes:
- EPv2 gets the AsyncLL path, LL128 small-token kernels and EP16/EP32 tuning on gfx1250, and plans that report the buffer sizes they need.
- CCO adds a family of fused GEMM + collective operators.
- CCO SDMA gains 2D transfers, wait and broadcast packets (gfx950 and gfx1250), and a complete device API for FlyDSL and Triton.
- GDA gains doorbell coalescing.
- MORI-IO adds a libfabric backend for Slingshot/CXI and recovers from a failed RDMA QP.
- UMBP reworks its SSD tier and its eviction.
Highlights
-
Portable wheels. Wheels are built on manylinux_2_28 (#852): one wheel per Python version, for any glibc 2.28+ distro (RHEL 8/9, Ubuntu 20.04 and later).
- UMBP's gRPC is now its own extension module (#751), so
mori.ops,mori.ioandmori.shmemno longer need gRPC at all. This removes the v1.2.4 known issue (ImportError: libgrpc++.so.1). - The wheel vendors UMBP's gRPC, together with the third-party license texts.
- The wheels carry gfx1250 code alongside gfx942 and gfx950.
- UMBP's gRPC is now its own extension module (#751), so
-
EPv2.
-
Fused GEMM + collective operators on CCO:
- GEMM + all-gather (#807);
- all-gather fused into the fp8 GEMM that consumes it (#809);
- GEMM + all-to-all (#695);
- all-to-all fused into the fp8 GEMM that consumes it (#868);
- GEMM + reduce-scatter for sequence parallelism (#802).
A guide to these operators is in #913.
-
CCO SDMA and GDA.
-
MORI-IO.
Features
- feat(ep/v2): AsyncLL dispatch/combine on cco, with cross-node and gfx125x support (#768); plans report the arena and staging sizes they need (#844); a dispatch plan gets a default staging buffer that its caller can replace (#764); module
__device__staging is replaced by a shared buffer (#703) - feat(ep/v2, gfx1250): LL128 small-token dispatch/combine, plus EP16/EP32 dispatch tuning (#843)
- feat(ep/v1): opt-in
MORI_EP_DISABLE_RDMA_ATOMICSfor InterNodeV1/V1LL (#736) - feat(cco): fused GEMM + collective operators: gemm_ag (#807), ag_gemm (#809), gemm_a2a (#695), a2a_gemm (#868), GEMM + reduce-scatter (#802)
- feat(cco): 2D SDMA put/get via the LINEAR_LARGE_SUB_WINDOW packet (#805)
- feat(cco): SDMA POLL_REGMEM wait and COPY broadcast packets, gfx950 (#814) and gfx1250 (#880); fused SDMA wait+put post (#866)
- feat(cco): expose ccoSdma's full put/get API to FlyDSL and Triton (#792);
cco_sdma_put_ptr_rsig__threadandcco_sdma_signal_slotbitcode (#786) - feat(cco): GDA doorbell coalescing and compile-time doorbell modes (#895)
- feat(io): multi-NIC OFI/libfabric backend for Slingshot/CXI (#674); async NIC-event fault injection and port-down fail-fast (#815); simulated RDMA/TCP/XGMI backends (#867)
- feat(rdma): register Ionic CQ/SQ/RQ rings and the atomic ibuf via dma-buf (#649)
- feat(umbp):
- BatchPrefetch stages a queued request's keys in local DRAM (#823);
- large host DRAM buffers are registered as several NIC memory regions (#810);
- remote writes are placed on the writer's NUMA node (#855);
- a local ranged GET off SSD reads only its extent (#781);
- SSD demote/promote skip staging (#776);
- eviction converges onto one path, and nodes evict locally by default (#741);
- masterless
/metrics(#683); - per-medium byte counters for tier moves (#837);
- hugepage-backed ranged scratch arenas (#830).
- feat(umbp), more:
- an SSD backend can keep small objects off it (
min_object_size, #824); - the master routes small objects away from tiers they are under the floor of (#829);
- the standalone server's ranged scratch is laid out by NUMA node (#856);
- simulated storage and local/peer KV transfers (#862);
- a warning when a NIC memory region outgrows what an AINIC translates (#896);
- opt-in DRAM residency reporting to SGLang's KV Indexer (#851).
- an SSD backend can keep small objects off it (
- examples and POCs:
Performance
- perf(gemm-ar): mxfp8 with row-major scales staged through LDS (#806); load mxfp8 scales one K step ahead at K >= 4096 (#857); opt-in optimization paths (#748)
- perf(ep/asyncll): fuse put-signal and drop drains (#801)
- perf(EPv2/gfx1250): dispatch rows for EP4, hidden 2048, topk 8 (#881)
- perf(umbp):
- transfer plans are built incrementally (#758);
- unbound DRAM tiers are prefaulted in parallel across NUMA nodes (#718);
- the default DRAM eviction band is now 0.95/0.9 (#832);
- SSD extent staging is freed when its hold drops, with a next-fit span search (#838);
- SSD batch reads get their own io_uring rings per drive (#901);
- local readers of the same extent share one SSD staging span (#902).
Bug Fixes
- fix(anvil/sdma):
- topology is read from sysfs, and libhsakmt gets its own node id, which fixes SDMA queue setup in containers that were given only some of the host's GPUs (#841);
- SDMA queues are keyed on the KFD node id instead of the HIP device ordinal (#623);
- on gfx1250, loopback is spread over the peer-link engines when there is no CPU link (#766).
- fix(cco): two SDMA device-post deadlocks (#711); never fill the SDMA ring completely (#819); clear the HIP last-error after a failed fabric handle probe (#761)
- fix(ep/v2): AsyncLL fused-receive deadlock under two streams, plus early notification by default on gfx950 (#875); each AsyncLL op gets its own SDMA queue per peer (#892)
- fix(ep/v2): three gfx1250 combine slow paths at topk 9 (#886); clamp the untuned FlyDSL combine block count to the flag slots (#876); routing handles get their own buffers on request (#733)
- fix(ep): close two InterNodeV1/V1LL dispatch/combine races, and add stress_accuracy (#796); fail fast for cross-node kernels on the CCO backend (#894)
- fix(io): register host memory with ibv_reg_mr alone, never dma-buf (#890); a failed notification post waits for the transfer's data WRs (#836); XGMI IPC handle device ordinal (#803)
- fix(bootstrap): retry
SendMessage's ConnectSocket for cross-node reliability (#771) - fix(topology): load the GPU topology one comm at a time (#813)
- fix(gemm-ar): drain the peeled tail's producers (#785)
- fix(rdma): name the likely causes in fatal MR registration errors (#821)
- fix(umbp):
- SSD tier:
- the SSD tier no longer runs out of fds (#754);
- it gives disk back one segment at a time (#775);
- a single-flight SSD read joins any in-flight read that covers its extent (#827);
- SSD staging no longer fails local loads (#762);
- an SSD Clear keeps the staging spans that local readers still hold (#788);
- an in-place SSD read counts as an access (#793);
- SSD keys are counted without snapshotting them (#808);
- puts that a full SSD staging arena refuses are counted (#822).
- Standalone server:
- Eviction:
- Correctness and reporting:
- local clear is acknowledged when no master is configured (#769);
- the backend commit happens outside the pool lock (#804);
- a resolve counts only the reader's slice of the bytes (#835);
- a remote put of a key that another writer holds counts as stored (#820);
- MoriIoEngine turns MORI IO notifications off (#840);
- false put-failure noise is gone (#777).
- SSD tier:
Build / CI / Tests / Docs
- build(wheel): build the published wheels on manylinux_2_28 (#852)
- build: UMBP's pybind is its own module (#751); PCI topology is read from sysfs, and the libpci dependency is dropped (#724)
- build(flydsl): FlyDSL 0.4.0 in the dev image and examples (#817). FlyDSL stays an unpinned optional extra.
- ci: the nightly version is based on the numeric part of the last release tag (#847); a dispatch can run the UMBP hicache smoke test (#834); fix the flaky test_engine (EADDRINUSE) (#853)
- refactor(umbp): one incremental plan builder shared by every transfer engine (#774); track each SSD resolve read in one struct (#903); remove dead code (#778); port test_peer_pool's FailingEngine (#790)
- docs: a guide to the fused GEMM + collective operators (#913); UMBP blog in the news section (#828); acknowledge hipFile in the GDS transfer engine (#782)
Wheels: cp310 and cp312, both manylinux_2_28_x86_64 (glibc 2.28 or newer), published to PyPI as amd-mori==1.2.5.
Full changelog: v1.2.4...v1.2.5