Skip to content

MoRI v1.2.2

Choose a tag to compare

@jhchouuu jhchouuu released this 16 Jul 13:33
· 246 commits to main since this release
dafdcfc

MoRI v1.2.2

Feature-heavy release since v1.2.1: the new CCO (Collective Communication Object) module lands, EPv2 dispatch/combine gains a FlyDSL intranode path and ships as an installable package, a new fabric (UALink scale-up) IO backend is added, plus broad EP/IO/UMBP performance work, RDMA correctness fixes, and CI hardening.

Highlights

  • CCO (Collective Communication Object) module — mori's self-contained collective/communication layer (host + device API, LSA intra-node, GDA scale-out) (#449).
  • EPv2 dispatch/combine: FlyDSL intranode MoE dispatch/combine (preview) (#448), now distributed as an installable package (#482), with combine gather inner-unroll + tuned geometry by default and gfx1250 EP4 retune (#484), and gfx1250 + cross-node fabric support (#466).
  • IO: fabric (UALink scale-up) transfer backend (#467).
  • UMBP: cache remote fetches into local DRAM (dual-scheme) (#442), lower master RPC latency under load + kv-event bench (#440), and a distributed-mode flush API with hot-path optimizations (#419).

Features

  • feat(EPv2): combine gather inner-unroll + tuned geometry by default + gfx1250 EP4 retune (#484)
  • feat(ep): distribute dispatch_combine_v2 as an installable package (#482)
  • feat(io): add fabric (UALink scale-up) transfer backend (#467)
  • feat(EPv2): gfx1250 + cross-node fabric support (#466)
  • feat(ep): add blockwise-FP8 combine to the AsyncLL (low-latency) path (#465)
  • feat(ep): add blockwise FP4 (E2M1) combine transport (#461)
  • feat(EPv2): [preview] FlyDSL intranode MoE dispatch/combine (#448)
  • feat(cco): add Collective Communication Object (CCO) module (#449)
  • feat(umbp): cache remote fetches into local DRAM (dual-scheme) (#442)
  • feat(rdma): add dmabuf-first MR registration with MORI_DISABLE_DMABUF_REG (#355)
  • feat(ccl): add param-contiguous SDMA allgather
  • feat(shmem): bind ranks to GPU-local CPUs by default (#401)
  • feat(sdma): use KFD recommended engine mask for SDMA engine selection (#429)
  • feat(io): enable worker chunking for GPU single-NIC sessions (#436)
  • feat(umbp): benchmark / distributed-mode flush API / hot-path optimization (#419)

Performance

  • perf(rdma): default to ibv_reg_mr, opt into dmabuf via MORI_ENABLE_DMABUF_REG (#453)
  • perf(io): eliminate SQ full spin with futex admission (#439)
  • perf(umbp): reduce master RPC latency under load, add kv-event bench (#440)
  • perf(ep): improve intranode dispatch kernel load balancing and eliminate repeated loads (#412)
  • tune(cco): re-tune MI355X (gfx950) EP8 schedule for vec4 combine (#472)

Bug Fixes

  • fix(cco): guard hipMemAllocationTypeUncached for older ROCm (#480)
  • fix(io/xgmi): honor sub-region base offset in IPC remapping (#416, fixes #415)
  • fix(jit): invalidate kernel cache on #included source changes (#469)
  • fix(cco): serialize handle import under the VMM process lock (#476)
  • fix(cco): deterministic release of per-op symmetric memory (#474)
  • fix(cco): fix HIP VMM memory leak and crash (#455)
  • fix(umbp): fix umbp tests error (#470)
  • fix(bnxt): only define true/false for C in bnxt_re_hsi.h (#457)
  • fix(ionic): cap max_send_wr to 32767 (#451)
  • fix(mlx5): fix collapsed-CQ bandwidth spikes (#435)
  • fix(rdma): harden RDMA control-plane resilience (#432)
  • fix(io): set QP max_inline_data for notification SENDs (#424)

CI / Build / Tools

  • ci(nightly): install wheels from artifact, stop bloating gh-pages with wheels (#483)
  • tools: add NIC firmware version checks + mori check CI step (#471)
  • ci(cco): move runner to MI355X-AINIC (#477)
  • ci: prevent GPU VRAM leaks via tini reaping + graceful container teardown (#473)
  • ci: recover podman state before internode build on MI355X_AINIC (#462)
  • ci(cco): enable cco python test (#456)
  • feat(cco): update mori-cco dockerfile (#458)
  • ci: move ci runner 325 -> 108 (#447)
  • ci: migrate bnxt 300x CI to 325 platform (#438)

Full changelog: v1.2.1...v1.2.2