MoRI v1.2.3

MoRI v1.2.3

Large feature + performance release since v1.2.2 (92 commits). Major themes: EPv2 (dispatch/combine v2) maturing with a HIP/JIT backend and broad gfx1250 (MI450) bring-up, a UMBP refactor to backend/transport-agnostic distributed mode, CCO gaining torch-SymmetricMemory/allocator integration, plus extensive EP tuning and RDMA/CCO correctness fixes.

Highlights

• EPv2 dispatch/combine: HIP/JIT backend for intranode dispatch/combine + gfx1250 TDM (#548), FP4 dispatch (#517), per-token scale row forwarding (#593), wide-EP (>32) intranode (#606) and rack-level wide-EP (#597).
• gfx1250 (MI450) bring-up: MoE dispatch/combine on gfx1250 (#532), tuning + kernel optimizations (#530, #599, #577, #567, #568), and build/anvil enablement (#550).
• UMBP: refactored distributed mode to be backend- and transport-agnostic (#540) and rebased the metadata store (#437); adds standalone process mode (#460) and tunable read leases (#452).
• CCO allocator / symmetric memory: register mori as a torch SymmetricMemory backend (#544), back symm_mem.rendezvous() with a CCO window (#628), and register external VMM allocations (e.g. torch.symm_mem) into the flat LSA space (#492) — plus CCO SDMA transport (#454, #513) and Triton device API bindings (#594) underneath.

Features

• feat(EPv2): HIP/JIT backend for intranode dispatch/combine + gfx1250 TDM (#548)
• feat(EPv2): support wide EP (>32) in intranode kernels (#606); extend to rack-level wide-EP (#597)
• feat(EPv2): enable fp4 dispatch (#517); forward a per-token scale row with dispatch (#593)
• feat(ep/intranode): bring MoE dispatch/combine up on gfx1250 (MI450) (#532)
• feat(cco): register external VMM allocations (torch.symm_mem) into flat LSA space (#492)
• feat(cco): Triton device API bindings (#594); gfx1250 SDMA support + anvil fixes (#513)
• feat(cco): honor reqs.sdmaQueueCount, clamp to HW max, guard queueId (#501)
• feat(shmem): CPU host-proxy RDMA transport alongside IBGDA (#558); MORI_ENABLE_RAIL_ONLY (#591)
• feat(allocator): back symm_mem.rendezvous() with a CCO window (#628); register mori as a torch SymmetricMemory backend (#544)
• feat(umbp): standalone process mode (#460); tunable read leases + tighter master lease default (#452)
• feat(io): optional ROCTX profiling ranges (#510)
• feat(tools): Ionic QoS setup configurable (#499); adapt mori check/setup for Mellanox NICs (#486)
• ccl: hierarchical cross-node AllGather (intra-node SDMA + inter-node RDMA)
• support UALink bandwidth testing (#552); MORI-IO nixl-style cpp bench (#504)

Performance

• perf(EPv2): gfx1250 combine tuning + LDS-budget pull tile (#608, #500, #530, #577, #487)
• perf(ep): MI350X/MI300X IntraNode/IntraNodeLL tuning configs (#586, #538, #464); EPv1 MI355X EP8 tuning keyed by topk (#511)
• perf(ep): InterNodeV1LL small-token latency on EP16 (#613); rework EP tuning pick/save + cut small-token host overhead (#604)
• perf(ep): small-batch dispatch cost cuts on gfx1250 (#568, #567, #562, #599)
• perf(io): hotpath improvements (#583)

Bug Fixes

• fix(EPv2): align staging pool capacity with EpMaxRecv (#617); clamp idle lanes in gather-combine (#545); intranode correctness/perf fixes (#522); flydsl 0.3.0 offset shim (#536)
• fix(cco): initialize packaged ROCm runtime before native load (#609); fix flaky SDMA data corruption (#519)
• fix(ep): AsyncLL slot double-allocation when top-k doesn't divide warpSize (#505); comb_bar cross-call race (#494)
• fix: silent cross-node combine corruption in InterNodeV1 (#546); SDMA AllReduce corruption across HIP graph replays (#559)
• fix(rdma): dmabuf offset for sub-allocated GPU buffers (LOC_PROT on ionic RoCE) (#509)
• fix(security): harden bootstrap and CI against scan findings (#605)
• fix(umbp): support N host regions per worker in standalone-process mode
• fix(build): cap setuptools before build_ext lifecycle change (#535); auto-disable UMBP when gRPC/Protobuf missing (#488)
• fix(bootstrap): configurable rendezvous timeout via MORI_BOOTSTRAP_TIMEOUT (#493)
• bugfix: mlx5 ci hung (#622); intranode bw check in mixed-vendor env (#590); guard ModuleLogger shared state with a mutex (#528)

Build / CI / Tools / Docs

• build: adapt mori to ROCm 7.1.4 container (#541); recognise gfx1250 for anvil_device.hpp (#550)
• compat(EPv2): support FlyDSL 0.3.0 alongside 0.2.x (#529)
• CI: enable mlx5 nightly (#635); enable MLX+MI300 CI (#607); remove bnxt ci (#516); perf report after nightly (#512)
• feat(tools/env-check): GPU memory check (#566)
• refactor(umbp): backend/transport-agnostic distributed mode (#540); metadata store rebase (#437)
• [JAX] drop local xla_ffi headers in favor of jax-backed header path (#584)
• docs: known-issues (ROCm 7.2.x clr bugs) (#640, #595), Kimi collaboration news (#495), io tuning guide (#485)

Full changelog: v1.2.2...v1.2.3