v0.2.1

First release on PyPI. Install with pip install mstar-ai (mstar-project and mstar-serve are alias packages that install the same thing). The import package and the CLI are still mstar.

What's Changed

• communication: vendored Rust transport (rust/) + RustZMQCommunicator (RFC #130 Step 1) by @npuichigo in #164
• [Fix] Conductor SIGINT, piecewise KV capture, and batched AC rollout split by @merceod in #176
• Register for send signature change by @stephen-dwq in #177
• Add ASR model support: whisper-large-v3 and higgs-audio-v3-stt by @zhudianGG in #159
• Pin worker executor threads to the worker's CUDA device by @merceod in #180
• Drain API result reads ahead of preprocess and fail requests on deliv… by @merceod in #181
• streaming: guarantee a terminal StreamBuffer chunk after producer-done by @stephen-dwq in #179
• Worker Bug Fixes by @NSagan271 in #183
• pin ruff + ignore Too many positional arguments by @NSagan271 in #187
• fix(client): decode streamed lines when Content-Type has no charset by @Sanjays2402 in #190
• Fix KeyError in eager batched execution for packed-output submodules by @NSagan271 in #195
• SHM tensor transport over a Rust shared-memory arena (RFC #130, Step 2) by @npuichigo in #168
• Qwen3-Omni Audio Understanding Bug Fixes by @NSagan271 in #196
• Cosmos3 droid by @merceod in #188
• Pluggable routers by @stephen-dwq in #189
• Add Qwen3-TTS-12Hz-0.6B-CustomVoice support by @qaqjx in #184
• Fail requests on worker error by @NSagan271 in #200
• Add Wan2.2: text/image-to-video (issue #126) by @Gaurav-Shah05 in #165
• Allow Multiple Sampling Configs Per Node by @NSagan271 in #201
• local reduce before all-reduce in tp moe blocks by @stephen-dwq in #206
• Perf(qwen3-tts): fuse CodePredictor RoPE and reuse decode masks by @qaqjx in #204
• Rust HTTP frontend behind --rust-frontend (RFC #130, Step 3) by @npuichigo in #169
• tests: modular: fix WorkerGraphsManager API drift by @vishwanath1306 in #229
• Dynamo backend by @merceod in #216
• [1/3] Extend device support from CUDA-only by using torch.accelerator in worker runtime by @yao-matrix in #219
• Refactor of Resources -> Resource Pools by @NSagan271 in #228
• api: pass raw request through image edits by @kamahori in #227
• test: update stale orpheus modular test to the decode_finished contract by @vasilevklart in #205
• fix(position): respect device-specific autocast state by @ryux1 in #240
• feat(#209): Order-preserving multimodal input adapter by @t-avil in #226
• api: accept unpadded base64 in input_audio.data by @appleweiping in #235
• Pypi packaging by @merceod in #242
• Ragged prefill wrapper by @NSagan271 in #236
• api: return 400 for malformed model kwargs by @appleweiping in #245
• sampling: write master rows with a scalar fill_, not through a reused pinned row by @vasilevklart in #222
• worker: async scheduling for TP nodes (MSTAR_TP_ASYNC_SCHED) by @vasilevklart in #224
• docs: resolve version from the m-star distribution by @Gaurav-Shah05 in #248
• Extend Double-Buffering to the Piecewise Cuda Graph Runner by @NSagan271 in #243
• Fix Broken Tests by @NSagan271 in #246
• Register tensor memory with mooncake under TCP, not just RDMA by @Gaurav-Shah05 in #252
• Feature/tp support crossattention by @Raphistolage in #250
• [2/N] Add BAGEL tensor-parallel serving on Intel GPU(XPU) by @yao-matrix in #220
• Bug fix: gate tensor transport teardown behind a reader-drain barrier (fixes abort-time FileNotFoundError) by @NSagan271 in #234
• Publish the engine as mstar-ai instead of m-star by @merceod in #259
• Release 0.2.1 and publish the aliases from the workflow by @merceod in #260

New Contributors

• @Sanjays2402 made their first contribution in #190
• @qaqjx made their first contribution in #184
• @Gaurav-Shah05 made their first contribution in #165
• @vishwanath1306 made their first contribution in #229
• @yao-matrix made their first contribution in #219
• @vasilevklart made their first contribution in #205
• @ryux1 made their first contribution in #240
• @t-avil made their first contribution in #226
• @appleweiping made their first contribution in #235
• @Raphistolage made their first contribution in #250

Full Changelog: v0.2.0...v0.2.1

M* v0.2.0

Highlights

• New model family: Cosmos3 (nvidia/Cosmos3-Nano, nvidia/Cosmos3-Super) text/image/video-to-video generation, robot-action modes, and optional synchronized sound; tensor- and sequence-parallel multi-GPU serving; CUDA-graph-captured denoising; mstar serve cosmos3 / mstar serve cosmos3_super; install via pip install mstar[cosmos3] (#121)
• PyTorch unpinned (torch>=2.9.1): the sgl-kernel dependency was replaced by a vendored, JIT-compiled CUDA kernel for the fused-MoE align op (#157)
• Per-request profiling support (#144)
• Piecewise CUDA-graph runner made model-agnostic (#154)
• In-flight requests abort when the client disconnects (#136)

Fixes

• KV-cache engine no longer leaks per-request submodule state (#146)
• Qwen3-Omni: Talker TTS embedding dtype mismatch (#138); flashinfer floor fixes the Talker sampler on graph-captured decode (#139)
• Submodule params allocate in the autocast dtype, avoiding fp32 weight-load OOM (#120)
• numba>=0.61 pin avoids an incompatible llvmlite (#152)
• food101 image-to-image benchmark used captioning prompts instead of edit prompts (#158)
• Setup and testing fixes (#116); torch.compile dynamic-param fix (#117); docs and README updates (#113, #114, #118, #122, #133, #134, #153)

Install notes

• New .[cosmos3] extra. Its fa3-fwd fast-path wheel is ABI-tied to your torch/CUDA build; on a mismatch the engine logs a warning and serves on the paged FlashInfer backend (see docs/models.rst).
• ninja is now required for Qwen3-Omni (first-use JIT build of the MoE align kernel; replaces sgl-kernel).
• flash-attn prebuilt wheels top out at torch 2.10 - on newer torch, build from source (see docs/installation.rst).
• torchcodec is no longer pinned so keep its version in lockstep with your torch.

Full changelog: v0.1.0...v0.2.0

M* v0.1.0

M* is a disaggregated, any-to-any multimodal inference engine. Models are declared as dataflow graphs and every request is a Walk over that graph; one runtime serves unified multimodal models, omni models, speech LMs, vision-language-action policies, and world models — via a Python SDK, an OpenAI-compatible API, and a native streaming endpoint.

Highlights

• Walk Graph execution: Sequential / Parallel / Loop + cross-partition streaming compose any model family.
• Supported models: BAGEL, Qwen3-Omni, Orpheus, Pi0.5, V-JEPA 2.
• Per-component fast paths: paged attention (FlashInfer) + continuous batching, CUDA-graph capture, classifier-free-guidance parallelism for diffusion, chunk streaming for audio codecs.
• Disaggregation: map components to GPUs per-component / per-walk via YAML; pluggable tensor transport (shared memory / TCP / RDMA).
• Interfaces: Python SDK (MStarClient), OpenAI-compatible API, native streaming.

Links

• Docs — https://mstar-project.github.io/mstar/
• Blog — https://mstar-project.github.io/
• Paper (PDF) — https://mstar-project.github.io/mstar.pdf

From Stanford University & the University of Washington.