Skip to content

v0.1.2

Latest

Choose a tag to compare

@Zhichenzzz Zhichenzzz released this 09 Oct 03:39
· 40 commits to main since this release

TL;DR

More frontier models. DeepSeek-V4.1 Flash and MiMo-V2.6-Flash-RL (309B MoE) now train on the standard Miles image.

A third training backend. torchtitan joins Megatron and FSDP. All three now share one layer for data, loss, metrics, checkpoints and weight sync to SGLang, so a new backend or model plugs into the same path.

Run Miles natively on Kubernetes (experimental feature). With the new experimental backend, --cluster-backend kubernetes, the cluster schedules every trainer, engine and controller of a run directly, with no Ray cluster in between. The same backend can deploy trainers and engines as separate releases and restart a run's orchestration without stopping training. Ray remains the default.

Train several policies in one run. One job can train multiple policies, each with its own trainer, engines and checkpoints; the gsm8k example trains a solver together with the verifier that grades it.

Faster attention for sparse and linear models. Sparse attention for GLM-5 and DeepSeek-V3.2/V4/V4.1 runs on merged, per-architecture kernels, and linear-attention models (Qwen3.5, Kimi-K3, GLM-5.3-Flash) get a tensor-parallel layer with faster backward kernels.

Score centering for off-policy RL. A new loss, --loss-type score_centering, corrects the gradient drift that appears when rollouts come from an older policy than the one being trained, using the sampler's top-k probabilities.

Highlights

417 PRs from 17 contributors.

Model Support

Model Training PRs Docs
DeepSeek-V4.1 Flash Full #3179 @Zhichenzzz link
MiMo-V2.6-Flash-RL Full #3648, #3743 @guapisolo link

DeepSeek-V4.1 Flash moves into mainline. It previously needed its own image and an SGLang fork. Its training plugin and launcher are now part of Miles, and the sglang-miles build in the standard image carries the model, so DeepSeek-V4.1 trains on radixark/miles like any other model (#3179 @Zhichenzzz, sgl-project/sglang#42408).

MiMo-V2.6-Flash-RL. A 309B MoE with per-layer KV heads, asymmetric Q/K/V and sliding-window sinks, trained in bridge mode with a BF16 rollout engine. prepare converts the official FP8/MXFP4 checkpoint to BF16; on 2x8 H200 the full model streams its optimizer state to NVMe (#3648, #3743 @guapisolo).

Also new: native Megatron DSA for DeepSeek-V3.2 and GLM-5/5.1/5.2 behind --dsa-impl megatron (#3866 @zianglih), and NVIDIA Vera Rubin support in the container and launchers (#3700 @kaixih).

Training Backends

torchtitan joins Megatron and FSDP. --train-backend torchtitan trains through torchtitan's own trainer and parallelisms, and models it does not ship can be added as Miles-side packages; GLM-4.7-Flash is the first, trained colocated with tensor and expert parallelism and routing replay. Megatron remains the default (docs, #3171, #3252 @Zhichenzzz).

One layer underneath all three. Before this release FSDP carried its own copy of the RL step, weight sync and offload, and a third backend would have meant a third copy. FSDP and torchtitan now share one actor and training-step interface, all three backends send weights to SGLang through the same updater, and miles/backends/training_utils/ holds the data, loss, metrics, replay and checkpoint code they have in common (#3171, #3248 @Zhichenzzz).

Features

Run Miles natively on Kubernetes (experimental feature). Until now a Miles run lived inside one Ray job, even on a Kubernetes cluster. With --cluster-backend kubernetes, Miles installs the run itself and lets the cluster schedule every trainer, engine and controller, so a run's pods are ordinary cluster workloads that the cluster's own scheduling and monitoring already see. A per-namespace workbench (python -m miles.utils.external_utils.miles_workbench) checks your permissions before installing, follows every pod's logs, and collects a diagnosis bundle when a run fails. The backend is still under active development; Ray remains the default and needs no change (docs, #2188, #2436, #2446 and the series under Cluster Backend: Kubernetes @fzyzcjy).

Keep a run alive while parts of it restarts. On Kubernetes, one run can be installed as separate releases for its trainers, its inference engines and its orchestration script (--deploy-component), so each part can be managed on its own. A live run can also replace its orchestration script and rollout executor while trainers and engines stay up, and trainers reload their state in place; e2e tests check both against an undisturbed run. The fault injector now crashes trainer cells and rollout engines on Ray and Kubernetes during soaks, and a deterministic comparison checks that rollout engine crashes leave the loss curve unchanged. More related pull requests will be merged to main. (split deployment, hot restart, #2952, #2963, #2974, #2919 @fzyzcjy).

Train several policies in one run. Some recipes need models that learn from each other, such as a solver and the verifier that grades it. --megatron-config now declares several trainers in one job; each policy gets its own placement, rollout queue, data buffer, weight updates and checkpoints, while the run coordinates a global checkpoint and a leader policy decides when the run ends. The gsm8k solver–verifier example shows the setup (example, #2642, #2643, #2645 @fzyzcjy).

Score centering. When rollouts come from an older policy than the one being trained, even a constant reward can push the policy in an unwanted direction. --loss-type score_centering removes that drift by subtracting the expected score under the sampler's top-k probabilities, with optional truncated or masked importance weights (--score-centering-is tis|mis). It works with native and session rollouts (example, #3621–#3626, #3732 @Shi-Dong, #3904 @guapisolo).

Agentic rollouts that keep moving. Each agent call now runs in its own process by default, so one episode stuck parsing a very large response no longer stalls every other episode on the rollout loop (--custom-agent-function-mode inline restores the old behavior). Agents also stop receiving per-token logprobs they did not ask for, which could exceed 100 MiB for a long reply with top-k logprobs on (#3717, #3716 @nblintao).

Attach existing SGLang engines. --rollout-external-engine-addrs points Miles at engines launched elsewhere; it discovers their topology from /server_info and still drives their weight updates (#2513, #2514 @fzyzcjy).

Optimizations

Sparse and linear attention. GLM-5, DeepSeek-V3.2, V4 and V4.1 used to carry separate copies of the same sparse-attention kernels. They now share one implementation that picks FlashMLA or TileLang per GPU architecture and shape, which is faster on every GLM-5 shape and on Blackwell, and no longer recompiles for each packed sequence length. Linear-attention models (Qwen3.5, Qwen3-Next, Kimi-K3, GLM-5.3-Flash) move to a tensor-parallel layer that shards heads across ranks, with faster short-conv and KDA backward kernels (#3607, #3608, #3609, #3610, #3632, #3634 @Zhichenzzz).

DeepSeek-V4 under context parallelism. The sparse-attention indexer used to leave the last CP rank with the most work. Rows are now balanced across ranks with identical top-k picks, which raises long-context training throughput (#3689 @HJSang).

Weight sync for large MoE models. --update-weight-transfer-mode broadcast_packed (opt-in) sends each bucket as one broadcast instead of one per tensor, and routed-expert weights are prepared in parallel across expert-parallel ranks before they are gathered (#3224, #3731 @zianglih).

Refactors

Kernels and launchers. Kernels move from the model plugins into miles/kernels (#3605, #3606 @Zhichenzzz). Every launch script now offers the same prepare, train and full-train commands, and controllers run as independent workers that the Kubernetes backend, split deployment and hot restart build on (#2431, #2432, #2319, #2324, #2325 @fzyzcjy).

Full release notes by category below; breaking changes are near the end.

Cluster Backend: Kubernetes

Split Deployment

Fault Tolerance

Hot Restart and Take-over

Workers and RPC

Multi-Policy Training

Training Backends

  • torchtitan as a third training backend; FSDP and torchtitan share one torch-native actor, step-runner seam and the WeightUpdater path: #3171 @Zhichenzzz
  • training_utils becomes the shared layer for Megatron, FSDP and torchtitan (data, loss, metrics, replay, checkpoint, weight_update); one spelling for common knobs: #3248 @Zhichenzzz
  • Miles-side torchtitan model packages, with GLM-4.7-Flash first: #3252 @Zhichenzzz

RL Algorithms & Training

Agentic & Session

  • Run each agent call in its own process, so one slow episode cannot stall the rest: #3717 @nblintao
  • Stop sending agents per-token logprobs they did not ask for: #3716 @nblintao
  • fix(session): omit meta_info from chat responses: #3877 @guapisolo

LoRA & Multi-LoRA

Weight Update

  • --update-weight-transfer-mode broadcast_packed: one NCCL byte broadcast per bucket (opt-in): #3224 @zianglih
  • Parallelize routed-expert weight updates across EP and expert DP: #3731 @zianglih
  • Gate inference engine weight checksums behind their own flag: #3022 @fzyzcjy
  • Hand the weight-update group the gpus each engine really has: #2510 @fzyzcjy
  • Stamp the startup weight sync with the restored rollout id: #3059 @fzyzcjy

Kernels & Performance

  • Kernels move under miles/kernels (quant, MoE, activation, norm, HC, RoPE, attention): #3605, #3606 @Zhichenzzz
  • One DSA kernel pair for GLM-5 / DeepSeek-V3.2 / V4 / V4.1, with per-arch dispatch and a FlashMLA sparse forward: #3607, #3608 @Zhichenzzz
  • Head-sharded linear-attention layer for GDN and KDA (Qwen3.5 / Qwen3-Next, Kimi-K3, GLM-5.3-Flash) plus per-arch short-conv and KDA backward kernels: #3609, #3610, #3632, #3634 @Zhichenzzz
  • perf(dsv4): balance the CSA indexer across contiguous CP ranks: #3689 @HJSang
  • feat(fsdp): load module-level compute kernels from the Hub: #2829 @adarshxs

Memory & Offload

  • fix(optimizer): convert NVMe-streamed gradients to fp32 one bucket at a time: #3743 @guapisolo
  • fix(ray): pin each trainer to the NUMA node of the GPU it actually uses: #3869 @guapisolo
  • test(e2e): cover distributed Muon for three MoE models: #3878 @guapisolo

Model Support & Fixes

Observability

  • Every process names itself; redacted environment audit events re-recorded periodically (--env-report-interval-seconds), uncommitted-diff hashes, what the Kubernetes launcher launched, and every SGLang engine's serving environment: #2503, #2504, #2505, #2506, #2507, #2508, #2509 @fzyzcjy

Launchers & Configuration

AMD / ROCm

Docker & Build

CI & Release Infrastructure

Docs & Legal

Dependencies

  • SGLang: sglang-miles locked at b84bec5761 (v0.1.1: 880e3d2453), the Miles patch stack rebased onto SGLang v0.5.21 (#3884 @guapisolo). It includes the DeepSeek-V4.1 bf16 MoE fix (sgl-project/sglang#42408), Kimi-K3 weight reloads after graph capture (sgl-project/sglang#41925), packed weight receive buffers (sgl-project/sglang#39265) and the MiMo colocated RL loaders.
  • Megatron-LM: radixark/Megatron-LM@miles-main locked at fd15ee20a4 (v0.1.1: f148a32b43): fused RoPE packing for MLA/DSA, the DeepSeek-V4 CSA core, a load-balanced CP DSA indexer, deterministic fused CSA top-k, and explicit DSA top-k backends (radixark/Megatron-LM#108).
  • Ray: constrained to >=2.56.0,<2.59.0; 2.59 removed the object-freeing API the RPC references use (#3895 @guapisolo).
  • Mooncake: the base image's mooncake-transfer-engine-cuda13==0.3.13 instead of an older dev wheel (#3738 @guapisolo).
  • TileLang: 0.1.14 patched for fp16 256-bit stores on Blackwell, pending tile-ai/tilelang#3158 in a release (#3647 @Zhichenzzz).

Docker

docker pull radixark/miles:v0.1.2
  • NVIDIA: CUDA 13 only, linux/amd64 and linux/arm64, built on lmsysorg/sglang:v0.5.21 (manifest digest sha256:c81d8ba7b91571aac8e0bb2be372c6a9664e1ff7b88f518a8891861bae83e3b7; pin with radixark/miles@sha256:c81d8ba7…). Sources: miles a22fd82419, sglang-miles b84bec5761, Megatron-LM fd15ee20a4. Automatic CUDA 12 builds are retired; previously published CUDA 12 tags remain available, and docker/Dockerfile.cu12 is kept as a deprecated manual recipe (#3895 @guapisolo).
  • The image now ships kubectl and helm for the Kubernetes backend (#2191 @fzyzcjy).
  • AMD: ROCm images build on the SGLang v0.5.20/v0.5.21 ROCm bases, and a new rocm10-mi30x (gfx942, MI300X/MI325X) variant replaces the retired rocm700-mi30x (#3873 @XinyuJiangCMU, #3858 @michaelzhang-ai).
  • Vera Rubin: docker/build.py --variant rubin builds an arm64 image from docker/Dockerfile.rubin (#3700 @kaixih).

Breaking Changes & Upgrade Notes

  • Launcher subcommands. Launch scripts now expose prepare, train and full-train. train only submits training (and copies converted checkpoints to node-local disk first); download/cast/conversion moved to prepare, and prepare-cp is gone. Scripts whose train used to run the whole pipeline (for example scripts/run_deepseek.py) → run full-train, or prepare once and then train (#2431 @fzyzcjy).
  • --rollout-external removed. External rollout is now implied by --rollout-external-engine-addrs (static engines discovered from /server_info) or --custom-inference-engine-provider-path → drop --rollout-external; for a PD-disaggregated external fleet add --rollout-external-router-pd. --eval-num-gpus cannot be combined with external engines (#2513 @fzyzcjy).
  • --freeze-indexer removed. It had no effect; there is no replacement (#3866 @zianglih).
  • Agent functions run in a child process by default. Their arguments and return values must be picklable, and module-level state lasts one call → make them picklable, or pass --custom-agent-function-mode inline for the old behavior (#3717 @nblintao).
  • Session chat replies are slimmer. meta_info is no longer included in /sessions/{id}/v1/chat/completions replies (session records keep it), and logprobs are returned only when the agent's request asks for them → set logprobs in the request if an agent reads them (#3877 @guapisolo, #3716 @nblintao).
  • CUDA 12 images are no longer built automatically → use the CUDA 13 image (#3895 @guapisolo).
  • Ray must be <2.59.0 (#3895 @guapisolo).
  • Licensing. LICENSE is the standard Apache-2.0 text; the Miles copyright and the Slime (Zhipu AI) attribution move to NOTICE (#3865 @maocheng23).

All Contributors

@adarshxs @anthonyhe-bot @Arist12 @fzyzcjy @guapisolo @HJSang @kaixih @maocheng23 @micahtyong @michaelzhang-ai @nblintao @Shi-Dong @XinyuJiangCMU @xiuhu17 @yueming-yuan @zianglih @Zhichenzzz

Full Changelog: v0.1.1...v0.1.2