Skip to content

v0.9.0

Latest
Latest

Choose a tag to compare

View all tags
@wuxibin89 wuxibin89 released this 14 Aug 07:55
v0.9.0
483b8a0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v0.9.0

Highlights

Training

Megatron

  • DeepSeek-V4 GRPO end-to-end with Megatron-Bridge actor/ref, vLLM rollout and FP8/MXFP4 weight transfer (#6473), plus a contiguous context-parallel layout (#7221) and CP fixes that make long-context DeepSeek-V4 runnable (#7297).
  • Megatron Lite (mlite) backend for DeepSeek-V4, GLM-5 and Kimi-K2.5/K2.6, with 256-GPU GRPO launchers (#6791, #7091).
  • Muon optimizer support via Megatron-Core TensorParallelMuon, with AdamW fallback for non-2D params and an opt-in muon_match_adamw_update_rms scale (#7120).
  • Dynamic context parallel scheduling: each packed micro-batch picks a smaller local CP group when sequences don't need the full CP size (#6555).
  • Megatron chunk entropy (#6446) and fused logprob/entropy migrated from a GPTModel.forward monkey-patch to the Megatron output_processor hook (#6933).
  • Packed-sequence throughput work: BSHD micro-batches padded to the mini-batch max seq_len (#6901) and bucketed packed sequence lengths (#7358).
  • Checkpoint manager rewrite with YAML checkpoint config APIs (#6014); optimizer states and DDP grad buckets aligned with model precision (#6526).

VeOmni

  • DeepSeek-V4 support (#7242) and GPT-OSS weight preservation without expert parallelism (#7243).
  • Weight-refit extra memory reduced from O(E) to O(E/ep_size) by replacing the expert all_gather with sequential intra-EP broadcasts (#6612).
  • pad_to_length bucketing for packed micro-batches, collapsing dynamic shapes so torch.compile / Triton / DeepGEMM autotune cost is paid once instead of every step (#7272).
  • EP-aware sharded delta export for fused expert stacks (#7085), DSD optimizer checkpoint state backfill (#7283), and the missing config surface in veomni.yaml (#6505).

FSDP

  • Per-unit LoRA summon with FSDP1/FSDP2 compatibility and strip-modules support (#6512); save_lora_only checkpoints (#7061).
  • Chunked gather-logsumexp for top-K distillation loss to avoid OOM at long context (#6593) and deferred gradient sync during accumulation (#7095).
  • FSDP2 weight export skips the whole-shard staging round trip (#7005).

Rollout

vLLM

  • Prefill-Decode disaggregated rollout (rollout.name=vllm_pd) routing GRPO rollouts across 1 prefill + N decode engines over NIXL or Mooncake (#6243).
  • Full determinism for vLLM rollout and reward-model inference, so two identical runs produce bitwise-aligned reward curves (#6572).
  • DeepSeek-V4 FP8/MXFP4 linear and MoE weight refit fixes, with the quantized weight-sync paths split per scheme (#7224).
  • Weight-sync restructuring (#7179), non-contiguous weight handling in bucketed transfer (#6919), and MTP drafter weights preserved across hybrid sleep (#6661).

SGLang

  • ROCm backend support out of the box via AITER attention/kernel defaults injected through the platform layer (#6664).
  • FP8 ignored-layer support for Qwen3.x GatedDeltaNet (#6906, #6915).
  • LoRA correctness: SGLang stays LoRA-free when model.lora.merge=True (#7234); NCCL buffer race guard in _compact_for_bucket (#7139).

CheckpointEngine

  • New delta_sharded checkpoint engine for disaggregated async training: each actor rank byte-diffs its own shard against a pinned shard-sized CPU snapshot, sparsely gathers only the changed (position, value) pairs, and the rollout side applies them in place through the stock custom-weight-loader hook — no rank ever holds a full-model snapshot (#6974).
  • Measured steady-state update_weights speedups vs full NCCL broadcast (H100, GSM8K GRPO, V1 separate_async, SGLang rollout): 2.4x at Qwen2.5-7B, 1.9x at 32B, 3.1x at 72B.
  • Generalized beyond flat Shard(0): block placements (Shard(k), multi-Shard meshes, manual splits) plus a backend-owned HF export contract (#7144), VeOmni FSDP2+EP (#7085), Megatron-Bridge param mappings covering TP+EP and hybrid-Mamba (#7181), and PP/VPP steady delta export (#7223).
  • Transport hardening: HCCL split_weight_chunks (#7205), NCCL broadcast bucket sizing (#7107), separate magic_recv buffer to prevent weight corruption (#6813), and CUDA stream synchronization on broadcast completion (#7083).

Trainer

Unified V1 trainer

  • New unified trainer abstraction covering sync and async training under verl/trainer/ppo/v1 (#6710), enabled by default in this release (#6823). sync, colocate_async and separate_async now share one control-flow implementation, replay buffer and metric surface.
  • Runnable separate_async trainer (#6790) aligned to the same step granularity as the other modes (#6977), with decoupled PPO support (#7188).
  • Staleness control strategies drop and wait against max_off_policy_threshold (#6778), and a unified replay-buffer eviction/refill matrix for stale, DAPO-filtered and failed rollout groups (#7082).
  • Streaming dataloader (data.gen_batch_size) plus async-trainer checkpoint recovery that restarts pending/running prompts on resume while preserving finished samples (#7037).
  • Colocated reward model for sync / colocate_async (#6818); SkipManager unified across skipping schemes (#6097) and wired into V1 (#6897, #7032).
  • Fully-async dynamic resource scheduling: trainer-node GPUs switch between rollout and training via a pluggable scaling policy (#6556).
  • New observability: off-policy metrics (#6736) and rollout MoE load-balance metrics (#6853, #7075).

Agentic RL

  • uni-agent is a reference stack for long-horizon agent RL on top of verl — bring any harness (Claude Code, Mini-SWE-Agent, or anything speaking an OpenAI/Anthropic-compatible endpoint) through the Uni-Agent Gateway, and run 1,000+ concurrent stateful sessions.
  • Continuous Token mechanism for multi-turn agentic rollout: a reusable builder layer that keeps token continuity across assistant output, tool/environment feedback and the next generation prompt, with Qwen / MiniMax / GLM boundary handling (#6779). Disabled by default.

Hardware support

  • Platform abstraction layer + plugin-based engine override (#6086): verl.plugin.platform exposes a hardware-agnostic device API, @PlatformRegistry.register(platform=...) registers new platforms, and EngineRegistry lets plugins override engines by (model_type, backend, device) — all without patching the verl source tree.
  • New verl-project/verl-hardware-plugin repository, jointly developed with the FlagOS community, ships reference platform and engine implementations for Cambricon MLU, MetaX, Iluvatar BI-V150, Intel XPU, Enflame GCU and FlagOS. Plugins are auto-discovered through the verl.plugins entry point.
  • AMD ROCm: PlatformROCm backend (#6702), DeepSeek-V4-Flash GRPO on AMD GPUs (#7050), pure-torch fast_hadamard_transform fallback for DSA (#7241), streamlined ROCm Dockerfile and README (#6619), corrected FLOPS tables (#6942), and an MI300 e2e_ppo_trainer CI workflow (#6668).
  • Ascend NPU: per-model NPU patches with fault isolation (#6777), Qwen3.5-122B support (#6582) and long-sequence launcher (#6807), MXFP8 training (#6562), fully-async adaptation for vLLM ≥0.18 and ≥0.19 (#6929, #6886), Atlas 950DT A5 installation guides, CANN 9.x images, and multi-node nightly CI (#6876).
  • New How to Extend verl guide covering reward functions, tools, agent loops, engines and platforms (#6825).

Algorithms & Reward

  • DRO losses (#7245) and token-sum loss aggregation (#7197).
  • Critic value loss normalized over the global mini-batch instead of per micro-batch (#6957); distillation loss micro-batch normalization fix (#7225).
  • Deterministic reward for user-defined generative RM paths (#7027) and an optional per-sample compute_score timeout in NaiveRewardManager (#6673).
  • REINFORCE++ carries running_return through observation spans (#7300).

Tools & Observability

  • RL-Insight integration: add rl_insight to trainer.logger and set RL_INSIGHT_SERVER_URL to get training metrics, RL state traces and rollout/TransferQueue subsystem metrics in unified Grafana dashboards — no training-loop changes (#6680, #7048).
  • torch profiler re-enabled with clearer output naming (#7099) and partial-token window profiling for rollout (#5760).
  • Gemma4 multimodal GRPO (#6715), Open-R1 multimodal and TinyLLaVA-Video-R1 preprocessing/training scripts (#6849), and a processor hook for multimodal RoPE kwargs (#7236).
  • Security: Qwen3 tool parser no longer allows arbitrary code execution (#6542).

Breaking Changes

  • The V1 PPO trainer (verl/trainer/ppo/v1) is now the default (#6823). separate_async also switches to the same step granularity as the other trainers (#6977).
  • Vanilla mBridge is deprecated; Megatron-Bridge is the default for the Megatron backend (#6951).
  • vLLM older than 0.18.0 is no longer supported; all compatibility branches at or below that version were removed (#7190).
  • Checkpoint engine: the misleading trainer argument is renamed to actor_wg (#6724).
  • The delta custom weight loader moved to verl.workers.rollout.sglang_rollout.delta_loader.apply_delta; update custom_weight_loader config strings (#6974).

What's Changed

  • [misc] chore: bump version to 0.9.0.dev by @wuxibin89 in #6553
  • [ci,veomni] fix: make router-replay install test independent of veomn… by @hjshi84 in #6557
  • [megatron] fix: Fix GPU memory leak in ref model offload by explicitly releasing storage by @ZLiao097 in #6447
  • [ci] chore: continue to replace the qwen25 model with the qwen3 model by @daikang6 in #6554
  • [reward] fix: release worker count when NaiveRouter request fails after retries by @fuergaosi233 in #6544
  • [vllm] fix: reset all caches after weight updates by @s-isaev in #6522
  • [megatron, ckpt] refactor: optimize mcore ckpt manager impl, checkpoint yaml config APIs by @ETOgaosion in #6014
  • [ci] chore: migrate pip→uv in all Ascend CI workflows by @KadenZhang3321 in #6535
  • [megatron, ckpt] fix: handle None param_data in get_megatron_module_device when use_distributed_optimizer=False by @khazic in #5782
  • [doc] chore: Update ascend_sglang_best_practices.rst by @hustmf in #6573
  • [docker] fix: align stable sglang image for GB200 by @kaixih in #6581
  • [ci] fix: stablize main CIs by @ETOgaosion in #6568
  • [model] feat: add Qwen3.5-122B Ascend support by @zjchenn in #6582
  • [doc] feat: add Qwen3.5-122B NPU guide link by @zjchenn in #6588
  • [ci] fix: chang sglang8.5.0 a3 by @daikang6 in #6586
  • [tool, rollout] feat: Use SkipManager to uniformly manage the skipping schemes. by @tardis-key in #6097
  • [vllm, megatron] fix: mxfp8 trainning support on Ascend NPU by @quancs in #6562
  • [hardware] feat: add platform abstraction layer and plugin-based engine override system by @physics31415926 in #6086
  • [tool] fix: update npu profiling test scrip by @yyyy2000 in #6606
  • [perf] feat: support partial-token window profiling for rollout in verl by @mengchengTang in #5760
  • [veomni] feat: reduce extra memory from O(E) to O(E/ep_size) during weight refit by @wuxibin89 in #6612
  • [data] fix: default image_patch_size to processor's real patch_size in filter_overlong_prompts by @zengxingchen in #6595
  • [rollout] chore: change default to calculate_log_probs=True by @wuxibin89 in #6616
  • [fsdp] fix: do not manually move model to GPU by @pavelgein in #6604
  • [doc] chore: Update index.rst by @hustmf in #6617
  • [doc] chore: update amd rocm index by @wuxibin89 in #6622
  • [misc] fix: revert run_qwen3_8b_fsdp_npu.sh by @hustmf in #6601
  • [megatron] fix: clamp num_tokens=0 in MTP loss & add normalized scale for MTP per token loss by @arvyanh in #6464
  • [ci] fix: remove uv usage from A2 vlm_rl_job, use pip instead by @KadenZhang3321 in #6618
  • [fsdp] feat: add Qwen3.5-4B on-policy distillation FSDP script by @cls1206 in #6638
  • [ci] fix: Modify the log path of npu nightly ci by @daikang6 in #6600
  • [rollout] fix: avoid arbitrary code execution in Qwen3 tool parser by @fuergaosi233 in #6542
  • [veomni] fix: pass mixed precision config to veomni by @Luosuu in #6644
  • [ci] fix: several CI syntax errors by @HollowMan6 in #6649
  • [megatron] fix: official moe_enable_routing_replay flag in addition by @HollowMan6 in #6647
  • [megatron] fix: Fix Megatron optimizer offloading when use_precision_aware_optimizer=True by @rao-ashish in #6635
  • [data] fix: support path multimodal placeholders by @le-czs in #6631
  • [data] chore: Update TransferQueue version to 0.1.8 by @0oshowero0 in #6652
  • [megatron] fix: MTP compatible with latest mcore by @HollowMan6 in #6648
  • [vllm] fix: use data-parallel rank in vLLM ZMQ handles by @he-yufeng in #6620
  • [veomni] fix: add veomni backend docs for ascend by @phdddd in #6657
  • [vllm, fsdp] fix: apply FSDP buffer updates during rollout weight sync by @chenshui223 in #5801
  • [doc] chore: Update ascend_vllm_best_practices.rst by @wangshuyang31 in #6640
  • [doc] feat: update Qwen3.5 NPU guide by @zjchenn in #6667
  • [veomni, cfg] feat: add missing config fields to veomni.yaml by @mikequan0425 in #6505
  • [fsdp] feat: chunked gather-logsumexp for top-K loss to avoid OOM at long context by @kekellllll in #6593
  • [ci] fix: ignore one ut which failed in npu ci by @yyyy2000 in #6679
  • [ci, hardware] feat: add AMD ROCm (MI300) e2e_ppo_trainer workflow by @PeterYang12 in #6668
  • [trainer, megatron] fix: fix MindSpeed repatch config bug by @chengminhua in #6674
  • [rollout] fix: propagate ignore_eos to vLLM sampling params by @Mengyuyang in #6630
  • [megatron] fix: propagate disabled MTP config to bridge providers by @Mengyuyang in #6626
  • [ci] chore: add npu's veomni ci by @daikang6 in #6701
  • [ci] test: add qwen3.5 2b ascend nightly ci by @Zhang1Sheng in #6637
  • [fsdp, trainer] fix: detach model_output and loss metrics to stop per-micro-batch graph retention in actor update by @YoungZSh in #6699
  • [fsdp] fix: handle missing chunked top-k config by @Luosuu in #6704
  • [rollout] fix: clone LoRA weights out of the reused IPC buffer before add_lora by @HaozheZhang6 in #6688
  • [hardware] feat: add ROCm/HIP platform backend (PlatformROCm) by @xiaohong42 in #6702
  • [reward] fix: add optional per-sample compute_score timeout to NaiveRewardManager by @zqinglin in #6673
  • [trainer] feat: add unify trainer abstraction for sync and async training by @wuxibin89 in #6710
  • [trainer] fix: use FullyAsyncLLMServerClient for async trainer by @wuxibin89 in #6716
  • [fully_async] fix: check audio forwarding contract in agent loop by @Luosuu in #6718
  • [reward] fix: only require max_resp_len when DAPO overlong penalty is enabled by @discobot in #6709
  • [rollout, trainer] fix: handle empty token sequences in _pad_token_ids and metric timing by @GJWu-zyx in #6675
  • [rollout, vllm] fix: preserve MTP drafter weights during hybrid sleep by @sunnweiwei in #6661
  • [model] feat: support Gemma4 multimodal models in RL (GRPO) by @dbuos in #6715
  • [rollout] fix: Adjust cuda graph capture sizes config logic for vLLM >= 0.11.1 compatibility by @ConanZH429 in #6728
  • [BREAKING][ckpt] refactor: rename misleading trainer arg to actor_wg by @PeterSH6 in #6724
  • [worker] feat: add tinker training worker primitives by @Luosuu in #6717
  • [trainer] feat: add off_policy metrics by @wuxibin89 in #6736
  • [rollout] fix: skip redundant clone in get_named_tensor_buckets to avoid OOM during SGLang weight sync by @HaozheZhang6 in #6738
  • [docker] feat: bump stable trtllm image to 1.3.0rc15 by @Superjomn in #6730
  • [ci] chore: add two-node Ascend RayJob E2E workflow by @wangdongleix in #6672
  • [rollout, sglang] feat: support sglang ROCm backend via aiter defaults and ray init env by @xiaohong42 in #6664
  • [docker, doc] refactor: streamline ROCm Dockerfile build and add ROCm README by @PeterYang12 in #6619
  • [doc] chore: ascend update install_guidance.rst by @hustmf in #6774
  • [model] fix: NPU patches for the Qwen3-MoE compatible with different transformers by @mikequan0425 in #6708
  • [trainer] feat: support staleness control strategy by @wuxibin89 in #6778
  • [trainer] feat: A runnable separate async trainer by @Begunner in #6790
  • [data] fix: offload process_vision_info to a thread executor to unblock the agent-loop event loop by @huaiyizhao in #6789
  • [doc] fix: update OPD docs by @Dmovic in #6788
  • [megatron,doc] feat: support DeepseekV4, GLM5, KimiK2.5 via Megatron Lite by @ISEEKYAN in #6791
  • [worker] feat: add per-step optimizer param overrides by @Luosuu in #6765
  • [ci] fix: fix oom in fullyasync+fsdp test by @tardis-key in #6784
  • [megatron] feat: Support Megatron chunk entropy by @ZLiao097 in #6446
  • [docker, misc] chore: upgrade vllm, nccl and numpy > 2.0.0, to resolve packages dependency error by @ETOgaosion in #6551
  • [fsdp, model] feat: per-unit LoRA summon, FSDP1/2 compatibility, and strip-modules support by @qinganrice in #6512
  • [model] feat: add qwen3-122b long seq launch script for ascend by @zjchenn in #6807
  • [fully_async, trainer] fix: align aggregated metrics logging with current step by @huaiyizhao in #6796
  • [rollout] feat: add Continuous Token for Agentic Rollout by @gxlvera in #6779
  • [ci] chore: solve dapo error and add three baselines for npu nightly ci by @daikang6 in #6711
  • [megatron] feat: align optimizer states and DDP grad bucket with model precision by @kolehma8 in #6526
  • [rollout, reward] feat: add full determinism support for vLLM rollout and reward model by @KaisennHu in #6572
  • [ci] fix: fix Megatron-Bridge version in e2e_ppo_trainer_megatron_sglang_ascend.yml and update megatron for sglang ascend by @yyyy2000 in #6816
  • [hardware] refactor: per-model NPU patches with fault isolation by @tardis-key in #6777
  • [reward] feat: colocated reward model for v1 sync/colocate_async trainer by @Begunner in #6818
  • [rollout] fix: Enable Graphics Mode on qwen3vl_verl_veomni by @Seren-hao in #6819
  • [doc] chore: add verl extension guide by @wuxibin89 in #6825
  • [doc] chore: modify the profiling documentation of ascend by @zhouhengan1211 in #6826
  • [BREAKING][trainer, cfg] chore: enable V1 trainer by default by @Begunner in #6823
  • [ci] fix: fix ascend ci bug by @yyyy2000 in #6838
  • [ckpt] fix: write HF config before bridge save to fix shard inference by @wizkdc in #6842
  • [env] chore: add verl release 0.8.0 docker for ascend by @yyyy2000 in #6820
  • [ci] fix: Update e2e_ppo_trainer_megatron_sglang_ascend.yml by @wucong25 in #6841
  • [rollout, algo] fix: compute rollout_is_seq_fraction from raw weights by @EazyReal in #6806
  • fix typo: clip_cvo_ratio to clip_cov_ratio by @GZYZG in #6855
  • [misc] chore: fix npu parameters and typo by @hustmf in #6858
  • [fsdp] fix: re-wrap fused log_probs/entropy when use_remove_padding=False by @Mecoli1219 in #6846
  • [worker] fix: Update external_lib config annotation in HFModelConfig to allow a list of string by @wyettzeng in #6862
  • [trainer, rollout] feat: log rollout moe load-balance metrics by @Luosuu in #6853
  • [ci] chore: add Ascend multinode nightly CI by @wangdongleix in #6876
  • [fully_async] fix: correct the use of partial_rollout by @zpltys in #6882
  • [model] fix: tokenizer.hf_processor: GLM-4V arm uses wrong processor class name (#6814) by @yushuosun in #6873
  • [doc] fix: stop sidebar from scrolling in sync with main content by @javidmz in #6871
  • [fully_async, doc] fix: ignore temperature config for teacher prompt_logprobs and warn when non-default value is set by @kenkenpa2126 in #6867
  • [ci] chore: add qwen3_5 megatron images by @wucong25 in #6889
  • [doc, algo] feat: add Ascend NPU GSPO script for Qwen3-32B by @chengminhua in #6890
  • [hardware] fix: add flops estimation for ascend a2g3 device by @Mengyuyang in #6892
  • [ci] fix: Add more test cases in e2e_ppo_trainer_megatron_sglang_ascend.yml by @xiazhahe in #6877
  • [megatron,doc] docs: point Megatron Lite examples to verl-project/Megatron-LM by @flameagainst in #6898
  • [vllm] fix: crash in start_profile/stop_profile on non-master nodes when nnodes > 1 by @kyle-zhangchi in #6861
  • [fsdp, megatron, trainer] fix: enhance mem footprint for forward_kl_topk OPD by @dimjava in #6848
  • [ci] chore: fix vllm_ascend ci by @wucong25 in #6839
  • [data, rollout, worker] feat: add Open-R1 multimodal and TinyLLaVA-Video-R1 preprocessing and training scripts by @lihanwen7 in #6849
  • [doc] refactor: update ascend quick_start, add quickstart scripts for 4 training-inference backend combinations by @yyyy2000 in #6900
  • [fully_async] feat: Adapt vLLM 0.19+ for Ascend NPU by @fh188 in #6886
  • [docs] chore: add verl-speco pre-release news by @tpx818 in #6910
  • [megatron, perf] feat: pad BSHD micro-batches to mini-batch max seq_len by @YaoweiFan in #6901
  • [rollout] fix: support SGLang FP8 ignored layers for Qwen3.x GatedDeltaNet in rollout by @gem-mint in #6906
  • [fsdp] fix: Fix Qwen3 MoE FSDP weight sync for vLLM rollout in Transformers 5 by @lxb007981 in #6896
  • [docs] add SGLang FP8 ignored layers usage by @gem-mint in #6915
  • [ci] chore: add some NPU's UT/ST by @daikang6 in #6831
  • [doc] fix: Update ascend quick_start.rst by @yyyy2000 in #6918
  • [doc] chore: update ascend Retool/DAPO/GSPO best practice version compatibility by @chengminhua in #6924
  • [doc] fix: update docs for deepseekv3 support by @xiazhahe in #6926
  • [fully_async] feat: Adapt vLLM >=0.18 and vLLM < 0.19 for Ascend NPU by @zhouhengan1211 in #6929
  • [megatron] fix: guard ModelType.encoder_and_decoder for Megatron-Core >= 0.18 compatibility by @chengcuiping in #6927
  • [trainer] fix: Update latest TorchtitanEngine by @acisseJZhong in #6916
  • [rollout] fix: restore turn separator dropped at multi-turn tool agent loop boundaries by @abtonmoy in #6921
  • [vllm, rollout] fix: handle non-contiguous weights in bucketed transfer by @Mecoli1219 in #6919
  • [cfg] chore: enable FULL_DECODE_ONLY cudagraph mode in Ascend PPO example by @chengminhua in #6938
  • [fsdp] fix: FSDP logits temperature scaling for view tensors by @zjchenn in #6935
  • [rollout] fix: probe list-of-parts content in initialize_turn_separator for multimodal processors by @abtonmoy in #6937
  • [hardware, perf] fix: update AMD GPU FLOPS values by @Vivicai1005 in #6942
  • [trainer] fix: spmd_types and activation checkpointing composability bug by @acisseJZhong in #6950
  • [vllm, rollout] fix: require vLLM>=0.22.0 for Qwen3.5 router replay (R3) by @dafu-wu in #6922
  • [fsdp] fix: fused-kernel gradient tracking bug. by @kolehma8 in #6913
  • [megatron] fix: free Transformer-Engine FP8 weight workspaces on CPU offload by @alexxu-roblox in #6952
  • [algo] fix: normalize critic value loss over the global mini-batch, not per micro-batch by @abtonmoy in #6957
  • [perf] feat: Add Qwen3.5 MFU flops estimation by @liangxuZhang in #6389
  • [doc] feat: add installation instructions for Atlas 950DT A5 by @fh188 in #6966
  • [ckpt] fix: use separate magic_recv buffer to prevent weight corruption by @KunWuLuan in #6813
  • [ci] feat: add GSPO Qwen3-8B FSDP2 vLLM nightly test for Ascend by @chengminhua in #6956
  • [trainer] fix: stabilize Qwen3 Next 80B FSDP rollout on NPU by @zjchenn in #6972
  • [trainer] fix: Add readme and CI test for Titan Engine by @acisseJZhong in #6954
  • [env] chore: Adapt qwen3.5 to Docker containers for Verl release 0.8.0 by @ruanhao566 in #6955
  • [trainer] fix: only set CUDA_DEVICE_MAX_CONNECTIONS=1 for megatron in Hopper/Ampere by @wuxibin89 in #6976
  • [rollout] feat: vLLM Prefill-Decode disaggregated rollout (NIXL + Mooncake wiring) by @aoshen02 in #6243
  • [BREAKING][trainer] fix: separate_async should use the same step granularity with other trainers by @Begunner in #6977
  • [doc] feat: update installation instructions for Atlas 950DT A5 by @fh188 in #6981
  • [fsdp] fix: drop per-micro-batch model_output in training to fix actor-update OOM by @ChangyiYang in #6984
  • [BREAKING][megatron] chore: deprecate vanilla mBridge and make Megatron Bridge default by @HollowMan6 in #6951
  • [doc] feat: update installation instructions for Atlas 950DT A5 by @fh188 in #6983
  • [trainer,doc] fix: workaround vLLM multimodal cache in verl 0.8.0, add Qwen3.5 397B by @ruanhao566 in #6982
  • [veomni] fix: use default moe param handler by @wuxibin89 in #6993
  • [doc] feat: update installation instructions for Atlas 950DT A5 by @fh188 in #7002
  • [doc] fix: correct vllm-ascend commit id in A5 install doc by @zjchenn in #6998
  • [megatron, trainer] feat: add HY v3 Megatron GRPO example by @xhx1022 in #7003
  • [tool, rollout] feat: Adapt SkipManager on Trainer V1 by @mikequan0425 in #6897
  • [fsdp] fix: skip the whole-shard staging round trip in FSDP2 weight export by @ChangyiYang in #7005
  • [doc] refactor: update ascend docker name by @yyyy2000 in #7008
  • [model] fix: fix vocab_weights device and shard handling for qwen3_vl.py by @ocp8 in #6659
  • [doc] fix: update the readme about kimi-checkpoint-engine by @beirong8kmiles in #7018
  • [ci] chore: temporary disable torchtitan ci due to im-compatible torchversion by @wuxibin89 in #7025
  • [megatron] fix: return 3-tuple under calculate_per_token_loss to fix MoE aux/z-loss grad blowup at CP>1 by @EricMarcus-ai in #6836
  • [rollout] fix: disable vLLM multimodal processor cache for vLLM < v0.22.0 by @lxb007981 in #6932
  • [env] fix: Sync Qwen3.5 Dockerfile from 0.8.0 Branch to Main and Pin PyArrow Version by @ruanhao566 in #7028
  • [tool, rollout] feat: support parameter sync steps in Skip Manager by @mikequan0425 in #7032
  • [ci] chrore: fix veomni ci by @wuxibin89 in #7036
  • [megatron] feat: support DeepSeek V4 GRPO by @HollowMan6 in #6473
  • [rollout, vllm] fix: stop the policy from sampling vision placeholder tokens by @zhshj0110 in #7038
  • [trainer, ckpt] feat: support streaming dataloader and async trainer checkpoint recovery by @Begunner in #7037
  • [env] chore: add megatron-bridge in sglang docker env by @zhouhengan1211 in #7039
  • [fully_async] fix: qwen2.5-0.5b fully async OOM bug fix by @zhouhengan1211 in #7041
  • [ci] test: update ascend ci docker tag by @yyyy2000 in #7035
  • [ci] feat: add GRPO Qwen3.5-35B Megatron vLLM nightly test for Ascend by @chengminhua in #6973
  • [tool] feat: verl add rl insight by @mengchengTang in #6680
  • [doc] chore: add rl-insight news by @tardis-key in #7048
  • [doc] refactor: edit profiling docs of npu for usability by @zhouhengan1211 in #7024
  • [fsdp] fix: sync merged LoRA weights before context exit by @rongkunxue in #7014
  • [trainer] fix: gen_batch_size falls back to train_batch_size by @Begunner in #7051
  • [ci] chore: remove vllm_ascend patch and fix ci by @wucong25 in #7042
  • [ci, trainer] fix: Fix ci AssertionError for the environment variable CUDA_DEVICE_MAX_CONNECTIONS by @pengnuoheng in #7040
  • [ckpt,rollout] feat: sharded delta weight sync over NCCL for disaggregated rollout by @ChangyiYang in #6974
  • [hardware] fix: fix build_npu_attn_mask oom by @RichardFido in #7055
  • [ci] chore: fix npu and gpu ci by @wucong25 in #7066
  • [trainer] fix: exclude the never-routed final response token from rollout MoE load-balance metrics by @zsnoob in #7075
  • [ckpt] feat: add save_lora_only checkpoint support by @Secbone in #7061
  • [ci] chore: fix nightly ci of npu by @zhouhengan1211 in #7081
  • [vllm] fix: restore vllm patch by @tardis-key in #7076
  • [rollout] fix: handle malformed Qwen3 XML tool calls by @zzzzzzzxh in #7044
  • [doc] chore: ascend doc, fix typos, punctuation, broken links and code errors by @hustmf in #7093
  • [ckpt] fix: added cuda stream synchronization in NCCL broadcast wait for complete by @parinayc20 in #7083
  • [doc] refactor: point MLite DAPO example to canonical launcher by @ISEEKYAN in #7091
  • [trainer, perf] fix: include standalone rollout GPUs in throughput denominator for separate async by @mikequan0425 in #7049
  • [doc] fix: repair broken Ascend tutorial links and shell filename by @chengminhua in #7074
  • [fully_async] feat: support dynamic resource scheduling by @zpltys in #6556
  • [model, fsdp] fix: support Qwen3.5 linear attention under Ulysses SP by @sunnweiwei in #6660
  • [megatron] fix: Qwen3.5 LoRA & MTP support (with Megatron-Bridge) by @HollowMan6 in #5599
  • [veomni, fsdp] fix: skip manual model.to(device) under FSDP2 CPU offload by @cben484 in #7103
  • [perf] feat: re-support torch profiler and optimize output naming by @ETOgaosion in #7099
  • [fsdp, perf] feat: defer gradient sync during accumulation by @zhangxin81 in #7095
  • [doc] chore: update Ascend 950 references and disable nightly test by @wangdongleix in #7111
  • [doc] chore: warn against hard-wrapping PR body markdown in pr skill by @zhangxin81 in #7116
  • [doc] fix: fix ascend docker tag by @yyyy2000 in #7124
  • [ci] fix: tigger npu ci on the release branch by @tardis-key in #7125
  • [rollout, hardware] fix: fix rollout replica.py for device abstraction by @kahlun in #7128
  • [trainer] feat: V1 replay buffer eviction/refill handling for stale, DAPO-filtered, and failed rollout groups by @Begunner in #7082
  • [trainer] fix: isolate TaskRunner logs from root logger changes by @Begunner in #7133
  • [recipe] feat: Retain model output from tinker forward backward by @wyettzeng in #7109
  • [sglang] fix: use _base guard in _compact_for_bucket to prevent NCCL buffer race by @zpltys in #7139
  • [docker] chore: Rocm714 doc update by @mingjielu in #7140
  • [ckpt, fsdp] feat: sharded delta block placements + backend-owned HF export (engine core + FSDP) by @ChangyiYang in #7144
  • [ci] fix: update Ascend nightly CI by @Mengyuyang in #7148
  • [vllm] fix: guard legacy FusedMoE loader patch by @Mengyuyang in #7147
  • [worker] fix: close temporary TransferQueue event loop by @le-czs in #7157
  • [tool, megatron] fix: resolve PrecisionDebugger model chunks by @Tjh-UKN in #7104
  • [rollout] fix: TypeError merging numpy routed_experts on partial rollout resume by @Begunner in #7158
  • [fsdp] refactor: move unfuse_moe_params to FSDP backend by @wuxibin89 in #7161
  • [docker] feat: upgrade vllm and megatron version, add packages to support DeepSeek-V4 by @ETOgaosion in #7101
  • Revert "[megatron] fix: Qwen3.5 LoRA & MTP support (with Megatron-Bridge) (#5599)" by @wuxibin89 in #7173
  • [ci] fix: update Ascend nightly CI & docker by @kyle-zhangchi in #7176
  • [doc] chore: add multi machine task startup document for npu by @zhouhengan1211 in #7175
  • [vllm] refactor: clean up weight sync by @wuxibin89 in #7179
  • [veomni] feat: EP-aware sharded delta export (fused expert stacks) by @ChangyiYang in #7085
  • [doc, model] fix: refresh FSDP and SkyPilot guides by @WenZheWang in #7185
  • [trainer] feat: support decouple ppo for v1 separate_async by @zpltys in #7188
  • [vllm] chore: remove unused get_device_uuid from vllm rollout by @aoshen02 in #7189
  • [rollout, vllm, hardware]fix: add IPC check before invoking ipc_collect in BucketedWeightReceiver by @kahlun in #7184
  • [ckpt, model] fix: validate model merger outputs by @kaining-never-stop in #7193
  • [vllm] refactor: drop support for vLLM older than 0.18.0 by @aoshen02 in #7190
  • [perf] fix: Prevent creating new threads/event loop for each tqbridge call by @0oshowero0 in #7162
  • [megatron] feat: add Muon optimizer support (expose Megatron-Core TensorParallelMuon) by @ISEEKYAN in #7120
  • [doc] refactor: rename latest ascend docker name by @yyyy2000 in #7201
  • [megatron] feat: delta_sharded on Megatron-Bridge param mappings (TP+EP, hybrid-Mamba) by @ChangyiYang in #7181
  • [ckpt] feat: add hccl ckpt engine split_weight_chunks by @wucong25 in #7205
  • [rollout] fix: decode per-turn LLM tokens in traces by @YAO-001 in #7204
  • [ci] chore: remove gemini code review by @wuxibin89 in #7219
  • [rollout] fix: cap partial rollout resumes at the remaining response budget by @benyucong in #7207
  • [megatron] feat: support contiguous context-parallel layout for DeepSeek V4 by @ISEEKYAN in #7221
  • [vllm] feat: enhance DeepSeek V4 fp8/fp4 linear and moe weight refit by @wuxibin89 in #7224
  • [algo] fix: micro-batch normalization for distillation loss by @JacobHelwig in #7225
  • [megatron] fix: preserve generation config during merge by @KANIKIG in #7199
  • [megatron] feat: PP/VPP support for the steady delta export by @gxlvera in #7223
  • [ckpt] fix: save base model's code, not the PeftModel wrapper's, in FSDP checkpoints by @ZhiliangWu in #7117
  • [trainer] fix: build RL tokenizer/processor from HFModelConfig by @ZhiliangWu in #7220
  • [veomni] fix: preserve GPT-OSS weights without expert parallelism by @Luosuu in #7243
  • [algo, cfg] feat: add token-sum loss aggregation by @wyettzeng in #7197
  • [rollout, sglang] fix: keep SGLang LoRA-free when model.lora.merge=True by @jamesruio in #7234
  • [doc] fix: remove invalid Ascend model support entries by @zjchenn in #7248
  • [doc] feat: Update vllm/vllm-ascend and torch version by @LeoYao123 in #7238
  • [megatron, hardware] fix: pure-torch fast_hadamard_transform fallback for DSA on ROCm by @PeterYang12 in #7241
  • [data] feat: Add processor hook for multimodal RoPE kwargs by @ZihaoW123 in #7236
  • [ci] chore: remove RAY_DEDUP_LOGS=0 from GSPO Qwen3-8B FSDP2 NPU nightly test by @chengminhua in #7258
  • [veomni] feat: add DeepSeek V4 support by @wuxibin89 in #7242
  • [megatron] fix: pad multidimensional THD tensors along the sequence dimension by @yyDing1 in #7261
  • [algo, cfg, doc] feat: add DRO losses by @wyettzeng in #7245
  • [ckpt, megatron] fix: megatron save checkpoints with strict false when vanilla_bridge is false by @RichardFido in #7264
  • [doc] chore: Modified some guide documents for NPU by @zhouhengan1211 in #7265
  • [ci] chore: Update python&cann version, delete non-existent file by @LeoYao123 in #7268
  • [fsdp,veomni] feat: support pad_to_length to reduce jit compile time by @wuxibin89 in #7272
  • [fsdp,veomni] fix: backfill missing state for DSD optimizer checkpoint by @wuxibin89 in #7283
  • [ci] chore: Update npu ci docker image by @LeoYao123 in #7282
  • [fsdp] fix: add FSDP1 restore handler for Decoupled PPO by @wuxibin89 in #7306
  • [ci] chore: Update npu docker build file by @LeoYao123 in #7305
  • [rocm] feat: enable DeepSeek-V4-Flash GRPO on AMD GPUs by @PeterYang12 in #7050
  • [doc] fix: install NPU requirements by @Mengyuyang in #7310
  • [megatron] feat: migrate fused logprob/entropy from GPTModel.forward monkey-patch to Megatron output_processor hook by @chengcuiping in #6933
  • [ci] chore: Fix docker image upload by @LeoYao123 in #7311
  • [megatron] fix: forward mhc_multistream to MTP and skip activation reclaim for MTP checkpoints by @HollowMan6 in #7328
  • [ckpt]: nccl broadcast bucket size fix by @parinayc20 in #7107
  • [ray] fix: remove conflicting PYTHONPATH forwarding by @yyDing1 in #7313
  • [algo] fix: carry running_return through observation spans in REINFORCE++ (#7278) by @shotsan in #7300
  • [vllm] fix: resolve .base_layer on the vLLM receiver for non-merged LoRA sync by @HollowMan6 in #7327
  • [misc] fix: warn on engine backend import failure instead of silent skip by @ji-huazhong in #7334
  • [ci] chore: add three baselines for npu's nightly ci by @daikang6 in #7337
  • [ci] chore: Update npu docker image cann version by @LeoYao123 in #7338
  • [megatron] fix: make DeepSeek-V4 context parallelism actually runnable by @HaochenYuan in #7297
  • [doc] fix: note the megatron commit MTP needs for recompute_granularity=full by @gaohongkui in #7346
  • [reward] feat: support deterministic reward for user-defined generative RM paths by @KaisennHu in #7027
  • [megatron] feat: add dynamic context parallel scheduling by @xiaoyao0115 in #6555
  • [megatron] feat: cache the Megatron-Bridge HF export plan across weight updates by @Begunner in #7348
  • [megatron] fix: add backward compatibility with older Megatron-Bridge versions by @nuerxiati in #6682
  • [ci] chore: fix npu ci by @wucong25 in #7354
  • [megatron] feat: bucket packed sequence lengths by @ISEEKYAN in #7358
  • [ci] chore: Fix npu nightly ci by @LeoYao123 in #7345
  • [megatron] fix: bugfix qwen 3 qwen 3.5 router replay by @EricMarcus-ai in #7340
  • [megatron] fix: per-name, mapper-aware .base_layer strip in resolve_weight_name by @HollowMan6 in #7376
  • [misc] chore: bump version to 0.9.0 by @wuxibin89 in #7411

New Contributors

Full Changelog: v0.8.0...v0.9.0

Contributors

dbuos, discobot, and 121 other contributors
Assets 2
Loading