v0.1.0

We are excited to announce v0.1.0, the first versioned release of Relax — a service-oriented reinforcement learning engine for agentic and omni-modal post-training at scale.

Highlights

• Flexible execution modes — choose among memory-efficient Colocate, high-throughput Fully Async, and cost-conscious Hybrid execution. See the Fully Async guide for dynamic batching and --balance-data semantics.
• End-to-end omni-modal post-training — unified training paths for text, images, video, and audio, from vision-language models to omni-modal models.
• Agentic RL for external applications — connect agents through an OpenAI-compatible chat API, with multi-turn training, loss masking, partial rollout and resumption, tool interaction, and multimodal context carry-over.
• A broad training stack — policy optimization, on-policy distillation, SFT, LoRA RL, MTP RL, pluggable rewards, and GenRM LLM-as-a-judge workflows.

Omni-Modal Training and Model Integrations

Relax provides end-to-end post-training paths for text, images, video, and audio, including image-token controls, video-frame sampling, audio processing, and configurable modality-field mapping.

Documented model integrations and reference configurations include Qwen3, Qwen3-VL, Qwen3.5, Qwen3-Omni, Qwen3.6, GLM5, Kimi K2.6, and dots.mocr via its Megatron Core integration. Training runs on Megatron-LM, rollout runs on SGLang, and Megatron Bridge provides Hugging Face ↔ Megatron checkpoint conversion.

Algorithms and Training Extensions

• Policy optimization: PPO, GRPO, GSPO, SAPO, CISPO, REINFORCE++, and REINFORCE++-baseline.
• Distillation: On-Policy Distillation and Multi-Teacher OPD, with text and vision-language reference workflows.
• Additional workflows: SFT, LoRA RL, and MTP RL.
• Rewards: math, GPQA, F1, IFBench, multiple-choice, Open-R1 multimodal, custom reward functions, and GenRM LLM-as-a-judge.

Agentic RL and Reference Recipes

Agentic Rollout keeps the external agent application as a normal process while Relax manages scheduling, request locking and resumption, reward orchestration, trajectory transfer, and training metrics.

Reference recipes and integrations:

• DeepEyes and DeepEyes Agentic for multi-turn visual-agent training;
• DeepEyes V2 for multimodal agents with Jupyter-sandbox code execution and text or image search;
• Mini-SWE Agent for R2E-Gym and SWE-Bench-style software-engineering tasks;
• NVIDIA NeMo Gym recipes for Calendar, GSM8K, Workplace Assistant, and R2E-Gym;
• Agentic OPD workflows for WebShop and ALFWorld.

Accelerator Reference Paths

• NVIDIA CUDA — the primary CUDA 12.9 runtime path, with Megatron-LM distributed training and SGLang rollout inference.
• Ascend 910C NPU — a reproducible NPU Docker build and text DAPO reference paths for Qwen3-4B, Qwen3.5-9B, and Qwen3.5-35B-A3B. The NPU guide documents both Colocate and Fully Async execution, based on CANN, torch_npu, MindSpeed/MindSpeed-Bridge, and SGLang NPU.
• Kunlunxin P800 XPU — synchronous DAPO reference configurations for Qwen3, Qwen3.5, and Qwen3.6, including Qwen3.5 vision-language workloads across 8- and 16-P800 topologies.

Container Image

docker pull ghcr.io/redai-infra/relaxrl:v0.1.0-cu129

Use this CUDA 12.9 runtime image together with the v0.1.0 source tag. For full documentation, visit redai-infra.github.io/Relax.

Acknowledgements

_{Organizations and contributors are listed in no particular order.}

We gratefully acknowledge the NVIDIA, Ascend, and Kunlunxin teams for their collaboration and technical support.

Contributors (43)

@Yangruipis, @NINGBENZHE, @Aurelius84, @yxyOo, @dirtyDan0, @yuanlehome, @li126com, @xiaoliang0601, @liujia-cc, @yeqinghe609, @GUOGUOPOT, @fxsc03, @kkyyxhll, @hbamboo, @Lw135, @RexFlux, @tang27-1, @Saddss, @Dayuxiaoshui, @grootZ, @MaybeIcanShow, @Baiiiii126, @huxy1225-cloud, @sksdiw3, @leohuang257, @overloadedHenry, @sidneychang, @DreamEnding, @yulingma545-ctrl, @howtomakeaname, @xiaoh1024, @circleLZY, @mununumu, @fenta, @flyinglandlord, @anranxia, @YowatariChiaki, @jamesruio, @meiqingsui, @chenwhql, @A-Words, @xcosmosbox, @zheself, @ChinaCYong, @sqfanling, @maricoxu, @alexqdh, @qinyi810, @startweb-wz, @zackkui, @Manfred-Hu