RL360 v0.1.0

RL360: Reinforcement Learning for K2 Horizon

First public release of RL360, the reinforcement learning codebase used in the post-training pipeline for the K2 Horizon series.

RL360 trains language models on tasks that involve writing code, using tools, and working in a sandbox.

Highlights

• End-to-end agentic RL stack. Miles manages rollouts and training, Megatron-LM updates the policy, SMG routes requests to SGLang, and Harbor runs tasks and verifiers in sandboxes.
• Reproducible coding recipe. A configurable Slurm recipe that trains K2 Horizon 7B on 32 mixed coding tasks with a 128K context window.
• Docker-based setup. A public container build (docker/) with pinned reference dependency versions.
• Reward functions and utilities. Task-specific reward functions and grading utilities, token-level loss reduction, and retrieval index construction and serving.
• Model portability. Includes an example for switching to Qwen3-8B.

Reference run

As a small-scale demo, we trained K2 Horizon 7B to overfit 32 coding tasks, sampling eight attempts per task. Mean reward rose from 0.373 to 0.901 over 99 rollout iterations. See the public W&B workspace for the run and charts.

These are training-set results, and the reward includes a format penalty. The reference run used an earlier training checkpoint and a selected task set, so results with the public checkpoint will differ.

Getting started

Requirements: a Linux x86_64 cluster with NVIDIA H100/H200 GPUs, Slurm, Docker Engine, the NVIDIA Container Toolkit, and shared storage.

See the README and the recipe guide for step-by-step instructions.

License

Apache-2.0. The bundled components (Harbor, Megatron-LM, Miles, SGLang, SMG) keep their own licenses and copyright notices under components/. See THIRD_PARTY_NOTICES.

---

Developed by the Institute of Foundation Models (IFM) at MBZUAI. Questions and feedback are welcome on Discord.