v0.6.0

330 commits from 25 contributors since v0.5.0.

1. Inference Stack: llm-d Router, MoonCake & KV-Cache Offloading

A major overhaul of the inference serving stack. A new llm-d (EPP + Envoy) router backend is available as an option alongside the existing router, with configurable routing policies, tiered KV-cache offloading spills cache across CPU, disk, and a MoonCake distributed store, and the disaggregated prefill/decode (P/D) deployment was hardened with external load-balancing everywhere.

• llm-d EPP + Envoy router backend with consistent-hash routing
• Tiered KV-cache offloading: CPU + disk + MoonCake distributed store
• Disaggregated P/D hardening (external-LB, SLURM-gated deployment, KV-transfer fixes)
• Orchestrator can optionally co-locate on an inference node

#2697 — llm-d (EPP+Envoy) router backend
#2689 — Tiered KV-cache offloading + MoonCakeStore
#2256 — New A2A backends, KV-cache offloading, DBO
#2457 — KV-cache offloading for non-PD deployments
#2695 — RouterConfig on the base inference deployment
#2255 — Make vllm-router policy configurable
#2134 — Switch vllm-router policy to consistent_hash
#2696 — Use external-LB everywhere, drop hybrid-LB
#2694 — Require SLURM for disaggregated deployment
#2811 — Optionally run orchestrator on an inference node
#2257 — Collect per-engine inference server metrics
#2375, #2449 — vLLM engine liveness probe (configurable timeout)
#2466 — JSON logging for inference (vLLM)

2. FP8 Training

End-to-end FP8 training support landed via DeepGEMM, with an FP8 fused transpose-cast path and an FP8 weight-transfer format for trainer→inference sync.

#2166 — DeepGEMM FP8
#2724 — FP8 fused transpose-cast
#2454 — Support quantize_in_weight_transfer + EPLB
#2453 — Upcast addresses to int64 to avoid FP8 overflow
#2582 — Patch DeepGEMM SiLU quant addressing
#2591 — Set DeepGEMM MoE env for inference
#2442 — Escape dots in FP8 ignore patterns

3. Performance

A broad set of throughput and memory wins across context parallelism, MoE, and the trainer runtime — a new Ulysses all-to-all CP path alongside ring attention, FA4 in ring-attention CP mode, improved sequence packing, and DeepEP for expert parallelism.

#2418 — Ulysses all-to-all CP path alongside ring
#2307 — FA4 in ring-attention CP mode
#2723 — Improve sequence packing
#2119 — DeepEP
#2120 — New selective AC targets
#2145 — Deterministic GC to avoid stragglers
#2148 — signSGD memory-efficient optimizers
#2376 — Single-point TPS/MFU API
#2566 — Defer heavy imports so --help short-circuits
#2543 — Quack RMS norm cleanup

4. Orchestrator Async-Pipeline Rewrite ("orch v2")

The orchestrator was rewritten around a single shared concurrency limiter that overlaps train and eval rollouts instead of draining one before the other. The async semantics are now a design invariant rather than a config knob: the trainer always runs exactly one step ahead of inference, and the orchestrator always adopts the freshest checkpoint that doesn't break the one-step barrier. The legacy difficulty buffer is gone, replaced by composable pre-/post-batch filters, and the scheduler can now train on partial groups instead of dropping them on rollout error.

• Reduced event-loop lag and trimmed orchestrator memory growth (glibc heap trim, bounded monitor history)
• Pre-/post-batch filter lists replace the old difficulty buffer
• Per-env state_columns for extra rollout fields

#2639 — Refactor orchestrator to overlap train/eval
#2631 — Hardcode async-barrier semantics, drop max_async_level / strict_async_level
#2590 — Train on partial groups instead of dropping on rollout error
#2589 — Per-env state_columns for extra rollout fields
#2615 — Default X-Session-ID header to trajectory_id
#2609 — Reduce event-loop lag
#2459, #2484 — Drop groups after max_error_reschedule_attempts (default 3)
#2533 — Allow capped inflight rollout capacity

---

Other Improvements

Renderer-Based Tokenization

Per-model client-side renderers replace server-side tokenization: this rips out the MITO multimodal branch and the server-tokenized TITO token client, unifies the renderer and teacher endpoints onto vLLM's /inference/v1/generate, and packs multimodal pixel_values directly from the renderer. Renderers are now the default for rollouts and an opt-in path for SFT.

#2278 — Per-model client-side renderers for RL rollouts (+ /v1/generate)
#2473 — Renderer-only multimodal path (rip MITO branch)
#2408 — Unify renderer + teacher endpoints on /inference/v1/generate
#2521 — Make renderers default for rollouts
#2493, #2496 — Opt-in renderer tokenization for SFT
#2264 — Shared tokenizer config + chat_template for RL
#2549 — Deprecate use_token_client

New Model Support

GLM-5.1 / 5.2, Qwen3.5 (custom dense + VL MoE), NemotronH (Mamba context parallelism), Gemma 4 (VLM), Poolside Laguna, Nano V3, and DeepSeek Sparse Attention (DSA) variants.

#2851 — GLM 5.2 training
#2275 — GLM-5.1 parser auto-detect
#2225 — vLLM 0.19 GLM5 + DBO + CPU KV offload
#2505 — Custom Qwen3 dense
#2196 — qwen3_vl_moe VLM registry
#2231 — CP for NemotronH Mamba
#2844 — Gemma 4 as VLM
#2479 — Poolside Laguna
#2767 — Nano V3 reasoning parser
#2173, #2372 — CP for DSA / GlmMoeDsa
#2233 — Attention-only LoRA

vLLM 0.22

vLLM moved from 0.17 through 0.22, bringing new all-to-all backends and the new routed-experts format.

#2686 — Bump vLLM to 0.22
#2487 — New vLLM routed-experts format
#2632 — Delta-only routed-experts replay
#2621 — ModelExpress in CUDA image

Submodules & Config Overhaul

The sibling repos (verifiers, renderers, research-environments, pydantic-config) are now vendored as git submodules under deps/, config classes were split into a standalone prime-rl-configs workspace package, and the config system migrated off tyro onto prime-pydantic-config.

#2507 — Add sibling repos as submodules under deps/
#2556 — Add pydantic-config submodule
#2416 — Split prime-rl-configs workspace package
#2564 — Bump pydantic-config (drop tyro)
#2567 — Field descriptions to docstrings + simplification
#2569 — Auto-resolve tool-call/reasoning parsers from model name
#2599 — Rename rollouts_per_example → group_size
#2573 — Automated tagging + release-notes flow

Training Modes (SFT / OPD)

Loss selection is now driven by a per-sample training_mode (rl / opd / sft) set under [orchestrator]; the trainer is mode-agnostic and dispatches per batch. SFT and on-policy-distillation (OPD) are first-class modes, and SFT can run without a teacher.

#2546 — First-class SFT + OPD training modes
#2720 — Allow SFT without a teacher
#2271 — Per-batch loss type override for hosted SFT hard distill
#2494 — Accept tool_defs column for SFT
#2629 — Drop teacher_inference from rl entrypoint

Misc Improvements

• Per-env configs: more per-env knobs, separate eval client type, configurable rollout timeout. #2238, #2247, #2371
• Token export: trainer token JSONL export, written per step, multiplexed to Prime monitor. #2592, #2596, #2641
• Runtime source override: PRIME_RL_REF + runtime verifiers version override. #2850, #2274
• Save rollouts in verifiers format for train and eval. #2248
• Clean stale artifacts on resume + pre-launch SLURM cleanup. #2219, #2331
• Configurable grace period for SLURM multinode. #2670
• Pre-download model weights in launcher instead of relying on HF_HUB_OFFLINE. #2282
• Set process titles on all entrypoints. #2159
• Trainable vision encoder (opt-in) for VLMs. #2066
• New environments: Wordle, RLM search, RLM UUID CTF. #2704, #2783, #2832

---

Breaking Changes

See CHANGELOG.md for the full field-by-field migration notes. Highlights:

• Orchestrator async-pipeline rewrite (2026-05-29): orchestrator.seed, [orchestrator.buffer] (entire difficulty-pool block), ckpt.skip_buffer, and orchestrator.eval.skip_eval_on_resume removed. orchestrator.filters → orchestrator.post_batch_filters (+ new pre_batch_filters); orchestrator.eval.eval_base_model → skip_first_step (semantics inverted). Filters are train-only now. #2639
• max_async_level / strict_async_level removed (2026-05-25): async semantics are now invariants — the trainer always runs one step ahead. Drop these fields. #2631
• First-class training_mode + batch-driven loss dispatch (2026-05-19): orchestrator.use_sft_loss, orchestrator.teacher_rollout_model, trainer.loss.type = "sft", and trainer.loss.teacher_tau removed; orchestrator.model → orchestrator.student (aliased). Set [orchestrator] training_mode = "rl"|"opd"|"sft" instead. #2546
• teacher_inference removed from RL entrypoint (2026-05-25): the [teacher_inference] block and deployment.num_teacher_* fields are gone. Start external teachers out-of-band and point [orchestrator.teacher.client] at them. #2629
• orchestrator.use_token_client removed (2026-05-19): the server-tokenized TITO path is deprecated end-to-end (config flag, openai_chat_completions_token client, /v1/chat/completions/tokens route). Migrate to use_renderer = true (renderer TITO, default) or false (MITO). #2549
• [model.vlm] requires orchestrator.use_renderer = true (2026-05-19): VLMs must go through the renderer path. use_renderer already defaults to true. #2473
• inference.kv_cache_offload.cpu_bytes removed (2026-06-02): replaced by a backend-discriminated type config with composable cpu/disk tiers and a mooncake backend.
• rollouts_per_example → group_size (2026-05-22): renamed across orchestrator, eval, and per-env configs. Old name still parses as an alias. #2599
• sampling.min_tokens / repetition_penalty / seed removed and wandb.shared removed (2026-05-27): drop these fields; the rl entrypoint always uses shared W&B mode now.
• orchestrator.val / max_concurrent / verification / temp_scheduler removed (2026-04-05): scoring is always on; concurrency is managed by the limiter; use a fixed sampling.temperature.
• Config consolidation & relocation (2026-03-31): log.file / log.env_worker_logs removed (capture is deployment-level now); .stdout logs renamed to .log, SLURM logs moved to logs/. Added trainer.log.ranks_filter.

---

Bug Fixes

#2843 — Tolerate KV-transfer finish for already-freed (aborted) reqs
#2841 — Drain off-policy rollouts before pausing for weight update
#2830 — Fix routed SLURM inference DP-rank client config
#2813 — Strip routed_experts from chat-completions responses
#2804 — Raise vLLM DP-coordinator startup timeout
#2800 — Freeze DSA sparse indexer so checkpoint resume doesn't crash-loop
#2784 — Restore usage block on /inference/v1/generate
#2759 — Retry all admin ops with bounded timeouts
#2760 — Portable GLOO/UCX NIC selection in sbatch templates
#2729 — Fix weight reload to CPU optimizer
#2714 — Restore NemotronH mixer.D after vLLM 0.22 layerwise reload
#2713 — Disable NemotronH HF-Hub Mamba kernels for offline init
#2707 — Size dense multi-node NCCL world by inference GPU count
#2699 — Avoid concurrent-mkdir race on parallel FS during checkpointing
#2698 — Guard against validation-hook crash taking down trainer
#2694 — Require SLURM for disaggregated deployment
#2690 — Preserve Qwen3.5 broadcast weight names
#2687 — Drop cache_salt for SFT teacher rollouts
#2661 — Fix multi-run checkpoint dispatch
#2658 — Prevent path where expert biases are not loaded for AFMoE
#2636 — Sync SFT validation iteration to prevent FSDP deadlock
#2626 — Trigger evals on orchestrator step
#2594 — Preserve traceback in JSON sink
#2587 — Support aarch64 hosts with any compute capability
#2586 — Stop pre-launch cleanup srun from self-killing
#2584 — Deep-merge legacy model/client aliases; switch multinode template to canonical CLI
#2582 — Patch DeepGEMM SiLU quant addressing
#2581 — Drop group on rollout error instead of rescheduling
#2578 — Handle shared-mode breakage from wandb v0.26.1
#2577 — Release allocation when any component crashes
#2562 — Normalize loss by global token count across dp_cp
#2547 — Inject prime LM head + complete ring-attn restore
#2527 — Trim glibc heap after each step to prevent RSS growth
#2520 — Fix sparse MLA backward TileLang NaNs
#2506 — Scrub padded vLLM model inputs
#2490 — Thread kv_transfer_params into engine for /inference/v1/generate
#2482 — Patch vLLM layerwise reload alias-buffer handling
#2477 — Fix SLURM cleanup self-kill patterns
#2429 — Use FutureWarning so config deprecations stay visible
#2424 — Don't divide reported throughput/MFU by CP for SFT
#2417 — Make FA3 causal/window_size kwargs version-agnostic
#2392 — Fix CP-local cu_seqlens from position ids
#2378 — Align Qwen3.5-MoE _compute_attention with FA4 kwarg
#2374 — Retry wandb.init on transient CommError
#2373 — Bound monitor history and drop duplicate samples list
#2366 — Scope example_id by env_name to prevent cross-env collisions
#2352 — Adapt FA3 varlen wrappers to flash_attn_3 3.0.0 kwargs
#2346, #2356 — Qwen GDN varlen + 3D fixes
#2338, #2340 — NemotronH state-dict fixes
#2337 — Wait for NCCL_READY on all ranks before broadcast
#2310 — Serialize env-server spawn to avoid port race
#2308 — Check rollout error before empty trajectory in scheduler
#2305 — Pin Triton JIT cache to node-local /tmp per job
#2304 — Clean stale rollouts on fresh runs
#2280 — Add gpt-oss extra to Dockerfile uv sync; fix meta buffers
#2276 — Work around transformers lazy_load_kernel offline regression
#2261 — Skip samples with unstable incremental tokenization
#2242 — Prepend base_model.model. prefix to broadcast LoRA keys
#2241 — Let validation errors propagate as 400 on token route
#2240 — Unwrap list pad_token_id before huggingface_hub strict setter
#2237 — Use device index for NCCL worker rank instead of dp_group
#2236 — Auto-disable NCCL P2P/SHM in VM environments
#2229, #2230 — Bound orchestrator cleanup and admin client timeouts
#2221, #2223 — Fix DP/inference_world_size and api_server_count for NCCL broadcast
#2215 — Respect existing WANDB_SHARED_RUN_ID for run resumption
#2201, #2365 — GPT-OSS kernel fixes
#2198 — Handle chat-template errors in eval sample logging
#2185 — Always log per-task error and empty-rollout metrics
#2175 — Increase env-server shutdown timeout for sandbox cleanup
#2158 — Kill full process trees on RL cleanup
#2154 — Reclaim GPU memory after checkpoint load with CPU optimizer offload
#2142 — Add watchdog to packer to detect silent hangs
#2021 — Fix VLM missing arg

---

Misc

#2602 — Big docs revamp
#2116 — Revamp docs and README
#2619 — Fix stale SLURM config examples and references
#2576 — Slim skills, rename, regroup under training/
#2672 — Update BUGBOT.md with custom-model requirements
#2668 — Extend supply-chain cooldown exempt list with first-party deps
#2382 — Add 7-day supply-chain cooldown via exclude-newer
#2641 — Multiplex trainer token export to Prime monitor on orch side
#2563 — Reverse-text smoke tests for RL OPD + SFT training modes
#2512 — Fix recurring CI failures + speed up CI
#2385 — Manual ghcr image build workflow
#2717 — Bump to ubuntu-24.04 to fix arm64 builds with MoonCake
#2381 — FA build on arm64
#2210 — Add training token-usage reporting
#2118 — Log full rollout batches to monitoring backends
#2709 — Send Prime monitor display config to platform
#2556 — Add pydantic-config submodule
#2741 — Remove private configs submodule
#2184 — Add all optional-dependency group combining all extras
#2234 — Dependency updates (flash-attn-3, wheels, lockfile)
#2174 — Update CI Docker image to CUDA 13.0
Plus numerous verifiers / renderers submodule bumps and dependency pins.

---

Contributors

@mikasenghaas, @samsja, @hallerite, @S1ro1, @JannikSt, @Jackmin801, @faresobeid, @rasdani, @eexwhyzee, @alelpoan, @d42me, @snimu, @JohannesHa, @eligotts, @warner-benjamin, @tim0120, @sapiosaturn, @ngc92, @leopold-tzafon, @jupiterepoch, @hubert-marek, @EazyReal, @dzautner, @burnpiro, @andre-fu