Skip to content

Tunix v0.1.8

Latest

Choose a tag to compare

@tejaswin432 tejaswin432 released this 20 Nov 17:28

Highlights

Important

Graduated Distributed RL (tunix.dist_rl)

A production-ready, modular RL framework promoted from tunix/experimental that enables training at the 10k+ accelerator scale, complete with unified launchers, Kubernetes deployment support, and GSM8K, FrozenLake, and DeepSWE recipes (#2999, #3014, #3134, #3175, #3205, #3231).

Zero-TIM Canonical Pallas TPU Kernels

Achieves zero log-prob difference (bitwise-identical logprobs) between the trainer and sampler for the first time on TPU, unlocking truly stable on-policy RL. Read more in our blog post (#2600, #3194, #3302).

Score Centering for Disaggregated GRPO (score_centering)

Eliminates sampler–trainer distribution bias in disaggregated and off-policy GRPO via a zero-value gradient correction over sampler top-k logprobs and the rescaled trainer tail, complete with async rollout top-k collection, clipped token-IS composition, and CLI support. Read more in the paper (#3316, #3409).

Additional Highlights

  • Gemma 4 Multimodal & Scale Support: Added Vision (#1601, #1606, #1607), Audio (#1625, #1634), 12B/12B-it (#1570), and 26B/31B (#2875) support, Gemma4ChatTemplateParser (#1551), cross-layer KV cache sharing (#1896), chunked prefill with prefix KV cache reuse (#1906), Pallas Splash attention (#1890), and distributed RL support for Gemma 4 E2B (#2347, #3364).
  • Qwen 3.5 & Qwen 3: Added native Qwen 3.5 hybrid models (0.8B, 2B) with Flash Attention, sequence packing, and chunkwise Gated DeltaNet TPU optimization (#3088, #3096), plus Qwen 3.5 35B/397B-A17B configs, vLLM mappings, and distributed RL recipes (#1799, #1834, #3182, #3379) and Qwen 3 0.6B (#2064).
  • Raiden Weight Sync & Fault Tolerance: End-to-end Raiden FFI/Pathways weight sync across Tunix and MaxText trainers with multi-host discovery and MoE 128-lane TPU GMM interleaving (#2059, #2096, #2146–#2151, #2201, #2970, #3032), plus rollout worker fault eviction, automatic re-dispatch, late-rejoin bring-up, and capacity-floor guardrails enabled by default (#2722, #3050, #3191, #3213).
  • JAX GPU & v2 Continuous-Batching Sampler (vanilla_v2): Added single- and multi-host JAX GPU support with Tokamax FlashAttention/RaggedDot and vendored vLLM Triton paged attention (#3010, #3101, #3102, #3265), plus the experimental v2 continuous-batching LLMEngine with multi-rank DP, prefix caching, async scheduling, and 1.75× faster unconstrained decoding (#2917–#2929, #3051, #3057, #3272).
  • Sequence Packing & High-Throughput Data Plane: Unified sequence packing with a C++ nanobind (nogil, abi3) extension (#1673, #2135, #2720, #2723), MoE router replay with compact uint8/uint10 bit-packing and Orchestrator Relay (#2046, #2818, #3271, #3300), and out-of-band bulk TCP + parallel LZ4 tensor transport (#2774, #2776, #2817).
  • Trajectory Store & Observability: Added ATIF-backed In-Memory, File, and SQL (SQLite/PostgreSQL partitioned) trajectory & metrics stores (#1857, #2206, #2753, #2984, #2985, #3246), trajectory-explore CLI (#2282, #2933, #2953), OpenTelemetry worker metrics (#2174, #3132, #3240), and Google Cloud ML Diagnostics backend (#2165, #3085).

API Changes

  • Renamed RLCluster → RLEngine across the codebase (#1810) and graduated tunix.experimental distributed RL modules to tunix.dist_rl (#2999).
  • Removed sglang_jax rollout/sampler support; vanilla (vanilla / vanilla_v2) and vllm are now the supported rollout backends (#3204).
  • Removed DeepSWE OpenHands scaffold (r2egym and sweagent remain supported, #2761) and removed MLPerf RCP (mllog) integration (#2762).
  • Standardized multi-turn agentic RL on Token-In-Token-Out (TITO), removing the legacy re-tokenization path (exact_token_continuity, #3254).
  • Config & Interface Updates:
    • GRPOConfig.num_generations is now the single spelling for GRPO group size (#2203, #2237); max_seq_token_per_tpu migrated to max_seq_tokens_per_device (#3284, #3322).
    • Renamed AlgorithmConfig.reward_manager → reward_mode ("reward_fn" / "env", #3046) and replaced force_on_policy_ratio / log_sampler_trainer_agreement with unified in-loss agreement fusion & sampler_is/sampler_rs (#2578, #2616).
    • AbstractTrainer now uses explicit args instead of **kwargs (#3019) and callback-based with_fwd_only_fn / fwd_only(payload) (#2273, #3225, #3243).
    • RolloutWorker.sample_prompts() removed; generate() requires RolloutRequest (#2476), and RolloutWorker/RolloutManager require explicit config, tokenizer, sampler, and chat_parser (#3074, #3347).
    • InprocessVllmSampler / VllmSampler now build engine configs internally from RolloutConfig (#3170); cluster_config.k8s.trainer_extra_env replaced by cluster_config.role_env.<role> (#3318).
  • Dependencies: Updated flax>=0.12.5 (#1581), added JAX 0.11.0 support (#1963), added gymnasium (#1613) and sqlalchemy (#2204), and added C++ nanobind wheel build (requires-python >= 3.12, #2723).

What's Changed

  • Algorithms & Training: Enabled PPO across all model types (#1683, #1752, #1775), added model-agnostic JAX diffusion contracts (#1832), ORPO TRL parity (#1574), DAPO beta=0.0 (#2570), chunked logps (#1595), mixed-precision fp32 master / bf16 compute training (#2844, #2972), and Orbax V1 Checkpointer migration for SFT (#1571, #1679, #1690).
  • Rollouts & Agentic RL: Added partial rollouts (in-flight weight updates, #2750), prompt-arrival group ordering & max_version_span staleness enforcement (#2725, #3104, #3165), DeepSWE Agent Sandbox & SandboxFleet dynamic pre-warming (#1791, #2125, #2189, #3161), process-pool reward evaluation (#1904), and Qwen 3.5 <tool_call> parsing & timeout fixes (#2820, #2821, #2934).
  • Performance, Serialization & CI: Reduced optimizer/gradient HBM footprint (#1758, #1783), added Tokamax Splash Attention (#1943, #2962), GIL-free C++ trajectory serialization & compact JSON (#2731, #2739), safetensors thread-safety & bfloat16 fixes (#1627, #2016, #3139), notebook fixes (#1738–#1741), and faster sharded CPU/TPU CI (#2272, #3155, #3166, #3173).

New Contributors

Full Changelog: v0.1.7...v0.1.8