v0.9.0

What's Changed

• [llama4] fix the mm yaml, add scout single gpu yaml by @winglian in #2510
• upgrade transformers to 4.51.1 by @winglian in #2508
• fix: liger swiglu for llama4 by @NanoCode012 in #2504
• Add Llama4 maverick examples and wandb links by @winglian in #2512
• fix: allow merge lora on pre-quantized model by @NanoCode012 in #2511
• feat: add CNAME by @NanoCode012 in #2513
• Update rlhf.qmd by @bursteratom in #2519
• add mocks for loading datasets in cli train tests by @winglian in #2497
• Feat(examples): add deepcogito by @NanoCode012 in #2516
• feat(doc): explain deepspeed configs by @NanoCode012 in #2514
• chore: update doc links by @NanoCode012 in #2509
• [ci] make e2e tests a bit faster by reducing test split size by @winglian in #2522
• remove strict=false from example yamls by @winglian in #2523
• feat: add examples for deepcoder by @NanoCode012 in #2517
• make sure the all of the model is on the same device by @winglian in #2524
• feat: update cce to latest by @NanoCode012 in #2521
• Fix: add delinearization and make qlora work with fsdp2 by @NanoCode012 in #2515
• batch api HF adapter for ring-flash-attn; cleanup and improvements by @djsaunde in #2520
• re-enable DS zero3 ci with updated transformers by @winglian in #2533
• adding codecov reporting by @djsaunde in #2372
• fix: preprocess yielding whole dataset to each worker by @NanoCode012 in #2503
• fix(doc): cut cross entropy installation instructions broken in qmd by @NanoCode012 in #2532
• zero val fix for beta by @winglian in #2538
• fix: upgrade liger to 0.5.8 and use native Gemma3 patches by @chiwanpark in #2527
• don't run multigpu tests twice, run SP in separate test by @winglian in #2542
• Fixed Rex Scheduler Warm Up by @Catgat in #2535
• prevent rate limiting to hf when using dispatch batches by @winglian in #2536
• make sure to download fixtures for kd test by @winglian in #2541
• fix missing host/port for vllm by @winglian in #2543
• feat: add glm and glm4 multipack and cce by @NanoCode012 in #2546
• Codecov fixes / improvements by @djsaunde in #2549
• add base docker image with pytorch 2.7.0 and variant for cuda 12.8 by @winglian in #2551
• builds for torch==2.7.0 by @winglian in #2552
• Fix(doc): add delinearize instruction by @NanoCode012 in #2545
• disable codecov pr annotations by @djsaunde in #2556
• make sure to validate the config before normalizing so defaults get set by @winglian in #2554
• Sequence parallel training context manager by @djsaunde in #2553
• don't fail on codecov upload for external contributor PRs by @winglian in #2564
• fix: gradient checkpointing functools.partial object has no attribute self by @ekojsalim in #2563
• make cce default to true when using the plugin by @winglian in #2562
• chore(doc): minor update docker tags on doc by @NanoCode012 in #2559
• fix: crash when pretraining_dataset with dispatch_batches is false by @chiwanpark in #2558
• fix support for wandb run_name for rl trainers by @winglian in #2566
• add e2e smoke test for using activation/gradient checkpointing with offload by @winglian in #2565
• don't use is_main_process during config validation by @winglian in #2569
• update trl to 0.17.0 by @winglian in #2560
• Fix bug in grpo reward module import by @dhruvmullick in #2571
• fix(doc): clarify vllm usage with grpo by @NanoCode012 in #2573
• replace references to random 68m model w 135m smollm2 by @winglian in #2570
• Add runpod sls handler by @KAJdev in #2530
• Add Post_model_load, post_lora_load, post_train, post_train_unload function calls by @divyanshuaggarwal in #2539
• grab sys prompt too from dataset by @winglian in #2397
• feat: add eos_tokens and train_on_eot for chat_template EOT parsing by @NanoCode012 in #2364
• add preview-docs workflow by @djsaunde in #2432
• support val_set_size for splitting test split from train with DPO by @winglian in #2572
• Feat: Add qwen3 and CCE for qwen family by @NanoCode012 in #2518
• chat template and example for qwen3 by @winglian in #2577
• automatically split out reasoning trace from dataset by @winglian in #2579
• v0.9.0 release by @winglian in #2578

New Contributors

• @Catgat made their first contribution in #2535
• @ekojsalim made their first contribution in #2563
• @dhruvmullick made their first contribution in #2571
• @KAJdev made their first contribution in #2530
• @divyanshuaggarwal made their first contribution in #2539

Full Changelog: v0.8.0...v0.9.0