Skip to content

v0.9.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 May 21:53
· 764 commits to main since this release
v0.9.2
54bbc9b

🚨 Breaking Changes

No breaking changes in this release! 🎉

🎉 New Features

Activation Checkpointing with Disk Offloading

You can now significantly reduce VRAM usage by offloading activation checkpoints to disk with prefetching. This makes it possible to train larger models or use larger batch sizes on memory-constrained hardware. This is only recommended on resource constrained systems like Google Colab.

Support for Atropos

Atropos is a framework for Reinforcement Learning by NousResearch. The Atropos plugin for Axolotl adds Atropos' RL environments into Axolotl's training pipelines. This allows you to leverage Atropos for reinforcement learning while utilizing Axolotl's extensive features for model fine-tuning.

🔧 Major Fixes

  • LoRA Kernel Stability: The LoRA kernel is now disabled when lora_dropout is non-zero instead of being auto-enabled. (by @NanoCode012 in #2655)
  • RunPod Stability: Improved handling of environment variables for RunPod serverless deployments to prevent accidental deletion of secrets. (by @winglian in #2653)
  • General Bug Fixes: Pass save_only_model to RL trainer and improve mistral fft tests. (by @winglian in #2661)

Other Improvements

  • Documentation: Updated GRPO documentation for clarity. (by @winglian in #2649)

Full Changelog: https://github.com/axolotl-ai-cloud/axolotl/compare/v0.9.1.post1...v0.9.2

Contributors

winglian and NanoCode012
Assets 2
Loading