v3.1.0

Image: quay.io/modh/fms-hf-tuning:v3.1.0

Summary

1. Support GPT OSS class of models.
2. Support Granite 4 series of models.
3. Support Mamba and Hybrid Architecture Models.
4. Support Flash Attention3 via HuggingFace kernelhub.
5. Support loading MxFP4 Quantized models - Models in MxFP4 need to be dequantized before training as MxFP4 training is not supported in HuggingFace.
6. Fixed major peformance bug which caused memory usage to double in few cases #592
7. Support Alora directly from PEFT upstream.

Data Preprocessor changes

1. Supports passing chat template as a .jinja file inside data config.
2. Improved documentation and various bug fixes.

Additional Changes

1. Supports experimental Online Data Mixing Plugin as presented in Pytorch Conference'25 by IBM Research Team @kmehant @romitjain @seshapad

List of Changes

• fix: Update offline-data-preprocessing.md by @dushyantbehl in #589
• fix: use default dataprocessor if one is not provided by @dushyantbehl in #590
• fix: data processing job exiting with failure even though successful by @dushyantbehl in #593
• fix: Disable caching in transformers via use_cache flag to avoid unncessary memory overhead by @romitjain in #592
• fix: Rank is set to zero by default by @seshapad in #594
• chore: upgrade trl by @dushyantbehl in #601
• chore: add changes to support Granite 4 models by @YashasviChaurasia in #599
• feat: Support gpt-oss class of models with flash attention 3 support by @dushyantbehl in #603
• feat: Restructure README by @dushyantbehl in #598
• fix: add default optim arg in training arg by @YashasviChaurasia in #607
• fix: fix multiple typos by @dushyantbehl in #610
• fix: subclass Lora config from upstream peft.LoraConfig by @romitjain in #609
• fix: update fms-accel by @YashasviChaurasia in #608
• docs: Update advanced-data-preprocessing.md by @dushyantbehl in #613
• feat: add ckpt conversion script fp32-bf16 by @YashasviChaurasia in #614
• feat: Allow chat template to be specified via a path in data config. by @YashasviChaurasia in #615
• feat: add online data mixing plugin by @kmehant in #612
• feat: Adopt resumption feature of online data mixing by @kmehant in #617
• feat: alora migration to peft by @YashasviChaurasia in #618
• feat: ensure fms-acceleration callbacks run before TrainerController by @YashasviChaurasia in #620
• fix: add on_init_end before adding tc callback by @YashasviChaurasia in #621
• fix: add hf compatible path update by @YashasviChaurasia in #622
• fix: avoid updating path kwarg by @YashasviChaurasia in #624
• fix: directly save final ckpt in save_model_dir by @YashasviChaurasia in #626
• fix: Add free up disk space to gh runners by @dushyantbehl in #628
• fix: image build failure due to flash attn by @dushyantbehl in #629
• chore: Merge set of changes for release v3.1.0 by @dushyantbehl in #630
• chore: tag peft to a rc instead of commit tag by @dushyantbehl in #631
• fix: release pypi build failure due to peft tagging to a commit by @dushyantbehl in #632
• fix: package version config bug by @dushyantbehl in #635
• chore: merge pypi fixes for the release by @dushyantbehl in #637

New Contributors

• @romitjain made their first contribution in #592

Full Changelog: v3.0.0...v3.1.0