v2.1.0

Image released: quay.io/modh/fms-hf-tuning:v2.1.0

New Major Feature

• Support for GraniteForCausalLM model architecture

Dependency upgrades

• Upgraded transformers to version 4.45.2, now supports GraniteForCausalLM models. Note that if a model is trained with transformers v4.45, you need the same version transformers>=4.45 to load the trained model, prior versions of transformers will not be compatible.
• Upgraded accelerate to version 1.0.1.
• Limit upper bound of torch to be under v2.5.0, not including v2.5.0, so it's compatible with flash-attention-2.
• Upgraded fms_acceleration_peft to v0.3.1 which includes disabling offloading state dict which caused ephemeral storage issues when loading large models with QLoRA. Also includes setting defaults when target_modules=None.

Additional bug fix

• Fix for crash when running a multi GPU training with a non-existent output dir.

Full list of Changes

• ci: run unit tests, fmt, image build on release branch by @anhuong in #361
• chore: update code owners by @anhuong in #363
• fix: crash when output directory doesn't exist by @HarikrishnanBalagopal in #364
• refactor: move tokenizer_data_utils with the rest of utils, add further unit testing. by @willmj in #348
• build(deps): update transformers and accelerate deps by @anhuong in #355
• build(deps): Update peft requirement from <0.13,>=0.8.0 to >=0.8.0,<0.14 by @dependabot in #354
• build(deps): Upgrade accelerate requirement to allow version 1.0.0 by @willmj in #371
• build: Set triton environment variables by @willmj in #370
• build(deps): torch<2.5 due to FA2 error with new version by @anhuong in #375
• chore: merge set of changes for v2.1.0 by @aluu317 in #376

Full Changelog: v2.0.1...v2.1.0