v0.1.0

Finetrainers v0.1.0 🧪

output.mp4

New models supported

• CogView4
• Wan 2.1

Checkpoints released

• https://huggingface.co/finetrainers/Wan2.1-T2V-1.3B-crush-smol-v0
• https://huggingface.co/finetrainers/Wan2.1-T2V-1.3B-3dgs-v0
• https://huggingface.co/finetrainers/CogView4-6B-rider-waite-tarot-v0
• https://huggingface.co/finetrainers/CogView4-6B-rider-waite-tarot-v0-shifted-sigmas

Other major changes

• Fully support for Accelerate as a parallelization backend again
• Opt-in precomputation. Helpful for smaller datasets
• Better remote dataset loading support
• Support for datasets>=3.4.0
• Bug fix: Layerwise Casting now works with Wan
• Bug fix: LTX Video training now works with batch_size > 1

What's Changed

• calling save_model_card correctly by @vivkul in #275
• 3D Parallel + Model Spec API by @a-r-r-o-w in #245
• CogVideoX ModelSpec by @a-r-r-o-w in #280
• Remove unused CogVideoX code by @a-r-r-o-w in #282
• Wan T2V ModelSpec by @a-r-r-o-w in #281
• update docs by @a-r-r-o-w in #284
• Improve local dataset loading by @a-r-r-o-w in #289
• Add 3DGS dataset example for Wan by @a-r-r-o-w in #290
• HunyuanVideo ModelSpec by @a-r-r-o-w in #287
• Fix full rank config by @a-r-r-o-w in #292
• update docs by @a-r-r-o-w in #294
• CogView4 ModelSpec by @a-r-r-o-w in #297
• PyPI package by @a-r-r-o-w in #298
• Minor cleanup and doc updates by @a-r-r-o-w in #304
• Remove forceful precomputation behaviour; Improvements to data loading by @a-r-r-o-w in #303
• Webdataset improvements; CogView4 example with The Simpsons webdataset by @a-r-r-o-w in #305
• Fix for #306 by @a-r-r-o-w in #307
• Fix Wan scaling due to upstream changes by @a-r-r-o-w in #308
• Fix CogVideoX config error for invert_scale_latents by @a-r-r-o-w in #309
• add valid names to dataset docs by @neph1 in #318
• Fix Layerwise Casting by @a-r-r-o-w in #316
• Fix enable_model_cpu_offload problems by @a-r-r-o-w in #320
• Patch WanTimeTextImageEmbedding forward only with fp8 by @a-r-r-o-w in #327
• Lower memory requirements on single GPU by @a-r-r-o-w in #321
• Respect report_to if set to none by @a-r-r-o-w in #331
• Cleanup model load methods by @a-r-r-o-w in #333
• add init.py to sft_trainer/ to fix #335 by @jbilcke-hf in #336
• Add version guard for datasets 3.4.0; Fix argument documentation by @a-r-r-o-w in #332
• Update dataset.py by @a-r-r-o-w in #337
• Remove hard-coded batch size configuration in LTXModel by @Linyou in #340
• Fix missing comma in loading hunyuan video tokenizer by @a-r-r-o-w in #341
• Add back accelerate compatibility by @a-r-r-o-w in #339
• Relative -> Absolute imports by @a-r-r-o-w in #343
• Prepare for v0.1.0 release by @a-r-r-o-w in #322

New Contributors

• @vivkul made their first contribution in #275
• @neph1 made their first contribution in #318
• @jbilcke-hf made their first contribution in #336
• @Linyou made their first contribution in #340

Full Changelog: v0.0.1...v0.1.0