Version 0.2.12

Highlights

📝 Code Quality: Ruff Formatting: PR #538

We introduced Ruff for unified linting and formatting across the codebase. This standardization improves contributor experience, ensures consistent style, and streamlines the review process. Thank you @arledesma for this great contribution!

⚡ CPU Offloading for Gradient Checkpointing: PR #537

Added support for activation CPU offloading during gradient checkpointing. This feature reduces VRAM usage by up to 20–30% in large-scale video training, enabling larger batch sizes. The trade-off is slightly slower training (a few percent to ~20%).

🚀 Faster Model Loading with MemoryEfficientSafeOpen: PR #556

Improved .safetensors loading with np.memmap and non-blocking GPU transfer, making model load times up to 1.5× faster. This significantly reduces waiting time for large model initialization.

🔬 FP8 Quantization with Block-wise Scaling: PR #575

Changed the --fp8_scaled option from per-tensor quantization to block-wise scaling, resulting in improved accuracy and stability. For Qwen-Image LoRA training, this reduces VRAM usage by about 5GB. Training and inference speed may be slightly slower.

What's Changed

• Configure ruff format for code quality standardization by @arledesma in #538
• doc: update contributing guidelines for clarity and consistency by @kohya-ss in #541
• feat: add CPU offloading support for gradient checkpointing by @kohya-ss in #537
• docs: update README to include ruff code analysis and activation CPU offloading by @kohya-ss in #545
• bugfix: fix mask initialization for wan flf2v inference by @LittleNyima in #548
• feat: faster MemoryEfficientSafeOpen by @kohya-ss in #556
• from_file with qwen_image_generate_image is broken by @nmfisher in #553
• fix: Qwen-Image training not working with fp8_base by @kohya-ss in #559
• chore: add .vscode/settings.json to .gitignore by @kohya-ss in #562
• feat: faster LoRA merging and stable fp8 max calculation by @kohya-ss in #563
• fix(wan): FlashAttention-3 call error by @jimlee2048 in #570
• Fix qwen image from file generation by @kohya-ss in #557
• fix: add dtype handling for fp8 model weights in weight_hook_func by @kohya-ss in #582
• fix: VAE tiling is always enabled for FramePack by @kohya-ss in #583
• feat: FP8 quantization with block-wise scaling by @kohya-ss in #575
• chore: bump version to 0.2.12 by @kohya-ss in #584

New Contributors

• @LittleNyima made their first contribution in #548
• @nmfisher made their first contribution in #553
• @jimlee2048 made their first contribution in #570

Full Changelog: v0.2.11...v0.2.12