v4.7.0 - MiniMax H3, verified AnyFlow distillation

Features

• Hailuo MiniMax H3 ref (v2v, i2v) + fl2v (t2v, t2i, etc)
  • Use the provided example configurations to enable the h3_drift distillation preserving loss augmentation
  • Verified via 1000 step training run
• Webshart datasets have video support now, tested with OpenVid 1M and MiniMax H3
• EMA copy-through mode to ensure EMA remains identical to the hot trained weights until its training is scheduled to begin
• Better lora_format=comfyui support for most models
• MultiGPU FP8 RamTorch support (tested with MiniMax H3)
• MultiGPU Musubi block swap (tested with MiniMax H3 + int8-convrot + FP8 weights)
• constant_with_warmup (or constant with warmup configured) learning rate can now be adjusted on resume

Bugfixes

• Flux2 attention mixed dtype fix to resolve errors with more strict attention_mechanism (non-default)
• SDNQ quantised models cannot re-quantise an already-quantised model (eg. native fp8 or int8-convrot checkpoints)
• Mixed-rank LoRA now function with init_lora to continue training
• Hugging Face Hub model card metadata fixes for various parameter names and corrections to inconsistencies or missing metadata
• MultiGPU (DDP) context-parallel synchronisation fix to ensure all mesh-local ranks receive the same batch
• LTX 2.3 dynamic shift now works correctly
• Custom optimizer_config will pass betas through correctly
• SDNQ compile mode now set before importing its module so that it applies properly
• Empty benchmarks/ subfolders now trigger benchmark regeneration instead of skipping
• validate_on_startup now runs after loading the resumed weights
• NCCL process group timeout can now be specified as env var to override hardcoded value
• Distillers now prepare modules before PEFT is configured for eg. AnyFlow's extra parameters it adds
• Validation samples now have the step count labeled correctly when benchmark is missing or disabled
• AnyFlow has been refactored to ensure we accurately mirror NVIDIA's reference implementation
• Distillation framework can now train inverse-flow models with the correct sign with less smelly code

What's Changed

• Fix fp32 dtype leak from QK-norm in Flux2 attention processors by @AL3708 in #3002
• Respect model flow target direction in distillers by @bghira in #3005
• Avoid double-quantizing SDNQ base models by @bghira in #3006
• Fix Webshart video sample and cache handling by @bghira in #3009
• Fix AnyFlow objective implementation by @bghira in #3010
• Fix LTX2 VAE widths and dynamic shift coverage by @bghira in #3008
• Patch Diffusers LoRA and attention overrides by @bghira in #3007
• Support mixed-rank LoRA adapter metadata by @bghira in #3004
• Preserve optimizer beta overrides from config strings by @bghira in #3015
• Stabilize Webshart video metadata and validation logging (TensorBoard) by @bghira in #3019
• Preserve validation context in text embed cache by @bghira in #3030
• Document public text privacy safeguards by @bghira in #3012
• Fix Hub training metadata commit message by @bghira in #3011
• Upload checkpoint Hub artifacts under checkpoint folders by @bghira in #3017
• Synchronize standalone context-parallel training batches by @bghira in #3022
• Improve AnyFlow CFG conditioning and target geometry by @bghira in #3032
• Split Anima AnyFlow examples into staged configs by @bghira in #3033
• Constrain duplicated conditioning paths by @bghira in #3013
• Add EMA copy-through warmup steps by @bghira in #3016
• Configure SDNQ compile mode before import by @bghira in #3014
• Respect active model dropout cache policy in collate by @bghira in #3031
• Refactor ComfyUI LoRA conversion through model hooks by @bghira in #3029
• Run base benchmark validation in eval mode by @bghira in #3026
• Treat empty validation benchmark folders as absent by @bghira in #3018
• Run startup validation after checkpoint resume by @bghira in #3025
• Add rank-local training diagnostics hooks by @bghira in #3024
• Allow process group timeout overrides from the environment by @bghira in #3023
• MultiGPU (DDP): support FP8 RamTorch and Musubi block swap by @bghira in #3021
• Initialize distillation adapter modules before PEFT setup by @bghira in #3028
• Handle SDNQ fake-mode detection failures by @bghira in #3020
• Restore constant-with-warmup LR on resume by @bghira in #3027
• Hailuo MiniMax H3 training support by @bghira in #3003
• add missing sample label in stitched validation outputs by @bghira in #3036
• merge by @bghira in #3037

New Contributors

• @AL3708 made their first contribution in #3002

Full Changelog: v4.6.1...v4.7.0