v21.5.4

[bmaltais]

• 2023/04/17 (v21.5.4)
  • Fixed a bug that caused an error when loading DyLoRA with the --network_weight option in train_network.py.
  • Added the --recursive option to each script in the finetune folder to process folders recursively. Please refer to PR #400 for details. Thanks to Linaqruf!
  • Upgrade Gradio to latest release
  • Fix issue when Adafactor is used as optimizer and LR Warmup is not 0: lr_warmup should not be passed when adafactor is used as the optimizer #617
  • Added support for DyLoRA in train_network.py. Please refer to here for details (currently only in Japanese).
  • Added support for caching latents to disk in each training script. Please specify both --cache_latents and --cache_latents_to_disk options.
    • The files are saved in the same folder as the images with the extension .npz. If you specify the --flip_aug option, the files with _flip.npz will also be saved.
    • Multi-GPU training has not been tested.
    • This feature is not tested with all combinations of datasets and training scripts, so there may be bugs.
  • Added workaround for an error that occurs when training with fp16 or bf16 in fine_tune.py.
  • Implemented DyLoRA GUI support. There will now be a new 'DyLoRA Unitslider when the LoRA type is selected askohya DyLoRA` to specify the desired Unit value for DyLoRA training.
  • Update gui.bat and gui.ps1 based on: Error output when WD14 Captioning #188
  • Update setup.bat to install torch 2.0.0 instead of 1.2.1. If you want to upgrade from 1.2.1 to 2.0.0 run setup.bat again, select 1 to uninstall the previous torch modules, then select 2 for torch 2.0.0