merged checkpoint works but lora weights does not work with kohya-ss sd-webui-additional-networks

[harrywang]

I trained a model - training completed successfully. The merged model works (shows beckham):

however, when I use the generated Lora weights using https://github.com/kohya-ss/sd-webui-additional-networks, it has no effect:

I used another Lora weights I got from Civitai.com and it works - to confirm the extension is working.

The files are:

I found the following error in the log:

Loading weights [1a189f0be6] from /home/hjwang/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned.safetensors
Creating model from config: /home/hjwang/stable-diffusion-webui/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Model loaded in 1.2s (create model: 0.4s, apply weights to model: 0.3s, apply half(): 0.2s, move model to device: 0.3s).
activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x7ff3fc4817f0>]: TypeError
Traceback (most recent call last):
  File "/home/hjwang/stable-diffusion-webui/modules/extra_networks.py", line 75, in activate
    extra_network.activate(p, extra_network_args)
  File "/home/hjwang/stable-diffusion-webui/extensions-builtin/Lora/extra_networks_lora.py", line 23, in activate
    lora.load_loras(names, multipliers)
  File "/home/hjwang/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 151, in load_loras
    lora = load_lora(name, lora_on_disk.filename)
  File "/home/hjwang/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 83, in load_lora
    sd = sd_models.read_state_dict(filename)
  File "/home/hjwang/stable-diffusion-webui/modules/sd_models.py", line 224, in read_state_dict
    sd = get_state_dict_from_checkpoint(pl_sd)
  File "/home/hjwang/stable-diffusion-webui/modules/sd_models.py", line 197, in get_state_dict_from_checkpoint
    pl_sd = pl_sd.pop("state_dict", pl_sd)
TypeError: pop expected at most 1 argument, got 2

I guess this is the cause but don't know why - Can anyone help? What did I miss?

BTW, I found the following two closed related issues:

#695
#723

[gammagoat]

The additional networks feature in a1111 was written by for kohya_ss model format. The main of this model uses the cloneofsimo implementation for LoRA and is not compatible. There is an option in dev to also have additional networks Loras for the non-extended version if you want to give it a try

[ArrowM]

Also that non-extended version should get merged to main sometime today or tomorrow

[harrywang]

The additional networks feature in a1111 was written by for kohya_ss model format. The main of this model uses the cloneofsimo implementation for LoRA and is not compatible. There is an option in dev to also have additional networks Loras for the non-extended version if you want to give it a try

Thanks!

[harrywang]

@gammagoat @ArrowM I see a PR merge #1014 did this merge "additional networks Loras for the non-extended version"? how to use/enable this? Any pointer? Thanks a lot!

[harrywang]

I pulled from dev and don't know which saving option will make lora compatible with https://github.com/kohya-ss/sd-webui-additional-networks

[gammagoat]

Generate Lora weights for extra networks will save a file in the model/lora directory that will work (as long as you don't use extended Lora). I recommend deleting or moving all the old incompatible .pt files out of there.

[harrywang]

Generate Lora weights for extra networks will save a file in the model/lora directory that will work (as long as you don't use extended Lora). I recommend deleting or moving all the old incompatible .pt files out of there.

awesome - will try and report back! Thanks!!

[harrywang]

@gammagoat I trained again and the new lora weights work perfectly!! Thanks!