Apply to Checkpoint

[d8ahazard]

Hi @cloneofsimo. Could you kindly provide an example of applying lora weights to a diffusers model and saving for conversion to a standard ckpt file?

Still having some troubles in this department.

[cloneofsimo]

Ok ill make a detailed blogpost on this but I'm really onto other things right now so give me a week or so please!

[qunash]

Hi @cloneofsimo, could you also provide some details of your training process? I'm trying to test this method by training a new style, but so far the results aren't as good as in your examples. Interested in info on the parameters you used, training dataset, etc.

[cloneofsimo]

@d8ahazard ok, so I've just made another mode that converts to ckpt format that works with v2.0 model.

Here is a full example with lora_illust.pt, LoRA that was trained on 10 illust images for 30000 steps with learning rate of 1e-4 :

lora_add --path_1 stabilityai/stable-diffusion-2-base --path_2 lora_illust.pt --mode upl-ckpt-v2 --alpha 1.2 --output_path merged_model.ckpt

I'm not sure if this works with v1.x tho, because I haven't trained lora on them yet. Put the checkpoint to for example, A1111's webui, and you get :

Which is exactly same output and configuration that you can find in the recently updated notebook scripts/run_inference.ipynb:

(So this proves that transformation works perfectly I guess?)

[cloneofsimo]

@qunash Train with relatively large learning rate! This seems to be the common problem that everyone is having.
By the way, I didn't play around with the parameters enough to conclusively say anything about "best training configurations" other than the fact that it works reasonably well.

[cloneofsimo]

All the loras in the example were trained with large learning rate, without prior preservation.

[qunash]

Thanks for the info!