How to set lora scale when loading a LoRA model?

[pietrobolcato]

Hey there, first of all thanks for your fantastic work!

I am loading LoRA weights, and I would like to set the scale of them being applied. Checking the code, it appears to be possible as shown here.

How can we do it in practice? Is it possible to provide a small code snippet?

Thank you so much! Really appreciate your help :)

[levi]

You want to do something like this:

image = pipe(prompt=prompt, cross_attention_kwargs={"scale":0.5}).images[0]

[pietrobolcato]

Sounds great thanks a lot! Appreciate a lot your help!

[Gynjn]

Thanks for your help. Then if I load the lora weight with load_lora_weights and use image = pipe(prompt=prompt, cross_attention_kwargs={"scale":0.5}).images[0] it only change the weight of the cross_attn_module not text_emb right??

[max-fofanov]

No, it also does for text_encoder to. See https://github.com/huggingface/diffusers/blob/1edd0debaa3103e0dc230551c36e22ff60a56af4/src/diffusers/pipelines/controlnet/pipeline_controlnet_img2img.py#L935C6-L935C6 for example