Converting lora .safetensors to diffusers

[tentothedollar1]

Describe the bug

Hi there,

I'm trying to convert a civitai model to diffusers format using this script in your repository: https://github.com/huggingface/diffusers/blob/main/scripts/convert_lora_safetensor_to_diffusers.py

I'm trying to convert the file like this:

python convert.py --checkpoint_path /Users/isaac/Desktop/convert/model.safetensors --dump_path /Users/isaac/Desktop/convert --base_model_path stabilityai/stable-diffusion-2-1

But after a while running I get the error: ""AttributeError: 'CLIPTextModel' object has no attribute 'cond_stage_model'".

I'm not super good with this stuff so a wiki on how to use these convert scripts would be useful. Am I calling the command correctly? Is the base_model_path supposed to be a huggingface path like "stabilityai/stable-diffusion-2-1" or is that wrong?

Would appreciate any help. Thanks

I'm running on a Mac OS.

Reproduction

Try to convert .safetensors on a Mac OS, using the command

python convert.py --checkpoint_path /Users/isaac/Desktop/convert/model.safetensors --dump_path /Users/isaac/Desktop/convert --base_model_path stabilityai/stable-diffusion-2-1

Logs

No response

System Info

Mac Os, Python 3.11.2

[sayakpaul]

Cc: @haofanwang

[sertdfyguhi]

Same problem over here. Here's the full traceback:

Traceback (most recent call last):
  File "/Users/a/programming/convert_lora_safetensor_to_diffusers.py", line 55, in convert
    curr_layer = curr_layer.__getattr__(temp_name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'CLIPTextModel' object has no attribute 'cond_stage_model'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/a/programming/ai/convert_lora_safetensor_to_diffusers.py", line 125, in <module>
    pipe = convert(base_model_path, checkpoint_path, lora_prefix_unet, lora_prefix_text_encoder, alpha)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/a/programming/ai/convert_lora_safetensor_to_diffusers.py", line 62, in convert
    temp_name += "_" + layer_infos.pop(0)
                       ^^^^^^^^^^^^^^^^^^
IndexError: pop from empty list

[patrickvonplaten]

We should probably allow loading the civitai/A1111 tensor directly into the load lora function

[sayakpaul]

@patrickvonplaten there's some very valuable discussion here #3064 (comment) by @pdoane.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.