ValueError: Checkpoint not supported because layer lora_unet_down_blocks_0_downsamplers_0_conv.alpha not supported.

[cjt222]

Describe the bug

Hello, I'm loading the lora model from civtai, which includes convolutional layers, cause error. Are diffusers not supported now?

ValueError: Checkpoint not supported because layer lora_unet_down_blocks_0_downsamplers_0_conv.alpha not supported.

Reproduction

from diffusers import StableDiffusionXLPipeline, LCMScheduler
pipe = StableDiffusionXLPipeline.from_pretrained(
base_model_path,
torch_dtype=torch.float16,
add_watermarker=False,
)

pipe.load_lora_weights(lcm_lora_id, weight_name="XL_LoRA_LCM_Sampler2.safetensors")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

Logs

No response

System Info

• diffusers version: 0.25.0.dev0
• Platform: Linux-5.4.0-48-generic-x86_64-with-glibc2.27
• Python version: 3.10.13
• PyTorch version (GPU?): 2.1.0+cu121 (True)
• Huggingface_hub version: 0.20.1
• Transformers version: 4.36.2
• Accelerate version: 0.24.1
• xFormers version: 0.0.22.post7
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

No response

[cjt222]

@sayakpaul

[sayakpaul]

Can you provide code that we can reproduce? What you have provided doesn't tell us what lcm_lora_id is.

[cjt222]

lcm_lora_id is https://civitai.com/models/216190/lcmandturbomix-lora-only-12mb-8-step-sampling-effect-is-superior-to-using-lcm-or-turbo-alone

[itrieu]

@sayakpaul
I got the same issue.
Is there any solution?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[sayakpaul]

Please provide a fully reproducible code snippet.

[painebenjamin]

I ran into this as well. Full code snippet to reproduce on diffusers==0.26.3:

from requests import get
from diffusers import StableDiffusionXLPipeline

sdxl_pipeline_id = "stabilityai/stable-diffusion-xl-base-1.0"
lora_url = "https://civitai.com/api/download/models/247778?type=Model&format=SafeTensor"
lora_path = "./lcm-turbo-mix.safetensors"

with open(lora_path, "wb") as fh:
  data = get(lora_url, stream=True)
  for chunk in data.iter_content(chunk_size=8192):
    fh.write(chunk)

pipe = StableDiffusionXLPipeline.from_pretrained(
  sdxl_pipeline_id,
  variant="fp16"
)
pipe.load_lora_weights(lora_path)

Output:

Traceback (most recent call last):
  File "/home/benjamin/test.py", line 17, in <module>
    pipe.load_lora_weights(lora_path)
  File "/home/benjamin/miniconda3/envs/enfugue/lib/python3.10/site-packages/diffusers/loaders/lora.py", line 1442, in load_lora_weights
    state_dict, network_alphas = self.lora_state_dict(
  File "/home/benjamin/miniconda3/envs/enfugue/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/benjamin/miniconda3/envs/enfugue/lib/python3.10/site-packages/diffusers/loaders/lora.py", line 298, in lora_state_dict
    state_dict = _maybe_map_sgm_blocks_to_diffusers(state_dict, unet_config)
  File "/home/benjamin/miniconda3/envs/enfugue/lib/python3.10/site-packages/diffusers/loaders/lora_conversion_utils.py", line 57, in _maybe_map_sgm_blocks_to_diffusers
    raise ValueError(f"Checkpoint not supported because layer {layer} not supported.")
ValueError: Checkpoint not supported because layer lora_unet_down_blocks_0_downsamplers_0_conv.alpha not supported.

[yiyixuxu]

@sayakpaul
I think their checkpoint contains extra layers - we should be able to support this, no?