Can't convert from controlNet model to diffusers

[brurpo]

Describe the bug

I was able to do so before the PR merge, when it was being developed.

now it throws the following error:

Traceback (most recent call last):
  File "E:\BRP_STABLE_DIFFUSION\bin\convert_original_stable_diffusion_to_diffusers.py", line 129, in <module>
    pipe = load_pipeline_from_original_stable_diffusion_ckpt(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1251, in load_pipeline_from_original_stable_diffusion_ckpt
    ctrlnet_config = create_unet_diffusers_config(original_config, image_size=image_size, controlnet=True)
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 234, in create_unet_diffusers_config
    unet_params = original_config.model.params.control_stage_config.params
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\dictconfig.py", line 355, in __getattr__
    self._format_and_raise(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\base.py", line 231, in _format_and_raise
    format_and_raise(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\_utils.py", line 899, in format_and_raise
    _raise(ex, cause)
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\_utils.py", line 797, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\dictconfig.py", line 351, in __getattr__
    return self._get_impl(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\dictconfig.py", line 442, in _get_impl
    node = self._get_child(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\basecontainer.py", line 73, in _get_child
    child = self._get_node(
  File "E:\BRP_STABLE_DIFFUSION\bin\lib\site-packages\omegaconf\dictconfig.py", line 480, in _get_node
    raise ConfigKeyError(f"Missing key {key!s}")
omegaconf.errors.ConfigAttributeError: Missing key control_stage_config
    full_key: model.params.control_stage_config
    object_type=dict

Reproduction

Just try to convert an existing controlNet pth

Logs

No response

System Info

2990wx
2080ti

[Pirog17000]

Any hint how to convert pth with diffusers at all? I mean, without conversion will be great.. To make it run locally with '.from_pretrained()'

[takuma104]

Hi @brurpo @Pirog17000 ,In the latest version of the main branch, you can do the conversion like this:

python scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path .../control_sd15_canny.pth --original_config_file .../cldm_v15.yaml --to_safetensor --controlnet --dump_path .../control_wd15_canny/

The arguments can be specified like this:

• checkpoint_path: Checkpoint file of the conversion source
• original_config_file: Config file from https://github.com/lllyasviel/ControlNet/blob/main/models/cldm_v15.yaml
• controlnet: Need to convert controlnet
• dump_path: Distination folder

[Pirog17000]

@takuma104 what does the script trying to download? 🤔