Kandinsky 2.2 fails to load

[vladmandic]

Describe the bug

i've tried both DiffusionPipeline and AutoPipelineForText2Image and Kandinsky 2.2 fails to load:

sd_model = diffusers.DiffusionPipeline.from_pretrained('models/Diffusers/models--kandinsky-community--kandinsky-2-2-decoder')
sd_model = diffusers.AutoPipelineForText2Image.from_pretrained('models/Diffusers/models--kandinsky-community--kandinsky-2-2-decoder')

error is:

pipeline_kandinsky2_2_combined.KandinskyV22CombinedPipeline'> 
expected {'prior_image_processor', 'prior_image_encoder', 'prior_tokenizer', 'prior_prior', 'prior_text_encoder', 'prior_scheduler', 'movq', 'unet', 'scheduler'}, 
but only {'unet', 'movq', 'scheduler'} were passed.

note that kandinsky 2.1 works fine (and so do most other models)

Reproduction

import os
import torch
import diffusers

cache_dir = '/home/vlado/dev/sdnext/models/Diffusers'
model_path = 'models--kandinsky-community--kandinsky-2-2-decoder/snapshots/824f8d584960715056dc509c119d6a33cde34889'

pipe = diffusers.AutoPipelineForText2Image.from_pretrained(os.path.join(cache_dir, model_path), torch_dtype=torch.float16, cache_dir=cache_dir)

Logs

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/vlado/dev/sdnext/cli/test-diffuser.py:8 in <module>                                        │
│                                                                                                  │
│    5 cache_dir = '/home/vlado/dev/sdnext/models/Diffusers'                                       │
│    6 model_path = 'models--kandinsky-community--kandinsky-2-2-decoder/snapshots/824f8d5849607    │
│    7                                                                                             │
│ ❱  8 pipe = diffusers.AutoPipelineForText2Image.from_pretrained(os.path.join(cache_dir, model    │
│    9 pipe = pipe.to("cuda")                                                                      │
│   10                                                                                             │
│   11 prompt = "portrait of a young women, blue eyes, cinematic"                                  │
│                                                                                                  │
│ /home/vlado/.local/lib/python3.11/site-packages/diffusers/pipelines/auto_pipeline.py:331 in      │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   328 │   │   text_2_image_cls = _get_task_class(AUTO_TEXT2IMAGE_PIPELINES_MAPPING, orig_class   │
│   329 │   │                                                                                      │
│   330 │   │   kwargs = {**load_config_kwargs, **kwargs}                                          │
│ ❱ 331 │   │   return text_2_image_cls.from_pretrained(pretrained_model_or_path, **kwargs)        │
│   332 │                                                                                          │
│   333 │   @classmethod                                                                           │
│   334 │   def from_pipe(cls, pipeline, **kwargs):                                                │
│                                                                                                  │
│ /home/vlado/.local/lib/python3.11/site-packages/diffusers/pipelines/pipeline_utils.py:1197 in    │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   1194 │   │   │   │   init_kwargs[module] = passed_class_obj.get(module, None)                  │
│   1195 │   │   elif len(missing_modules) > 0:                                                    │
│   1196 │   │   │   passed_modules = set(list(init_kwargs.keys()) + list(passed_class_obj.keys()  │
│ ❱ 1197 │   │   │   raise ValueError(                                                             │
│   1198 │   │   │   │   f"Pipeline {pipeline_class} expected {expected_modules}, but only {passe  │
│   1199 │   │   │   )                                                                             │
│   1200                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

System Info

• diffusers version: 0.21.1
• Platform: Linux-6.1.21.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
• Python version: 3.11.1
• PyTorch version (GPU?): 2.1.0.dev20230903+cu121 (True)
• Huggingface_hub version: 0.16.4
• Transformers version: 4.31.0
• Accelerate version: 0.20.3
• xFormers version: not installed
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@patrickvonplaten @sayakpaul @williamberman

[patrickvonplaten]

Thanks for the nice issue!

The following works for me:

from diffusers import DiffusionPipeline
import torch

cache_dir = DiffusionPipeline.download("kandinsky-community/kandinsky-2-2-decoder", torch_dtype=torch.float16, variant="fp16", load_connected_pipeline=True)
pipe = DiffusionPipeline.from_pretrained(cache_dir, load_connected_pipeline=True)

where cache_dir is a local path to "/home/patrick/.cache/huggingface/hub/models--kandinsky-community--kandinsky-2-2-decoder/snapshots/44eb212bdc717528a43a52b61beeb2fd98766fe4"

Could you try passing load_connected_pipeline=True? (We should always pass this for UIs I think)

Also what could happen here is that you have an older version of Kandinsky on your local disk, could you maybe try re-downloading it?

Please let me know if this works - if yes, I'll make sure we document load_connected_pipeline better

[vladmandic]

load_connected_pipeline was already set, i just forgot to include in the example above.
i wasn't aware that there was an update to model itself, i just deleted my copy and re-downloaded it - it works fine.