AttributeError: 'list' object has no attribute '__module__' when calling from_pretrained using file system

[dferguson992]

Discussed in #7610

^{Originally posted by dferguson992 April 8, 2024}
I downloaded the stabilityai/stable-cascade model from HuggingFace and saved it to my file system using the following:

model = StableCascadeCombinedPipeline.from_pretrained(HF_MODEL_ID, 
                                                      variant="bf16",
                                                      torch_dtype=torch.float16)
model_path = "model/"
code_path = "code/"

if not os.path.exists(model_path):
    os.mkdir(model_path)
if not os.path.exists(code_path):
    os.mkdir(code_path)

model.save_pretrained(save_directory=model_path)

My plan is to deploy this to a SageMaker endpoint but am having trouble re-loading the model from the local file system. I keep getting the following error when I call model = StableCascadeCombinedPipeline.from_pretrained("model/", variant="bf16", torch_dtype=torch.float16):

Been at this for several days now, not sure what I'm doing wrong.

[standardAI]

As of my investigation, one can add this (as a fast and temporary action) at the 506. line of src/diffusers/pipelines/pipeline_loading_utils.py:

     not_compiled_module = _unwrap_model(module)
+    if isinstance(not_compiled_module, list): return (not_compiled_module[0], not_compiled_module[1])
     library = not_compiled_module.__module__.split(".")[0]

[dferguson992]

I don't want to modify the source library as a work-around.

Are you suggesting this is a bug of some kind? Or am i using the library incorrectly? The below is the code I'm using to generate this error:

model = StableCascadeCombinedPipeline.from_pretrained("model/", variant="bf16", torch_dtype=torch.float16)

[DN6]

@dferguson992 Taking a look into this.

[DN6]

Hi @dferguson992 So the issue here is that StableCascadeCombinedPipeline is a connected pipeline composed of the StableCascadePriorPipeline and StableCascadeDecoderPipeline.

When using from_pretrained with the model id on the hub, we inspect the the model card on the Hub to get information on whether this model is a connected pipeline, as well as which pipeline to load. By default, calling from_pretrained will save your model to the Hugging Face cache on your machine (which is also where the model card will be saved with the connected pipeline information)

When saving to your local directory with save_pretrained the model card information isn't saved there, leading to the error when trying to load the model again.

A potential solution to the issue.

You can configure the location of the cache by passing in the cache_dir argument to from_pretrained.

import os

import torch
from diffusers import StableCascadeCombinedPipeline

MODEL_ID = "stabilityai/stable-cascade"
CACHE_DIR = os.getenv("CACHE_DIR", "my_cache_dir")

model = StableCascadeCombinedPipeline.from_pretrained(
    MODEL_ID,
    variant="bf16",
    torch_dtype=torch.float16,
    cache_dir=CACHE_DIR,
)

The first time you run this snippet it will download the model to the specified cache directory. If you run it again, it won't download the model again, instead it will load the model from the cache_dir.

[dferguson992]

Hey @DN6 thank you so much for your explanation, it was very helpful.

This creates the "my_cache_dir" as you imply, but my goal is to package the model contents into a zip file, deploy the model onto an endpoint, and have the endpoint unzip the model contents for loading in a script.

The script on my machine unzips the model tarball into a directory called "model/". When I load the model, the script calls

// remote function call executed by my script on the remote host. 
model = StableCascadeCombinedPipeline.from_pretrained(model_dir)

where model_dir is "model/". I've been trying to test this locally as well, still getting the screenshotted issue above. Even when I extend the local test to include the cache_dir, I still get the screenshotted error when I run the tests locally.

// local test, using the local model/ directory to load from and specifying the cache directory
model = StableCascadeCombinedPipeline.from_pretrained("model/", 
                                                      variant="bf16",
                                                      torch_dtype=torch.float16,
                                                      cache_dir="cache_dir")

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[DN6]

Hi @dferguson992 You could zip the entire cache dir and unzip on your endpoint machine/script.

If you're using the HF Cache mechanism, and the models have been saved to the specified cache dir, you would need to pass in the name of the model repo id when using from_pretrained since that's what Diffusers will look for in the cache.

// local test, using the local model/ directory to load from and specifying the cache directory
model = StableCascadeCombinedPipeline.from_pretrained("stabilityai/stable-cascade", 
                                                      variant="bf16",
                                                      torch_dtype=torch.float16,
                                                      cache_dir="cache_dir")

If you need to move folders around, I would recommend saving the prior and decoder pipeline separately since it's a bit tricky to use the combined pipelines without making use of the HF caching mechanism.

What you can try to do is download the model folders for the prior and decoder pipelines, and build the combined pipeline from the components. e.g

Saving locally

from diffusers import (
    StableCascadeCombinedPipeline,
    StableCascadeDecoderPipeline,
    StableCascadePriorPipeline,
)

prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior")
decoder = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade")

prior.save_pretained("my_local_sd_cascade_prior")
decoder.save_pretrained("my_local_sd_cascade_decoder")

Loading

prior = StableCascadePriorPipeline.from_pretrained("my_local_sd_cascade_prior")
decoder = StableCascadeDecoderPipeline.from_pretrained("my_local_sd_cascade_decoder")

StableCascadeCombinedPipeline.from_pretrained(
    tokenizer=decoder.tokenizer,
    text_encoder=decoder.text_encoder,
    decoder=decoder.decoder,
    scheduler=decoder.scheduler,
    vqgan=decoder.vqgan,
    prior_prior=prior.prior,
    prior_text_encoder=prior.text_encoder,
    prior_tokenizer=prior.tokenizer,
    prior_scheduler=prior.scheduler,
    prior_feature_extractor=prior.feature_extractor,
    prior_image_encoder=prior.image_encoder,           
)