Improve single loading file #4041
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
patrickvonplaten
changed the title
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
| # ) | ||
| config = CLIPTextConfig.from_pretrained(config_name, **config_kwargs) | ||
|
|
||
| with init_empty_weights(): |
There was a problem hiding this comment.
This seems to break downstream if you don't have accelerate and we probably need a if is_accelerate_available(): check
There was a problem hiding this comment.
Great catch! I think we should actually just force the user here to install accelerate since this method only exists for PyTorch anyways and there is no harm in installing accelerate for PyTorch
There was a problem hiding this comment.
Any chance we can avoid forcing an accelerate install ? We ship nightly HF libraries (transformers, diffusers) and then export out via torch-mlir to SHARK so we don't use PyTorch/Accelerate and would like to avoid adding the dependency to our shipping binaries if possible.
There was a problem hiding this comment.
Fixed in: #4132
| @@ -1318,25 +1340,45 @@ def download_from_original_stable_diffusion_ckpt( | |||
| # Convert the UNet2DConditionModel model. | |||
| unet_config = create_unet_diffusers_config(original_config, image_size=image_size) | |||
| unet_config["upcast_attention"] = upcast_attention | |||
| unet = UNet2DConditionModel(**unet_config) | |||
| with init_empty_weights(): | |||
There was a problem hiding this comment.
we probably need to qualify this with if is_accelerate_available(): check if we don't have accelerate
|
It seems that when I use StableDiffusionPipeline to load another model with from_pretrained, and then try to load an SDXL model with StableDiffusionXlPipeline using from_single_file, the program gets stuck for a long time. I have already updated to the latest code. |
|
Can you open a new issue and add a reproducible? :-) |
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
* start improving single file load * Fix more * start improving single file load * Fix sd 2.1 * further improve from_single_file
What does this PR do?
This PR improves the single model loading file so that it now works very well for:
All unnecessary downloads are removed and loading is sped up as much as possible.
StableDiffusionXLPipeline.from_single_file(...)now only takes 2 seconds which is almost as fast as the official diffusers format:StableDiffusionXLPipeline.from_pretrained(...)which takes 0.7 seconds.Fixes