Checkpoint conversion script from Diffusers => Stable Diffusion (CompVis)

[jachiam]

I added a script that converts from the Diffusers save format to the Stable Diffusion checkpoint format.

Some notes: it only handles the UNet, the VAE, and the Text Encoder. Nothing else. No optimization state is preserved.

I think it works. I'm not ten million percent sure. It works locally in that after a conversion, all of the keys for the new checkpoint's state_dict do indeed match a corresponding key in a Stable Diffusion model, I can load it, and an image generated using the SD checkpoint appears basically right. I have not tested this beyond generating two images.

This would solve Issue #672.

Looking for advice on what would constitute a stronger test here / make this a passable script.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[jachiam]

A few folks in the Stable Diffusion discord are also reporting that this works for them.

[jachiam]

Hm, also worth noting it only handles the Stable Diffusion v1-4 architecture, and would not be generally applicable to other architectures.

[hopibel]

Looking for advice on what would constitute a stronger test here

The simplest test would be to do a side-by-side comparison of images from an unconverted CompVis model vs the same model converted to diffusers format and back again using your script, all using the same prompt and seed. They should generate identical images.

[jachiam]

Would we consider it acceptable if I just took a Diffusers model, converted to CompVis format, made an image with the CompVis format, converted back to Diffusers format, converted again to CompVis format, and made a comparison image in CompVis format? TBH I mostly did this to avoid having to download an original CompVis format checkpoint file, because an hour into the first 1.5 hour download the internet cut out and the download had to restart. (Yes, I spent ~8 hours coding a script to avoid a 1.5 hours download.)

[hopibel]

Should be fine imo. The goal is just to show the conversion is correct by showing it's reversible without affecting its output

[patil-suraj]

Thanks a lot for the PR !From a quick look it looks good, will play around a bit and then we can merge.

[patrickvonplaten]

Cool looks good to me!

[patrickvonplaten]

Thanks @jachiam !

[tedliosu]

Thank you so much for the work you've put into making this script, but unfortunately now the script no longer works because the weights under text_encoder, unet, and vae directories are saved using Huggingface's new safetensors file format instead of pytorch's traditional bin file format. Also stable diffusion frontends like AUTOMATIC1111's stable-diffusion-webui now expects a yaml config file for every custom model like the one here, so this script will have to also generate at least one additional file for every custom model generated using dreambooth as well if I'm not mistaken.

Please let us know when you may get around to updating this script so that it works with safetensors files too instead of just the traditional pytorch bin files.

[tedliosu]

Thank you so much for the work you've put into making this script, but unfortunately now the script no longer works because the weights under text_encoder, unet, and vae directories are saved using Huggingface's new safetensors file format instead of pytorch's traditional bin file format. Also stable diffusion frontends like AUTOMATIC1111's stable-diffusion-webui now expects a yaml config file for every custom model like the one here, so this script will have to also generate at least one additional file for every custom model generated using dreambooth as well if I'm not mistaken.

Please let us know when you may get around to updating this script so that it works with safetensors files too instead of just the traditional pytorch bin files.

My utmost apologies; I did not see this commit which has updated this script so that it works with safetensors files as well.