From 4a958d0d507d4a0fedcbfe892e5bf774c13400b0 Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Mon, 6 May 2024 15:06:38 -0700 Subject: [PATCH 1/5] files and formats --- docs/source/en/_toctree.yml | 4 +- .../en/using-diffusers/other-formats.md | 237 +++++++++++------- .../en/using-diffusers/using_safetensors.md | 84 ------- 3 files changed, 145 insertions(+), 180 deletions(-) delete mode 100644 docs/source/en/using-diffusers/using_safetensors.md diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index 2f4651ba3417..df6709ae2005 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -29,10 +29,8 @@ title: Load community pipelines and components - local: using-diffusers/schedulers title: Load schedulers and models - - local: using-diffusers/using_safetensors - title: Load safetensors - local: using-diffusers/other-formats - title: Load different Stable Diffusion formats + title: Model files and formats - local: using-diffusers/loading_adapters title: Load adapters - local: using-diffusers/push_to_hub diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index 13efe7854ddc..89d6714b0786 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -10,167 +10,218 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o specific language governing permissions and limitations under the License. --> -# Load different Stable Diffusion formats +# Model files and formats [[open-in-colab]] -Stable Diffusion models are available in different formats depending on the framework they're trained and saved with, and where you download them from. Converting these formats for use in ๐Ÿค— Diffusers allows you to use all the features supported by the library, such as [using different schedulers](schedulers) for inference, [building your custom pipeline](write_own_pipeline), and a variety of techniques and methods for [optimizing inference speed](../optimization/opt_overview). +Diffusion models are saved in various file types and organized in different layouts. Diffusers prefers storing model weights as `.safetensors` files in a multifolder layout, but it also supports loading file types (can be `.safetensors`, `.ckpt`, etc.) from a single file layout which is commonly used in the diffusion ecosystem. - +Each layout has its own benefits and use cases, and this guide will show you how to load the different file types and layouts, and how to convert them. -We highly recommend using the `.safetensors` format because it is more secure than traditional pickled files which are vulnerable and can be exploited to execute any code on your machine (learn more in the [Load safetensors](using_safetensors) guide). +## Files - +Model weights are stored in various file types such as `.safetensors`, `.ckpt`, or `.bin`. PyTorch model weights are typically saved with Python's [pickle](https://docs.python.org/3/library/pickle.html) utility. However, pickle is not secure and pickled files may contain malicious code that can be executed. This vulnerability became an increasing concern as model sharing platforms like the Hugging Face Hub gained popularity. Most of the earlier models are saved this way, as `.ckpt` files. To address this security issue, the [Safetensors](https://hf.co/docs/safetensors) library was developed as a secure alternative to pickle, which saves models as `.safetensors` files. -This guide will show you how to convert other Stable Diffusion formats to be compatible with ๐Ÿค— Diffusers. +### Safetensors -## PyTorch .ckpt +> [!TIP] +> Learn more about the design decisions and why Safetensors is the preferred file format for saving and loading model weights in the [Safetensors audited as really safe and becoming the default](https://blog.eleuther.ai/safetensors-security-audit/) blog post. -The checkpoint - or `.ckpt` - format is commonly used to store and save models. The `.ckpt` file contains the entire model and is typically several GBs in size. While you can load and use a `.ckpt` file directly with the [`~StableDiffusionPipeline.from_single_file`] method, it is generally better to convert the `.ckpt` file to ๐Ÿค— Diffusers so both formats are available. +[Safetensors](https://hf.co/docs/safetensors) is a safe and fast file type for securely storing and loading tensors. Safetensors restricts the header size to limit certain types of attacks, supports lazy loading (useful for distributed setups), and generally faster loading speeds. -There are two options for converting a `.ckpt` file: use a Space to convert the checkpoint or convert the `.ckpt` file with a script. +Make sure you have the [Safetensors](https://hf.co/docs/safetensors) library installed. -### Convert with a Space +```py +!pip install safetensors +``` -The easiest and most convenient way to convert a `.ckpt` file is to use the [SD to Diffusers](https://huggingface.co/spaces/diffusers/sd-to-diffusers) Space. You can follow the instructions on the Space to convert the `.ckpt` file. +Safetensor files are stored as `.safetensors`. Diffusers loads `.safetensors` files by default if they're available and the Safetensors library is installed. There are two ways `.safetensors` files can be organized: -This approach works well for basic models, but it may struggle with more customized models. You'll know the Space failed if it returns an empty pull request or error. In this case, you can try converting the `.ckpt` file with a script. +1. all the model weights may be saved in a single file (check out the [WarriorMama777/OrangeMixs](https://hf.co/WarriorMama777/OrangeMixs/tree/main/Models/AbyssOrangeMix) repository as an example) +2. there may be several separate `.safetensors` files, one for each pipeline component (text encoder, UNet, VAE), organized in subfolders (check out the [runwayml/stable-diffusion-v1-5](https://hf.co/runwayml/stable-diffusion-v1-5/tree/main) repository as an example) -### Convert with a script + + -๐Ÿค— Diffusers provides a [conversion script](https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py) for converting `.ckpt` files. This approach is more reliable than the Space above. +Use the [`~DiffusionPipeline.from_pretrained`] method to load a model with `.safetensors` files stored in multiple folders. -Before you start, make sure you have a local clone of ๐Ÿค— Diffusers to run the script and log in to your Hugging Face account so you can open pull requests and push your converted model to the Hub. +```py +from diffusers import DiffusionPipeline -```bash -huggingface-cli login +pipeline = DiffusionPipeline.from_pretrained( + "runwayml/stable-diffusion-v1-5", + use_safetensors=True +) ``` -To use the script: - -1. Git clone the repository containing the `.ckpt` file you want to convert. For this example, let's convert this [TemporalNet](https://huggingface.co/CiaraRowles/TemporalNet) `.ckpt` file: + + -```bash -git lfs install -git clone https://huggingface.co/CiaraRowles/TemporalNet -``` +Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to load a model with all the weights stored in a single `.safetensors` file. -2. Open a pull request on the repository where you're converting the checkpoint from: +```py +from diffusers import StableDiffusionPipeline -```bash -cd TemporalNet && git fetch origin refs/pr/13:pr/13 -git checkout pr/13 +pipeline = StableDiffusionPipeline.from_single_file( + "https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/Models/AbyssOrangeMix/AbyssOrangeMix.safetensors" +) ``` -3. There are several input arguments to configure in the conversion script, but the most important ones are: + + - - `checkpoint_path`: the path to the `.ckpt` file to convert. - - `original_config_file`: a YAML file defining the configuration of the original architecture. If you can't find this file, try searching for the YAML file in the GitHub repository where you found the `.ckpt` file. - - `dump_path`: the path to the converted model. +#### LoRA files - For example, you can take the `cldm_v15.yaml` file from the [ControlNet](https://github.com/lllyasviel/ControlNet/tree/main/models) repository because the TemporalNet model is a Stable Diffusion v1.5 and ControlNet model. +[LoRA](https://hf.co/docs/peft/conceptual_guides/adapter#low-rank-adaptation-lora) is a lightweight adapter that is fast and easy to train, making them especially popular for generating images in a certain way or style. These adapters are commonly stored in a `.safetensors` file, and are widely popular on model sharing platforms like [civitai](https://civitai.com/). LoRAs are loaded into a base model with the [`~loaders.LoraLoaderMixin.load_lora_weights`] method. -4. Now you can run the script to convert the `.ckpt` file: +```py +from diffusers import StableDiffusionXLPipeline +import torch -```bash -python ../diffusers/scripts/convert_original_stable_diffusion_to_diffusers.py --checkpoint_path temporalnetv3.ckpt --original_config_file cldm_v15.yaml --dump_path ./ --controlnet -``` +# base model +pipeline = StableDiffusionXLPipeline.from_pretrained( + "Lykon/dreamshaper-xl-1-0", torch_dtype=torch.float16, variant="fp16" +).to("cuda") -5. Once the conversion is done, upload your converted model and test out the resulting [pull request](https://huggingface.co/CiaraRowles/TemporalNet/discussions/13)! +# download LoRA weights +!wget https://civitai.com/api/download/models/168776 -O blueprintify.safetensors -```bash -git push origin pr/13:refs/pr/13 +# load LoRA weights +pipeline.load_lora_weights(".", weight_name="blueprintify.safetensors") +prompt = "bl3uprint, a highly detailed blueprint of the empire state building, explaining how to build all parts, many txt, blueprint grid backdrop" +negative_prompt = "lowres, cropped, worst quality, low quality, normal quality, artifacts, signature, watermark, username, blurry, more than one bridge, bad architecture" + +image = pipeline( + prompt=prompt, + negative_prompt=negative_prompt, + generator=torch.manual_seed(0), +).images[0] +image ``` -## Keras .pb or .h5 +
+ +
- +### ckpt -๐Ÿงช This is an experimental feature. Only Stable Diffusion v1 checkpoints are supported by the Convert KerasCV Space at the moment. +> [!WARNINNG] +> Pickled files may be unsafe because they can be exploited to execute malicious code. It is recommended to use `.safetensors` files instead where possible, or convert the weights to `.safetensors` files. - +PyTorch's [torch.save](https://pytorch.org/docs/stable/generated/torch.save.html) function uses Python's [pickle](https://docs.python.org/3/library/pickle.html) utility to serialize and save models. You may see these files saved with the `.ckpt` extension and they contain the entire models weights. Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to directly load a `.ckpt` file. -[KerasCV](https://keras.io/keras_cv/) supports training for [Stable Diffusion](https://github.com/keras-team/keras-cv/blob/master/keras_cv/models/stable_diffusion) v1 and v2. However, it offers limited support for experimenting with Stable Diffusion models for inference and deployment whereas ๐Ÿค— Diffusers has a more complete set of features for this purpose, such as different [noise schedulers](https://huggingface.co/docs/diffusers/using-diffusers/schedulers), [flash attention](https://huggingface.co/docs/diffusers/optimization/xformers), and [other -optimization techniques](https://huggingface.co/docs/diffusers/optimization/fp16). +```py +from diffusers import StableDiffusionPipeline + +pipeline = StableDiffusionPipeline.from_single_file( + "https://huggingface.co/runwayml/stable-diffusion-v1-5/blob/main/v1-5-pruned.ckpt" +) +``` -The [Convert KerasCV](https://huggingface.co/spaces/sayakpaul/convert-kerascv-sd-diffusers) Space converts `.pb` or `.h5` files to PyTorch, and then wraps them in a [`StableDiffusionPipeline`] so it is ready for inference. The converted checkpoint is stored in a repository on the Hugging Face Hub. +## Storage layout -For this example, let's convert the [`sayakpaul/textual-inversion-kerasio`](https://huggingface.co/sayakpaul/textual-inversion-kerasio/tree/main) checkpoint which was trained with Textual Inversion. It uses the special token `` to personalize images with cats. +There are two ways model files are organized, either in multiple folders or in a single file. Diffusers uses the multifolder layout by default where each component file (text encoder, UNet, VAE) is stored in a separate subfolder. Diffusers also supports loading models from a single file where all the components are bundled together. -The Convert KerasCV Space allows you to input the following: +### Multifolder -* Your Hugging Face token. -* Paths to download the UNet and text encoder weights from. Depending on how the model was trained, you don't necessarily need to provide the paths to both the UNet and text encoder. For example, Textual Inversion only requires the embeddings from the text encoder and a text-to-image model only requires the UNet weights. -* Placeholder token is only applicable for textual inversion models. -* The `output_repo_prefix` is the name of the repository where the converted model is stored. +The multifolder layout is the default storage layout for Diffusers. Each component's (text encoder, UNet, VAE) weights are stored in a separate subfolder. The weights can be stored as any file type such as `.safetensors` or `.ckpt`. -Click the **Submit** button to automatically convert the KerasCV checkpoint! Once the checkpoint is successfully converted, you'll see a link to the new repository containing the converted checkpoint. Follow the link to the new repository, and you'll see the Convert KerasCV Space generated a model card with an inference widget to try out the converted model. +
+
+ +
multifolder layout
+
+
+ +
UNet subfolder
+
+
-If you prefer to run inference with code, click on the **Use in Diffusers** button in the upper right corner of the model card to copy and paste the code snippet: +To load from a multifolder layout, use the [`~DiffusionPipeline.from_pretrained`] method. ```py from diffusers import DiffusionPipeline pipeline = DiffusionPipeline.from_pretrained( - "sayakpaul/textual-inversion-cat-kerascv_sd_diffusers_pipeline", use_safetensors=True -) + "stabilityai/stable-diffusion-xl-base-1.0", + torch_dtype=torch.float16, + variant="fp16", + use_safetensors=True, +).to("cuda") ``` -Then, you can generate an image like: +Benefits of using the multifolder layout include: -```py -from diffusers import DiffusionPipeline +1. Faster to load each component file individually or in parallel. +2. Easier to update individual components. For example, once you've downloaded and cached a model, Diffusers only downloads and updates components that have been changed. This saves storage because you don't have to download the entire model again if you only need one component of the model. +3. Replace a component in the model with a newer or better version. -pipeline = DiffusionPipeline.from_pretrained( - "sayakpaul/textual-inversion-cat-kerascv_sd_diffusers_pipeline", use_safetensors=True -) -pipeline.to("cuda") + ```py + from diffusers import DiffusionPipeline, AutoencoderKL -placeholder_token = "" -prompt = f"two {placeholder_token} getting married, photorealistic, high quality" -image = pipeline(prompt, num_inference_steps=50).images[0] -``` + vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16, use_safetensors=True) + pipeline = DiffusionPipeline.from_pretrained( + "stabilityai/stable-diffusion-xl-base-1.0", + vae=vae, + torch_dtype=torch.float16, + variant="fp16", + use_safetensors=True, + ).to("cuda") + ``` + +4. More visibility and information about a model's components which are stored in a [config.json](https://hf.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/unet/config.json) file in each component subfolder. + +### Single file -## A1111 LoRA files +The single file layout stores all the model weights in a single file. All the model components (text encoder, UNet, VAE) weights are kept together instead of separately in subfolders. This single file can be any file type such as `.safetensors` or `.ckpt`. -[Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) (A1111) is a popular web UI for Stable Diffusion that supports model sharing platforms like [Civitai](https://civitai.com/). Models trained with the Low-Rank Adaptation (LoRA) technique are especially popular because they're fast to train and have a much smaller file size than a fully finetuned model. ๐Ÿค— Diffusers supports loading A1111 LoRA checkpoints with [`~loaders.LoraLoaderMixin.load_lora_weights`]: +
+ +
+ +To load from a single file layout, use the [`~loaders.FromSingleFileMixin.from_single_file`] method. ```py -from diffusers import StableDiffusionXLPipeline import torch +from diffusers import StableDiffusionXLPipeline -pipeline = StableDiffusionXLPipeline.from_pretrained( - "Lykon/dreamshaper-xl-1-0", torch_dtype=torch.float16, variant="fp16" +pipeline = StableDiffusionXLPipeline.from_single_file( + "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors", + torch_dtype=torch.float16, + variant="fp16", + use_safetensors=True, ).to("cuda") ``` -Download a LoRA checkpoint from Civitai; this example uses the [Blueprintify SD XL 1.0](https://civitai.com/models/150986/blueprintify-sd-xl-10) checkpoint, but feel free to try out any LoRA checkpoint! +Benefits of using a single file layout include: -```py -# uncomment to download the safetensor weights -#!wget https://civitai.com/api/download/models/168776 -O blueprintify.safetensors -``` +1. Easy compatibility with diffusion interfaces such as [ComfyUI](https://github.com/comfyanonymous/ComfyUI) or [Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) which commonly use a single file layout. +2. Easier to manage (download and share) a single file. -Load the LoRA checkpoint into the pipeline with the [`~loaders.LoraLoaderMixin.load_lora_weights`] method: +## Convert layout and files -```py -pipeline.load_lora_weights(".", weight_name="blueprintify.safetensors") +Diffusers provides many scripts and methods to convert storage layouts and file types to enable broader support across the diffusion ecosystem. + +Take a look at the [diffusers/scripts](https://github.com/huggingface/diffusers/tree/main/scripts) collection to find a script that fits your conversion needs. + +> [!TIP] +> Scripts that have `to_diffusers` appended at the end mean they convert a model to the multifolder format used by Diffusers. Each script has their own specific set of arguments to configure the conversion, so make sure you check what type of arguments are available! + +For example, to convert a Stable Diffusion XL model stored in Diffusers multifolder format to a single file type, run the [convert_diffusers_to_original_sdxl.py](https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py) script. Provide the path to the model to convert, and the path to save the converted model to. You can optionally specify whether you want to save the model as a `.safetensors` file and whether to save the model in half-precision. + +```bash +python convert_diffusers_to_original_sdxl.py --model_path path/to/model/to/convert --checkpoint_path path/to/save/model/to --use_safetensors ``` -Now you can use the pipeline to generate images: +You can also save a model to the multifolder format with the [`~DiffusionPipeline.save_pretrained`] method. This creates a directory for you if it doesn't already exist, and it also saves the files as a `.safetensors` file by default. ```py -prompt = "bl3uprint, a highly detailed blueprint of the empire state building, explaining how to build all parts, many txt, blueprint grid backdrop" -negative_prompt = "lowres, cropped, worst quality, low quality, normal quality, artifacts, signature, watermark, username, blurry, more than one bridge, bad architecture" +from diffusers import StableDiffusionXLPipeline -image = pipeline( - prompt=prompt, - negative_prompt=negative_prompt, - generator=torch.manual_seed(0), -).images[0] -image +pipeline = StableDiffusionXLPipeline.from_single_file( + "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors", +) +pipeline.save_pretrained() ``` -
- -
+Lastly, there are also Spaces, such as [SD To Diffusers](https://hf.co/spaces/diffusers/sd-to-diffusers) and [SD-XL To Diffusers](https://hf.co/spaces/diffusers/sdxl-to-diffusers), that provide a more user-friendly interface for converting models to the multifolder format. This is the easiest and most convenient option for converting layouts, and it'll open a PR on your model repository with the converted files. However, this option is not as reliable as running a script, and the Space may fail for more complicated models. diff --git a/docs/source/en/using-diffusers/using_safetensors.md b/docs/source/en/using-diffusers/using_safetensors.md deleted file mode 100644 index a9ab7b8ae431..000000000000 --- a/docs/source/en/using-diffusers/using_safetensors.md +++ /dev/null @@ -1,84 +0,0 @@ - - -# Load safetensors - -[[open-in-colab]] - -[safetensors](https://github.com/huggingface/safetensors) is a safe and fast file format for storing and loading tensors. Typically, PyTorch model weights are saved or *pickled* into a `.bin` file with Python's [`pickle`](https://docs.python.org/3/library/pickle.html) utility. However, `pickle` is not secure and pickled files may contain malicious code that can be executed. safetensors is a secure alternative to `pickle`, making it ideal for sharing model weights. - -This guide will show you how you load `.safetensor` files, and how to convert Stable Diffusion model weights stored in other formats to `.safetensor`. Before you start, make sure you have safetensors installed: - -```py -# uncomment to install the necessary libraries in Colab -#!pip install safetensors -``` - -If you look at the [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main) repository, you'll see weights inside the `text_encoder`, `unet` and `vae` subfolders are stored in the `.safetensors` format. By default, ๐Ÿค— Diffusers automatically loads these `.safetensors` files from their subfolders if they're available in the model repository. - -For more explicit control, you can optionally set `use_safetensors=True` (if `safetensors` is not installed, you'll get an error message asking you to install it): - -```py -from diffusers import DiffusionPipeline - -pipeline = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", use_safetensors=True) -``` - -However, model weights are not necessarily stored in separate subfolders like in the example above. Sometimes, all the weights are stored in a single `.safetensors` file. In this case, if the weights are Stable Diffusion weights, you can load the file directly with the [`~diffusers.loaders.FromSingleFileMixin.from_single_file`] method: - -```py -from diffusers import StableDiffusionPipeline - -pipeline = StableDiffusionPipeline.from_single_file( - "https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/Models/AbyssOrangeMix/AbyssOrangeMix.safetensors" -) -``` - -## Convert to safetensors - -Not all weights on the Hub are available in the `.safetensors` format, and you may encounter weights stored as `.bin`. In this case, use the [Convert Space](https://huggingface.co/spaces/diffusers/convert) to convert the weights to `.safetensors`. The Convert Space downloads the pickled weights, converts them, and opens a Pull Request to upload the newly converted `.safetensors` file on the Hub. This way, if there is any malicious code contained in the pickled files, they're uploaded to the Hub - which has a [security scanner](https://huggingface.co/docs/hub/security-pickle#hubs-security-scanner) to detect unsafe files and suspicious pickle imports - instead of your computer. - -You can use the model with the new `.safetensors` weights by specifying the reference to the Pull Request in the `revision` parameter (you can also test it in this [Check PR](https://huggingface.co/spaces/diffusers/check_pr) Space on the Hub), for example `refs/pr/22`: - -```py -from diffusers import DiffusionPipeline - -pipeline = DiffusionPipeline.from_pretrained( - "stabilityai/stable-diffusion-2-1", revision="refs/pr/22", use_safetensors=True -) -``` - -## Why use safetensors? - -There are several reasons for using safetensors: - -- Safety is the number one reason for using safetensors. As open-source and model distribution grows, it is important to be able to trust the model weights you downloaded don't contain any malicious code. The current size of the header in safetensors prevents parsing extremely large JSON files. -- Loading speed between switching models is another reason to use safetensors, which performs zero-copy of the tensors. It is especially fast compared to `pickle` if you're loading the weights to CPU (the default case), and just as fast if not faster when directly loading the weights to GPU. You'll only notice the performance difference if the model is already loaded, and not if you're downloading the weights or loading the model for the first time. - - The time it takes to load the entire pipeline: - - ```py - from diffusers import StableDiffusionPipeline - - pipeline = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", use_safetensors=True) - "Loaded in safetensors 0:00:02.033658" - "Loaded in PyTorch 0:00:02.663379" - ``` - - But the actual time it takes to load 500MB of the model weights is only: - - ```bash - safetensors: 3.4873ms - PyTorch: 172.7537ms - ``` - -- Lazy loading is also supported in safetensors, which is useful in distributed settings to only load some of the tensors. This format allowed the [BLOOM](https://huggingface.co/bigscience/bloom) model to be loaded in 45 seconds on 8 GPUs instead of 10 minutes with regular PyTorch weights. From 8d8ae1e073baa6d4baf9ebd00ee383cbf8c2a12e Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Mon, 6 May 2024 15:25:45 -0700 Subject: [PATCH 2/5] fix callout --- docs/source/en/using-diffusers/other-formats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index 89d6714b0786..f6cc1c2a420c 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -105,7 +105,7 @@ image ### ckpt -> [!WARNINNG] +> [!WARNING] > Pickled files may be unsafe because they can be exploited to execute malicious code. It is recommended to use `.safetensors` files instead where possible, or convert the weights to `.safetensors` files. PyTorch's [torch.save](https://pytorch.org/docs/stable/generated/torch.save.html) function uses Python's [pickle](https://docs.python.org/3/library/pickle.html) utility to serialize and save models. You may see these files saved with the `.ckpt` extension and they contain the entire models weights. Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to directly load a `.ckpt` file. From 5b5ecabdee2c7a27956b2e8266e25ec55dd58005 Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Wed, 15 May 2024 12:51:48 -0700 Subject: [PATCH 3/5] feedback --- docs/source/en/_toctree.yml | 2 +- .../en/using-diffusers/other-formats.md | 54 +++++++++++-------- 2 files changed, 33 insertions(+), 23 deletions(-) diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index df6709ae2005..828a79f63cfd 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -30,7 +30,7 @@ - local: using-diffusers/schedulers title: Load schedulers and models - local: using-diffusers/other-formats - title: Model files and formats + title: Model files and layouts - local: using-diffusers/loading_adapters title: Load adapters - local: using-diffusers/push_to_hub diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index f6cc1c2a420c..6d4450fdd33d 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -10,24 +10,24 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o specific language governing permissions and limitations under the License. --> -# Model files and formats +# Model files and layouts [[open-in-colab]] -Diffusion models are saved in various file types and organized in different layouts. Diffusers prefers storing model weights as `.safetensors` files in a multifolder layout, but it also supports loading file types (can be `.safetensors`, `.ckpt`, etc.) from a single file layout which is commonly used in the diffusion ecosystem. +Diffusion models are saved in various file types and organized in different layouts. Diffusers stores model weights as safetensors files in a multifolder layout and it also supports loading files (like safetensors and ckpt files) from a single file layout which is commonly used in the diffusion ecosystem. -Each layout has its own benefits and use cases, and this guide will show you how to load the different file types and layouts, and how to convert them. +Each layout has its own benefits and use cases, and this guide will show you how to load the different files and layouts, and how to convert them. ## Files -Model weights are stored in various file types such as `.safetensors`, `.ckpt`, or `.bin`. PyTorch model weights are typically saved with Python's [pickle](https://docs.python.org/3/library/pickle.html) utility. However, pickle is not secure and pickled files may contain malicious code that can be executed. This vulnerability became an increasing concern as model sharing platforms like the Hugging Face Hub gained popularity. Most of the earlier models are saved this way, as `.ckpt` files. To address this security issue, the [Safetensors](https://hf.co/docs/safetensors) library was developed as a secure alternative to pickle, which saves models as `.safetensors` files. +PyTorch model weights are typically saved with Python's [pickle](https://docs.python.org/3/library/pickle.html) utility as ckpt or bin files. However, pickle is not secure and pickled files may contain malicious code that can be executed. This vulnerability is a serious concern given the popularity of model sharing. To address this security issue, the [Safetensors](https://hf.co/docs/safetensors) library was developed as a secure alternative to pickle, which saves models as safetensors files. ### Safetensors > [!TIP] > Learn more about the design decisions and why Safetensors is the preferred file format for saving and loading model weights in the [Safetensors audited as really safe and becoming the default](https://blog.eleuther.ai/safetensors-security-audit/) blog post. -[Safetensors](https://hf.co/docs/safetensors) is a safe and fast file type for securely storing and loading tensors. Safetensors restricts the header size to limit certain types of attacks, supports lazy loading (useful for distributed setups), and generally faster loading speeds. +[Safetensors](https://hf.co/docs/safetensors) is a safe and fast file format for securely storing and loading tensors. Safetensors restricts the header size to limit certain types of attacks, supports lazy loading (useful for distributed setups), and has generally faster loading speeds. Make sure you have the [Safetensors](https://hf.co/docs/safetensors) library installed. @@ -35,15 +35,15 @@ Make sure you have the [Safetensors](https://hf.co/docs/safetensors) library ins !pip install safetensors ``` -Safetensor files are stored as `.safetensors`. Diffusers loads `.safetensors` files by default if they're available and the Safetensors library is installed. There are two ways `.safetensors` files can be organized: +Safetensor stores weights in a safetensors file. Diffusers loads safetensors files by default if they're available and the Safetensors library is installed. There are two ways safetensors files can be organized: 1. all the model weights may be saved in a single file (check out the [WarriorMama777/OrangeMixs](https://hf.co/WarriorMama777/OrangeMixs/tree/main/Models/AbyssOrangeMix) repository as an example) -2. there may be several separate `.safetensors` files, one for each pipeline component (text encoder, UNet, VAE), organized in subfolders (check out the [runwayml/stable-diffusion-v1-5](https://hf.co/runwayml/stable-diffusion-v1-5/tree/main) repository as an example) +2. there may be several separate safetensors files, one for each pipeline component (text encoder, UNet, VAE), organized in subfolders (check out the [runwayml/stable-diffusion-v1-5](https://hf.co/runwayml/stable-diffusion-v1-5/tree/main) repository as an example) -Use the [`~DiffusionPipeline.from_pretrained`] method to load a model with `.safetensors` files stored in multiple folders. +Use the [`~DiffusionPipeline.from_pretrained`] method to load a model with safetensors files stored in multiple folders. ```py from diffusers import DiffusionPipeline @@ -57,7 +57,7 @@ pipeline = DiffusionPipeline.from_pretrained( -Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to load a model with all the weights stored in a single `.safetensors` file. +Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to load a model with all the weights stored in a single safetensors file. ```py from diffusers import StableDiffusionPipeline @@ -72,7 +72,9 @@ pipeline = StableDiffusionPipeline.from_single_file( #### LoRA files -[LoRA](https://hf.co/docs/peft/conceptual_guides/adapter#low-rank-adaptation-lora) is a lightweight adapter that is fast and easy to train, making them especially popular for generating images in a certain way or style. These adapters are commonly stored in a `.safetensors` file, and are widely popular on model sharing platforms like [civitai](https://civitai.com/). LoRAs are loaded into a base model with the [`~loaders.LoraLoaderMixin.load_lora_weights`] method. +[LoRA](https://hf.co/docs/peft/conceptual_guides/adapter#low-rank-adaptation-lora) is a lightweight adapter that is fast and easy to train, making them especially popular for generating images in a certain way or style. These adapters are commonly stored in a safetensors file, and are widely popular on model sharing platforms like [civitai](https://civitai.com/). + +LoRAs are loaded into a base model with the [`~loaders.LoraLoaderMixin.load_lora_weights`] method. ```py from diffusers import StableDiffusionXLPipeline @@ -106,9 +108,11 @@ image ### ckpt > [!WARNING] -> Pickled files may be unsafe because they can be exploited to execute malicious code. It is recommended to use `.safetensors` files instead where possible, or convert the weights to `.safetensors` files. +> Pickled files may be unsafe because they can be exploited to execute malicious code. It is recommended to use safetensors files instead where possible, or convert the weights to safetensors files. + +PyTorch's [torch.save](https://pytorch.org/docs/stable/generated/torch.save.html) function uses Python's [pickle](https://docs.python.org/3/library/pickle.html) utility to serialize and save models. These files are saved as a ckpt file and they contain the entire model's weights. -PyTorch's [torch.save](https://pytorch.org/docs/stable/generated/torch.save.html) function uses Python's [pickle](https://docs.python.org/3/library/pickle.html) utility to serialize and save models. You may see these files saved with the `.ckpt` extension and they contain the entire models weights. Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to directly load a `.ckpt` file. +Use the [`~loaders.FromSingleFileMixin.from_single_file`] method to directly load a ckpt file. ```py from diffusers import StableDiffusionPipeline @@ -120,11 +124,11 @@ pipeline = StableDiffusionPipeline.from_single_file( ## Storage layout -There are two ways model files are organized, either in multiple folders or in a single file. Diffusers uses the multifolder layout by default where each component file (text encoder, UNet, VAE) is stored in a separate subfolder. Diffusers also supports loading models from a single file where all the components are bundled together. +There are two ways model files are organized, either in a multifolder layout or in a single file layout. Diffusers uses the multifolder layout by default where each component file (text encoder, UNet, VAE) is stored in a separate subfolder. Diffusers also supports loading models from a single file layout where all the components are bundled together. ### Multifolder -The multifolder layout is the default storage layout for Diffusers. Each component's (text encoder, UNet, VAE) weights are stored in a separate subfolder. The weights can be stored as any file type such as `.safetensors` or `.ckpt`. +The multifolder layout is the default storage layout for Diffusers. Each component's (text encoder, UNet, VAE) weights are stored in a separate subfolder. The weights can be stored as safetensors or ckpt files.
@@ -153,8 +157,14 @@ pipeline = DiffusionPipeline.from_pretrained( Benefits of using the multifolder layout include: 1. Faster to load each component file individually or in parallel. -2. Easier to update individual components. For example, once you've downloaded and cached a model, Diffusers only downloads and updates components that have been changed. This saves storage because you don't have to download the entire model again if you only need one component of the model. -3. Replace a component in the model with a newer or better version. +2. Reduced memory usage because you only load the components you need. For example, models like [SDXL Turbo](https://hf.co/stabilityai/sdxl-turbo), [SDXL Lightning](https://hf.co/ByteDance/SDXL-Lightning), and [Hyper-SD](https://hf.co/ByteDance/Hyper-SD) have the same components except for the UNet. You can reuse their shared components with the [`~DiffusionPipeline.from_pipe`] method without consuming any additional memory (take a look at the [Reuse a pipeline](./loading#reuse-a-pipeline) guide) and only load the UNet. This way, you don't need to download redundant components and unnecessarily use more memory. + + ```py + + ``` + +3. Reduced storage requirements because if a component, such as the SDXL [VAE](https://hf.co/madebyollin/sdxl-vae-fp16-fix), is shared across multiple models, you only need to download and store a single copy of it instead of downloading and storing it multiple times. For 10 SDXL models, this can save ~3.5GB of storage. The storage savings is even greater for newer models like PixArt Sigma, where the [text encoder](https://hf.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS/tree/main/text_encoder) alone is ~19GB! +4. Flexibility to replace a component in the model with a newer or better version. ```py from diffusers import DiffusionPipeline, AutoencoderKL @@ -169,11 +179,11 @@ Benefits of using the multifolder layout include: ).to("cuda") ``` -4. More visibility and information about a model's components which are stored in a [config.json](https://hf.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/unet/config.json) file in each component subfolder. +5. More visibility and information about a model's components, which are stored in a [config.json](https://hf.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/unet/config.json) file in each component subfolder. ### Single file -The single file layout stores all the model weights in a single file. All the model components (text encoder, UNet, VAE) weights are kept together instead of separately in subfolders. This single file can be any file type such as `.safetensors` or `.ckpt`. +The single file layout stores all the model weights in a single file. All the model components (text encoder, UNet, VAE) weights are kept together instead of separately in subfolders. This can be a safetensors or ckpt file.
@@ -200,20 +210,20 @@ Benefits of using a single file layout include: ## Convert layout and files -Diffusers provides many scripts and methods to convert storage layouts and file types to enable broader support across the diffusion ecosystem. +Diffusers provides many scripts and methods to convert storage layouts and file formats to enable broader support across the diffusion ecosystem. Take a look at the [diffusers/scripts](https://github.com/huggingface/diffusers/tree/main/scripts) collection to find a script that fits your conversion needs. > [!TIP] -> Scripts that have `to_diffusers` appended at the end mean they convert a model to the multifolder format used by Diffusers. Each script has their own specific set of arguments to configure the conversion, so make sure you check what type of arguments are available! +> Scripts that have "`to_diffusers`" appended at the end mean they convert a model to the multifolder format used by Diffusers. Each script has their own specific set of arguments for configuring the conversion, so make sure you check what arguments are available! -For example, to convert a Stable Diffusion XL model stored in Diffusers multifolder format to a single file type, run the [convert_diffusers_to_original_sdxl.py](https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py) script. Provide the path to the model to convert, and the path to save the converted model to. You can optionally specify whether you want to save the model as a `.safetensors` file and whether to save the model in half-precision. +For example, to convert a Stable Diffusion XL model stored in Diffusers multifolder format to a single file type, run the [convert_diffusers_to_original_sdxl.py](https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py) script. Provide the path to the model to convert, and the path to save the converted model to. You can optionally specify whether you want to save the model as a safetensors file and whether to save the model in half-precision. ```bash python convert_diffusers_to_original_sdxl.py --model_path path/to/model/to/convert --checkpoint_path path/to/save/model/to --use_safetensors ``` -You can also save a model to the multifolder format with the [`~DiffusionPipeline.save_pretrained`] method. This creates a directory for you if it doesn't already exist, and it also saves the files as a `.safetensors` file by default. +You can also save a model to the multifolder format with the [`~DiffusionPipeline.save_pretrained`] method. This creates a directory for you if it doesn't already exist, and it also saves the files as a safetensors file by default. ```py from diffusers import StableDiffusionXLPipeline From 5f270484f843c5849e4bc4d049e127c1ab48a4e1 Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Wed, 15 May 2024 14:55:52 -0700 Subject: [PATCH 4/5] code sample --- .../en/using-diffusers/other-formats.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index 6d4450fdd33d..3d0d091cf6c6 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -160,7 +160,39 @@ Benefits of using the multifolder layout include: 2. Reduced memory usage because you only load the components you need. For example, models like [SDXL Turbo](https://hf.co/stabilityai/sdxl-turbo), [SDXL Lightning](https://hf.co/ByteDance/SDXL-Lightning), and [Hyper-SD](https://hf.co/ByteDance/Hyper-SD) have the same components except for the UNet. You can reuse their shared components with the [`~DiffusionPipeline.from_pipe`] method without consuming any additional memory (take a look at the [Reuse a pipeline](./loading#reuse-a-pipeline) guide) and only load the UNet. This way, you don't need to download redundant components and unnecessarily use more memory. ```py + import torch + from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel, EulerDiscreteScheduler + # download one model + sdxl_pipeline = StableDiffusionXLPipeline.from_pretrained( + "stabilityai/stable-diffusion-xl-base-1.0", + torch_dtype=torch.float16, + variant="fp16", + use_safetensors=True, + ).to("cuda") + + # switch UNet for another model + unet = UNet2DConditionModel.from_pretrained( + "stabilityai/sdxl-turbo", + subfolder="unet", + torch_dtype=torch.float16, + variant="fp16", + use_safetensors=True + ) + # reuse all the same components in new model except for the UNet + turbo_pipeline = StableDiffusionXLPipeline.from_pipe( + sdxl_pipeline, unet=unet, + ).to("cuda") + turbo_pipeline.scheduler = EulerDiscreteScheduler.from_config( + turbo_pipeline.scheduler.config, + timestep+spacing="trailing" + ) + image = turbo_pipeline( + "an astronaut riding a unicorn on mars", + num_inference_steps=1, + guidance_scale=0.0, + ).images[0] + image ``` 3. Reduced storage requirements because if a component, such as the SDXL [VAE](https://hf.co/madebyollin/sdxl-vae-fp16-fix), is shared across multiple models, you only need to download and store a single copy of it instead of downloading and storing it multiple times. For 10 SDXL models, this can save ~3.5GB of storage. The storage savings is even greater for newer models like PixArt Sigma, where the [text encoder](https://hf.co/PixArt-alpha/PixArt-Sigma-XL-2-1024-MS/tree/main/text_encoder) alone is ~19GB! From 8535d50d297a5a1d69a8bb2bfbbc29bdf8e1260a Mon Sep 17 00:00:00 2001 From: Steven Liu Date: Thu, 16 May 2024 10:29:06 -0700 Subject: [PATCH 5/5] feedback --- .../en/using-diffusers/other-formats.md | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/source/en/using-diffusers/other-formats.md b/docs/source/en/using-diffusers/other-formats.md index 3d0d091cf6c6..c0f2a76b459d 100644 --- a/docs/source/en/using-diffusers/other-formats.md +++ b/docs/source/en/using-diffusers/other-formats.md @@ -14,7 +14,7 @@ specific language governing permissions and limitations under the License. [[open-in-colab]] -Diffusion models are saved in various file types and organized in different layouts. Diffusers stores model weights as safetensors files in a multifolder layout and it also supports loading files (like safetensors and ckpt files) from a single file layout which is commonly used in the diffusion ecosystem. +Diffusion models are saved in various file types and organized in different layouts. Diffusers stores model weights as safetensors files in *Diffusers-multifolder* layout and it also supports loading files (like safetensors and ckpt files) from a *single-file* layout which is commonly used in the diffusion ecosystem. Each layout has its own benefits and use cases, and this guide will show you how to load the different files and layouts, and how to convert them. @@ -22,10 +22,10 @@ Each layout has its own benefits and use cases, and this guide will show you how PyTorch model weights are typically saved with Python's [pickle](https://docs.python.org/3/library/pickle.html) utility as ckpt or bin files. However, pickle is not secure and pickled files may contain malicious code that can be executed. This vulnerability is a serious concern given the popularity of model sharing. To address this security issue, the [Safetensors](https://hf.co/docs/safetensors) library was developed as a secure alternative to pickle, which saves models as safetensors files. -### Safetensors +### safetensors > [!TIP] -> Learn more about the design decisions and why Safetensors is the preferred file format for saving and loading model weights in the [Safetensors audited as really safe and becoming the default](https://blog.eleuther.ai/safetensors-security-audit/) blog post. +> Learn more about the design decisions and why safetensor files are preferred for saving and loading model weights in the [Safetensors audited as really safe and becoming the default](https://blog.eleuther.ai/safetensors-security-audit/) blog post. [Safetensors](https://hf.co/docs/safetensors) is a safe and fast file format for securely storing and loading tensors. Safetensors restricts the header size to limit certain types of attacks, supports lazy loading (useful for distributed setups), and has generally faster loading speeds. @@ -35,10 +35,10 @@ Make sure you have the [Safetensors](https://hf.co/docs/safetensors) library ins !pip install safetensors ``` -Safetensor stores weights in a safetensors file. Diffusers loads safetensors files by default if they're available and the Safetensors library is installed. There are two ways safetensors files can be organized: +Safetensors stores weights in a safetensors file. Diffusers loads safetensors files by default if they're available and the Safetensors library is installed. There are two ways safetensors files can be organized: -1. all the model weights may be saved in a single file (check out the [WarriorMama777/OrangeMixs](https://hf.co/WarriorMama777/OrangeMixs/tree/main/Models/AbyssOrangeMix) repository as an example) -2. there may be several separate safetensors files, one for each pipeline component (text encoder, UNet, VAE), organized in subfolders (check out the [runwayml/stable-diffusion-v1-5](https://hf.co/runwayml/stable-diffusion-v1-5/tree/main) repository as an example) +1. Diffusers-multifolder layout: there may be several separate safetensors files, one for each pipeline component (text encoder, UNet, VAE), organized in subfolders (check out the [runwayml/stable-diffusion-v1-5](https://hf.co/runwayml/stable-diffusion-v1-5/tree/main) repository as an example) +2. single-file layout: all the model weights may be saved in a single file (check out the [WarriorMama777/OrangeMixs](https://hf.co/WarriorMama777/OrangeMixs/tree/main/Models/AbyssOrangeMix) repository as an example) @@ -124,11 +124,11 @@ pipeline = StableDiffusionPipeline.from_single_file( ## Storage layout -There are two ways model files are organized, either in a multifolder layout or in a single file layout. Diffusers uses the multifolder layout by default where each component file (text encoder, UNet, VAE) is stored in a separate subfolder. Diffusers also supports loading models from a single file layout where all the components are bundled together. +There are two ways model files are organized, either in a Diffusers-multifolder layout or in a single-file layout. The Diffusers-multifolder layout is the default, and each component file (text encoder, UNet, VAE) is stored in a separate subfolder. Diffusers also supports loading models from a single-file layout where all the components are bundled together. -### Multifolder +### Diffusers-multifolder -The multifolder layout is the default storage layout for Diffusers. Each component's (text encoder, UNet, VAE) weights are stored in a separate subfolder. The weights can be stored as safetensors or ckpt files. +The Diffusers-multifolder layout is the default storage layout for Diffusers. Each component's (text encoder, UNet, VAE) weights are stored in a separate subfolder. The weights can be stored as safetensors or ckpt files.
@@ -141,7 +141,7 @@ The multifolder layout is the default storage layout for Diffusers. Each compone
-To load from a multifolder layout, use the [`~DiffusionPipeline.from_pretrained`] method. +To load from Diffusers-multifolder layout, use the [`~DiffusionPipeline.from_pretrained`] method. ```py from diffusers import DiffusionPipeline @@ -154,7 +154,7 @@ pipeline = DiffusionPipeline.from_pretrained( ).to("cuda") ``` -Benefits of using the multifolder layout include: +Benefits of using the Diffusers-multifolder layout include: 1. Faster to load each component file individually or in parallel. 2. Reduced memory usage because you only load the components you need. For example, models like [SDXL Turbo](https://hf.co/stabilityai/sdxl-turbo), [SDXL Lightning](https://hf.co/ByteDance/SDXL-Lightning), and [Hyper-SD](https://hf.co/ByteDance/Hyper-SD) have the same components except for the UNet. You can reuse their shared components with the [`~DiffusionPipeline.from_pipe`] method without consuming any additional memory (take a look at the [Reuse a pipeline](./loading#reuse-a-pipeline) guide) and only load the UNet. This way, you don't need to download redundant components and unnecessarily use more memory. @@ -213,15 +213,15 @@ Benefits of using the multifolder layout include: 5. More visibility and information about a model's components, which are stored in a [config.json](https://hf.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/unet/config.json) file in each component subfolder. -### Single file +### Single-file -The single file layout stores all the model weights in a single file. All the model components (text encoder, UNet, VAE) weights are kept together instead of separately in subfolders. This can be a safetensors or ckpt file. +The single-file layout stores all the model weights in a single file. All the model components (text encoder, UNet, VAE) weights are kept together instead of separately in subfolders. This can be a safetensors or ckpt file.
-To load from a single file layout, use the [`~loaders.FromSingleFileMixin.from_single_file`] method. +To load from a single-file layout, use the [`~loaders.FromSingleFileMixin.from_single_file`] method. ```py import torch @@ -235,9 +235,9 @@ pipeline = StableDiffusionXLPipeline.from_single_file( ).to("cuda") ``` -Benefits of using a single file layout include: +Benefits of using a single-file layout include: -1. Easy compatibility with diffusion interfaces such as [ComfyUI](https://github.com/comfyanonymous/ComfyUI) or [Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) which commonly use a single file layout. +1. Easy compatibility with diffusion interfaces such as [ComfyUI](https://github.com/comfyanonymous/ComfyUI) or [Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) which commonly use a single-file layout. 2. Easier to manage (download and share) a single file. ## Convert layout and files @@ -247,15 +247,15 @@ Diffusers provides many scripts and methods to convert storage layouts and file Take a look at the [diffusers/scripts](https://github.com/huggingface/diffusers/tree/main/scripts) collection to find a script that fits your conversion needs. > [!TIP] -> Scripts that have "`to_diffusers`" appended at the end mean they convert a model to the multifolder format used by Diffusers. Each script has their own specific set of arguments for configuring the conversion, so make sure you check what arguments are available! +> Scripts that have "`to_diffusers`" appended at the end mean they convert a model to the Diffusers-multifolder layout. Each script has their own specific set of arguments for configuring the conversion, so make sure you check what arguments are available! -For example, to convert a Stable Diffusion XL model stored in Diffusers multifolder format to a single file type, run the [convert_diffusers_to_original_sdxl.py](https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py) script. Provide the path to the model to convert, and the path to save the converted model to. You can optionally specify whether you want to save the model as a safetensors file and whether to save the model in half-precision. +For example, to convert a Stable Diffusion XL model stored in Diffusers-multifolder layout to a single-file layout, run the [convert_diffusers_to_original_sdxl.py](https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_to_original_sdxl.py) script. Provide the path to the model to convert, and the path to save the converted model to. You can optionally specify whether you want to save the model as a safetensors file and whether to save the model in half-precision. ```bash python convert_diffusers_to_original_sdxl.py --model_path path/to/model/to/convert --checkpoint_path path/to/save/model/to --use_safetensors ``` -You can also save a model to the multifolder format with the [`~DiffusionPipeline.save_pretrained`] method. This creates a directory for you if it doesn't already exist, and it also saves the files as a safetensors file by default. +You can also save a model to Diffusers-multifolder layout with the [`~DiffusionPipeline.save_pretrained`] method. This creates a directory for you if it doesn't already exist, and it also saves the files as a safetensors file by default. ```py from diffusers import StableDiffusionXLPipeline @@ -266,4 +266,4 @@ pipeline = StableDiffusionXLPipeline.from_single_file( pipeline.save_pretrained() ``` -Lastly, there are also Spaces, such as [SD To Diffusers](https://hf.co/spaces/diffusers/sd-to-diffusers) and [SD-XL To Diffusers](https://hf.co/spaces/diffusers/sdxl-to-diffusers), that provide a more user-friendly interface for converting models to the multifolder format. This is the easiest and most convenient option for converting layouts, and it'll open a PR on your model repository with the converted files. However, this option is not as reliable as running a script, and the Space may fail for more complicated models. +Lastly, there are also Spaces, such as [SD To Diffusers](https://hf.co/spaces/diffusers/sd-to-diffusers) and [SD-XL To Diffusers](https://hf.co/spaces/diffusers/sdxl-to-diffusers), that provide a more user-friendly interface for converting models to Diffusers-multifolder layout. This is the easiest and most convenient option for converting layouts, and it'll open a PR on your model repository with the converted files. However, this option is not as reliable as running a script, and the Space may fail for more complicated models.