How to replace and embed custom VAEs

Overview

Depending on the VAE format, we will have several embedding possibilities:

• VAE in MLMODELC format
• VAE in BIN format
• VAE in PT or CKPT format
  • Starting from a SafeTensors model
  • Starting from a CKPT model

VAE in MLMODELC format

Simply replace the VAEDecoder.mlmodelc and VAEEncoder.mlmodelc files in your model folder. You can do that with VAEs found in other models, or the ones found here.

↑ Back to top

VAE in BIN format

Convert your model to Diffusers as shown in SD model → Diffusers, and replace the existing VAE in the <YOUR-DIFFUSERS-NAME>/vae folder before converting to MLMODELC. It's also possible to find models already converted as Diffusers, especially on Hugging Face.

↑ Back to top

VAE in PT or CKPT format

Starting from a SafeTensors model

Conversion to CKPT is required.
You can do that either by using online tools such as this one or this one, or do as follows:

git clone https://github.com/diStyApps/Safe-and-Stable-Ckpt2Safetensors-Conversion-Tool-GUI.git

cd Safe-and-Stable-Ckpt2Safetensors-Conversion-Tool-GUI

Change the Python version to 3.8 in the Safe-and-Stable-Ckpt2Safetensors-Conversion-Tool-GUI/conda.yaml file

conda env create -f conda.yaml

conda activate st_gui

python run_app_gui.py

An interface will launch, and you will be able to convert your SafeTensors file to CKPT.

Starting from a CKPT model

1. Download this script

2. Put your CKPT and your VAE in the same folder as the script

3. On line 8 of the replace_vae.py file, change the CKPT and VAE names with yours

4. On line 11, change the name of the future embedded file. For example: <MODEL-NAME-VAE>.ckpt

5. Save the file and close it

6. Activate the Conda environment

   conda activate coreml_stable_diffusion

7. Navigate to the folder where the script is located via cd /<YOUR-PATH> (you can also type cd and then drag the folder into Terminal), and launch the script

   python replace_vae.py

8. A new CKPT file with the VAE embedded will be generated

9. Convert your new CKPT to MLMODELC

↑ Back to top