chore(docs): explain custom VAE

ssube · Feb 17, 2023 · 03a579a · 03a579a
1 parent d42de16
commit 03a579a
Show file tree

Hide file tree

Showing 3 changed files with 64 additions and 8 deletions.
diff --git a/README.md b/README.md
@@ -24,10 +24,12 @@ Please [see the User Guide](https://github.com/ssube/onnx-web/blob/main/docs/use
 - web app to generate and view images
   - [hosted on Github Pages](https://ssube.github.io/onnx-web), from your CDN, or locally
   - keeps your recent image history as you change tabs
-- supports many common pipelines
-  - txt2img mode
-  - img2img mode
-  - inpainting mode
+- supports many `diffusers` pipelines
+  - txt2img
+  - img2img
+  - inpainting
+  - upscale (with ONNX acceleration)
+- blending mode for combining images
 - supports upscaling and face correction
   - upscaling with Real ESRGAN or Stable Diffusion
   - face correction with CodeFormer or GFPGAN

diff --git a/docs/server-admin.md b/docs/server-admin.md
@@ -40,6 +40,8 @@ These extra images can be helpful when debugging inpainting, especially poorly b
 
 ### Environment Variables
 
+Paths:
+
 - `ONNX_WEB_BUNDLE_PATH`
   - path where client bundle files can be found
 - `ONNX_WEB_MODEL_PATH`
@@ -48,15 +50,26 @@ These extra images can be helpful when debugging inpainting, especially poorly b
   - path where output images should be saved
 - `ONNX_WEB_PARAMS_PATH`
   - path to the directory where the `params.json` file can be found
+
+Others:
+
+- `ONNX_WEB_ANY_PLATFORM`
+  - whether or not to include the `any` option in the platform list
+- `ONNX_WEB_BLOCK_PLATFORMS`
+  - comma-delimited list of platforms that should not be presented to users
+  - further filters the list of available platforms returned by ONNX runtime
+  - can be used to prevent CPU generation on shared servers
+- `ONNX_WEB_CACHE_MODELS`
+  - the number of recent models to keep in memory
+  - setting this to 0 will disable caching and free VRAM between images
 - `ONNX_WEB_CORS_ORIGIN`
   - comma-delimited list of allowed origins for CORS headers
+- `ONNX_WEB_DEFAULT_PLATFORM`
+  - the default platform to show in the client
+  - overrides the `params.json` file
 - `ONNX_WEB_NUM_WORKERS`
   - number of background workers for image pipelines
   - this should be equal to or less than the number of available GPUs
-- `ONNX_WEB_BLOCK_PLATFORMS`
-  - comma-delimited list of platforms that should not be presented to users
-  - further filters the list of available platforms returned by ONNX runtime
-  - can be used to prevent CPU generation on shared servers
 
 ### Server Parameters
 

diff --git a/docs/user-guide.md b/docs/user-guide.md
@@ -30,6 +30,7 @@ Please see [the server admin guide](server-admin.md) for details on how to confi
     - [Adding your own models](#adding-your-own-models)
     - [Model sources](#model-sources)
       - [Downloading models from Civitai](#downloading-models-from-civitai)
+    - [Using a custom VAE](#using-a-custom-vae)
   - [Tabs](#tabs)
     - [Txt2img tab](#txt2img-tab)
       - [Scheduler parameter](#scheduler-parameter)
@@ -258,6 +259,46 @@ which you can see and copy from the right-click menu:
 You want the Pruned SafeTensor, if one is available. Be careful downloading PickleTensors, they may contain unsafe
 code. The original, non-pruned models are much larger but are better for training.
 
+### Using a custom VAE
+
+You can use a custom VAE when converting models. Some models require a specific VAE, so if you get weird results,
+check the model card for a specific VAE. This works for both diffusers directories and original SD checkpoints. You
+can use the `sources` field in the extras file to download the VAE file or provide a HuggingFace model name.
+
+```json
+{
+  "diffusion": [
+    {
+      "name": "diffusion-stablydiffused-aesthetic-v2-6-ema",
+      "source": "civitai://6266?type=Pruned%20Model&format=SafeTensor",
+      "format": "safetensors",
+      "vae": ".cache/vae-ft-mse-840000-ema-pruned.safetensors"
+    },
+    {
+      "name": "stable-diffusion-onnx-v1-4",
+      "source": "CompVis/stable-diffusion-v1-4",
+      "vae": "stabilityai/sd-vae-ft-ema"
+    }
+  ],
+  "correction": [],
+  "upscaling": [],
+  "sources": [
+    {
+      "name": "vae-ft-mse-840000-ema-pruned",
+      "source": "https://huggingface.co/stabilityai/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.safetensors",
+      "format": "safetensors"
+    }
+  ]
+}
+```
+
+Some common VAE models include:
+
+- https://huggingface.co/stabilityai/sd-vae-ft-ema
+- https://huggingface.co/stabilityai/sd-vae-ft-ema-original
+- https://huggingface.co/stabilityai/sd-vae-ft-mse
+- https://huggingface.co/stabilityai/sd-vae-ft-mse-original
+
 ## Tabs
 
 ### Txt2img tab