Don't use built-in VAE

[IdiotSandwichTheThird]

The built in VAE is watermarked and degrades image quality, leading to noticeable aliasing and chromatic aberration like effects, as seen in this example https://i.imgur.com/SCY5P5z.png .

[AshtakaOOf]

Changing VAEs will not change the watermark on SDXL.

[lllyasviel]

Hello @IdiotSandwichTheThird and @AshtakaOOf , we are aware of the opinions of XL’s VAE is watermarked by some methods.

Nevertheless, we do not have enough evidences to support this. We do not know any existing research or methods that can watermark a latent diffusion’s first stage model.

The artifacts in your images are probably caused by the distribution of training data of XL’s vae. The XL’s vae is (likely to be) trained with many images with crisp and sharp texts in it so that its performance can be better for encoding/decoding texts, and the downside is probably some of your mentioned “chromatic aberration” because those training images have many “subpixels” for crisp and sharp texts.

Feel free to take a look at “subpixels” https://en.wikipedia.org/wiki/Subpixel_rendering to understand the “chromatic aberration” and text rendering, and this probably explains XL’s vae behaviors.

Finally, if anyone has stronger evidences to support the claim of “vae watermark”, please let us know.

[VolaNova]

The VAE issue with "watermarking" was an error that was fixed 2 days after the SDXL 1.0 release, if you download the model or VAE from their repository after that time then there will be no "watermarking" effect as the model's baked VAE has been reverted to the .9 version and re-uploaded.

[MoonRide303]

@lllyasviel Stability AI released official SDXL 1.0 with VAE from 0.9 to solve artifacts problems in their original repo (sd_xl_base_1.0_0.9vae.safetensors and sd_xl_refiner_1.0_0.9vae.safetensors) - you can check out discussion in diffusers issue #4310, or just compare some images from original, and fixed release by yourself. By default I'd suggest to go with updated versions (those with VAE from 0.9).

I've locally replaced paths the model files (in modules/default_pipeline.py) to fixed versions, and launched following test:

positive prompt: photography of a modern sports car, raw photo, high quality, high resolution

negative prompt: drawing, painting, illustration, rendered, low quality, low resolution

seed: 0, resolution: 1024x1024, mode: quality, style: cinematic-default

Results - output from original SDXL 1.0:

and from models with fixed VAE - artifacts near mirror are gone:

[lllyasviel]

This looks more convincing. We will have some actions on this.

[lllyasviel]

fixed in 1.0.17

https://github.com/lllyasviel/Fooocus/blob/main/update_log.md