Tiled VAE?

[valery016]

Hello,
I understand there's no such thing as Tiled VAE in SD.Next? I tried to generate a 1024x1024 image in txt2image with Hires Fix enabled and 4x upscale, but the process ran out of CUDA memory. In Forge, in comparison, it switches to a tiled VAE mode automatically when it detects that it has run out of memory, and there is also that Never OOM extension, for switching that on manually. What about SD.Next?

[Woisek]

https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111

[valery016]

Thanks, I know MultiDiffusion and use it. It's not exactly a substitute to solve Hires Fix running out of CUDA memory, but ok. I can't get MultiDiffusion installed under SD.Next however.

[Disty0]

Diffusers Settings -> VAE Tiling
Compute Settings -> Attention optimization method -> Dynamic Attention SDP

Tho i would also recommend HyperTile UNet to speed things up.

Never OOM is pretty much the same thing as Sequential Offload + VAE Tiling.
Sequential Offload has pretty high performance hit.
You can enable Sequential Offload either from Diffusers Settings or using --lowvram.

Generating a 6552x6552 image with VAE Tiling + Dynamic Attention SDP + Sequential Offload takes 5-6 GB with 7.5 GB peaks: