Question about VRam usage compared to other forks

[HyunJae5463]

I have been using this fork for quite some time now which allows me to generate images like 440 x 780 with 2x Upscale. The downside is that its rather slow due to the lack of optimization and its not getting any updates either, which can result in Extensions breaking etc.

With this version however i have to go down like a hundred pixels on both sides to not run out of memory. In both versions i have been using med vram and quad attention. I have also tried different processing methods in this version but i still run out of Vram when i use the same resolutions as in Aloereeds SD version.

I have an Arc A770 btw with 16 GB VRam.

[vladmandic]

cc: @Disty0 as he did all of ipex code

[Disty0]

You can use IPEX instead of DirectML:

https://www.technopat.net/sosyal/konu/using-stable-diffusion-webui-with-intel-arc-gpus.2593077/

Resolution: 2048x2048
OS: Arch Linux 6.4.1
GPU active 10316 MB reserved 13512 MB

Also don't go above 2032x2032.
It has high tendency to generate white images if you go above 2032x2032.
2048x(Ratio) or (Ratio)x2048 is fine though.

[HyunJae5463]

IPEX only works on Linux, im on Windows. If its that much better though i may get some external drive and put Linux on it.

[vladmandic]

off topic, but generating images bigger than ~1k pixels will likely result in bad artifacts in general as diffuser models start duplicating things - its much better to use some kind of tiling to get to higher resolutions. hires fix is basic, tiled diffusion is better, etc.

[FullBleed]

IPEX only works on Linux, im on Windows. If its that much better though i may get some external drive and put Linux on it.

Probably easier to use WSL.

[whchan05]

For those who are interested, Vik from Intel Insider Discord found that VRAM leak in WSL2 is caused by unexpected behavior of torch.xpu.empty_cache(). I have then replaced it with

# gc.collect()
# os.system('sync;sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"')
pass

I successfully generated 100 pictures without running out of VRAM or system memory.
Until it is properly reported to and fixed by Intel, you guys may give it a shot to see if that works for you.
He also said it probably is not related to the code I replaced with but I am too lazy to figure out if that is the case.

edit: simply commenting out torch.xpu.empty_cache() is sufficient

[vladmandic]

if there is an issue in torch.xpu.empty_cache(), we'll have to wait for intel to publish a fix.
what you're doing is dropping slab objects and pagecache from linux kernel globally - extremely costly operation as pretty much every handle needs to be recreated.

[whchan05]

Yep I have little idea what I have replaced original code with - but inference speed is not affected and I consider that a win

[vladmandic]

because in latest versions of sdnext, gc is actually skipped during generation unless high-memory utilization threshold is detected (and then you'd see warning message printed about that). i've done that to avoid unnecessary gc calls during generate as it always slows down things, even if just by a little bit.

[Disty0]

Pushed this commit to disable empty_cache in WSL2:
c3a4293