[Bug] I am running out of memory after generating several batches

[gibsonfan2332]

I noticed this issue in two different occurrences.

1. When I set the highest resolution I could handle (1600x960 on 8GB Vram), but also set my batch count to more than one. This produced one image, but then ran out of memory trying to produce the second. As if the memory was not being freed up for the next batch (one image at a time).
2. When I set my image resolution lower, but tried to set my batch size to 3. This produced a batch of 3 images, but then ran out of memory attempting to produce the next batch of 3. Again as if the memory was not being freed.

If I do the same as #2^ but just one image at a time, I can create countless images without any issues.

Shouldn't the memory be freed up for the next batch each time? Meaning if I can produce one batch of 3 images, it should not run out of memory attempting the next batch right?

What I am actually wondering is if it needs memory to create images in the "txt2img-grids" directory. I notice its generating a grid of my batches combined. Maybe as that gets larger it is needing more memory just for that? That might explain why I can generate countless amounts of lower res images one at a time without issue, but not the highest resolution images more than once. Or perhaps this is some other kind of bug.

Is there any way to disable generating "txt2img-grids" files? I haven't been able to find out what those are even for, and if its using any more memory or resources I certainly don't want it generating those. Not to mention taking up storage space.

I would also like to point out I have no issues creating countless batches of multiple images using the original Stable Diffusion. It only seems to happen with this repo.

[gibsonfan2332]

For more info I am using these arguments for my 8GB VRAM
set COMMANDLINE_ARGS=--medvram --opt-split-attention

[AUTOMATIC1111]

the option to disable grids is right there in settings.

[vinyvince]

I have a RTX 3090 and even with no previous iteration , a single batch of 896x896 is resulting is cude out of memory on my side while i didn't face this issue with other repo . If i use set COMMANDLINE_ARGS=--opt-split-attention with yesterday built, it works but it's very slow...

Not sure hat to do now excepted getting back to another repo

[TheEnhas]

This is a recurring problem that I've had both with this and hlky's repo.

Generations work fine but sometimes I get a 2.0 GB VRAM spike at the very end that causes OOM (and prevents the images from saving). I've tried disabling GFPGAN and grids but it doesn't make any difference, yet I'm sure it's one or both of those that's causing it because I've never had this problem on basujindal's repo (because it doesn't have GFPGAN or grids). There's a definite memory leak or problem somewhere that hasn't been found yet.

[gibsonfan2332]

I don't know how I missed seeing the option to disable grids. But I can confirm it did not make a difference in my original issue. I still run out of memory after disabling it.

[allkhor]

@AUTOMATIC1111 Thanks for great tools!
I workaround this by replacing two files from stable-diffusion-webui/repositories/stable-diffusion/ to Doggettx autocast-improvements branch. Before any replacing backup original files!
https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/diffusionmodules/model.py
https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/attention.py
And after this run only with --medvram.
Maybe need replace only attention.py need more tests.
In my machine i don't saw any negative performance inpact.

[powderdba]

I can't set my resolution to anything above 576x576 else I get the cuda out of memory error.

But over time I have to set it back down to 512 again. And over time, I can't do higher batch counts and sizes over 1 though Ican when I first start using it. Happens over time.

This is on a 3070RTX TI 8GB.

[BrahRah]

@AUTOMATIC1111 Thanks for great tools! I workaround this by replacing two files from stable-diffusion-webui/repositories/stable-diffusion/ to Doggettx autocast-improvements branch. Before any replacing backup original files! https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/diffusionmodules/model.py https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/attention.py And after this run only with --medvram. Maybe need replace only attention.py need more tests. In my machine i don't saw any negative performance inpact.

I've tried your approach but I get:

stable-diffusion-webui/repositories/stable-diffusion/ldm/modules/attention.py", line 10, in
from ldm.modules.attention import LinearAttention
ImportError: cannot import name 'LinearAttention' from partially initialized module 'ldm.modules.attention' (most likely due to a circular import) (/media/r2d2/4TB/000 stable diffusion/stable-diffusion-webui/repositories/stable-diffusion/ldm/modules/attention.py)

[allkhor]

@BrahRah Something wrong with your attention.py there is no line 10 with "import from ldm.modules.attention import LinearAttention" but model.py has this line.
Did you correct replaced both files? I imagine you replaced attention.by by model.py.

/stable-diffusion-webui/repositories/stable-diffusion/ldm/modules/attention.py must be replaced to attention.py from Doggettx repo
/stable-diffusion-webui/repositories/stable-diffusion/ldm/modules/diffusionmodules/model.py  must be replaced to model.py from Doggettx repo

[gibsonfan2332]

@AUTOMATIC1111 Thanks for great tools! I workaround this by replacing two files from stable-diffusion-webui/repositories/stable-diffusion/ to Doggettx autocast-improvements branch. Before any replacing backup original files! https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/diffusionmodules/model.py https://github.com/Doggettx/stable-diffusion/blob/autocast-improvements/ldm/modules/attention.py And after this run only with --medvram. Maybe need replace only attention.py need more tests. In my machine i don't saw any negative performance inpact.

This seems to have made a big difference, thanks! Seems much quicker now and I can do higher resolutions.

[C43H66N12O12S2]

duplicate of #170