CUDA out of memory

[MrRaymondLee]

I'm receiving the following error but unsure how to proceed. This is the output of setting --n_samples 1!

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 6.13 GiB already allocated; 0 bytes free; 6.73 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[kybercore]

You have too little VRAM. Use smaller resolutions or the optimized fork

https://github.com/basujindal/stable-diffusion

[breadbrowser]

or just use this https://huggingface.co/spaces/stabilityai/stable-diffusion

[ayyar]

PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.6,max_split_size_mb:128

Then run the command with:

--n_samples 1

[cjauvin]

@ayyar I believe garbage_collection_threshold is only available with PyTorch >= 1.12, whereas it is pinned at 1.11 in the conda environment of this project. I upgraded PyTorch to 1.12, but since I had still memory issues, I switched to https://github.com/basujindal/stable-diffusion, as suggested by @parsec501, which works at least with my RTX 2070 (8GB) on Ubuntu 20.04.

[ninjoala]

@ayyar where is the configuration file that I can set this? I'm new to python environments.

[tuwonga]

@ayyar where is the configuration file that I can set this? I'm new to python environments.

me too...

[saadmuk123]

I'm receiving the following error but unsure how to proceed. This is the output of setting --n_samples 1!

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 6.13 GiB already allocated; 0 bytes free; 6.73 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

try using --W 256 --H 256 as part of you prompt. the default image size is 512x512, which may be the reason why you are having this issue.

[tuwonga]

I'm receiving the following error but unsure how to proceed. This is the output of setting --n_samples 1!

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 6.13 GiB already allocated; 0 bytes free; 6.73 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

try using --W 256 --H 256 as part of you prompt. the default image size is 512x512, which may be the reason why you are having this issue.

Well,
I use now basujindal optimizedSD and I can make 1280x832.
Try it !

[slymeasy]

I have this same issue and looking at my memory usage it's at 5GBs before I even try to generate any images at it just stays there. Anyway to get it to let go of that memory so I can use the program?

[tuwonga]

I have this same issue and looking at my memory usage it's at 5GBs before I even try to generate any images at it just stays there. Anyway to get it to let go of that memory so I can use the program?

try basujindal optimizedSD

[slymeasy]

try basujindal optimizedSD

Do you have a link to a Latent Diffusion Upscaler that I can install on Windows? That's what I was trying to install using a Stable Diffusion webUI installation tutorial I found on the internet. I need that upscaler.

[tuwonga]

try basujindal optimizedSD

Do you have a link to a Latent Diffusion Upscaler that I can install on Windows? That's what I was trying to install using a Stable Diffusion webUI installation tutorial I found on the internet. I need that upscaler.

do you mean REALESRGAN?

[tuwonga]

try basujindal optimizedSD

Do you have a link to a Latent Diffusion Upscaler that I can install on Windows? That's what I was trying to install using a Stable Diffusion webUI installation tutorial I found on the internet. I need that upscaler.

Please use this stable diffusion gui and you will solve your upscaling issue
stable-diffusion-webgui

[slymeasy]

Please use this stable diffusion gui and you will solve your upscaling issue stable-diffusion-webgui

Thanks, got the same issue with CUDA error, but that guy had a lot of helpful pointers in the trouble shooting section and I managed to get it working.

I was already using REALESRGAN. I was looking for the Latent Diffusion upscaler because it supposedly adds more details and "fixes" the picture. There are some images I get back from Stable Diffusion that just don't have enough detail or are too messed up to be any good. I this this option "SD Upscale" that might be what I was looking for.

[tuwonga]

Actually I fixed just replacing attention.py and module.py with the ones from basujindal.