CUDA out of memory.

[realcarlos]

However I am using a station with 4 x A100(40G)

if_I = IFStageI('/IF/deepfloyd-if/IF-I-XL-v1.0', device='cuda:0')
if_II = IFStageII('/IF/deepfloyd-if/IF-II-L-v1.0', device='cuda:1')
if_III = StableStageIII('/IF/deepfloyd-if/stable-diffusion-x4-upscaler', device='cuda:2')
t5 = T5Embedder(device="cuda:3")

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 39.39 GiB total capacity; 29.37 GiB already allocated; 6.90 GiB free; 30.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[realcarlos]

solved by adding "os.environ["FORCE_MEM_EFFICIENT_ATTN"] = "1""

[YeHaijia]

对我有用，赞

[404-xianjin]

hi，I got the same error, my error code is as follows,
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB (GPU 0; 23.99 GiB total capacity; 17.99 GiB already allocated; 3.23 GiB free; 18.43 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

After seeing your solution, I don't know where to set this parameter? Can you tell me? Thanks @realcarlos

[404-xianjin]

大哥，我点开你的主页发现你也在北京，我直接

目前看好像可以了。

[Delicious-Bitter-Melon]

大哥，我点开你的主页发现你也在北京，我直接 目前看好像可以了。

I have the same problem using the 3090. Although I set the ""os.environ["FORCE_MEM_EFFICIENT_ATTN"] = "1""", I still have the problem. Do you also use the 3090 with the 24GB?