CUDA memory requirement

[HeChengHui]

@yxlu-0102
I would like to check what is the minimum vram to run this model. I have a 2070Super and running into OOM error.

    skip = torch.cat([x, skip], dim=1)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.84 GiB (GPU 0; 8.00 GiB total capacity; 5.63 GiB already allocated; 455.00 MiB free; 5.65 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[yxlu-0102]

I used two 3090ti GPUs to train the MP-SENet model, each with a memory size of 24 GB.

[HeChengHui]

@yxlu-0102
Is it the same for inference?

[yxlu-0102]

One GPU with a memory size of 24 GB is enough for the inference process.

[HeChengHui]

@yxlu-0102
I just tried inferencing on my 3090ti (24gb vram), and I am still getting OOM error. Is there a way to reduce the memory usage?

[yxlu-0102]

Due to the variable length of speech, the memory consumption for generating each speech is not fixed, and it doesn't depend on whether the GPU is a 3090 or not. It primarily depends on the amount of GPU memory you have.

During training, you can reduce GPU memory usage by adjusting the batch size and segment length, but this may have an impact on the training results.