OOM && Large GPU usage

[arnavmehta7]

Hi, I am using this, but it doesn't seems to release the gpu vram. I am using large-v2 model and it's using 16gigs of GPU ram. Which I don't think is NORMAL at all, moreover it doesn't free up the ram. Is this normal?

[m-bain]

Memory scales with length of input segment to wav2vec2 model. With current version, adding
--vad_filter
Ensures that no segments greater than 30s are inputted to the w2v model. You can further reduce gpu reqs by
a. Using smaller w2v align_model
b. Limit input segment duration

I will add b as a new feature at some point

[arnavmehta7]

I am using a older commit, and its now using 28gigs of GPU ram, which is TOOOO much. Any idea how to fix this?

[arnavmehta7]

Files are being processed in a loop, and seems like after every file, GPU ram usage pops up by ~2gbs ram.

[arnavmehta7]

@m-bain I just ran a very long audio and it showed that it tried to allocate 30gbs of GPU ram, is there any clean fast fix?
Thankyou!