Out of memory in Colab (free)

[gnmarten]

Thanks for the Colab.

I ran into the dreaded out memory while

processing the embedding files saved to nemo_outputs/speaker_outputs/embeddings

Just trying to understand: My audio file is only 1h18 minutes long, but its embeddings amount to: "Dataset loaded with 9209 items, total duration of 2.55 hours." Since segmentations and subsegmentations are done automatically, I wonder what the upper limit filesize for the script to work on Colab would be.

Full error:
RuntimeError: CUDA out of memory. Tried to allocate 7.61 GiB (GPU 0; 14.75 GiB total capacity; 7.81 GiB already allocated; 5.65 GiB free; 7.83 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[KevinGeLe]

Thats weird havent had that problem on long audio files. Try with another smaller whisper model "--whisper-model" Default is "medium.en", so try with "small.en" it will reduce the accuracy but should use 3x less memory.

Edit: adding "torch.cuda.empty_cache()", will free up memory thats not in use.

[MahmoudAshraf97]

Hi, this is strange as nemo automatically segments the file to avoid OOM errors, setting the max split size might help because there's plenty of memory available

[rashi-budati]

Hello, I would like your help to know where to set the max split size. in the create_config function, i could not find any such parameter.

[MahmoudAshraf97]

@rashi-budati check max_split_size_mb Here

[kksgandhi]

If you're having OOM issues and don't mind a performance hit, always remember that you can run this with --device cpu, that way you aren't worrying about your GPU at all.

I'm running whisper-diarization locally though, unsure how this would interact with GColab