Inference on long files

[databill86]

Hello,

Thank you for this great library!
Is there any way we can chunk the initial audio into shorter samples, let's say 50 seconds each, run inference on those, and end up with a final reconstruction.
I came across this article and I wonder if it's possible to get it working here.
Any ideas if this is possible ?

[guillaumekln]

Hi,

The Whisper transcription loop already handles long files using a sliding 30-second window while keeping the context. So you don't need to do anything to transcribe long files.

[databill86]

Thank you. So is it normal that the transcription time is considerably long for long files ?

[guillaumekln]

Yes, the transcription time depends on the audio file duration. Long files will take longer.

[databill86]

Sorry I closed and reopened the issue. I just have one last thing about the longer files.
If we use the "gpu" as a device, is there any way we can avoid OOM for these longer files ?

[guillaumekln]

What is your GPU and what model size are you running?

[databill86]

It's a NVIDIA GeForce GTX 1070 Ti 8Go, I was running the large-v2 model on a 18min file. But even with 4min file I have OOM.

[guillaumekln]

Try running the model with 8-bit quantization:

model = WhisperModel(model_path, device="cuda", compute_type="int8")

[databill86]

Wow, just like that! it's a lot faster, and no OOM!!!
Thank you!
I will close the issue now for good :)