support other audio format

[khursani8]

Might solve #1549

Currently I let pydub handle file that is not .wav format because there is something I'm not understand with current implementation.

1. "samples.transpose()" before init with cls
2. User also need to install ffmpeg to be able to load non wav file format, I don't know where to put the warning or write a script when user pip install
3. I'm not sure if NeMo dataloader will cache the loaded samples because it might convert using ffmpeg everytime it try to load non wav file.

So far I only tested inference with mp3 audio from Common Voice with model trained on wav file format, there is no problem with the prediction result.

[blisc]

Thanks for the PR. It looks good to me.
I just added a slight check to ensure that it will continue to work with io.bufferedreader that we use for tarred datasets.

[yoks]

This MR, make training so much slower. Pydub takes too much of CPU to convert media files. Need to make sure pydub is only used for unsupported media formats, right now we using FLAC, and it goes through pydub, but libsnd supports flac.

Part in the middle is with MR, and parts before and after are without it. Training done on DGX A100.

[blisc]

Ok that's a problem. I'll go ahead and change the logic a bit

[khursani8]

Hi @blisc,
I create hotfix for this bug

[blisc]

@yoks, can you try #1630 and see if performance returns?

[yoks]

Will test as soon as we will have free video card available on DGX (tomorrow most likely)

[yoks]

Yea looks like it works as before (maybe a little bit slower, but can be related to task/node load).

[rbracco]

Is there a reason this isn't using torchaudio? It supports multiple formats in a single library.

[khursani8]

I don’t have experience using torchaudio, after take a look at their repo I think torchaudio will be better for this, support multiple audio format including kaldi format. Maybe someone know how to use torchaudio better can contribute to this

[mthrok]

I don’t have experience using torchaudio, after take a look at their repo I think torchaudio will be better for this, support multiple audio format including kaldi format. Maybe someone know how to use torchaudio better can contribute to this

Hi

torchaudio maintainer here. I commented in #1549 but the current release of torchaudio does not support file-like object. I am adding that support right now and if will be released in the next release. (Early March).
Is there any other requirements that NeMo has?

[khursani8]

Hmmm, I'm not related to NeMo actually, I also don't know the requirements, I think @blisc can answer this question.

[rbracco]

Theres a very detailed and helpful response from @titu1994 on why torchaudio isn't used as a dependency.