Windows Support

[vincentqb]

To bring Windows support with mp3 support, we need

• Activate build for wheels and conda package on CircleCI for Windows without SoX, see Add Windows CI #394
• Activate SoX tests only when SoX available, see testing with sox only when sox is available #419
• Fix kaldi_io NameError for Windows in comment.
• Activate CircleCI tests on Windows (e.g. Turn on tests when building through conda-build #493)
• Implement per-file-format backend dispatch mechanism.
• Add minimp3 for mp3 support, see comment.
• Activate all tests that depend on mp3 support.
• Activate nightly upload for wheels and conda package, e.g. Apply 'nightly' branch filter to binary uploads #385.
• If MinGW is supported by pytorch, then fix Can't compile on Windows (MinGW)  #42.

If and only if no backend support mp3 on Windows after the above:

• Compile SoX on Windows
• Activate build for wheels and conda package on CircleCI for Windows with SoX usinc cpp extension, e.g. Add Windows CI #394.

Closes #50, closes #219, closes #258.

cc @peterjc123, @chauhang, pytorch/pytorch#24344

[peterjc123]

The kaldi_io test is passing on Windows now. BTW, I think it's hard to compile Sox on Windows. Other things sound reasonable to me.

[vincentqb]

Thanks for the input. Can you share the output of CircleCI where the kaldi_io tests are passing?

If SoX is not possible to compile on Windows, we'll need to identify an alternative backend that offers similar file support on Windows: mp3, flac, wav, at least. soundfile unfortunately doesn't support mp3. See e.g. comparison.

[peterjc123]

Thanks for the input. Can you share the output of CircleCI where the kaldi_io tests are passing?

Sure. It was posted here: #419 (comment).

If SoX is not possible to compile on Windows, we'll need to identify an alternative backend that offers similar file support on Windows: mp3, flac, wav, at least.

What about this one? https://github.com/beetbox/audioread or https://github.com/librosa/librosa?

[vincentqb]

What about this one? https://github.com/beetbox/audioread or https://github.com/librosa/librosa?

aubio seems to perform better than librosa, according to this, and supports more format than audioread. Thoughts?

[peterjc123]

Well, it looks good to me except its package on pypi is a source package. However, if we use the C/C++ part then we should be okay.

[vincentqb]

Well, it looks good to me except its package on pypi is a source package. However, if we use the C/C++ part then we should be okay.

What is the implication of a source package?

[peterjc123]

As you can see from https://pypi.org/project/aubio/#files, only the file ends with .tar.gz is available.

[vincentqb]

What about this one? https://github.com/beetbox/audioread or https://github.com/librosa/librosa?

Both seems good then. Let's go for audioread then, since it appears to be faster than librosa. I've updated the description above to reflect the choice of audioread over sox for windows.

[dachosen1]

Have you looked into pydub? https://github.com/jiaaro/pydub

I've been using it on windows, and it works great for mp3 and wav files. The installation is a bit involved since it requires the user to add ffmpeg to the environment path?

[faroit]

@vincentqb Just some small remarks...

For the various use cases of audio i/o there are two scenarios where loading is used within torchaudio:

1. Training

Here, loading and decoding performance is crucial and easily becomes the bottleneck of dataloaders that deal with raw audio. Typically expensive compression formats should be avoided and simple formats such as wav, flac and mp3 should be used instead. Furthermore seeking support is crucial to load chunked audio from original (larger tracks)
In this use-case we already have libsndfile, interfaced with pysoundfile that cover wav and flac (at one point it would make sense to directly interface libsndfile to avoid numpy). Regarding MP3 support (+windows) I just discovered minimp3 that ticks all boxes. Also it is ridiculously fast and therefore could easily be the best tradeoff between loading and decoding speed.

2. Inference

Here, performance is not that crucial but support for various formats such as m4a/mp4/aac would be beneficial. As we often discussed in torchaudio-contrib, I still don't see any way around ffmpeg. ;-)

To sum up, I don't think it make sense to add another python package for audio i/o and instead focus on more low level and faster alternatives such as minimp3 that also come with less dependencies. What do you think?

[vincentqb]

In this use-case we already have libsndfile, interfaced with pysoundfile that cover wav and flac (at one point it would make sense to directly interface libsndfile to avoid numpy). Regarding MP3 support (+windows) I just discovered minimp3 that ticks all boxes. Also it is ridiculously fast and therefore could easily be the best tradeoff between loading and decoding speed.

@faroit -- Have you run your benchmark with minimp3? I'd love to see how it compares.

You are suggesting having a mix of backend for different format? That could be an option, yes. However, the context of this particular pull request is to make torchaudio available on Windows with the same features as the other OSs supported, and so this particular pull request doesn't push the boundaries of speed :)

1. Inference

Here, performance is not that crucial but support for various formats such as m4a/mp4/aac would be beneficial. As we often discussed in torchaudio-contrib, I still don't see any way around ffmpeg. ;-)

To sum up, I don't think it make sense to add another python package for audio i/o and instead focus on more low level and faster alternatives such as minimp3 that also come with less dependencies. What do you think?

I agree that there are already many python libraries loading audio files. In particular, those that load into numpy can be then used to load into pytorch, since pytorch can convert tensors from/to numpy at no cost. This means most users that want some very specific audio file can already do so.

It is still convenient for the users to get support for some common audio file format directly in torchaudio. But we can focus on the most critical format (wav, flac, mp3), and support them well and fast.

In that context, since ffmpeg is a heavy dependency, I would avoid depending on it for as long as I can. :)

[peterjc123]

@vincentqb Actually both audioread and aubio relies on ffmpeg.

[vincentqb]

Ah, good point. Has any of you faced any challenges such as this installing audioread? If not, I'd say we move forward anyway.

By the way, torchvision is also moving toward ffmpeg for video.

@cpuhrsch -- You voiced not being in favor of ffmpeg in the past. Any comments?

[peterjc123]

@vincentqb It will be easy for conda users because they can simply do conda install -c conda-forge ffmpeg. To make it convenient for other users, we may just distribute the DLLs for them.