Info length and rate returns different values for different backends

[faroit]

🐛 Bug

torchaudio.info returns the info objects directly from the respective backend. Due to same property naming, users might forget to check how the metadata is calculated. This results in metadata being reported differently depending on which backend is reported.
E.g. sox calculates the length summed across channels whereas soundfile does this per channel (correct)

I would propose to add wrapper for the info objects that - independent of the backend - the most important metadata (length and rate) is identical.

Currently, the sox backend reports a missleading length and the rate parameter is of type float instead of int.

To Reproduce

path =  "any/wavfile.wav"
# soundfile
torchaudio.set_audio_backend("soundfile")
info = torchaudio.info(path)
print(si.length)
print(type(si.rate))

# sox
torchaudio.set_audio_backend("sox")
info = torchaudio.info(path)
print(si.length)
print(type(si.rate))

Expected behavior

soundfile reports the correct metadata, sox should be corrected so that:

# sox
torchaudio.set_audio_backend("sox")
info = torchaudio.info(path)
print(si.length // si.channels)
print(int(si.rate))

Environment

torchaudio==0.5.0 from pypi

[vincentqb]

Relates to #236?

[vincentqb]

I agree that both backend should return the same values. We need to make sure we document the change of either so that users are aware of any changes. Thoughts?

[mthrok]

I could reproduce this;

code

import torchaudio

path = "test/assets/vad-go-stereo-44100.wav"

# soundfile
torchaudio.set_audio_backend("soundfile")
si, _ = torchaudio.info(path)
print(si.length)
print(type(si.rate))

# sox
torchaudio.set_audio_backend("sox")
si, _ = torchaudio.info(path)
print(si.length)
print(type(si.rate))

43339
<class 'int'>
86678
<class 'float'>

[mthrok]

I checked #236, and it does not fix the issue.
I think it's more appropriate to handle it on Python side, as suggested.
(Copy data from sox backend C++ struct to SignalInfo Python class and apply the fix there.)
That way, info function returns the same class regardless of backend.

[faroit]

That way, info function returns the same class regardless of backend.

we had already used the sox backend in production (before soundfile was merge in), so we made the modifications to info in our code (should have reported earlier!). If we change the info object now, this might cause unexpected results for users. I would rather solve this by

• clarify documentation on torchaudio.info(...), stating that this returns the native info object of each backend and each can contain a different number and definitions of metadata (e.g. some metadata also depends on the codec/container being used).
• create a new torchaudio info object that holds just the most common important infos: sample_rate, nb_samples and nb_channels. No idea how to name that, though...
• mark torchaudio.info() as deprecated in future torchaudio versions

I could provide a PR

[vincentqb]

we had already used the sox backend in production (before soundfile was merge in), so we made the modifications to info in our code (should have reported earlier!). If we change the info object now, this might cause unexpected results for users. I would rather solve this by

You mean that you adjusted your code for sox, soundfile, or both?

[faroit]

You mean that you adjusted your code for sox, soundfile, or both?

I had nb_samples = si.length // si.channels calculated for sox. With the introduction of the soundfile backend, I was required to change this.

I looked into the unit tests of the torchaudio.info and I guess the error was not caught before as the unit tests only cover mono signals (where sox and soundfile give the same output). Can we add a stereo (and possibly multichannel) regression test for info?

[mthrok]

I looked into the unit tests of the torchaudio.info and I guess the error was not caught before as the unit tests only cover mono signals (where sox and soundfile give the same output). Can we add a stereo (and possibly multichannel) regression test for info?

@faroit Sure. We should add the test. Let us know if you would like to open a PR. If you are busy I will add it to my backlog.

[vincentqb]

There are two issues here.

• "When doing torchaudio.info() on a multi channel audio file, the length is multiplied by however many channels there are." as in Length Inaccurate on Multi-Channel Audio Files #236.
• sox and soundfile backend return similar object, but with different values.

For the first, I'm concerned about such change for backward compatibility for sox. I would recommend instead adding a key nb_samples = si.length // si.channels to the object returned in python.

For the second, I second adding a stereo test for consistency of course. :) One of the two would need to be updated to match the other. Since sox has been there longer, soundfile's si.length is the one that should be adjusted. A warning when someone invokes info with soundfile would need to be raised, and the documentation aligned.

I prefer this solution to creating a new abstraction, simply since it keeps us closer to the intended set up so far.

Thoughts? Do you want to open pull requests for this?

[faroit]

Regarding the first issue, this is probably a just a matter of setting the right vocabulary to make a formal distinction between frames and samples as it's done in libsndfile. Over there:

A frame is just a block of samples, one for each channel.

and

...seeks are defined in number of (multichannel) frames. Therefore, a seek in a stereo file from the current position forward with an offset of 1 would skip forward by one sample of both channels.

which makes totally sense to me (also soundfile is the defacto standard when it comes to proper handling of audio I/O). However this would probably lead to too many changes here but it makes sense to put the definition that is used here ("we define samples are the number of frames in an audio signal per channel").

For the second, I second adding a stereo test for consistency of course. :) One of the two would need to be updated to match the other. Since sox has been there longer, soundfile's si.length is the one that should be adjusted. A warning when someone invokes info with soundfile would need to be raised, and the documentation aligned.

I agree this is probably the simplest solution

Do you want to open pull requests for this?

I started with a new test #639 that is expected to fail and can propose a fix for this as well ( in the same PR?)