performance - is it as bad as reported?

[chananshgong]

According to this paper, librosa performance are the worst by far among audio feature extraction packages.

Is that true? too bad, I liked the design.

See figure 2:

[dpwe]

Hmm, I wonder what they used for resampling in librosa? Pre-resampy, and installed without libSRC, librosa would run ok, yet be very slow. Pretty annoying if that's what they did. And, absent that, the result looks suspicious to me; there's nothing drastically inefficient about doing computation using numpy via librosa. DAn.

…

On Mon, May 8, 2017 at 12:27 PM, Hanan Shteingart ***@***.***> wrote: According to this paper <https://www.ntnu.edu/documents/1001201110/1266017954/DAFx-15_submission_43_v2.pdf/06508f48-9272-41c8-9381-7639a0240770>, librosa performance are the worst by far among audio feature extraction packages. Is that true? too bad, I liked the design. See figure 2: [image: image] <https://cloud.githubusercontent.com/assets/24369534/25813946/405d475c-3424-11e7-80bb-367a24ae0a1b.png> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#572>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAhs0Yh4ddBfRCZKgPBiL7V-1KKbR9vMks5r30JmgaJpZM4NULzD> .

[bmcfee]

They're citing v0.3.1, so I wouldn't put much stock in the comparison.

[rafaelvalle]

I've emailed the authors and asked them to share the source code and data such that I can try producing new results...

[bmcfee]

Is there any reason to keep this issue open?

[chananshgong]

Maybe a call for new benchmark

…

On Jul 17, 2017 10:12 PM, "Brian McFee" ***@***.***> wrote: Is there any reason to keep this issue open? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#572 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AXPZfmMNNoYm1m8eP4cT71IaPAlH8-Zkks5sO7IZgaJpZM4NULzD> .

[bmcfee]

Seeing as the authors of that study have not produced source code and do not report what exact features they're benchmarking only benchmark mfcc calculation, I don't feel much inclination to try reproducing their setup.

As mentioned above, they cite v0.3.1, which at this point, bears little similarity to the modern code base. Especially when it comes to features like resampling and constant-Q transforms, where there have been significant speedups introduced.

This paper has a more recent benchmark (against 0.4.3): http://www.aes.org/e-lib/browse.cfm?elib=18769 . The load benchmark is a complicated case because of librosa's interface to audioread. If you want fast loading, you can read through soundfile instead and get a speedup at the cost of losing mp3 support. Otherwise, things are roughly in the small-constant-factor ballpark that you would expect when comparing llvm/compiled code to interpreted python.

[rafaelvalle]

FYI: I never heard back from the authors...