Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Spectrogram fft fix #505
What changes were proposed in this pull request?
Fixes for issue #502
Length of each timewindow in Spectrogram now is same sa fftSampleSize.
How was this patch tested?
New testcase for FFT added, although it has magic numbers in it. Manual inspection in debugger to see that change of real to complex format expected by FFT is correct.
@saudet Glad I could help!
Just a tip regarding the FFT: I recently converted my project to use Jtransform for transforms. It's licensed under BSD-2 Clause which from what I understand is a nicer license compared to GPL as it is non-viral. They claim comparable performance to FFTW (although the min execution time benchmark methodology looks somewhat fishy to me). Having a mix java and c in your project is bound to fail anyways /jk :)
I haven't done any measurements, but to me, there is no perceptable difference for ETL when training on raw waveforms vs spectrograms/MFSC/MFCC etc. It seems to be dominated by the file reading. For "online" inference (classifying audio stream from soundcard) even with my worst model ensembles the program spends ~70% of the time idling while the sound buffer fills up. All in all, FFT performance does not seem to be that important since other things (GPU memory for me) is bottlenecking.
I also saw that NDArrays have a built in fft method. I haven't tried it yet though but I guess it could be a kind of nice self-containment thing.
https://github.com/wendykierp/JTransforms looks awesome! If you could integrate that it would be great! FFT isn't currently implemented in ND4J and since this is currently only for ETL, I think it makes sense to just use JTransforms as is in DataVec. But if you'd like to create the API in ND4J for FFT first, and then use that in DataVec, that would be fine too.