Inverse Spectrogram and Mel-Spectrogram Layer?

[AI-Guru]

Namaste!

kapre has become an integral part of all my audio Deep Learning experiments. Powerful! Thanks for providing such a great software!

I was thinking... I guess it would make sense to have layers for inverse spectrogram and inverse mel-spectrogram. Thinking about Autoencoders, this would be even more powerful. I know that reconstructing samples from spectrograms is not the best, but it is possible to a certain degree.

What do you think about that feature request?

Best,
Tristan

[keunwoochoi]

Hi, good to know that Kapre is useful for you! I think that's good idea, although the development of Kapre has been paused for a while. I'll get back to work in a month and will make some update then.

[douglas125]

Maybe make a Griffin-Lim spectrogram inversion? BTW congrats, great work.

[jonnor]

Btw, librosa 0.7+ has Griffin-Lim and inverse melspectrogram. Not using GPU though. https://librosa.github.io/librosa/generated/librosa.feature.inverse.mel_to_audio.html#librosa.feature.inverse.mel_to_audio

[AI-Guru]

Wow! Is there a way to push a gradient through this?

[jonnor]

Forward pass probably not a problem. No idea about backpropagation though. Probably hard, as Griffin-Lim is a iterative algorithm. One-shot approximations for spectrogram inversion based on neural networks have been explored in the literature though, for example Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks. Those would be backprop-able by construction

[HudsonHuang]

Maybe we have STFT before this line ?
So, the inverse of Spectrogram and Mel-Spectrogram might be diffcult, but the inverse of STFT might be easy? A pytorch version is here

[keunwoochoi]

It's on my to-do now and will be added on 0.3.x.