Network working with STFT real and imag

[nperraud]

[andimarafioti]

Here.
I did refactored the code to remove some responsibility from the simulation. The dividing into gap and sides part still seems too local to add it to the model class. And the model class starts to look too fat. Some refactoring is needed for sure.

[andimarafioti]

I'm running two tests on this one with learning rate 1e-3. First test without dropout, second with. Will report tomorrow.

[andimarafioti]

I ran two experiments last night:

Both trained for around 300k steps in 20hs. Orange one doesn't have any regularization term, blue one has the one we discussed yesterday: (1 + 5 / (norm_orig+tf.constant(1e-2, dtype=norm_orig.dtype). None of them have dropout.
In both cases we have some over-fitting but not too mayor.

I took the network with regularization and did some reconstructions on a subset from the testing set. The mean for the SNR here is 15 dB.

For this signal the reconstruction is pretty good (21dB in 1D):

Here with the difference in its own scale:

We now observe again (it didn't happen for the mag network) some weird distribution around 0 for the SNRs:

And for a signal with a worse reconstruction (8dB on 1D):

[andimarafioti]

This network is doing better in terms of SNR but right now we are not able to hear sounds since the processing adds a bunch of noise (even after the border effect, the constant multiplier that should be 1.5 has an oscillation of around 7%).

[andimarafioti]

Nicki pointed out that I should be using the full spectogram for the istft (taking the original from the sides). With that setting the SNR on the 1280 samples we get from the reconstructed spectrogram is 17dB and for the actual gap of 1024 samples of 16dB.

The 1D plots from before now are:

and

We also don't get the weird distribution:

[andimarafioti]

I continued training with learning rate 1e-4. Now the mean for the SNR on the subset of the training set is 17.8dB (for the 1024 gap samples). Where it performs worse is where the signals are too simple on the spectrogram (so mostly constant frequencies). We heard some examples with Nicki and we don't hear the same noise artifacts from before, which is really good.

[nperraud]

That looks very promising! We are slowly getting where we want. Here are you still cheating or not? (I mean with the borders...)

[andimarafioti]

Yes, we're still cheating, but I'm already training one that doesn't