network working with the STFT magnitude only

[nperraud]

[andimarafioti]

Is this: https://github.com/andimarafioti/audioContextEncoder/blob/master/simulations/runNatStftMarchTest.py what you meant? I trained it starting with 1e-3 and 1e-4 as learning rates. Results aren't great. I still have a bug with the tensorflow graph growing beyond 2Gb after 200k steps.

[nperraud]

To be honnest. I do not know. Your code start to be less and less clear.
In this code:

    stft = tf.contrib.signal.stft(signals=signal, frame_length=fft_frame_length, frame_step=fft_frame_step)

    sides_stft = tf.stack((stft[:, :15, :], stft[:, 15+7:, :]), axis=3)
    print(sides_stft)

    mag_stft = tf.abs(sides_stft)    # (256, 15, 257, 2)
    mag_stft = tf.reshape(mag_stft, (batch_size, 15, 257, 2))
aModel.setOutputTo(mag_stft)

mag_stft should be the input and tf.abs(stft[:, 15:22, :]) the output. Right?
So what is the last line doing?

[andimarafioti]

That reshape doesn't do anything, I already removed it. The setOutputTo function is a way I found to add computations to the model without needing to write staff inside that class. I just take the output at some point (

audioContextEncoder/simulations/runNatStftMarchTest.py

Line 18 in 6ad0397

signal = aModel.output()

) apply some computations and then set the output to what I want it to be.

[andimarafioti]

Just to clarify, after setting the output you can continue to add computations to the graph. Is just at that point that the output is what you set it to be.

[nperraud]

I do not think it is this since the decoder do 1 dimentional deconvolution.

[andimarafioti]

That is not correct. The decoder does 2D convolutions. After the fully connected layer the height is fixed to 1 and the width to 7 (out frames for the gap). The decoder changes the channels finishing with 257 (the desired fft bins) and finally there's a reshape to go to (batch_size, number_of_fft, fft_bins). I know it's not a good architecture, it's just a proof of concept. But it is not using 1d deconvolutions. It's just using strides of 1 for the height.

[nperraud]

This does not imply 1 dimensional deconv?

filter_widths = [(1, 5), (1, 9)]

[andimarafioti]

Only if you think fixing the height to 1 implies 1d deconv. But I would argue that it's just a technicality. If this proof of concept does what you wanted it to do but you're not happy with the hyper-parameters I'm very willing to change them. I just wanted to finish this before going home yesterday so I could run it over night.

[nperraud]

Okay good! I think it is what I wanted. I will put some parameter this afternoon.

[andimarafioti]

I uploaded and I'm testing with a more sensible choice of hyper-parameters. Here's the excel table I used to guide me:

MAG 5	Conv+ relu	Conv+ relu	Conv+ relu	Conv+ relu	Conv+ relu	Reshape	Linear fully	Reshape	Deconv + Relu	Deconv + Relu	Reshape	Deconv + Relu	Reshape	Deconv
in_channels	2	32	64	128	128	1	3200	1	1	64	1	256	1	16
out	32	64	128	128	200	1	1799	1	64	16	1	4	1	1
filter_h	7	3	2	1	1	1	1	1	5	3	1	3	1	7
filter_w	89	17	6	5	3	1	1	1	189	33	1	33	1	257
params	39872	104448	98304	81920	76800	1	5756800	1	60480	101376	1	101376	1	28784	6450164
h	15	8	4	2	2	2	1	7	7	14	28	7	14	7
w	257	129	43	15	8	8	1	257	257	514	1028	257	514	257

I find it a little difficult to work with the stft on the deconv because I need to produce a 7*257 signal and those are both prime numbers.

[andimarafioti]

For some reason this is incredibly slow

[nperraud]

Could you give me the result of this?

    model_vars = tf.trainable_variables()
    slim.model_analyzer.analyze_vars(model_vars, print_info=True)

[andimarafioti]

Total size of variables: 6455032
Total bytes of variables: 25820128