Output of SPADE

[gduthe]

Hi,
Would you mind explaining why in the implementation of SPADE you have:
out = normalized * (1 + gamma) + beta

This is different to what is described in the paper, where I understood it as:
out = normalized *gamma + beta

Thank you.

[taesungp]

It's just because the initialization of the last layer outputs something close to zero. In case of gamma, we initialized it to output something close to one, so that in the beginning of training SPADE will preserve normalized activations.

Another way of having the same effect would be initializing the bias of the last layer to be 1.

Just to reiterate your issue that it's different from the paper, it doesn't really matter because gamma is learned. You can have gamma learn x or 1 + x.

[gduthe]

Got it. Thanks!