Batch Normalization

[AmirMahdiNassiri]

Hello dear authors,

In your source code, we can set the normalization parameter to 'batch', 'instance' normalization or the default 'None'.

Can I ask that is there a reason why normalization is off by default? Did you have the time to experiment with normalization as well? Didn't it contribute to better results?

Any information is greatly appreciated.
Thanks.

[shaohua0116]

I have experimented with both batch norm and instance norm, which shows worse results. In my implementation (ops.py bn_act()), I apply activation functions before normalizing the values, which is widely used. I believe that some people argue that we should first normalize the output and then apply the activation functions. I haven’t tried this yet.