loss just not converge when adding a L2 norm layer after Encoding layer

[tengteng95]

Hi, I am very interested in your great work. I try to make use of Encoding Layer in my own dataset, but I found that training loss just not decrease(so so slow) when L2 norm layer is used after Encoding Layer.

Could you please tell me what role the L2 norm plays in your experiments and whether it fluences the performance or not?

[zhanghang1989]

It depends on the scale of your experiments. For cifar experiment with 10 classes, L2 norm works best. For large scale problem such as ImageNet, please try using BatchNorm.

[tengteng95]

Great thanks to you. BN does alleviate the problem a lot.