Add GroupNormLayer

[2wins]

Checklist

• I've tested that my changes are compatible with the latest version of Tensorflow.
• I've read the Contribution Guidelines
• I've updated the documentation if necessary.

Motivation and Context

Description

This is for a new normalization layer, Group Normalization Layer.
Even if there is tf.contrib.layers.group_norm, it has an unsolved issue (link).

[zsdonghao]

@2wins Thanks.

Could you also change these two files:

• https://github.com/tensorlayer/tensorlayer/blob/master/docs/modules/layers.rst

• https://github.com/tensorlayer/tensorlayer/blob/master/CHANGELOG.md

[DEKHTIARJonathan]

@zsdonghao if its fine with you this PR will be merged only after TL 2.0

[wagamamaz]

It seen this layer only support 2D images?

mean, var = tf.nn.moments(x, [1, 2, 4], keep_dims=True)

[2wins]

@2wins Thanks.

Could you also change these two files:

• https://github.com/tensorlayer/tensorlayer/blob/master/docs/modules/layers.rst
• https://github.com/tensorlayer/tensorlayer/blob/master/CHANGELOG.md

@zsdonghao OK. I'll reflect it to them.

It seen this layer only support 2D images?

mean, var = tf.nn.moments(x, [1, 2, 4], keep_dims=True)

@wagamamaz Yes, I only consider 2D input case like InstanceNormLayer.

[wagamamaz]

@2wins I see, could you raise an exception if the input doesn't fix the shape?
Also, could we support channel_first using data_format? Many thanks.

[zsdonghao]

@DEKHTIARJonathan I think this one can be merged first, we need to support these kinds of layers, and I have no idea when the 2.0 can be finished.. 😭

[2wins]

@wagamamaz We need to discuss data_format. Many layers we have do not support it normally as can be seen in #640.

[wagamamaz]

@2wins in the latest version, it seen that conv2d and many other layers already support channel_first, also the batch norm layer

[wagamamaz]

This one will have error when two layers have similar name. Use the following instead:

https://github.com/tensorlayer/tensorlayer/blob/master/tensorlayer/layers/convolution/simplified_conv.py#L96

[2wins]

The commit Support in GroupNormLayer 27224ce
should be Support data_format in GroupNormLayer.