ModeNormalization(axis=-1, k=2, momentum=0.99, epsilon=1e-3, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None)
Mode Normalization Lucas Deecke, Iain Murray, Hakan Bilen - 2018
Normalize the activations of the previous layer at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1 for K different modes.
- axis: Integer, the axis that should be normalized (typically the features axis). For instance, after a
Conv2Dlayer withdata_format="channels_first", setaxis=1inModeNormalization. - k: Integer, the number of modes of the normalization.
- momentum: Momentum for the moving mean and the moving variance.
- epsilon: Small float added to variance to avoid dividing by zero.
- center: If True, add offset of
betato normalized tensor. If False,betais ignored. - scale: If True, multiply by
gamma. If False,gammais not used. When the next layer is linear (also e.g.nn.relu), this can be disabled since the scaling will be done by the next layer. - beta_initializer: Initializer for the beta weight.
- gamma_initializer: Initializer for the gamma weight.
- moving_mean_initializer: Initializer for the moving mean.
- moving_variance_initializer: Initializer for the moving variance.
- beta_regularizer: Optional regularizer for the beta weight.
- gamma_regularizer: Optional regularizer for the gamma weight.
- beta_constraint: Optional constraint for the beta weight.
- gamma_constraint: Optional constraint for the gamma weight.
- Arbitrary. Use the keyword argument
input_shape(tuple of integers, does not include the samples axis) when using this layer as the first layer in a model.
- Same shape as input.
pytest