-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
InstanceNorm1D、InstanceNorm2D、InstanceNorm3D #48940
Merged
Merged
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
ba5f15d
modified: python/paddle/nn/layer/norm.py
Ayuan2021 948fa62
modified: python/paddle/nn/layer/norm.py
Ayuan2021 706b6cc
modified: python/paddle/nn/layer/norm.py
Ayuan2021 f59556e
modified: python/paddle/nn/layer/norm.py
Ayuan2021 e0fb93f
modified: python/paddle/nn/layer/norm.py
Ayuan2021 9daa876
modified: python/paddle/nn/layer/norm.py
Ayuan2021 c3e32a2
test=docs_preview
Ayuan2021 49bddca
InstanceNorm2D中文档格式修改
Ayuan2021 ebb9b58
test=docs_preview
Ayuan2021 ddb8d06
modified: python/paddle/nn/functional/loss.py
Ayuan2021 e435259
test=docs_preview
Ayuan2021 1b6ada9
test=docs_preview
Ayuan2021 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -327,19 +327,20 @@ def layer_norm( | |||||
x, normalized_shape, weight=None, bias=None, epsilon=1e-05, name=None | ||||||
): | ||||||
""" | ||||||
see more detail in paddle.nn.LayerNorm | ||||||
nn.LayerNorm is recommended. | ||||||
For more information, please refer to :ref:`cn_api_nn_LayerNorm` . | ||||||
|
||||||
Parameters: | ||||||
x(Tensor): Input Tensor. It's data type should be float32, float64. | ||||||
normalized_shape(int|list|tuple): Input shape from an expected input of | ||||||
size :math:`[*, normalized_shape[0], normalized_shape[1], ..., normalized_shape[-1]]`. | ||||||
If it is a single integer, this module will normalize over the last dimension | ||||||
which is expected to be of that specific size. | ||||||
epsilon(float, optional): The small value added to the variance to prevent | ||||||
division by zero. Default: 1e-05. | ||||||
weight(Tensor, optional): The weight tensor of batch_norm. Default: None. | ||||||
bias(Tensor, optional): The bias tensor of batch_norm. Default: None. | ||||||
name(str, optional): Name for the LayerNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. | ||||||
epsilon(float, optional): The small value added to the variance to prevent | ||||||
division by zero. Default: 1e-05. | ||||||
name(str, optional): Name for the LayerNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . | ||||||
|
||||||
Returns: | ||||||
None | ||||||
|
@@ -448,7 +449,7 @@ def instance_norm( | |||||
name=None, | ||||||
): | ||||||
""" | ||||||
See more detail in nn.layer.InstanceNorm2D. | ||||||
It is recommended to use :ref:`cn_api_nn_InstanceNorm1D` , :ref:`cn_api_nn_InstanceNorm2D` , :ref:`cn_api_nn_InstanceNorm3D` to call this method internally. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
Parameters: | ||||||
x(Tensor): Input Tensor. It's data type should be float32, float64. | ||||||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -127,25 +127,25 @@ class InstanceNorm1D(_InstanceNormBase): | |||||
\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ | ||||||
y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift | ||||||
|
||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
|
||||||
Parameters: | ||||||
num_features(int): Indicate the number of channels of the input ``Tensor``. | ||||||
epsilon(float, optional): A value added to the denominator for | ||||||
numerical stability. Default is 1e-5. | ||||||
momentum(float, optional): The value used for the moving_mean and moving_var computation. Default: 0.9. | ||||||
weight_attr(ParamAttr|bool, optional): The parameter attribute for Parameter `scale` | ||||||
of instance_norm. If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
weight_attr(ParamAttr|bool, optional): The parameter attribute for Parameter `scale` of instance_norm. | ||||||
If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as weight_attr, the name of scale can be set in ParamAttr. | ||||||
If the Initializer of the weight_attr is not set, the parameter is initialized | ||||||
one. If it is set to False, will not create weight_attr. Default: None. | ||||||
one. If it is set to False, will not create weight_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
bias_attr(ParamAttr|bool, optional): The parameter attribute for the bias of instance_norm. | ||||||
If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as bias_attr, the name of bias can be set in ParamAttr. | ||||||
If the Initializer of the bias_attr is not set, the bias is initialized zero. | ||||||
If it is set to False, will not create bias_attr. Default: None. | ||||||
If it is set to False, will not create bias_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
data_format(str, optional): Specify the input data format, may be "NC", "NCL". Default "NCL". | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . | ||||||
|
||||||
|
||||||
Shape: | ||||||
|
@@ -170,6 +170,26 @@ class InstanceNorm1D(_InstanceNormBase): | |||||
|
||||||
""" | ||||||
|
||||||
def __init__( | ||||||
self, | ||||||
num_features, | ||||||
epsilon=0.00001, | ||||||
momentum=0.9, | ||||||
weight_attr=None, | ||||||
bias_attr=None, | ||||||
data_format="NCL", | ||||||
name=None, | ||||||
): | ||||||
super().__init__( | ||||||
num_features, | ||||||
epsilon, | ||||||
momentum, | ||||||
weight_attr, | ||||||
bias_attr, | ||||||
data_format, | ||||||
name, | ||||||
) | ||||||
|
||||||
def _check_input_dim(self, input): | ||||||
if len(input.shape) != 2 and len(input.shape) != 3: | ||||||
raise ValueError( | ||||||
|
@@ -198,7 +218,7 @@ class InstanceNorm2D(_InstanceNormBase): | |||||
\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ | ||||||
y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift | ||||||
|
||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
|
||||||
Parameters: | ||||||
num_features(int): Indicate the number of channels of the input ``Tensor``. | ||||||
|
@@ -209,14 +229,14 @@ class InstanceNorm2D(_InstanceNormBase): | |||||
of instance_norm. If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as weight_attr, the name of scale can be set in ParamAttr. | ||||||
If the Initializer of the weight_attr is not set, the parameter is initialized | ||||||
one. If it is set to False, will not create weight_attr. Default: None. | ||||||
one. If it is set to False, will not create weight_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
bias_attr(ParamAttr|bool, optional): The parameter attribute for the bias of instance_norm. | ||||||
If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as bias_attr, the name of bias can be set in ParamAttr. | ||||||
If the Initializer of the bias_attr is not set, the bias is initialized zero. | ||||||
` If it is set to False, will not create bias_attr. Default: None. | ||||||
If it is set to False, will not create bias_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
data_format(str, optional): Specify the input data format, could be "NCHW". Default: NCHW. | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . | ||||||
|
||||||
Shape: | ||||||
- x: 4-D tensor with shape: (batch, num_features, height, weight). | ||||||
|
@@ -239,6 +259,26 @@ class InstanceNorm2D(_InstanceNormBase): | |||||
print(instance_norm_out) | ||||||
""" | ||||||
|
||||||
def __init__( | ||||||
self, | ||||||
num_features, | ||||||
epsilon=0.00001, | ||||||
momentum=0.9, | ||||||
weight_attr=None, | ||||||
bias_attr=None, | ||||||
data_format="NCHW", | ||||||
name=None, | ||||||
): | ||||||
super().__init__( | ||||||
num_features, | ||||||
epsilon, | ||||||
momentum, | ||||||
weight_attr, | ||||||
bias_attr, | ||||||
data_format, | ||||||
name, | ||||||
) | ||||||
|
||||||
def _check_input_dim(self, input): | ||||||
if len(input.shape) != 4: | ||||||
raise ValueError( | ||||||
|
@@ -250,7 +290,7 @@ class InstanceNorm3D(_InstanceNormBase): | |||||
r""" | ||||||
Create a callable object of `InstanceNorm3D`. Applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization . | ||||||
|
||||||
DataLayout: NCHW `[batch, in_channels, D, in_height, in_width]` | ||||||
DataLayout: NCDHW `[batch, in_channels, D, in_height, in_width]` | ||||||
|
||||||
|
||||||
:math:`input` is the input features over a mini-batch. | ||||||
|
@@ -265,7 +305,7 @@ class InstanceNorm3D(_InstanceNormBase): | |||||
\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ | ||||||
y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift | ||||||
|
||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
Where `H` means height of feature map, `W` means width of feature map. | ||||||
|
||||||
Parameters: | ||||||
num_features(int): Indicate the number of channels of the input ``Tensor``. | ||||||
|
@@ -276,14 +316,14 @@ class InstanceNorm3D(_InstanceNormBase): | |||||
of instance_norm. If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as weight_attr, the name of scale can be set in ParamAttr. | ||||||
If the Initializer of the weight_attr is not set, the parameter is initialized | ||||||
one. If it is set to False, will not create weight_attr. Default: None. | ||||||
one. If it is set to False, will not create weight_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
bias_attr(ParamAttr|bool, optional): The parameter attribute for the bias of instance_norm. | ||||||
If it is set to None or one attribute of ParamAttr, instance_norm | ||||||
will create ParamAttr as bias_attr, the name of bias can be set in ParamAttr. | ||||||
If the Initializer of the bias_attr is not set, the bias is initialized zero. | ||||||
If it is set to False, will not create bias_attr. Default: None. | ||||||
If it is set to False, will not create bias_attr. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
data_format(str, optional): Specify the input data format, could be "NCDHW". Default: NCDHW. | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. | ||||||
name(str, optional): Name for the InstanceNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . | ||||||
|
||||||
Shape: | ||||||
- x: 5-D tensor with shape: (batch, num_features, dims, height, weight). | ||||||
|
@@ -306,6 +346,26 @@ class InstanceNorm3D(_InstanceNormBase): | |||||
print(instance_norm_out.numpy) | ||||||
""" | ||||||
|
||||||
def __init__( | ||||||
self, | ||||||
num_features, | ||||||
epsilon=0.00001, | ||||||
momentum=0.9, | ||||||
weight_attr=None, | ||||||
bias_attr=None, | ||||||
data_format="NCDHW", | ||||||
name=None, | ||||||
): | ||||||
super().__init__( | ||||||
num_features, | ||||||
epsilon, | ||||||
momentum, | ||||||
weight_attr, | ||||||
bias_attr, | ||||||
data_format, | ||||||
name, | ||||||
) | ||||||
|
||||||
def _check_input_dim(self, input): | ||||||
if len(input.shape) != 5: | ||||||
raise ValueError( | ||||||
|
@@ -503,11 +563,11 @@ class LayerNorm(Layer): | |||||
division by zero. Default: 1e-05. | ||||||
weight_attr(ParamAttr|bool, optional): The parameter attribute for the learnable | ||||||
gain :math:`g`. If False, weight is None. If is None, a default :code:`ParamAttr` would be added as scale. The | ||||||
:attr:`param_attr` is initialized as 1 if it is added. Default: None. | ||||||
:attr:`param_attr` is initialized as 1 if it is added. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
bias_attr(ParamAttr|bool, optional): The parameter attribute for the learnable | ||||||
bias :math:`b`. If is False, bias is None. If is None, a default :code:`ParamAttr` would be added as bias. The | ||||||
:attr:`bias_attr` is initialized as 0 if it is added. Default: None. | ||||||
name(str, optional): Name for the LayerNorm, default is None. For more information, please refer to :ref:`api_guide_Name`.. | ||||||
:attr:`bias_attr` is initialized as 0 if it is added. Default: None. For more information, please refer to :ref:`cn_api_fluid_ParamAttr` . | ||||||
name(str, optional): Name for the LayerNorm, default is None. For more information, please refer to :ref:`api_guide_Name` . | ||||||
|
||||||
Shape: | ||||||
- x: 2-D, 3-D, 4-D or 5-D tensor. | ||||||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.