split vector-matrix norm #5478
Merged
split vector-matrix norm #5478
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MARD1NO
requested changes
Jul 13, 2021
oneflow/python/nn/modules/norm.py
Outdated
| def __init__(self, ord=None, dim=None, keepdim=False) -> None: | ||
| super().__init__() | ||
|
|
||
| self.ord = ord | ||
| self.ord = 2 if ord == None else ord |
There was a problem hiding this comment.
后续感觉还需要一个if else判断，让他变成float输入到后续
if not isinstance(ord, str):
ord = float(ord)
oneflow/python/nn/modules/norm.py
Outdated
| ), | ||
| 1.0 / ord, | ||
| ) | ||
|
|
||
| elif isinstance(ord, int): |
There was a problem hiding this comment.
这个分支处理没有必要，前面做一个float转换。因为实际上int分支和float分支的计算逻辑是一样的
oneflow/python/nn/modules/norm.py
Outdated
| @@ -51,6 +58,19 @@ def _vector_norm(self, x, ord, dim): | |||
| else: | |||
| raise ValueError("Invalid norm order: {}".format(ord)) | |||
|
|
|||
| def forward(self, x): | |||
| return self._vector_norm(x.reshape((1, -1))[0], ord = self.ord, dim=self.dim) | |||
There was a problem hiding this comment.
reshape两维是不正确的，torch是可以支持多维的。
oneflow/python/nn/modules/norm.py
Outdated
| def __init__(self, ord=None, dim=None, keepdim=False) -> None: | ||
| super().__init__() | ||
|
|
||
| self.ord = ord | ||
| self.ord = 2 if ord == None else ord | ||
| self.dim = dim | ||
| self.keepdim = keepdim | ||
|
|
||
| def _vector_norm(self, x, ord, dim): | ||
| if isinstance(ord, str) and ord in ["fro", "nuc"]: |
MARD1NO
reviewed
Jul 16, 2021
| self.ord = ord | ||
| if ord == None: | ||
| self.ord = 2.0 | ||
| elif isinstance(ord, (int, float)): |
There was a problem hiding this comment.
去除了 float 的话，传入 float 就会报错了吧
oneflow/python/nn/modules/norm.py
Outdated
|
|
||
|
|
||
| class Vector_Norm(Module): | ||
| def __init__(self, ord, dim, keepdim) -> None: |
There was a problem hiding this comment.
oneflow/python/nn/modules/norm.py
Outdated
| num_dims = len(x.shape) | ||
| dim = check_dim(num_dims, self.dim) | ||
| if dim == None: | ||
| return self._vector_norm(x.reshape((1, -1))[0], ord = self.ord, dim=self.dim, keepdim= self.keepdim) |
There was a problem hiding this comment.
reshape((1, -1))这里可以改成flatten吧
oneflow/python/nn/modules/norm.py
Outdated
| def _matrix_norm(self, x, ord, dim): | ||
|
|
||
| class Matrix_Norm(Module): | ||
| def __init__(self, ord, dim, keepdim) -> None: |
There was a problem hiding this comment.
oneflow/python/nn/modules/norm.py
Outdated
| raise TypeError("linalg_matrix_norm(): argument 'ord' must be Number, not {}".format(type(ord))) | ||
| if isinstance(dim,tuple) and len(dim) == 2 and dim[0] != dim[1]: | ||
| self.dim = dim | ||
| elif dim == None: |
There was a problem hiding this comment.
这部分按照前面设置好默认值，可以去掉这部分判断逻辑，不用判断dim==None
There was a problem hiding this comment.
这边感觉不能去掉吧，linalg.norm 的默认 dim == None，在那个里面调 Matrix_norm 的话就会把 none 传进来了（line 157）。或者就是我在 linalg.norm 里判断一下
oneflow/python/nn/modules/norm.py
Outdated
| raise NotImplementedError | ||
| elif ord == "fro": | ||
| return flow.experimental.sqrt( | ||
| flow.experimental.sum(flow.experimental.square(x), dim=dim, keepdim= keepdim) |
oneflow/python/nn/modules/norm.py
Outdated
|
|
||
| def _norm_min_max(input, ord,dim,keepdim): | ||
| if ord > 0: | ||
| return flow.experimental.max(input, dim= dim,keepdim = keepdim) |
oneflow/python/nn/modules/norm.py
Outdated
|
|
||
| def _norm_min_max(input, ord,dim,keepdim): | ||
| if ord > 0: | ||
| return flow.experimental.max(input, dim= dim,keepdim = keepdim) |
MARD1NO
approved these changes
Jul 16, 2021
| self.ord = ord | ||
| if ord == None: | ||
| self.ord = 2.0 | ||
| elif isinstance(ord, (int, float)): |
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
oneflow-ci-bot
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
torch.norm后续会弃用，因此选择与torch.linalg.norm接口对齐。vector_norm支持 float 类型 order