This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
[MXNET-101] Support float16 in LeakyReLU operator #10169
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
haojin2
force-pushed
the
fp16_leaky_relu
branch
2 times, most recently
from
0fcaa7d
to
512feea
Compare
marcoabreu
reviewed
Mar 20, 2018
| @@ -217,7 +237,7 @@ class LeakyReLUOp : public Operator { | |||
| }; // class LeakyReLUOp | |||
|
|
|||
| template<typename xpu> | |||
| Operator* CreateOp(LeakyReLUParam type); | |||
| Operator* CreateOp(LeakyReLUParam type, int dtype); | |||
There was a problem hiding this comment.
Sorry, I'm not familiar with the cpp code, but is there no typedef for dtype - int is kind of arbitrary.
There was a problem hiding this comment.
The types are represented by integers, please see leaky_relu.cc:43 for how this function is being called. Also you can refer to https://github.com/dmlc/mshadow/blob/b3771de20ed36f90ba7b8436ae4b79ea298a687a/mshadow/base.h#L288 for more info about how each integer map to a type. This change is written in a similar way as #2322 #3011 #10078 #10125 .
There was a problem hiding this comment.
Great, thanks for the explanation :)
haojin2
force-pushed
the
fp16_leaky_relu
branch
5 times, most recently
from
fb0c6c2
to
5390b43
Compare
haojin2
force-pushed
the
fp16_leaky_relu
branch
from
5390b43
to
d44a588
Compare
ashokei
pushed a commit
to ashokei/incubator-mxnet
that referenced
this pull request
Mar 27, 2018
* support for any datatype in leaky ReLU * test for LeakyReLU operators * make lint * clean up unnecessary prints * fix for amalgamation build failure * add InferType for Leaky ReLU and slight modification to the tests
jinhuang415
pushed a commit
to jinhuang415/incubator-mxnet
that referenced
this pull request
Mar 30, 2018
* support for any datatype in leaky ReLU * test for LeakyReLU operators * make lint * clean up unnecessary prints * fix for amalgamation build failure * add InferType for Leaky ReLU and slight modification to the tests
rahul003
pushed a commit
to rahul003/mxnet
that referenced
this pull request
Jun 4, 2018
* support for any datatype in leaky ReLU * test for LeakyReLU operators * make lint * clean up unnecessary prints * fix for amalgamation build failure * add InferType for Leaky ReLU and slight modification to the tests
zheng-da
pushed a commit
to zheng-da/incubator-mxnet
that referenced
this pull request
Jun 28, 2018
* support for any datatype in leaky ReLU * test for LeakyReLU operators * make lint * clean up unnecessary prints * fix for amalgamation build failure * add InferType for Leaky ReLU and slight modification to the tests
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add support for any datatype for Leaky ReLU operator, mentioned in issue #2302.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.
Changes
Comments
The test for rrelu is harder to write as rrelu contains random components in forward pass, will figure out a way to write a test for it later.