Add float16 support to relu op #9267
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
helinwang
reviewed
Mar 21, 2018
| def test_check_output(self): | ||
| if core.is_compiled_with_cuda(): | ||
| place = core.CUDAPlace(0) | ||
| if core.is_float16_supported(place): |
There was a problem hiding this comment.
Just to double check: will the condition ever become True?
There was a problem hiding this comment.
It will be true if the compute capability of GPU is >= 5.3, which means all pascal and volta GPUs will be tested.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix #9266
I originally plan to add one single line in
activation_op.cuto add fp16 forward kernel for all activation ops. However, this brings up tons of eigen error message requiring lots of modification tofloat16.h. Because of the currently limited time, I am temporarily putting this plan on hold and simply add fp16 support for relu op in this PR.