MKLDNN layout: Support for batch norm operator #11098
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced May 31, 2018
Merged
mozga-intel
force-pushed
the
mozga-intel/Batch_norm_mkldnn_layout
branch
from
69b8f56
to
7d56435
Compare
|
@luotao1 The code is prepared to the code-review process. |
tensor-tang
reviewed
Jun 11, 2018
| bool is_diff_dst_reordered = false; | ||
| if (diff_dst_pd != user_diff_dst_memory.get_primitive_desc()) { | ||
| diff_dst_memory = memory(diff_dst_pd); | ||
| reorder_diff_dst = reorder(user_diff_dst_memory, diff_dst_memory); |
There was a problem hiding this comment.
@tensor-tang Yes.
There was a problem hiding this comment.
But does mkldnn reorder support inplace?
There was a problem hiding this comment.
"Both forward and backward passes support in-place operation, i.e. src and dst point to the same memory for forward, and diff_dst and diff_src point to the same memory for backward pass."
info
There was a problem hiding this comment.
Actually my point is does mkldnn::reorder support inplace, not batchnorm.
AFAIK, mkldnn::reorder does not support inplace yet.
While, you are not using inplace reorder here.
auto diff_dst_memory = user_diff_dst_memory;
...
diff_dst_memory = memory(diff_dst_pd);
This will allocate new memory.
So that's fine.
tensor-tang
approved these changes
Jun 13, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Waiting for supports of the mkldnn’s layout.
Please have a look at this batch norm operator supported by the MKLDNN’s layout and assess if we can keep this design of the code.
This code uses the implementation of layout which is available in the last pull-request. Therefore some of the function in this code are not available in this pull-request, but are available here.
This version of code can be merged into the main branch when the pull-request with layout is accepted.
The concept of splits of the long code was proposed by @luotao1
Pull-request is related to #11040