v4.0.x: Cherry pick ob1 fixes from master #6634
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In case of using a btl_put in ob1, the handle of the locally registered memory is sent with a PUT control message. In the current master code the sent handle is necessary the handle in the frag but if the handle has been successfully registered in the request, the frag structure does not have any valid handle and all fragments use the request one. I suggest to check if the handle in the fragment is valid and if not to send the handle from the request. Signed-off-by: Brelle Emmanuel <emmanuel.brelle@atos.net> (cherry picked from commit e630046)
|
The IBM CI (XL Compiler) build failed! Please review the log, linked below. Gist: https://gist.github.com/90c02b30b0acdf60d49f6eff5f77e769 |
|
The IBM CI (GNU Compiler) build failed! Please review the log, linked below. Gist: https://gist.github.com/e21e4f2797e937a188a1f6df3cd01bed |
The rdma_frag attached to the send request was not correctly released upon request completion, leaking until MPI_Finalize. A quick solution would have been to add RDMA_FRAG_RETURN at different locations on the send request completion, but it would have unnecessarily made the sendreq completion path more complex. Instead, I added the length to the RDMA fragment so that it can be completed during the remote ack. Be more explicit on the comment. The rdma_frag can only be freed once when the peer forced a protocol change (from RDMA GET to send/recv). Otherwise the fragment will be returned once all data pertaining to it has been trasnferred. NOTE: Had to add a typedef for "opal_atomic_size_t" from master into opal/threads/thread_usage.h into this cherry pick (it is in opal/include/opal_stdatomic.h on master, but that file does not exist here on the v4.0.x branch). Signed-off-by: George Bosilca <bosilca@icl.utk.edu> (cherry picked from commit a16cf0e) Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
jsquyres
force-pushed
the
pr/v4.0.x/ob1-fixes
branch
from
97bb75c to
48f8243
Compare
Member
|
Lets discuss on Tuesday. Perhaps we should pull this PR, and then do a separate PR for #6633 content later. |
Member
|
@hppritcha We didn't get to this PR in today's web-ex. Lets bump discussion to next week. |
Member
|
at 6/25/19 devel meeting we decided this can be merged in but it doesn't fix issues with the put protocol pipelining (it fixes a different OB1 PUT protocol issue). |
bosilca
approved these changes
Jun 25, 2019
This was referenced Jun 28, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #6633 for a full explanation.
NOTE: The cherry-pick of a16cf0e had to include an extra
typedefinopal/threads/thread_usage.hfrom elsewhere in the tree (and not part of a16cf0e), clearly noted in the commit.FYI @EmmanuelBRELLE