Backport/datatype #8837
Merged
Backport/datatype #8837
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bosilca
force-pushed
the
backport/datatype
branch
from
597f63a
to
f15f4e2
Compare
Signed-off-by: George Bosilca <bosilca@icl.utk.edu> (cherry picked from commit e8ebe13)
Signed-off-by: Christoph Niethammer <niethammer@hlrs.de> (cherry picked from commit 9901325)
Signed-off-by: Austen Lauria <awlauria@us.ibm.com> (cherry picked from commit ef28e8d)
This commit fixes the support for heterogeneous environments and specifically for external32. The root cause was that during the datatype optimization process types that are contiguous in memory are collapsed together in order to decrease the number of conversion (or memcpy) function calls. The resulting type however, does not have the same conversion rules as the types it replaced, leading to an incorrect (or absent) conversion in some cases. This patch marks the datatypes where types have been collapsed during the optimization process with a flag, allowing the convertor to detect if the optimized type can be used in heterogeneous setups. Signed-off-by: George Bosilca <bosilca@icl.utk.edu> (cherry picked from commit 73d64cb)
When unpacking a partial predefined element check the boundaries of the description vector type, and adjust the memory pointer accordingly (to reflect not only when a single basic type was correctly unpacked, but also when an entire blocklen has been unpacked). Signed-off-by: George Bosilca <bosilca@icl.utk.edu> (cherry picked from commit fb07960)
bosilca
force-pushed
the
backport/datatype
branch
from
f15f4e2
to
85d2dbc
Compare
Signed-off-by: George Bosilca <bosilca@icl.utk.edu> bot:notacherrypick
bosilca
force-pushed
the
backport/datatype
branch
from
85d2dbc
to
565d72e
Compare
|
@open-mpi/ucx Could you guys verify that this fixes this issue for you on v4.1.x ASAP? |
|
Confirmed: issue is resolved. |
bwbarrett
approved these changes
Apr 22, 2021
|
@hoopoepg Could you do me a huge favor (since I don't have access to UCX/IB networks)? Could you try that reproducer on v4.0.0? I'd like to know how far back this issue goes. |
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brings some updates on the datatype engine into the 4.1. Among these the most critical is the partial unpack bug from #8466.
Here are the commits from master that are covered by this PR:
e8ebe13
9901325
ef28e8d
73d64cb
fb07960
It must be noted that this PR does not bring the support for MPI_LONG and MPI_UNSIGNED_LONG in external32, because it would have required to break the ABI (because of the 2 new datatypes #define added).
Unfortunately, I had to import 2 additional commits in order to be able to build and run on an M1: 4f2dde0 and 73aae14.
Fixes #8466.
One of these commits is intentionally not a cherry pick: bot:notacherrypick