Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement hierarchical MPI_Gatherv and MPI_Scatterv #12376

Merged
merged 3 commits into from
Mar 23, 2024
Merged

Implement hierarchical MPI_Gatherv and MPI_Scatterv #12376

merged 3 commits into from
Mar 23, 2024

Commits on Mar 22, 2024

  1. coll/han: refactor mca_coll_han_get_ranks function 

    Relax the function requirement to allow null low/up_rank output
pointers, and rename the arguments because the function works for
non-root ranks as well.

Signed-off-by: Wenduo Wang <wenduwan@amazon.com>
    @wenduwan
    wenduwan committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    b22e5fa View commit details
    Browse the repository at this point in the history

  2. coll/han: implement hierarchical gatherv 

    Add gatherv implementation to optimize large-scale communications on
multiple nodes and multiple processes per node, by avoiding high-incast
traffic on the root process.

Because *V collectives do not have equal datatype/count on every
process, it does not natively support message-size based tuning without
an additional global communication.

Similar to gather and allgather, the hierarchical gatherv requires a
temporary buffer and memory copy to handle out-of-order data, or
non-contiguous placement on the output buffer, which results in worse
performance for large messages compared to the linear implementation.

Signed-off-by: Wenduo Wang <wenduwan@amazon.com>
    @wenduwan
    wenduwan committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    48c125e View commit details
    Browse the repository at this point in the history

  3. coll/han: implement hierarchical scatterv 

    Add scatterv implementation to optimize large-scale communications on
multiple nodes and multiple processes per node, by avoiding high-incast
traffic on the root process.

Because *V collectives do not have equal datatype/count on every
process, it does not natively support message-size based tuning without
an additional global communication.

Similar to scatter, the hierarchical scatterv requires a
temporary buffer and memory copy to handle out-of-order data, or
non-contiguous placement on the send buffer, which results in worse
performance for large messages compared to the linear implementation.

Signed-off-by: Jessie Yang <jiaxiyan@amazon.com>
    @jiaxiyan @wenduwan
    jiaxiyan authored and wenduwan committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    2152b61 View commit details
    Browse the repository at this point in the history