Add communication modes to specialize send routines #30
Conversation
|
We may want to draft this for now as I haven't updated the corresponding documentation yet. |
|
The PR now does the communication mode dispatch in the It now also lets users override the default communication mode to alias |
src/impl/KokkosComm_isend.hpp
Outdated
| if constexpr (SendMode == CommMode::Standard) { | ||
| MPI_Isend(KCT::data_handle(args.view), args.count, args.datatype, dest, | ||
| tag, comm, &req.mpi_req()); | ||
| } else if constexpr (SendMode == CommMode::Ready) { | ||
| MPI_Irsend(KCT::data_handle(args.view), args.count, args.datatype, dest, | ||
| tag, comm, &req.mpi_req()); | ||
| } else if constexpr (SendMode == CommMode::Synchronous) { | ||
| MPI_Issend(KCT::data_handle(args.view), args.count, args.datatype, dest, | ||
| tag, comm, &req.mpi_req()); | ||
| } else if constexpr (SendMode == CommMode::Default) { | ||
| #ifdef KOKKOSCOMM_FORCE_SYNCHRONOUS_MODE | ||
| MPI_Issend(KCT::data_handle(args.view), args.count, args.datatype, dest, | ||
| tag, comm, &req.mpi_req()); | ||
| #else | ||
| MPI_Isend(KCT::data_handle(args.view), args.count, args.datatype, dest, | ||
| tag, comm, &req.mpi_req()); | ||
| #endif | ||
| } | ||
| req.keep_until_wait(args.view); |
There was a problem hiding this comment.
I'd rather define a lambda so you don't have to repeat the logic for tyes that need packing or not.
|
@masterleinad Can you tell me if the lambda definitions look ok? Using |
MPI's `Buffered` mode requires the user to attach a buffer using `MPI_Buffer_attach`. We do not want to bother finding a way to support this cleanly for now, hence its removal.
Remove `static_assert` from unreachable path
Co-authored-by: Cédric Chevalier <cedric.chevalier019@proton.me>
Renamed the enum to `CommMode` to better reflect what it is. Renamed the `Default` variant to `Standard` to match wording in the MPI spec. Added comments to explain the semantics of each communication mode.
This change reduces code duplication in the `Impl` namespace, where the dispatch is now performed in a single interface. We have also added an environment variable `KOKKOSCOMM_FORCE_SYNCHRONOUS_SEND_MODE` to ensure that all operations with `CommMode::Default` mode instead alias to synchronous sends instead of standard ones (mainly for debugging purposes).
If send operations do not specify a communication mode, use the default behavior. If the `KokkosComm_FORCE_SYNCHRONOUS_MODE` is defined (through the pre-processor), the default behavior is to use `Synchronous` mode. Otherwise, it defaults to `Standard` mode.
Co-authored-by: Daniel Arndt <arndtd@ornl.gov>
Removes reference-captures and adds explicit typing.
* mpi: MPI_Barrier
Also fixes some errors in the docs
|
I rebased the latest commits from After discussing with @cedricchevalier19, we believe it's simpler to skip tests for ready-mode send operations for now. We should merge this PR and open an issue to fix ready-send tests while we wait for #32 to be merged. |
|
Works for me, sorry for the delay on #32. Open such an issue and then we will merge once conflicts are resolved! |
|
Issue #43 for the skipped tests is opened. I think we can merge this now 👍 |
This PR adds an enum to allow users to specify the communication mode of send routines (both blocking and non-blocking) as proposed by @cedricchevalier19 in #10. The
CommModetemplate parameter is only used for the user-facing API: there are dedicated lean wrappers for each underlying MPI call in theImplnamespace (as suggested by @cwpearson, also in #10). Relevant unit tests have been added too.The changes fulfill the requirements for supporting
rsendandssendfrom #10.We do not support "Buffered" mode (at least for now), as it requires the user to manually attach a buffer using
MPI_Buffer_attach(see documentation here).