New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MPI_PROC_NULL in Topology Creation #4675
Comments
My understanding of the MPI standard description of the topology creation functions is that the list of neighbors is rank based, and should only contains the meaningful neighbors. Thus, you cannot have MPI_PROC_NULL as a neighbor in any type of topology. |
It is true that the distributed graph constructors specify
I'm not quite sure what you mean by meaningful neighbors—that term isn't used in the specification, and is certainly vague, as what is 'meaningful' can vary between implementations. The specification in fact explicitly allows edges to be defined multiple times for the same Regarding
This would appear to imply that |
meaningful neighbor is a neighbor that define an edge where a communication will take place. This does not prevent 2 ranks of being neighbors multiple times, not a rank of being it's own neighbor. Communications with MPI_PROC_NULL are meaningless, and except for adding gaps into the communication buffers I cant see why you want to specify them. This discussion pertains to the MPI standardization effort, I would suggest to ask the question on the MPI Forum mailinglist (mpi-forum@lists.mpi-forum.org). |
I ended up posting this on the MPI Comments mailing list: Behavior of [Distributed] Graph Topology Constructors when a neighbor is MPI_PROC_NULL Adding gaps in communication buffers is exactly the point here; it allows one to say "every process has
This is useful for e.g. binary tree topologies (as described above). |
Allowing MPI_PROC_NULL as a neighbor in any topology allows us to add gaps on the send and recv buffers. This does make the traditional neighbor collective have a similar behavior as the V version, but in same time it allows the users to skip the step where they prepare the counts and the displacement array. For more info please take a look at issue open-mpi#4675. Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
Allowing MPI_PROC_NULL as a neighbor in any topology allows us to add gaps on the send and recv buffers. This does make the traditional neighbor collective have a similar behavior as the V version, but in same time it allows the users to skip the step where they prepare the counts and the displacement array. For more info please take a look at issue open-mpi#4675. Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
From a practical perspective I see your point. You want a similar level of flexibility as the V version of the neighbor collective calls, but without having to provide an array of counts, nor compute locally the neighbors displacements. I make a PR #4898. Give it a try and let us know. Meanwhile I will try to get convinced by the MPI forum that this is the right approach. |
I'll take a look at the PR. I'll also post on the MPI issues repository, as it seems more active than the mailing lists. |
It appears that MPICH actually supports this behavior. I've submitted an issue to the MPI Issues repository to clarify the wording to make this explicit. |
I'm trying to create a binary tree topology using
MPI_Dist_graph_create_adjacent()
, simplifying the graph boundaries by usingMPI_PROC_NULL
. This allows specifying all nodes in a consistent way.I'm not sure this is allowed by the specification; I could find no information either way.
However, the neighborhood collectives specify that the borders of a cartesian topology act as though they send and receive from
MPI_PROC_NULL
. It could be useful to be able to obtain similar behavior in a generic graph.Example code is given below:
Currently this code results in the following error:
The text was updated successfully, but these errors were encountered: