Nonblocking Communicator Duplication

[mpiforumbot]

Originally by htor on 2009-07-06 13:47:56 -0500

---

MPI_Comm_idupAuthor: Torsten Hoefler

Description

Nonblocking communicator creating is needed for implementing nonblocking communication libraries such as LibNBC. For a full explanation of the issues see Hoefler, Snir: " Writing Parallel Libraries with MPI - Common Practice, Issues, and Extensions"

History

it was brought up for MPI-2.2 but deferred to MPI-3.0

Proposed Solution

after MPI_Comm_dup add on page 202:21 (MPI-2.2):

MPI_COMM_IDUP(comm, newcomm, request)
IN comm communicator (handle)
OUT newcomm copy of comm (handle)
OUT request communication request (handle)

int MPI_Comm_idup(MPI_Comm comm, MPI_Comm *newcomm, MPI_Request *request)

MPI_COMM_IDUP(COMM, NEWCOMM, REQUEST, IERROR)
INTEGER COMM, NEWCOMM, REQUEST, IERROR

MPI_COMM_IDUP is a nonblocking variant of MPI_COMM_DUP. The semantics 
of MPI_COMM_IDUP are as if MPI_COMM_DUP was executed at the time 
that MPI_COMM_IDUP is called. For example, attributes changed
after MPI_COMM_IDUP will not be copied to the new communicator. All 
restrictions and assumptions for nonblocking collective 
operations (see Section 5.12) apply to MPI_COMM_IDUP and 
the returned request.

It is erroneous to use the communicator \mpiarg{newcomm} as an 
input argument to other MPI functions before the MPI_COMM_IDUP 
operation completes.

Rationale: This functionality is crucial for the development 
of purely nonblocking libraries (see~\ref{MPIlibraries}).

Impact on Implementations

Low, can be easily built on top of nonblocking collectives.

Impact on Applications / Users

None if not used.

Alternative Solutions

none

Entry for the Change Log

Added MPI_COMM_IDUP

[mpiforumbot]

Originally by rsthakur on 2009-07-06 13:52:34 -0500

---

You can't do anything with the communicator returned by this function until test/wait, can you? In that case, what will you do in the nonblocking collective communication call that uses this function?

[mpiforumbot]

Originally by htor on 2011-11-24 10:49:27 -0600

---

Replying to rsthakur:

You can't do anything with the communicator returned by this function until test/wait, can you?
No, you can't and I made this explicitly erroneous (for performance reasons).

In that case, what will you do in the nonblocking collective communication call that uses this function?
The nonblocking collective needs to contain a state-machine (in its progress) that will not start any communication before the communicator creation completed. It's a very simple extension to libNBC.

Torsten

[mpiforumbot]

Originally by jsquyres on 2012-01-06 13:35:48 -0600

---

The sentence "This functionality is crucial for the development of purely nonblocking libraries (see~\ref{MPIlibraries})." seems like it should be in a Rationale section, not in the middle of the text. Moving that text later then makes the beginning part of the next sentence redundant.

Additionally, the "It is erroneous..." sentence is quite important. I would make it a new paragraph.

Finally, "...the MPI_ICOMM_DUP operation completes." Do we use that kind of terminology elsewhere in the text? Or do we refer to the completion of the request returned by a non-blocking function?

Here's what I would suggest:

MPI_ICOMM_DUP is a nonblocking variant of MPI_COMM_DUP. All restrictions and assumptions for nonblocking collective operations (see Section 5.12) apply to MPI_ICOMM_DUP and the returned request.

It is erroneous to use the communicator \mpiarg{newcomm} as input argument to other MPI functions before the MPI_ICOMM_DUP operation completes.

Rationale. This functionality is crucial for the development of purely nonblocking libraries (see~\ref{MPIlibraries}). End of Rationale.

(possibly with changes about "...the MPI_ICOMM_DUP operaton completes", depending on Torsten's reply...)

[mpiforumbot]

Originally by RolfRabenseifner on 2012-01-07 12:54:32 -0600

---

Ticket should include also a Fortran interface (for mpif.h and mpi module).

[mpiforumbot]

Originally by htor on 2012-01-07 13:18:51 -0600

---

Replying to jsquyres:

The sentence "This functionality is crucial for the development of purely nonblocking libraries (see~\ref{MPIlibraries})." seems like it should be in a Rationale section, not in the middle of the text. Moving that text later then makes the beginning part of the next sentence redundant.
Good point, changed.

Additionally, the "It is erroneous..." sentence is quite important. I would make it a new paragraph.
ok

Finally, "...the MPI_ICOMM_DUP operation completes." Do we use that kind of terminology elsewhere in the text? Or do we refer to the completion of the request returned by a non-blocking function?
Yes, all over in the nonblocking p2p and collective chapter (see intro to nonblocking colls).

Here's what I would suggest:

MPI_ICOMM_DUP is a nonblocking variant of MPI_COMM_DUP. All restrictions and assumptions for nonblocking collective operations (see Section 5.12) apply to MPI_ICOMM_DUP and the returned request.

It is erroneous to use the communicator \mpiarg{newcomm} as input argument to other MPI functions before the MPI_ICOMM_DUP operation completes.

Rationale. This functionality is crucial for the development of purely nonblocking libraries (see~\ref{MPIlibraries}). End of Rationale.

(possibly with changes about "...the MPI_ICOMM_DUP operaton completes", depending on Torsten's reply...)
Yep.

Ticket is updated.

Thanks & Best,
Torsten

[mpiforumbot]

Originally by htor on 2012-01-07 13:20:37 -0600

---

Replying to RolfRabenseifner:

Ticket should include also a Fortran interface (for mpif.h and mpi module).
It had (and has) the old Fortran bindings (mpif.h?). I don't really know what you mean by "mpi module". The bindings I added are similar to what we have been using in MPI-2.2 and MPI-3.0 so far.

Can you please provide those bindings for this small function?

Thanks & Best,
Torsten

[mpiforumbot]

Originally by jsquyres on 2012-01-10 22:16:58 -0600

---

Torsten got some minor ticket 0 feedback; he will update the ticket.

[mpiforumbot]

Originally by goodell on 2012-03-02 18:13:39 -0600

---

This is now basically implemented in MPICH2 as of r9556 on the trunk:

https://trac.mcs.anl.gov/projects/mpich2/changeset/9556

The only caveat here is that I did not make the implementation safe when multithreading, that is a substantially more complicated task. The current implementation just throws an error when MPIX_Comm_idup is called if MPI was initialized with a thread level greater than MPI_THREAD_SINGLE.

[mpiforumbot]

Originally by htor on 2012-03-07 12:18:59 -0600

---

Edited during the forum, the Forum agreed that this was a ticket 0 change.

[mpiforumbot]

Originally by htor on 2012-07-04 16:42:20 -0500

---

Set chapter author

[mpiforumbot]

Originally by gropp on 2012-07-18 10:29:18 -0500

---

MPI_Comm_idup is now in the document. Please review.

[mpiforumbot]

Originally by davesolt on 2012-07-18 11:46:12 -0500

---

Reviewed the pdf generated from approved@1400

[mpiforumbot]

Originally by RolfRabenseifner on 2013-01-07 11:02:46 -0600

---

Since Sep. 21, 2012, this ticket is included in MPI-3.0 and the pdf is checked according https://svn.mpi-forum.org/svn/mpi-forum-docs/trunk/meetings/2012-07-jul/mpi3-tickets.xlsx

Therefore, by proxy / on behalf of the ticket owner, I close it with priority "Ticket complete", resolution "Text committed".