Info key(s) for Start Ordering Persistent Collective Communication

[tonyskjellum]

Problem

We want to add one or more info keys, such as one that guarantees that the persistent collective operation will only be started in order across the group of its communicator. Other info keys relevant to persistent collective initialization or operations may also be considered in addition to this one.

Note: this ticket actually describes the ordering rules for persistent collectives with and without this assertion with regard to all other collective communication on the same communicator.

Proposal

Specify at least one info key for ordered-across-group in persistent collectives at communicator scope. Others TBD.

Changes to the Text

Appropriate text will be added that defines the info key and its semantics. If other keys are conceived of quickly, they may be added too. So far, just one info key,

Impact on Implementations

For the specific info key specified thus far, it is an assertion. Implementations can ignore this assertion, or exploit it for greater performance. The assertion is mpi_assert_strict_persistent_collective_ordering ; it requires that programs order persistent collective starts [defined via a communicator that makes this assertion] under the same rules as other collectives. If an implementation can exploit this assertion for greater performance, it can do so, or do legally do nothing.

Impact on Users

Provides an info key; requires that the programs meet the restricted semantics if a given communicator with this assertion is used to generate persistent collective operations.

References

The original pull request for this ticket is https://github.com/mpi-forum/mpi-standard/pull/278
Merged too early: https://github.com/mpi-forum/mpi-standard/pull/661
Reverted by: https://github.com/mpi-forum/mpi-standard/pull/662

This is the PR that fixes this issue: https://github.com/mpi-forum/mpi-standard/pull/666

PDF File: mpi40-report-ticket83-12sep20.pdf

Updated PDF:
mpi40-report-ticket83-04may22.pdf

[tonyskjellum]

See page 217 of initial text...

mpi32-pers-infokeys-report-01mar18.pdf

[tonyskjellum]

Verified today that this info key is at the communicator scope.

[tonyskjellum]

mpi32-report-ticket83.pdf

This is the public copy.

[tonyskjellum]

We need to make updates based on the reading attempt on June 13, 2018 (all minor). We will present a new reading in Barcelona.

[puribangalore]

Here is the latest PDF file for reading at Barcelona:
mpi32-report-ticket83-sep2018.pdf

[tonyskjellum]

Outcome of reading

1. Make clear that the info key provides a total ordering of all collective calls on a given communicator, not just a partial order involving Blocking/Nonblocking and another involving all Persistent. In other words, clarify the semantic impact.
2. Refer to the semantic terms chapter for clear definition of start; do not restate it here.
   The semantic terms chapter is getting an additional virtual meeting before December f2f meeting.
3. Add an example of what is "not legal" once we turn on this flag, for a given communicator. This will complement Puri's example(s) being worked for Ticket Reorganizing the "Collective Communication" Chapter #90 .
4. Make sure it is clear that MPI_Startall is just a part of the story, not the whole story.

[tonyskjellum]

Rudiments of the example/counter-examples:

Restrictions with key:

    persistent_op, persistent_op2 are specified on comm1

Rank 0:

    MPI_Bcast(...,comm1)
    MPI_Start(persistent_op)
    MPI_Reduce(,...,comm1)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

All other ranks must do the exact same dispatch order for these collectives.

illegal:

    MPI_Bcast(...,comm1)
    MPI_IReduce(,...,comm1)
    MPI_Start(persistent_op)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

Without key:

Rank 0:

    MPI_Bcast(...,comm1)
    MPI_Start(persistent_op)
    MPI_IReduce(,...,comm1)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

Rank 1:

    MPI_Start(persistent_op)
    MPI_Bcast(...,comm1)
    MPI_IReduce(,...,comm1)
    MPI_Gather(...,comm1)
    MPI_Start(persistent_op2)

legal.

Rank N-1:
MPI_Bcast(...,comm1)
MPI_IReduce(,...,comm1)
MPI_Start(persistent_op2)
MPI_Gather(...,comm1)
MPI_Start(persistent_op)

Legal

Legal --> Other collectives still are issued in same order across group.

[tonyskjellum]

mpi32-report-ticket83-19nov18.pdf

Here is the rewritten and revised text.

[tonyskjellum]

We did the reading in San Jose.

1. @dholmes-epcc-ed-ac-uk and I will clarify with word 'defined' to the word 'started' in the text (change bar line 4314)

2. we will also copy the definition of the total ordering into the section (line 4287-4297)... that will change the overall text by repetition. Then we will streamline to remove
   redundancy.

3. @abouteiller reminded us to make the examples 'compilable'. So, we will make the examples compilable.

The group agrees that we will take ticket 0 votes in Chattanooga on these changes above, which do not change the syntax or semantics of the info key, and then if it passes, take the first vote in Chattanooga in March, 2019. If ticket 0 votes fails, we will re-read in Chattanooga.

[wesbland]

This was merged because of my mistake. I'll reopen this issue and work with @wgropp and @tonyskjellum to create a new PR.

[Wee-Free-Scot]

@tonyskjellum we owe this start ordering info key from the persistent collective operations work. It would be great if this hit the MPI-4.1 release. I updated the PR by merging mpi-4.x into it. This should not need further changes, but we will need to restart the procedure with a reading in the Dec 22 meeting.

[tonyskjellum]

Yes we should read again in December … thank you for doing this merging … this is my albatross (cf, Rime of Ancient Mariner).

On Thu, Oct 20, 2022 at 12:30 PM Dan Holmes ***@***.***> wrote: @tonyskjellum <https://github.com/tonyskjellum> we owe this start ordering info key from the persistent collective operations work. It would be great if this hit the MPI-4.1 release. I updated the PR by merging mpi-4.x into it. This should not need further changes, but we will need to restart the procedure with a reading in the Dec 22 meeting. — Reply to this email directly, view it on GitHub <#83 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7ZCNKBZH3ABDGTXLMOETWEFXTLANCNFSM4ETCZGJQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Anthony Skjellum, PhD ***@***.*** Cell: +1-205-807-4968

[wesbland]

This had a reading on 2022-12-07. There were some no-no items raised (e.g., the changelog) that will be re-read at the March 2023 meeting.

[Wee-Free-Scot]

Latest PDF (with fixed change log and moved AtoU) for Feb'23 voting meeting:
mpi41-report_Issue083_ticket83-2022redo.pdf

[wesbland]

This passed a no-no vote on 2023-02-01.

Yes	No	Abstain
31	0	0

[wesbland]

This passed a first vote on 2023-02-08.

Yes	No	Abstain
27	0	4

[mpiforumbot]

This passed a 2nd vote.

Yes	No	Abstain
30	0	1