Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Info key(s) for Start Ordering Persistent Collective Communication #83

Open
tonyskjellum opened this issue Mar 1, 2018 · 15 comments
Open

Info key(s) for Start Ordering Persistent Collective Communication #83

tonyskjellum opened this issue Mar 1, 2018 · 15 comments

Comments

@tonyskjellum
Copy link

@tonyskjellum tonyskjellum commented Mar 1, 2018

Problem

We want to add one or more info keys, such as one that guarantees that the persistent collective operation will only be started in order across the group of its communicator. Other info keys relevant to persistent collective initialization or operations may also be considered in addition to this one.

Note: this ticket actually describes the ordering rules for persistent collectives with and without this assertion with regard to all other collective communication on the same communicator.

Proposal

Specify at least one info key for ordered-across-group in persistent collectives at communicator scope. Others TBD.

Changes to the Text

Appropriate text will be added that defines the info key and its semantics. If other keys are conceived of quickly, they may be added too. So far, just one info key,

Impact on Implementations

For the specific info key specified thus far, it is an assertion. Implementations can ignore this assertion, or exploit it for greater performance. The assertion is mpi_assert_strict_persistent_collective_ordering ; it requires that programs order persistent collective starts [defined via a communicator that makes this assertion] under the same rules as other collectives. If an implementation can exploit this assertion for greater performance, it can do so, or do legally do nothing.

Impact on Users

Provides an info key; requires that the programs meet the restricted semantics if a given communicator with this assertion is used to generate persistent collective operations.

References

The pull request for this ticket is mpi-forum/mpi-standard#278

PDF File: mpi40-report-ticket83-12sep20.pdf

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Mar 1, 2018

See page 217 of initial text...

mpi32-pers-infokeys-report-01mar18.pdf

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Mar 21, 2018

Verified today that this info key is at the communicator scope.

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Jun 6, 2018

mpi32-report-ticket83.pdf

This is the public copy.

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Jun 14, 2018

We need to make updates based on the reading attempt on June 13, 2018 (all minor). We will present a new reading in Barcelona.

@puribangalore
Copy link

@puribangalore puribangalore commented Sep 5, 2018

Here is the latest PDF file for reading at Barcelona:
mpi32-report-ticket83-sep2018.pdf

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Sep 23, 2018

Outcome of reading

  1. Make clear that the info key provides a total ordering of all collective calls on a given communicator, not just a partial order involving Blocking/Nonblocking and another involving all Persistent. In other words, clarify the semantic impact.
  2. Refer to the semantic terms chapter for clear definition of start; do not restate it here.
    The semantic terms chapter is getting an additional virtual meeting before December f2f meeting.
  3. Add an example of what is "not legal" once we turn on this flag, for a given communicator. This will complement Puri's example(s) being worked for Ticket #90 .
  4. Make sure it is clear that MPI_Startall is just a part of the story, not the whole story.
@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Nov 1, 2018

Rudiments of the example/counter-examples:

Restrictions with key:

    persistent_op, persistent_op2 are specified on comm1

Rank 0:

    MPI_Bcast(...,comm1)
    MPI_Start(persistent_op)
    MPI_Reduce(,...,comm1)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

All other ranks must do the exact same dispatch order for these collectives.

illegal:

    MPI_Bcast(...,comm1)
    MPI_IReduce(,...,comm1)
    MPI_Start(persistent_op)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

Without key:

Rank 0:

    MPI_Bcast(...,comm1)
    MPI_Start(persistent_op)
    MPI_IReduce(,...,comm1)
    MPI_Start(persistent_op2)
    MPI_Gather(...,comm1)

Rank 1:

    MPI_Start(persistent_op)
    MPI_Bcast(...,comm1)
    MPI_IReduce(,...,comm1)
    MPI_Gather(...,comm1)
    MPI_Start(persistent_op2)

legal.

Rank N-1:
MPI_Bcast(...,comm1)
MPI_IReduce(,...,comm1)
MPI_Start(persistent_op2)
MPI_Gather(...,comm1)
MPI_Start(persistent_op)

Legal

Legal --> Other collectives still are issued in same order across group.

@tonyskjellum

This comment has been hidden.

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Nov 19, 2018

mpi32-report-ticket83-19nov18.pdf

Here is the rewritten and revised text.

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Dec 5, 2018

We did the reading in San Jose.

  1. @dholmes-epcc-ed-ac-uk and I will clarify with word 'defined' to the word 'started' in the text (change bar line 4314)

  2. we will also copy the definition of the total ordering into the section (line 4287-4297)... that will change the overall text by repetition. Then we will streamline to remove
    redundancy.

  3. @abouteiller reminded us to make the examples 'compilable'. So, we will make the examples compilable.

The group agrees that we will take ticket 0 votes in Chattanooga on these changes above, which do not change the syntax or semantics of the info key, and then if it passes, take the first vote in Chattanooga in March, 2019. If ticket 0 votes fails, we will re-read in Chattanooga.

@tonyskjellum tonyskjellum changed the title Info key(s) for Persistent Collective Communication Info key(s) for Start Ordering Persistent Collective Communication Dec 5, 2018
@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Dec 6, 2018

Latest text, post meeting, thanks to @dholmes-epcc-ed-ac-uk for the work on post-reading edits.
I still have to fix the examples to be compilable.

mpi32-report-ticket83-06dec18.pdf

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Mar 11, 2020

I need to retarget for MPI-4.1

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Mar 25, 2020

We are going to compare with the send+recv explanation in the pt2pt chapter to decide how much to elaborate our example on pp 219-220 of the https://github.com/mpi-forum/mpi-issues/files/2654199/mpi32-report-ticket83-06dec18.pdf draft.

@tonyskjellum
Copy link
Author

@tonyskjellum tonyskjellum commented Sep 12, 2020

As of today, per discussion on the Forum and WG, a new PR has been made, and the assertion is done on Chapter 6 (Communicators). This obviates prior discussion of an example.

@wesbland
Copy link
Member

@wesbland wesbland commented Oct 7, 2020

@tonyskjellum Can you mention the new PR here so we can link the two together?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants