Philosophical difference with current classification of barriers #89
Comments
|
Originally by htor on 2008-12-30 18:22:37 -0600 I accept your invitation to a beer. And even though I slightly disagree with the justification (an MPI_Allreduce with arbitrary input would also act like a barrier, i.e., '1' is not necessary), I agree to the proposal. I found this also confusing and would vote to change it. I don't think it's necessary to give a rationale why barrier is in the All-to-All group because it's clear that all processes contribute to the result and all processes receive the result. The "result" of this operation is, as defined later, the knowledge that all processes called the operation. We never defined "result" as explicit data, so we can stretch it. I would propose to introduce a new category for the scan operations because "Other" seems rather clueless. It could be named (Prefix-)Scan or Many-To-Many (which is very close to "Other"). |
|
Originally by bronis on 2008-12-30 18:43:10 -0600 I agree with Torsten's comments. I think replacing "Other" would be good. I think it would be useful to think how other proposals will fit into these categories. Although we may not adopt them, I am thinking of the sparse/topological collectives in particular. These also seem to fit into a Many-To-Many category. Would they be valid for intercommunicators? It seems unlikely since topologies return intracommunicators ("topologies cannot be added to intercommunicators" page 241:17-18). It would be a further argument for defining the category if we can make the rule "Operations in the Many-To-Many category are not valid for intercommunicators." So far we have two examples for which that holds; with a third, the rule would be proven ;-). Seriously, I have trouble conceiving of how to define a Many-To-Many operation on an intercommunicator. Perhaps that is just my limited imagination... |
|
Originally by htor on 2008-12-30 19:13:52 -0600 It depends on how Many-To-Many is defined. I would say it's a very generic communication operation where some (>0) processes contribute to the result(s) and some processes (>0) get the result(s). In this sense, all collectives fit the Many-To-Many category and intercommunicator semantics are possible. So can we come up with a formal definition to reduce the generic Many-To-Many, then we could probably say something like this. So my unchecked corollary is: Many-To-Many = any communication operation - One-To-Many - Many-To-One - All-To-All While I think about it: The vector operations does actually invalidate the definition in MPI-2.1. So I would argue that the definition in 5.2.2. in MPI-2.1 is wrong because for example not all processes need to contribute the the result in MPI_Alltoallv. Topological Collectives would not work on intercommunicators. Actually, the proposed nearest neighbor exchange can be emulated with a sparse nonscalable Alltoallv which can be performed on Intracommunicators. However, I am not proposing to consider this. Best, |
|
Originally by jsquyres on 2009-01-12 19:21:57 -0600 I'd be careful of the statement: "The barrier operation has implicit data movement between all processes." There may be no data movement between processes; instead, there may be some form of control signals. For example, a horrible but valid implementation of a barrier on a communicator on a single host is each process creating a file in a shared filesystem. When each process sees the file from all of the other processes, it knows that the barrier condition has been reached. That's likely to be a horribly-performing barrier, but it's valid and no data was exchanged between the processes (unless you include "visibility of files in the filesystem" in the definition of "data" ;-) ). |
|
Originally by bronis on 2009-01-12 21:27:14 -0600 Response to Jeff's comment: Yes, I include that in "implicit data movement". The implicit data is that the program point has been reached. I don't care how that data is communicated. So, using file creations would qualify. |
|
Originally by jsquyres on 2009-01-13 07:10:34 -0600 My last point in this minor quibble is that "data movement" may imply the actual movement of data to some people because we usually use the word "data" in the spec to refer to user buffers. "Information movement" may not carry the same implication...? If you still think it's ok, fine. I just wanted to get this off my chest. :-) |
|
Originally by bronis on 2009-02-06 11:05:58 -0600 Updating to Waiting for Reviews. |
|
Originally by gropp on 2009-02-06 11:49:05 -0600 Reviewed and ok (under the assumption that p 133, line 27-28 about Barrier are deleted. |
|
Originally by jsquyres on 2009-02-06 16:00:22 -0600 Reviewed ok. |
|
Originally by bronis on 2009-02-07 12:06:49 -0600 Put description in standard format; moved suggested replacement sentence (that Bill did not like) into alternative solutions. Torsten and Adam, please review. |
|
Originally by htor on 2009-02-09 12:50:23 -0600 I still agree to this change (I also agree to the deletion of the sentence) - reviewed ok |
|
Originally by moody20 on 2009-02-09 15:24:06 -0600 I agree with the text in the current proposal. I think a "many-to-many" classification would need more thought, since many of the v and w variants should be shifted to such a category, which I think just confuses things. |
|
Originally by bronis on 2009-02-09 15:25:44 -0600 Ready for reading. |
|
Originally by bronis on 2009-02-14 10:41:57 -0600 No implementation required |
|
Originally by traff on 2009-03-23 05:53:40 -0500 Although the proposal is "reviewed", I strongly disagree with the philosophical description/motivation for this suggestion. A barrier is conceptually a synchronization, not a data movement operation (it could be implemented by HW, in which case no data movement except in a very philosophical sense would take place) - and I therefore think it MUCH more useful to keep the distinction. As commented for ticket #134, I also think the usefulness of such classifications is much diminished by lumping things together only because they by some (ok, not totally wrong or unreasonable) interpretation could be seen as aspects of the same thing. I'd vote against this proposal Jesper |
|
Originally by jsquyres on 2009-04-02 14:03:07 -0500 I agree with Jesper; I voiced earlier in this ticket that I don't think that a barrier (always) has implicit data movement. I also don't think it's hugely important to move it out of "other" to "all-to-all". But the proposal itself is valid. :-) |
|
Originally by RolfRabenseifner on 2009-07-13 10:52:58 -0500 Obviously implemented. Therefore I wanted to change the priority to "Waiting for PDF reviews". PDF-review: Not okay: MPI-2.1 p133:25 was not deleted, i.e., MPI_BARRIER is currently shown twice. Therefore priority kept at "Passed". |
|
Originally by moody20 on 2009-07-16 13:17:00 -0500 Attachment added: |
|
Originally by RolfRabenseifner on 2009-07-17 03:45:18 -0500 PDF-review: okay. |
|
Originally by traff on 2009-07-17 08:57:17 -0500 PDF-review OK |
|
Originally by htor on 2009-07-17 12:36:13 -0500 PDF-review: OK |
|
Originally by jsquyres on 2009-07-20 20:22:58 -0500 PDF review: ok |
|
Originally by gropp on 2009-07-22 12:09:42 -0500 WDG - Reviewed and complete. |
|
Originally by bronis on 2009-07-24 09:05:31 -0500 PDF OK. |
|
Originally by bronis on 2009-08-29 18:33:02 -0500 I thought Adam was supposed to set this to ticket complete. I am going to do it as it clearly has enough reviews... |
|
Originally by jsquyres on 2010-09-18 05:05:11 -0500 This ticket is (long-since) complete; marking it resolved/text committed. |
Originally by bronis on 2008-12-30 17:21:39 -0600
Description[[BR]]
WARNING: This issue might best first be the topic for discussion over a beer. Nonetheless, I think we should make the fix I suggest below.
AS discussed in ticket #87, Section 5.2.2 introduces terms for the different types of collective operations. Those terms are "All-To-All", "All-To-One" and "One-To-All" as well as "Other" for operations that do not fit any of the three categories. While I would choose different terms ("Synchronizing", "Fan-Out" and "Fan-In"), I am happy to live with these terms. However, I must disagree with the classification of MPI_BARRIER as Other and the text that is included to justify that error. A barrier operation is the simplest of All-To-All operations, for which the definition reads "All processes contribute to the result. All processes receive the result." A barrier is best viewed as a bit-wise "and" Allreduce over a single bit with the restriction that the input bit must be set (i.e., "1"). Thus, the input and output bit are implicit in the call to the function. When implemented as a hardware barrier network, in fact, the operation is exactly that, with the turning on of a line occuring during the call. The distinction made here is imply fictious.
Overall, I would argue that this fix makes it much clearer why some operations are not valid for intercommunicators (and we could consider adding something to this effect). Specifically, the "Other" category has no clear meaning when applied to two distinct groups.
Extended Scope
None.
History
Appears to have existed prior to 2.1.
Proposed Solution
On page 133, move line 25 (which contains a bullet and "MPI_BARRIER") to follow immediately after line 10 (which contains a bullet and "MPI_ALLREDUCE, MPI_REDUCE_SCATTER").
Also delete the bogus sentence on page 133, lines 27-28 that currently reads:
The MPI_BARRIER operation does not fit into this classification since no data is being moved (other than the implicit fact that a barrier has been called).
Impact on Implementations
None.
Impact on Applications / Users
None.
Alternative Solutions
Instead of just deleting the sentence on 133:27-28, replace it with:
The barrier operation has implicit data movement between all processes.
The above change would simplify the changes needed for nonblocking collectives since we can then just make it "The barrier operations".
Entry for the Change Log
None.
The text was updated successfully, but these errors were encountered: