Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

P2300 std::execution #1054

Open
wg21bot opened this issue Jun 26, 2021 · 19 comments
Open

P2300 std::execution #1054

wg21bot opened this issue Jun 26, 2021 · 19 comments

Comments

@wg21bot
Copy link
Collaborator

@wg21bot wg21bot commented Jun 26, 2021

P2300R0 std::execution (Michał Dominiak, Lewis Baker, Lee Howes, Michael Garland, Eric Niebler, Bryce Adelstein Lelbach)

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Jul 6, 2021

2021-07-06 Joint Library Evolution and Concurrency Telecon

P2300R1: std::execution

2021-07-06 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Michael Garland

Minute Taker: Ben Craig

Start: 2021-07-06 10:08 Pacific

This week, the authors will present the proposal and we'll identify what questions we have and what areas we need to explore. In subsequent meetings, we'll dive into questions and feedback in greater detail.

The UK delegation feedback will be reviewed at a future meeting.

Start User Facing Presentation: 2021-07-06 10:11 Pacific

End User Facing Presentation: 2021-07-06 10:41 Pacific

Start Implementer Interface Presentation: 2021-07-06 10:54 Pacific

End Implementer Interface Presentation: 2021-07-06 11:05 Pacific

The motivation for both maybe eager and strictly lazy models needs to be clearer and strengthened.

Add discussion of the role, interactions, and compatibility of coroutines in senders/receivers.

Senders should probably be awaitable.

Sender adaptor piping vs coroutines.

Is sender adaptor piping fundamental to the proposal? It doesn't seem like it is.

What does shape mean in the context of bulk? What are the implications for chunking? Is chunking allowed? Clarify this.

Should bulk take an execution policy?

Add more examples of multi-shot vs single-shot.

Consider introducing a maybe_force that possibly starts, but is not required to.

Ensure that set_value, set_error, and set_done use tag_invoke.

Should we decouple the Networking TS from executors/senders/receivers?

Library Evolution needs to decide whether we are going to standardize the existing practice of Boost.Asio via the Networking TS.

Topics to Polls:

  • Distinct maybe eager and strictly lazy versions of sender algorithms.
  • Shipping the Networking TS.

End: 11:49

Summary

Library Evolution and Concurrency met for a joint review of P2300 std::execution. This was the first time that the proposal was reviewed, so this session was primarily an overview and tour through the proposal. After the author's presentation, we began discussing open questions and feedback.

One of the major areas of uncertainty was the existence of two variants of sender adaptors - maybe eager and strictly lazy. Some questioned the need for the maybe eager variants, given that we have facilities like ensure_started. The authors will need to include stronger motivation of this in future revisions, and we will definitely need to make a decision on whether we want to go this route. The alternative is to just have strictly lazy algorithms and a way to eagerize explicitly, e.g. ensure_started.

A number of questions were raised about how coroutines fit into the senders/receivers model presented by P2300. Some comparisons and contrasts where made between P2300's sender adaptor piping syntax and coroutines (in particular, coroutines as used in the Networking TS). The authors explained that while coroutines are powerful, they can't necessarily express all aspects of the sender/receiver model, such as the error and cancellation channels, and may have unnecessary overhead in some cases.

We discussed the removal of properties, which P2300 proposes. It was pointed out that there was not prior guidance to make this change, so this is a design change from P0443. The authors indicated that they believe that properties could be added to P2300 after it ships.

We also considered the interactions and relationship between P2300 and the Networking TS. Some asserted that it was unclear if P2300 could support the Networking TS or async I/O. The authors explained how P2300 could be used for async I/O and pointed to libunifex for implementation experience. They indicated that I/O schedulers could be written with senders as they are in P2300, although we may need additional scheduler concepts in the future.

Finally, we talked about whether the Networking TS and executors/senders/receivers should remain coupled. Some said out that we have been trying to develop a "grand unified" async model for years, and perhaps that is a mistake. Others pointed out that it may be a mistake to try and retrofit the Networking TS, which is based on established practice from Boost.Asio, with a new model. We could choose to ship the Networking TS with its current model and also develop and ship a new model (e.g. executors/senders/receivers). We could also choose to not ship the Networking TS, and ship a networking library built from the ground up around executors/senders/receivers.

The Networking TS has been in flight for a long time now. The C++ committee needs to make a decision about whether we are going to ship the Networking TS with its current model or whether we want a different networking story. We also need to decide the degree to which we want networking and executors/senders/receivers to be coupled.

The UK delegation is putting together a more complete set of feedback from their members, which will hopefully be ready for presentation at the next meeting, or the subsequent one.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Jul 27, 2021

2021-07-12 Joint Library Evolution and Concurrency Telecon

P2300R1: std::execution

2021-07-12 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Michal Dominiak

Minute Taker: Inbal Levi

Start: 2021-07-12 14:08 Pacific

BSI position slides should be published as a P paper.

Surfacing different kinds of cancellation:

  • Terminal
  • Partial - set_done(how much succeeded)
  • Total

There's an ask to be able to express partial results and to expose what types of cancellation a sender supports.

Who is responsible for handling exceptions?

Should set_value always be noexcept?

transfer algorithms need to be exception firewalls.

Expected errors vs unexpected errors.

Different completion schedulers for set_value, set_error, and set_done.

Do we need a way to ask that a scheduler run completion on a particular context?

Tail calls and sender authoring.

3 sender channels vs 1 sender channel.

End: 15:31 Pacific

Summary

We continued our review of P2300 with a presentation from members of the UK delegation explaining their feedback.

First, we discussed cancellation. It was suggested that we need to be able to surface and distinguish between different types of cancellation, such as terminal, partial, and total cancellation. Some expressed a need to support sending partial results or partial success, and there was a question of whether this is possible with the current proposal.

We also considered whether the three channels that senders/receivers support are sufficient and necessary. Some suggested that they could be condensed to a single channel, while others suggested that the three channels may be insufficient.

Finally, we discussed the matter of what context receiver functions will complete on, and how this may be complicated when some signals are mapped to other signals.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Jul 27, 2021

2021-07-26 Joint Library Evolution and Concurrency Telecon

P2300R1: std::execution

2021-07-26 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Michal Dominiak

Minute Taker: Ben Craig & Gasper Azman

Start: 2021-07-26 07:18 Pacific

Questions we should feel comfortable answering before we poll on P2300:

  • Is P2300 an improvement upon or a regression from P0443?
  • Is the cancellation model in P2300 acceptable?
  • Is the error handling model in P2300 acceptable?
  • Is a generic property system a pre-requisite to ship executors?
  • Should the paper include facilities for starting detached work? Should one-way execution be supported?
  • Do we need to support both a strictly lazy and maybe eager model? Is a strictly lazy model with an explicit way to eagerize acceptable?
  • Do we believe we can build ... on top of what is in P2300?
    • Parallel algorithms, including future extensions like asynchronous algorithms?
    • A future Standard C++ networking library?
    • A future Standard C++ asynchronous I/O library?
  • Do we understand the potential interactions of senders/receivers and ...
    • Coroutines?
    • The Networking TS?

Some are unclear how they would implement sender algorithms.

Some are concerned about the context in which a receiver signal will be called.

"If I'm writing a scheduler, what else will I have to implement at the same time?" Presumably a sender, an operation state, an execution context, and some CPOs.

What is set_error actually for? Expected errors? Unexpected errors?

How do you express partial success or errors with a side effect? Partial success calling set_value is unpalatable to some.

Should the default definition of bulk be parallel or serial?

Chris K should present the Networking TS compositional model to Library Evolution.

Coroutines vs senders/receivers as the base language for expressing parallelism.

Demonstrate that senders are awaitable in P2300.

Add example of simple scheduler to P2300.

End: 8:37

Summary

We continued today discussing the feedback from the UK delegation on P2300.

The chair presented some questions that he wants participants to feel comfortable answering before we poll:

  • Is P2300 an improvement upon or a regression from P0443?
  • Is the cancellation model in P2300 acceptable?
  • Is the error handling model in P2300 acceptable?
  • Is a generic property system a pre-requisite to ship executors?
  • Should the paper include facilities for starting detached work? Should one-way execution be supported?
  • Do we need to support both a strictly lazy and maybe eager model? Is a strictly lazy model with an explicit way to eagerize acceptable?
  • Do we believe we can build ... on top of what is in P2300?
    • Parallel algorithms, including future extensions like asynchronous algorithms?
    • A future Standard C++ networking library?
    • A future Standard C++ asynchronous I/O library?
  • Do we understand the potential interactions of senders/receivers and ...
    • Coroutines?
    • The Networking TS?

First, we focused again on how you can express partial successes with senders/receivers. It is unclear to some whether senders/receivers can express partial success. Others believe that it is trivial to do so via the arguments you pass to set_value. However, some desired a generic way to deal with partial success, although others questioned what the use case for that would be.

We also discussed the semantics of the default definition of bulk, which is serial. This was surprising to some. There was some question about what set of CPOs need to be customized by a scheduler and what things need to be defined when writing your own scheduler.

Once again, questions about the interoperability between senders and coroutines was mentioned. The authors explained that senders are awaitable, and that future revisions of P2300 will discuss this.

There was a claim that the Networking TS's async model was a viable alternative to senders/receivers and should be considered as well. The proponents of this claim agreed to produce a proposal and presentation explaining their thesis.

It became clear during our discussion that the Networking TS has an async model distinct from senders/receivers. This was true with P0443 and P2300, so this is not a regression in the new proposal. If we truly want a "grand unified async model", either we must adopt the Networking TS's async model, or modify the Networking TS. Some questioned the utility and need for a "grand unified async model", and suggested that we should perhaps decouple the Networking TS from senders/receivers.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

At the next meeting, we will discuss whether properties are essential and whether we need to have both strictly lazy and maybe eager forms of algorithms.

@wg21bot
Copy link
Collaborator Author

@wg21bot wg21bot commented Jul 30, 2021

P2300R1 std::execution (Michał Dominiak, Lewis Baker, Lee Howes, Kirk Shoop, Michael Garland, Eric Niebler, Bryce Adelstein Lelbach)

@brycelelbach brycelelbach changed the title P2300 std::execution P2300 std::execution Aug 1, 2021
@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Aug 4, 2021

2021-08-03 Joint Library Evolution and Concurrency Telecon

P2300R1: std::execution

2021-08-03 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach & Ben Craig

Champion: Corentin Jabot

Minute Taker: Ben Craig & Inbal Levi

Start: 2021-08-03 14:19 Pacific

Topics for today:

  • Is there a capability gap between P0443 and P2300? Is it acceptable?
  • Is a generic properties system a prerequisite for executors?
  • Do we need both a "strictly lazy" and "maybe eager" model?

POLL: We believe we need one grand unified model for asynchronous execution in the C++ Standard Library, that covers structured concurrency, event based programming, active patterns, etc...

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
4 9 5 5 1

Attendance: 29

Outcome: No consensus (leaning in favor).

N: I'd prefer one grand unified model, but it isn't a dealbreaker if we can't do it

SA: I think we are making too many compromises for the simple case in favor of the very complicated heterogeneous case

POLL: In P2300, in addition to ways to query the properties of schedulers, we think we need ways to transform schedulers into other schedulers, like P0443 did with prefer and require.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
2 1 14 6 2

Attendance: 29

Outcome: No consensus (leaning against).

WA: I'm leaning against because I don't think it's entirely necessary to have it in P2300, it could be subsequent

POLL: Knowing what we know today, we should continue working on P2300 instead of P0443 to provide structured concurrency for the C++ Standard Library.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
10 11 2 1 0

Attendance: 29

Outcome: Consensus in favor.

End: 15:37

Summary

Today we continued with our review of P2300, the new senders/receivers proposal.

We started out by discussing properties. Properties were a part of P0443, but P2300 instead uses CPO-based queries.

In P0443, properties could be queried, preferred, or required, and there was a full blown generic property system. Preferring or requiring a property of an executor would give you back a new executor.

In P2300, instead there are CPOs that act as queries, providing the same sort of information that could be accessed in P0443 via properties. However, with P2300 you cannot prefer or require something, you can only query. Additionally, there is no generic property mechanism. CPO queries can be forwarded through layers and can be made to work with type erasure as well.

Some felt that something fundamental is lost without properties. P0443's blocking property was cited as an example. You may wish to do a blocking submission in one part of your program, and a non-blocking submission in another part. The original executors proposals facilitated this by providing a wide API with many different execution functions. This became unwieldy, so properties were introduced as a way to request different semantics from a single execution function.

The P2300 authors believe that this does not represent a loss of capabilities. All of the important properties from P0443 are represented in P2300 as CPO queries. They also questioned the usefulness of requiring or preferring, which assumes that there is a suitable adaptation for the executor. Instead, they believe that one should use different schedulers for different execution semantics, and use queries to assert that the schedulers that have been provided meet the requirements that are needed.

The authors also pointed out that nothing prohibits the addition of more queries or the entire properties system in the future. P2300 does not include it because the authors do not believe it is essential.

We also had a short discussion of the dichotomy of having both a "strictly lazy" and "maybe eager" model in P2300. Some inquired whether we could just have one model of "maybe lazy". The authors explained that this would be problematic, but pointed out that if we want only a single model, "strictly lazy" with explicit eagerization may work. Some questioned the meaning of "eager" and "lazy" in P2300, claiming that both are really just two different forms of laziness. More discussion on this topic is certainly needed.

Finally, we took our first exploratory polls on P2300. We didn't have consensus on whether we felt we needed to have a grand unified model for asynchrony in the C++ Standard Library. We also didn't have consensus on whether we believe that we need to be able to prefer and require capabilities, not just query them. Finally, we had fairly robust consensus that we think that P2300 should replace P0443 as the solution for structured concurrency in the C++ Standard Libray.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Aug 18, 2021

2021-08-17 Joint Library Evolution and Concurrency Telecon

P2300R1: std::execution

2021-08-17 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Michael Garland

Minute Taker: Ben Craig

Start: 2021-08-17 14:13 Pacific

Examples of how to implement sender algorithms.

Examples of scheduler propagation.

More in-depth examples, but simpler than production libunifex code.

Show P2300 versions of examples from other executors papers.

End: 15:34

Summary

At this meeting, we wrapped up our initial review of P2300 std::execution.

First, the authors presented their summary of the feedback and the actions that they plan to take in the next revision, most notably improving the examples, explaining the relationship between senders/receivers and coroutines, adding an explanatory section on cancellation, clarifying how forward progress guarantees are exposed (in particular for bulk), and illustrating how schedulers will integrate with existing Standard Library parallel algorithms.

Then the floor was opened up for any additional questions, comments, or feedback. Some concerns were raised the teachability and complexity of the proposal. In particular, some felt that authoring senders, receivers, and sender algorithms may be too difficult.

We also continued discussing whether or not we need to have a grand unified model for asynchrony. It was claimed by some that the stakeholders for asynchrony may be too broad to be satisfied by a single model. Some stakeholders primarily care about structured concurrency and parallelism, while others primarily care about reactive/IO/networking. While most structured concurrency and parallelism stakeholders on the committee seem to be onboard with senders/receivers, the reactive/IO/networking stakeholders are split. Some expressed that it might be better to standardize P2300 with a specific scope of structured concurrency and parallelism and without mandating it for everything asynchronous in the Standard Library, allowing components like the Networking TS to pick a different model.

There was also an inquiry about whether senders/receivers was something worth standardizing. Some suggested that the need may be better served by a language feature like coroutines. The authors again reiterated that coroutines are not sufficient on their own and that coroutines will be used in and have a deep relationship with senders/receivers. It was mentioned that a future Standard Library coroutine task type would likely model sender.

There was a great desire for improved and expanded examples in future revisions of the P2300 to help Library Evolution and SG1 better understand senders/receivers and compare and contrast it to other models. The requests include: examples of how to implement schedulers, senders, receivers, and sender algorithms, examples of scheduler propagation, P2300 versions of all the examples from prior executors papers, and more in-depth examples demonstrating end use cases (echo server, etc) but simpler than production code like libunifex. The authors have indicated that they will make a concerted effort to improve the examples in future revisions.

Outcome

Bring a revision of P2300R1 (std::execution), with the guidance below, to Library Evolution for further design review:

  • Improve and expand the examples:
    • Add examples of how to implement schedulers, senders, receivers, and sender algorithms.
    • Add examples of scheduler propagation.
    • Add P2300 versions of all the examples from prior executors papers.
    • Add more in-depth examples demonstrating end use cases (echo server, etc) but simpler than production code like libunifex.
  • Better explain how partial success works with senders/receivers.
  • Explain the relationship between senders/receivers and coroutines.
  • Add an explanatory section on cancellation.
  • Clarify how forward progress guarantees are exposed, in particular for bulk.
  • Illustrate how schedulers will integrate with existing Standard Library parallel algorithms.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Sep 28, 2021

2021-09-20 Joint Library Evolution and Concurrency Telecon

P2444R0: Asio Asynchronous Model

P2300R1: std::execution

P2463R0: Asio Asynchronous Model Slides

2021-09-20 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Chris Kohlhoff

Minute Taker: Dietmar Kuhl

Start: 2021-09-20 14:15 Pacific

Is this proposal an alternative to senders/receivers?

What would be wrong with having both this Asio model and senders/receivers in the Standard Library?

How does this model support compute and structured concurrency? What about bulk execution?

POLL: Knowing what we know today, we should continue considering shipping the Networking TS in C++23, as is.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
10 6 8 3 3

Attendance: 36

Author Position: SF

Outcome: No consensus.

POLL: Knowing what we know today, we should continue considering shipping P2300 senders/receivers in C++23.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
11 6 5 5 4

Attendance: 36

Outcome: No consensus.

End: 15:40

Summary

A detailed presentation was given on Asio's asynchronous model, which the Networking TS is based on.

Afterwards, we briefly discussed whether the Networking TS needs to be linked to senders/receivers. We also started discussing whether we wanted to ship the Networking TS in C++23; more discussion will be needed before we reach a conclusion.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Sep 28, 2021

2021-09-28 Joint Library Evolution and Concurrency Telecon

P2444R0: Asio Asynchronous Model

P2463R0: Asio Asynchronous Model Slides

P2300R1: std::execution

2021-09-28 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Chris Kohlhoff

Minute Taker: Ben Craig

Start: 2021-09-28 7:12 Pacific

Set aside senders/recievers and Net TS interoperability for now.

POLL: We must have a single async model for the C++ Standard Library.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 9 10 11 5

Attendance: 49

Outcome: No consensus.

POLL: Knowing what we know today, we should continue considering shipping the Networking TS in C++23.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
9 9 8 8 3

Attendance: 48

Outcome: No consensus.

POLL: Knowing what we know today, we should continue considering shipping P2300 std::execution in C++23.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
11 9 6 6 6

Attendance: 47

Outcome: No consensus.

End: 8:40

Summary

We finished the presentation of Asio's asynchronous model at today's meeting, specifically the section covering completion tokens.

We then dove into a discussion of how to proceed with the Networking TS and senders/receivers. We considered decoupling the two, and what implications that would have on future planning and interoperability. If we shipped the Networking TS in C++23, would we be able to write wrappers that allowed it to interoperate with senders/receivers later? If we ship senders/receivers without properties, does anything preclude us from adding them later?

We also briefly discussed the ABI stability of the Networking TS, and how ABI stability needs could be problematic if we added TLS facilities to the Networking TS.

We still have not reached a conclusion of our discussions on these topics.

Outcome

The review of P2300 std::execution and discussions of executors, senders, receivers, networking, and the Networking TS will resume at future Library Evolution and Concurrency telecons.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Oct 5, 2021

2021-10-04 Joint Library Evolution and Concurrency Telecon

P2300R2: std::execution

P2444R0: The Asio asynchronous model

P2464R0: Ruminations on networking and executors

P2469R0: Response to P2464: The Networking TS is baked, P2300 Sender/Receiver is

2021-10-04 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Eric Niebler

Minute Taker: Gašper Ažman

Start P2300 Presentation: 2021-10-04 14:08 Pacific

End P2300 Presentation: 14:36

End P2300 Discussion: 14:57

Start P2464 Presentation: 15:04

End P2464 Presentation: 15:36

Start P2469 Presentation: 15:38

End P2469 Presentation: 15:49

End P2464/P2469 Discussion: 16:00

Networking and Executors Polls

To be taken electronically:

POLL 1: The Networking TS/Asio async model (P2444) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

POLL 2: The sender/receiver model (P2300) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

POLL 3: Stop pursuing the Networking TS/Asio design as the C++ Standard Library's answer for networking.

POLL 4: Networking in the C++ Standard Library should be based on the sender/receiver model (P2300).

POLL 5: It is acceptable to ship socket-based networking in the C++ Standard Library that does not support secure sockets (TLS/DTLS).

Summary

We finished this 3-week review of Networking and Executors with presentations of three papers:

  • P2300R2, an updated revision of the sender/receiver model addressing feedback from our summer review.
  • P2464R0, a position paper suggesting that there are deficiencies in the Networking TS/Asio async model and that it should not be adopted by the Standard Library.
  • P2469R0, a rebuttal of P2464 that argues that the Networking TS is proven, mature, and ready to ship, while the P2300 sender/receiver model is not, and suggesting that the sender/receiver model could be layered on top of the Networking TS/Asio model.

One major topic of today's discussion was layering - can sender/receiver be built on top of the Networking TS/Asio model, and vice versa. One of the principle questions with such a layering is how error channels and cancellation would be supported.

Error handling was a main point of debate between proponents of the two models. Some claimed that the Networking TS/Asio model did not provide sufficient mechanisms for handling errors during work submission.

Performance overheads and efficiency were another point of contention. Proponents of the Networking TS/Asio model feel that implementing and using async APIs with the sender/receiver model would come with unacceptable overheads associated with the construction of sender/receiver objects.

There were some questions about the change in P2300R2 from having a maybe-eager and always-lazy version of each algorithm to having only the strictly-lazy version. This warrants further discussion when we next look at the sender/receiver model in detail.

Outcome

We will take electronic polls on Networking and Executors to determine how to proceed. Details on how to vote have been sent to the lib-ext@ mailing list.

@ben-craig
Copy link
Collaborator

@ben-craig ben-craig commented Oct 18, 2021

2021-10-18 Library Evolution Supertelecon

P2300R2: std::execution

2021-10-18 Library Evolution Telecon Minutes

Chair: Ben Craig

Champion: Multiple

Minute Taker: Inbal Levi

Start: 2021-10-18

Summary

Discussed two high level design questions of P2300: tag_invoke, and strictly-lazy vs. possibly-eager execution.

Recent reflector discussions have questioned whether tag_invoke is suitable for standardization. The complexity and aesthetics have been the major facets of this discussion, but compiler throughput has also been raised as a question.

LEWG is very interested in a language solution, but the consensus is that LEWG can live with tag_invoke for now.

POLL: We can live with tag_invoke for P2300.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
12 10 3 2 0

Attendance: 32

# of Authors: 6

Author Position:

Outcome: Consensus

WA: Users perspective, this will be incomprehensible. Need to figure out if we really need all the features that tag_invoke offers. Since nobody has a better idea, only WA

N: Would have been SF if we were adding tag_invoke to existing capabilities.

WA: language would be great, but not sure that tag_invoke is the right way forward, as it isn't very simple or straightforward.

P2300R1 had eager and lazy versions of many algorithms. P2300R2 has only lazy versions. There was much discussion about whether eager execution (whether through algorithms or schedulers) could be added later, and whether eager is a net benefit in many cases. The committee is not sufficiently informed to provide direction at this time, and needs a discussion paper. We did take a feel-of-the-room poll there to verify that there is a lack of consensus.

POLL: (feel of the room) P2300 must include potentially-eager algorithms.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
4 2 6 7 6

Attendance: 32

Outcome: No consensus (needs exploration / discussion paper)

Outcome

tag_invoke does not appear to be a blocker for P2300.

The authors should collaborate with the eager evaluation contingent on a paper that discusses strictly-lazy and possibly-eager execution, so that LEWG can make an informed decision.

@ben-craig
Copy link
Collaborator

@ben-craig ben-craig commented Oct 19, 2021

2021-10-19 Library Evolution Supertelecon

P2300R2: std::execution

2021-10-19 Part 1 Library Evolution Telecon Minutes

2021-10-19 Part 2 Library Evolution Telecon Minutes

Chair: Ben Craig & Fabio Fracassi

Champion: Michał Dominiak

Minute Taker: Inbal Levi

Start: 2021-10-19

Open Questions:

Naming for start_detached (start_and_discard? start_and_forget?)

Start back up at execution.helpers (and discuss transfer more)

Topics to Poll:

POLL: LEWG should schedule meetings with the plan of getting P2300 into C++23.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
12 8 3 2 6

Attendance: 38

# of Authors: 6

Author Position: All in favor

Outcome: Weak Consensus

@wg21bot
Copy link
Collaborator Author

@wg21bot wg21bot commented Oct 26, 2021

P2300R2 std::execution (Michał Dominiak, Lewis Baker, Lee Howes, Kirk Shoop, Michael Garland, Eric Niebler, Bryce Adelstein Lelbach)

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Nov 3, 2021

2021-10 Library Evolution / Concurrency Polls on Networking and Executors

Consensus and outcomes determined by @ben-craig and @FabioFracassi.

Poll 1: The Networking TS/Asio async model (P2444) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 10 6 14 18

Outcome: Weak consensus against.

What this doesn't mean: It doesn't mean that the NetTS async model isn't a good fit for networking. There were many comments to the contrary.

What this means: If the authors continue to pursue a design similar to the current NetTS, they would have an easier time getting consensus by focusing on the networking side of things, rather than proposing the design for general asynchrony.

Poll 2: The sender/receiver model (P2300) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
24 16 3 6 3

Outcome: Consensus in favor.

What this doesn't mean: It doesn't mean that S&R is going into C++23 (though it might). It doesn't mean that P2300 as it stands today will be sent forward to LWG. It doesn't prevent us from adding new asynchronous models in the future.

What this means: Work will continue in LEWG on refining P2300, and LEWG will keep the various asynchronous use cases in mind while working on P2300.

Poll 3: Stop pursuing the Networking TS/Asio design as the C++ Standard Library's answer for networking.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
13 13 8 6 10

Outcome: No consensus

What this doesn't mean: The NetTS is not "dead".

What this means: WG21 will still work on networking in this general form, but the authors need to do a lot in order to build up consensus to get something like the TS merged into the standard. The bulk of this work should be done in SG4. Many of the people in favor of stopping work on the TS would like networking to be built on top of Senders and Receivers. Others were opposed to the lack of security through Transport Layer Security (TLS). It is highly unlikely that design changes to the networking TS can be made fast enough, and consensus gained fast enough, for networking to make C++23.

Poll 4: Networking in the C++ Standard Library should be based on the sender/receiver model (P2300).

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
17 11 10 4 6

Outcome: Weak consensus in favor.

What this doesn't mean: Work on networking using other models will still be reviewed and considered on its own merits. WG21 doesn't pause work on a concrete paper based off of the wish for another paper. Note that there are a high number of neutrals on this vote. Many of the neutrals (and some of the abstentions) would like to see a paper before picking a side here.

What this means: In the short term, this poll result doesn't mean much. We don't have a paper in hand that proposes networking based on the P2300 model. For paper authors though, this poll is encouragement to do work in the area of networking based on senders and receivers, or to be prepared with compelling new information on why networking should use a different model.

Poll 5: It is acceptable to ship socket-based networking in the C++ Standard Library that does not support secure sockets (TLS/DTLS).

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
9 13 5 6 13

Outcome: No consensus.

What this means: A networking library that does not support secure sockets will face significant headwinds getting through the standardization process.

What this doesn't mean: This doesn't make a statement on whether insecure networking should be included, the defaults of secure vs. insecure, or how things like ABI should managed.

Guidance for the Networking Study Group

Before bringing networking papers back to LEWG, two major areas need to be thoroughly addressed: Security, and the Senders and Receivers async model.

Those voting in favor of Poll 5 argued that the insecure parts are the building blocks for the secure parts, and that ABI is major concern that will plague TLS support. Those voting against Poll 5 argued that secure sockets are needed for communicating with many sites on the internet, and that shipping without secure sockets would be irresponsible. A networking proposal needs to address these concerns before coming to LEWG.

As for networking in combination with Senders and Receivers, I will show this older poll from the 2021-09-28 telecon:

POLL: We believe we need one grand unified model for asynchronous execution in the C++ Standard Library, that covers structured concurrency, event based programming, active patterns, etc.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
4 9 5 5 1

Outcome: No consensus (leaning in favor).

The combination of this "grand unified model" poll and Poll 4 heavily encourages the networking study group to produce a paper based on Senders and Receivers. There is room to produce a non-S&R paper, but such a paper would need to provide compelling new information in order to convince the "grand unified model" contingent that S&R can't get the job done suitably.

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Dec 8, 2021

2021-12-06 Part 1 Library Evolution Supertelecon

P2300R3: std::execution

2021-12-06 Part 1 Library Evolution Telecon Minutes

Chair: Nevin Liber

Champion: Michał Dominiak

Minute Taker: Inbal Levi

Summary

TODO (@nliber)

Outcome

TODO (@nliber)

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Dec 8, 2021

2021-12-06 Part 2 Library Evolution Supertelecon

P2300R3: std::execution

2021-12-06 Part 2 Library Evolution Telecon Minutes

Chair: Inbal Levi

Champion: Michał Dominiak & Eric Niebler

Minute Taker: Ben Craig

Outcome

POLL: LEWG supports exception_ptr overload of set_error being required by the receiver concept

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 10 4 4 0

Attendance: 38

# of Authors: 6

Author Position: 2xSF 4xWF

Outcome: Weak Consensus In Favor

Inbal: This is a weak consensus in favor, but freestanding should be addressed (and supported) to increase consensus.

Summary

Topics which should be addressed in next revisions:

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Dec 8, 2021

2021-12-07 Part 1 Library Evolution Supertelecon

P2300R3: std::execution

2021-12-07 Part 1 Library Evolution Telecon Minutes

Chair: Billy Baker

Champion: Michał Dominiak

Minute Taker: Ben Craig

TODO (@billy-baker)

Summary

TODO (@billy-baker)

Outcome

TODO (@billy-baker)

@brycelelbach
Copy link
Collaborator

@brycelelbach brycelelbach commented Dec 8, 2021

2021-12-07 Part 2 Library Evolution Supertelecon

P2300R3: std::execution

2021-12-07 Part 2 Library Evolution Telecon Minutes

Chair: Ben Craig

Champion: Michał Dominiak & Eric Niebler

Minute Taker: Inbal Levi

Summary

Largely discussing transfer and on. A use case that received much discussion that is not directly handled (but can be done with a combination of existing algorithms) is to send an item of work to another execution context, and then have that item of work's continuation run on the original execution context (there-and-back-again).

A naming suggestion was made to change upon_error to recover.

A request was made to have each algorithm in the paper point to an example, and not just on a per algorithm family basis.

Outcome

POLL: Non customizable callable entities (just family, into_variant, done_as_error) in P2300 should be CPOs (and not functions)

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
3 11 7 0 0

Attendance: 33

# of Authors: 5

Author Position: 1 SF 3 WF 1 N

Outcome: Strong consensus

@wg21bot
Copy link
Collaborator Author

@wg21bot wg21bot commented Dec 18, 2021

P2300R3 std::execution (Michał Dominiak, Lewis Baker, Lee Howes, Kirk Shoop, Michael Garland, Eric Niebler, Bryce Adelstein Lelbach)

@cplusplus cplusplus deleted a comment from RandomDSdevel Dec 22, 2021
@brycelelbach brycelelbach removed this from the 2021-telecon milestone Dec 22, 2021
@jensmaurer jensmaurer added this to the 2022-telecon milestone Jan 1, 2022
@ben-craig
Copy link
Collaborator

@ben-craig ben-craig commented Jan 13, 2022

D2300R4: std::execution

D2300R4: std::execution

2021-01-11 and 2021-01-12 Library Evolution Telecon Minutes

Chairs: Fabio Fracassi, Nevin Liber, Inbal Levi, and Ben Craig

Champion: Michał Dominiak & Eric Niebler

Minute Taker: Ben Craig and Inbal Levi

SUMMARY:
A paper has been promised to further investigating the usage of exception_ptr in the receiver concept.

If names are to be changed, then papers with rationale and motivation need to be supplied.

OUTCOME:

POLL: Remove with_awaitable_senders from P2300.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
0 1 8 10 3

Attendance: 31

# of Authors: 6

Author Position: 5xWA

Outcome: Keep with_awaitable_senders


POLL: Remove untyped senders from P2300 (typed senders declare what signals/types they complete with)

Attendance: 31

# of Authors: 6

Author Position:

Outcome: No objection to unanimous consent


POLL: Produce a P2300R4 (std::execution), modified as described below, and then send the revised paper to LWG for C++23 with priority 1 (to be confirmed with a Library Evolution electronic poll).

  1. Removing untyped senders from P2300
Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
15 9 1 3 6

Attendance: 39

# of Authors: 6

Author Position: 6xSF

Outcome: Weak Consensus

SA: Putting this in at this time of the C++ release is too late

SA: Too few reviews from embedded people

SA: maturity/readiness/tag_invoke and the numerous change in the last few meetings

SA: Lacking properties, with regards to nesting. Need type erased sender. Can't be done later

SA: Feels like ranges all over again. Needs maturity

WF: I didn’t vote SF although I like S/R (on F): my concern for not voting SF is that things are still somewhat in flux. I don’t think any paper update can address that, though. I’m torn between wanting S/R urgently and it not feeling fully baked.


POLL: LEWG prefers a hypothetical P2300 with the following changes:

  1. Removing untyped senders from P2300
  2. Remove the requirement that receivers use exception_ptr
  3. receiver's set_value is noexcept
Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 13 10 3 1

Attendance: 39

# of Authors: 6

Author Position: 1xSA, 1xWF, 3xN, 1xWA

Outcome: Weak consensus. Authors will investigate

SA: I've used exception_ptr based S&R in freestanding with -fno-exceptions and haven't seen a reason that it couldn't be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants