io_service question

[yfinkelstein]

It would be good to cover the vision and strategy behind io_service in some doc or wiki. This is needed to have organized approach for supporting a generic platform.

1. Why program platform-specific IO polling backend directly in this project instead of integrating with one of omni-platform IO polling libraries such as libevent, libev, libuv, or boost asio.

2. Re boost in particular, as I understand it, Networking TS is based on boost asio, and it has been accepted for C++20(?). So, would it not be best to integrate with boost io_service from this TS?

3. IO polling is inevitably related to multithreading. There are several popular models: single-threaded event loop (ex: libevent), single threaded polling with multithreaded event processing (ASIO), fully multithreaded polling and processing (GRPC). Where is this project stand in this regard? Is there some specific set of goals in mind? Perhaps, there is a perimeter beyond which this project should not go and leave the rest for custom integrations?

[lewissbaker]

Great questions! I'll give a brief answer here and try to write up something a bit more detailed in a document that's more easily accessible in the project.

1. The main reason I'm looking into implementing async IO handling in this project rather than using libuv, boost asio, etc. is that I believe there may be more efficient ways of implementing async I/O if we only need to target use in C++ coroutines. I want to explore this possibility.

For example, we can efficiently allocate memory for per-operation state on the coroutine frame as local variables rather than having to allocate storage for that state separately on the heap.
This can also make it possible to make certain operations noexcept that would otherwise be difficult to make noexcept due to the need to allocate memory. For example, the co_await io_service::schedule() operation can be made noexcept by borrowing some storage from the coroutine frame for storing the per-operation state for scheduling a coroutine.

2. You may be right. I have not studied the Networking TS enough to make a fully informed call on this.
   However, from the brief reading I have made of the TS, it doesn't currently seem possible to avoid memory allocation and synchronisation per-async-operation when used with coroutines. @GorNishanov wrote about this in P00551 and will be presenting a talk on using Coroutines TS with Networking TS at CppCon this year, so perhaps he could comment further on this.

AFAIK, the Networking TS doesn't yet add async file I/O to the standard so there is still a gap there to be filled.

There may also some potential performance improvements with regards to what can be done with executors and scheduling tasks if they can be built using coroutines.

3. My thoughts here are to provide two kinds of I/O scheduler.

The first is io_service which allows processing events from a thread under the control of the caller. This allows single-threaded or multi-threaded event processing (similar to libevent and asio).

The second is to add a thread_pool that supports processing I/O events as well as scheduled tasks, where the threads are managed by the scheduler. This could be mapped to the Win32 thread-pool on Windows or could have a largely generic work-stealing thread pool implementation based on std::thread and hooked up with the platform's OS async dispatcher (epoll/aio on Linux, IOCP on Windows).

Both kinds of schedulers can also be used for scheduling generalised parallel/concurrent tasks, not only for async I/O completion events.

I'm yet not sure where the line should be drawn between cppcoro and the other async I/O projects. However, there don't seem to be many (any?) other projects looking at what is possible if you implement async I/O purely on top of coroutines yet so I'm hoping to explore that space with cppcoro.

It may be worthwhile separating out the platform-specific I/O bits of cppcoro to a separate library (cppcoro_io?) so the core fundamental abstractions (task, async_generator, when_all(), etc.) can be used in isolation with another async I/O library if required without needing to pull in the cppcoro version.

[ericniebler]

It may be worthwhile separating out the platform-specific I/O bits of cppcoro to a separate library (cppcoro_io?) so the core fundamental abstractions (task, async_generator, when_all(), etc.) can be used in isolation with another async I/O library if required without needing to pull in the cppcoro version.

This is important, I think. The early adopters will want to integrate your abstractions into existing applications that already have an IO polling technology in place, and forcing them to change will be a barrier to adoption. To be clear, I'm 100% in favor of seeing what can be done purely with coroutines, but the IO stuff should be pluggable to the extent that's possible.

[lewissbaker]

I'll look at splitting cppcoro up into a couple of libraries: cppcoro.lib and cppcoro_io.lib.
This would include moving io_service, file* classes into cppcoro_io.lib, and possibly under cppcoro::io namespace.

I'm thinking of rearranging the source tree like:

libs/
  cppcoro/
    include/cppcoro/
      *.hpp
    source/
      *.cpp
      build.cake
    test/
      *.cpp
      build.cake
  cppcoro_io/
    include/cppcoro/io/
      *.hpp
    source/
      *.cpp
      build.cake
    test/
      *.cpp
      build.cake

[akoolenbourke]

I just want to add my vote for splitting out the IO. We are about to undertake a large re-architecture, likely using coroutines and upon a brief look at cppcoro I see some really interesting things we could use. However, we will be using ASIO (Networking TS specifically) so a way to integrate them both would be very beneficial to us.

[IronsDu]

@akoolenbourke asio already support co_await, but has't the core fundamental abstractions (task, async_generator, when_all(), etc.) . so if i want some async function to sync, it must need i am a coroutine expert.