Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

thread support #19

Closed
klemens-morgenstern opened this issue Jul 14, 2023 · 9 comments
Closed

thread support #19

klemens-morgenstern opened this issue Jul 14, 2023 · 9 comments
Labels
enhancement New feature or request

Comments

@klemens-morgenstern
Copy link
Collaborator

The library is currently strictly single threaded, and the former async::thread class has been removed. If we want multi-threading we need would need synchronization primitives, such as mutex etc. (the sam stuff, but simpler).

Potentially also thread-safe channels.

@klemens-morgenstern klemens-morgenstern added the enhancement New feature or request label Jul 14, 2023
@ned14
Copy link
Member

ned14 commented Jul 14, 2023

Surely the bare minimum viable support needed here is to reparent something from one executor to another? You can build anything else you want on top of that.

@jsaf0
Copy link

jsaf0 commented Jul 14, 2023

Simple synchronization primitives like Event and Lock from Python’s asyncio (https://docs.python.org/3/library/asyncio-sync.html) would still be very convenient to have as part of this library - also in a single-threaded context.

@klemens-morgenstern
Copy link
Collaborator Author

This was the thread type (that can be restored): a7fe353

boost::async::thread thr()
{
  boost::asio::steady_timer tim{co_await boost::asio::this_coro::executor, std::chrono::milliseconds(100)};

  auto exec = co_await boost::asio::this_coro::executor;
  co_await tim.async_wait(boost::asio::deferred);
}

boost::async::main co_main(int argc, char ** argv)
{

   co_await thr();
   co_return 0;
}

@klemens-morgenstern
Copy link
Collaborator Author

Actually all we need in addition is to be able to safely have co_await cross executors:

boost::async::thread thr(async::channel<int> foreign_channel) // < different executor
{
  co_await thread_safe(foreign_channel.read()); // has an associated executor
  co_await thread_safe(foreign_channel.get_executor(), foreign_channel.read());
}

klemens-morgenstern added a commit that referenced this issue Jul 15, 2023
klemens-morgenstern added a commit that referenced this issue Jul 15, 2023
klemens-morgenstern added a commit that referenced this issue Jul 15, 2023
@klemens-morgenstern
Copy link
Collaborator Author

Alternatively we could have a coro that is attached to a thread but can be awaited safely, like this:

thread::safe coro(thread & thread_to_run_on)
{
  // everything in here runs on `thread_to_run_on`
   co_await 
}

@klemens-morgenstern
Copy link
Collaborator Author

await_on doesn't work with select.

latch might do:

boost::async::thread thr(latch<int> l)
{
  l.notify(42);
}

boost::async::main co_main(int argc, char ** argv)
{
   latch<int> l;
   thread(l).detach();

   co_return co_await l;
}

Alternatively, sam could be ported. Unlike raw sam we can actually detect if we're recursive based onte thread_local and/or coroutine's own executor. Thus you won't accidentally dead-lock.

@klemens-morgenstern
Copy link
Collaborator Author

@madmongo1

@klemens-morgenstern
Copy link
Collaborator Author

Added an spsc-queue example, we don't need this with unique_handle and the other stuff being in the code.

klemens-morgenstern added a commit that referenced this issue Oct 18, 2023
```
In file included from libs\cobalt\test\strand.cpp:8:
In file included from .\boost/asio/strand.hpp:582:
In file included from .\boost/asio/io_context_strand.hpp:25:
.\boost/asio/detail/strand_service.hpp(112,15): warning: private field 'io_context_' is not used [-Wunused-private-field]
  io_context& io_context_;
              ^
fatal error: error in backend: failed to perform tail call elimination on a call site marked musttail
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: clang-cl.exe --target=i386-pc-windows-msvc libs\\cobalt\\test\\strand.cpp -c -Fobin.v2\\libs\\cobalt\\test\\strand.test\\clang-win-16.0.6\\debug\\address-model-32\\cxxstd-20-iso\\threading-multi\\strand.obj -TP /EHs /std:c++20 /GR /Z7 /Od /Ob0 /W3 /MDd -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_COBALT_USE_STD_PMR=1 -DWIN32_LEAN_AND_MEAN -I.
1.	<eof> parser at end of file
2.	Code generation
3.	Running pass 'Function Pass Manager' on module 'libs\cobalt\test\strand.cpp'.
4.	Running pass 'X86 DAG->DAG Instruction Selection' on function '@"?do_the_thing@@ya?AU?$promise@X@cobalt@boost@@XZ.resume"'
Exception Code: 0xE0000046
 #0 0x00007ffa840df1ac (C:\Windows\System32\KERNELBASE.dll+0x3f1ac)
 #1 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x625afa C:\Program Files\LLVM\bin\clang-cl.exe 0x64fbc9
 #2 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0xa731 C:\Program Files\LLVM\bin\clang-cl.exe 0x626b2c
 #3 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x6269f5 C:\Program Files\LLVM\bin\clang-cl.exe 0x2569828
 #4 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x23634ef C:\Program Files\LLVM\bin\clang-cl.exe 0x2386421
 #5 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x236b068 C:\Program Files\LLVM\bin\clang-cl.exe 0x23558ae
 #6 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x2348a18 C:\Program Files\LLVM\bin\clang-cl.exe 0x1d856b6
 #7 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x1d8512e C:\Program Files\LLVM\bin\clang-cl.exe 0x1d830cd
 #8 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x12fbc5c C:\Program Files\LLVM\bin\clang-cl.exe 0x8bd7cd
 #9 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x3dd07a C:\Program Files\LLVM\bin\clang-cl.exe 0x3e2d63
#10 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x3dd6f3 C:\Program Files\LLVM\bin\clang-cl.exe 0x17efdbf
#11 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x1b82a63 C:\Program Files\LLVM\bin\clang-cl.exe 0x33b032b
#12 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x1aed942 C:\Program Files\LLVM\bin\clang-cl.exe 0x79fbbd
#13 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x8360de C:\Program Files\LLVM\bin\clang-cl.exe 0xa131
#14 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x5504 C:\Program Files\LLVM\bin\clang-cl.exe 0x18d41a6
#15 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x625a15 C:\Program Files\LLVM\bin\clang-cl.exe 0x18d3993
#16 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x75a1f5 C:\Program Files\LLVM\bin\clang-cl.exe 0x75a53e
#17 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x777adb C:\Program Files\LLVM\bin\clang-cl.exe 0x4ab7
#18 0x00007ff6dde95afa C:\Program Files\LLVM\bin\clang-cl.exe 0x4626ac0 (C:\Program Files\LLVM\bin\clang-cl.exe+0x625afa)
#19 0x00007ff6dde95afa
#20 0x00007ff6dde95afa (C:\Program Files\LLVM\bin\clang-cl.exe+0x625afa)
0x00007FFA840DF1AC, C:\Windows\System32\KERNELBASE.dll(0x00007FFA840A0000) + 0x3F1AC byte(s), RaiseException() + 0x6C byte(s)
0x00007FF6DDE95AFA, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x625AFA byte(s)
0x00007FF6DDEBFBC9, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x64FBC9 byte(s)
0x00007FF6DD87A731, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0xA731 byte(s)
0x00007FF6DDE96B2C, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x626B2C byte(s)
0x00007FF6DDE969F5, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x6269F5 byte(s)
0x00007FF6DFDD9828, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x2569828 byte(s)
0x00007FF6DFBD34EF, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x23634EF byte(s)
0x00007FF6DFBF6421, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x2386421 byte(s)
0x00007FF6DFBDB068, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x236B068 byte(s)
0x00007FF6DFBC58AE, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x23558AE byte(s)
0x00007FF6DFBB8A18, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x2348A18 byte(s)
0x00007FF6DF5F56B6, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x1D856B6 byte(s)
0x00007FF6DF5F512E, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x1D8512E byte(s)
0x00007FF6DF5F30CD, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x1D830CD byte(s)
0x00007FF6DEB6BC5C, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x12FBC5C byte(s)
0x00007FF6DE12D7CD, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x8BD7CD byte(s)
0x00007FF6DDC4D07A, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x3DD07A byte(s)
0x00007FF6DDC52D63, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x3E2D63 byte(s)
0x00007FF6DDC4D6F3, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x3DD6F3 byte(s)
0x00007FF6DF05FDBF, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x17EFDBF byte(s)
0x00007FF6DF3F2A63, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x1B82A63 byte(s)
0x00007FF6E0C2032B, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x33B032B byte(s)
0x00007FF6DF35D942, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x1AED942 byte(s)
0x00007FF6DE00FBBD, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x79FBBD byte(s)
0x00007FF6DE0A60DE, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x8360DE byte(s)
0x00007FF6DD87A131, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0xA131 byte(s)
0x00007FF6DD875504, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x5504 byte(s)
0x00007FF6DF1441A6, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x18D41A6 byte(s)
0x00007FF6DDE95A15, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x625A15 byte(s)
0x00007FF6DF143993, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x18D3993 byte(s)
0x00007FF6DDFCA1F5, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x75A1F5 byte(s)
0x00007FF6DDFCA53E, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x75A53E byte(s)
0x00007FF6DDFE7ADB, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x777ADB byte(s)
0x00007FF6DD874AB7, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x4AB7 byte(s)
0x00007FF6E1E96AC0, C:\Program Files\LLVM\bin\clang-cl.exe(0x00007FF6DD870000) + 0x4626AC0 byte(s)
```
@pfeatherstone
Copy link

I'm struggling to see why threading has become an issue here, and why we would need synchronization primitives. I'm not an expert like you guys, so that's likely why... However, in Asio, you can use awaitable or coro in a multi-threaded environment safely, provided you carefully choose your executors, e.g. implicit/explicit strands, or channels. Moreover, users can "bring their own threads" by simply calling asio::io_context::run on multiple threads. You don't have to use special threads. I feel like this is causing more confusion to asio users. Maybe cppalliance's book on asio, concurrency and coroutines is much needed. What were @chriskohlhoff 's comments on this library? Thank you everyone for the fantastic work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants