Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Should we have a way to let some other coroutine runner take temporary control of a task? #649
async def somefunc(): await do_something_in_main_thread() async with in_worker_thread(): do_something_blocking() await do_something_in_main_thread()
Another possible application would be to allow trio-asyncio to switch back and forth between trio mode and asyncio mode within a single function.
This is kind of brain melting, but the implementation is surprisingly simple: a task's current execution is represented by a coroutine object. So, you suspend the coroutine, and from Trio's point of view it becomes "blocked"; then you take that coroutine object and give it to another coroutine runner to iterate for a while. Later, it gets handed back and Trio "resumes" it.
You can't actually implement something like this right now though, using Trio's public API. You can suspend a task, and get the coroutine object... but the coroutine is suspended inside
But it wouldn't be terribly hard to add a new suspend/resume API just for this. In
@types.coroutine def detach_task(abort_fn): yield some_magic_value(abort_fn) @types.coroutine def reattach_task(yield_value): got = yield yield_value do_whatever_wait_task_rescheduled_would_do_with(got)
So the idea is that to detach the task from trio, you do
I think this + #642 would make it possible to fully implement curio's "async threads" API as a trio library.
So... is it a good idea?
I've known about this trick for quite some time, and hesitated because
Doing the handoff part in reverse is fairly straightforward, I think, because asyncio's yield protocol is effectively public (IIRC:
async def start_in_trio_mode(): original_trio_task = trio.hazmat.current_task() async with aio_mode: original_aio_task = asyncio.current_task() async with trio_mode: # Can nesting return to the original task, instead of creating a new one? assert original_trio_task == trio.hazmat.current_task() async with aio_mode: original_aio_task = asyncio.current_task() # Back in trio mode now... # If we switch into aio mode several times, do we keep re-entering the same task? async with aio_mode: assert original_aio_task is asyncio.current_task()
I'm not sure how much this matters, but as we've seen with aiohttp, asyncio code does sometimes make strong assumptions about
Hmm. I was thinking more of
NB: your assertions will fail: you cannot re-purpose the original "outer" task, because there may be more than one concurrent "inner" task. So you need a new one. For the same reason you'll probably need a new context manager for each level, so
Yeah, it'd have to spawn it under the loop's nursery. I guess teardown might be an issue – like maybe we also need a way to say "don't just detach this task temporarily, actually forget about it entirely, pretend it just exited even though the coroutine wasn't exhausted". We need to prototype some stuff here to learn what the actual tricky bits are :-).
I don't understand. I'm imagining that this is all within the text of a single function, and by definition a single Python function is a purely sequential thing, no concurrency allowed.
I'm just being lazy with my pseudocode :-). No point in worrying about details like parentheses when we don't even know if the basic idea is possible or useful :-).
Yeah, in your example it's all within a single function, but conceptually it's the same as
async def foo(): async with aio_mode(): await bar() async def bar(): async with trio_mode(): await baz() async def baz(): await trio.sleep(0) trio_asyncio.run(foo)
and nothing prevents one from inserting a couple of nurseries and
I was imagining that we'd keep some kind of bidirectional mapping between asyncio and trio