asyncio: odd behaviour with TaskGroup.start()

[arthur-tacca]

Consider the following code:

import anyio

async def sleep_and_raise(task_status=anyio.TASK_STATUS_IGNORED):
    await anyio.sleep(3)
    raise RuntimeError

async def sleep_only(task_status=anyio.TASK_STATUS_IGNORED):
    await anyio.sleep(20)
    task_status.started()

async def foo():
    async with anyio.create_task_group() as run_tg:
        async with anyio.create_task_group() as start_tg:
            start_tg.start_soon(run_tg.start, sleep_and_raise)
            print("here")
            await run_tg.start(sleep_only)
            print("there")

anyio.run(foo)

sleep_and_raise raises an exception, which propagates into start_tg (because task_status.started() hasn't been called yet, so the task is still running start_tg). This happens after 3 seconds, so sleep_only() (which is also still running under start_tg) gets cancelled.

Except, sleep_only() doesn't get cancelled, and the whole thing waits for it to complete. At that point, the exception is noticed and the exception gets raised.

If you add await anyio.sleep(0) immediately before print("there") then a cancellation happens then instead, so it's not a matter of having to reach the end of the task group context manager.

Changing await run_tg.start(sleep_only) to await start_tg.start(sleep_only) also fixes the problem.

[arthur-tacca]

I should've made clear - this came up while I was writing an automated test (for aioresult), not because I need it for any real-world situation. Typically the start part of a task is something very quick, like opening a port of listening, so in practice this is perhaps not likely to be a big deal.

[agronholm]

There are corner cases where cancellation might not work as expected, but I'm working to resolve these for AnyIO 4.0. It's a daunting task (cancellation is hard to get right), so I'm not sure when it's going to be released.

[arthur-tacca]

"cancellation is hard to get right" I can imagine it is! And this is an especially obscure case, and doesn't even actually affect me. I'm happy to choose this of you like, I just wanted to let you know about it.

[agronholm]

I investigated this today, and the main task (foo()) isn't getting cancelled by the RuntimeError because it's in a shielded scope, waiting for await run_tg.start(sleep_only) to complete. Obviously this differs from Trio behavior, so I'm going to see if I can get this fixed for v4.0.

[agronholm]

While I wasn't able to figure out a proper fix yet, I've at least added a minimal reproducing test to the test suite.

[agronholm]

I did in fact figure it out! @arthur-tacca would you mind reviewing the PR?