-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inconsistent gather with child exception #81884
Comments
I found some issue that I suppose is a bug. Let us have long running coroutines. We use them in gather, and one of them raises an error. Since then we cannot cancel the gather anymore, thus remaining children are not cancelable and executed until complete or raise an exception themselves. === import asyncio
async def coro_with_error():
# Coro raises en error with 1 sec delay
await asyncio.sleep(1)
raise Exception('Error in coro')
async def cancellator(coro):
# We use this to cancel gather with delay 1 sec
await asyncio.sleep(1)
coro.cancel()
async def success_long_coro():
# Long running coro, 2 sec
try:
await asyncio.sleep(2)
print("I'm ok!")
return 42
except asyncio.CancelledError:
# Track that this coro is really cancelled
print('I was cancelled')
raise
async def collector_with_error():
gather = asyncio.gather(coro_with_error(), success_long_coro())
try:
await gather
except Exception:
print(f"WHOAGH ERROR, gather done={gather.done()}")
print(f'EXC={type(gather.exception()).__name__}')
# We want to cancel still running success_long_coro()
gather.cancel()
async def collector_with_cancel():
# Gather result from success_long_coro()
gather = asyncio.gather(success_long_coro())
# schedule cancel in 1 sec
asyncio.create_task(cancellator(gather))
try:
await gather
except Exception:
print(f"WHOAGH ERROR, gather done={gather.done()}")
print(f'EXC={type(gather.exception()).__name__}')
# We want to cancel still running success_long_coro()
gather.cancel()
return
# First case, cancel gather when children are running
print('First case')
loop = asyncio.get_event_loop()
loop.create_task(collector_with_cancel())
# Ensure test coros we fully run
loop.run_until_complete(asyncio.sleep(3))
print('Done')
# Second case, cancel gather when child raise error
print('Second case')
loop = asyncio.get_event_loop()
loop.create_task(collector_with_error())
# Ensure test coros we fully run
loop.run_until_complete(asyncio.sleep(3))
print('Done') === Actual output:
Expected output:
Documentations says:
From doc:
I believe asyncio should allow cancellation in that case. |
Hi Dimitri, I also believe that the documentation of gather should explicitly mention this. But, depending on the fact, whether this is an expected behaviour, current code base might also need changes. Therefore I have created two patches, one updating the current documentation according to the current functionality, and other changing the codebase which supports cancelling even after raising exceptions. I will try to contact one of the core developers on deciding which one is the way to go. |
The behavior change is not backward compatible, we cannot apply it. Also, please use github pull requests for proposing a patch: https://devguide.python.org/pullrequest/ |
Hi Andrew, I understand that the behavior change is not backward compatible. But the current behavior is a bit ambiguous ad mentioned by Dimitri, therefore I have updated the documentation and opened a Pull request. I would be very glad if you could review it. Also, would adding an optional argument like force_cancel=False to |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: