Skip to content

Commit

Permalink
gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncance…
Browse files Browse the repository at this point in the history 
…l() (#95253)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
  • Loading branch information
@ambv @graingert
ambv and graingert committed Oct 1, 2022
1 parent 273a819 commit f00645d
Show file tree
Hide file tree
Showing 3 changed files with 254 additions and 80 deletions.
202 changes: 128 additions & 74 deletions Doc/library/asyncio-task.rst
View file
Expand Up @@ -294,11 +294,13 @@ perform clean-up logic. In case :exc:`asyncio.CancelledError`
is explicitly caught, it should generally be propagated when
clean-up is complete. Most code can safely ignore :exc:`asyncio.CancelledError`.

Important asyncio components, like :class:`asyncio.TaskGroup` and the
:func:`asyncio.timeout` context manager, are implemented using cancellation
internally and might misbehave if a coroutine swallows
:exc:`asyncio.CancelledError`.
The asyncio components that enable structured concurrency, like
:class:`asyncio.TaskGroup` and :func:`asyncio.timeout`,
are implemented using cancellation internally and might misbehave if
a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code
should not call :meth:`uncancel <asyncio.Task.uncancel>`.

.. _taskgroups:

Task Groups
===========
Expand Down Expand Up @@ -1003,76 +1005,6 @@ Task Object
Deprecation warning is emitted if *loop* is not specified
and there is no running event loop.

.. method:: cancel(msg=None)

Request the Task to be cancelled.

This arranges for a :exc:`CancelledError` exception to be thrown
into the wrapped coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the
request by suppressing the exception with a :keyword:`try` ...
... ``except CancelledError`` ... :keyword:`finally` block.
Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
not guarantee that the Task will be cancelled, although
suppressing cancellation completely is not common and is actively
discouraged.

.. versionchanged:: 3.9
Added the *msg* parameter.

.. deprecated-removed:: 3.11 3.14
*msg* parameter is ambiguous when multiple :meth:`cancel`
are called with different cancellation messages.
The argument will be removed.

.. _asyncio_example_task_cancel:

The following example illustrates how coroutines can intercept
the cancellation request::

async def cancel_me():
print('cancel_me(): before sleep')

try:
# Wait for 1 hour
await asyncio.sleep(3600)
except asyncio.CancelledError:
print('cancel_me(): cancel sleep')
raise
finally:
print('cancel_me(): after sleep')

async def main():
# Create a "cancel_me" Task
task = asyncio.create_task(cancel_me())

# Wait for 1 second
await asyncio.sleep(1)

task.cancel()
try:
await task
except asyncio.CancelledError:
print("main(): cancel_me is cancelled now")

asyncio.run(main())

# Expected output:
#
# cancel_me(): before sleep
# cancel_me(): cancel sleep
# cancel_me(): after sleep
# main(): cancel_me is cancelled now

.. method:: cancelled()

Return ``True`` if the Task is *cancelled*.

The Task is *cancelled* when the cancellation was requested with
:meth:`cancel` and the wrapped coroutine propagated the
:exc:`CancelledError` exception thrown into it.

.. method:: done()

Return ``True`` if the Task is *done*.
Expand Down Expand Up @@ -1186,3 +1118,125 @@ Task Object
in the :func:`repr` output of a task object.

.. versionadded:: 3.8

.. method:: cancel(msg=None)

Request the Task to be cancelled.

This arranges for a :exc:`CancelledError` exception to be thrown
into the wrapped coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the
request by suppressing the exception with a :keyword:`try` ...
... ``except CancelledError`` ... :keyword:`finally` block.
Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
not guarantee that the Task will be cancelled, although
suppressing cancellation completely is not common and is actively
discouraged.

.. versionchanged:: 3.9
Added the *msg* parameter.

.. deprecated-removed:: 3.11 3.14
*msg* parameter is ambiguous when multiple :meth:`cancel`
are called with different cancellation messages.
The argument will be removed.

.. _asyncio_example_task_cancel:

The following example illustrates how coroutines can intercept
the cancellation request::

async def cancel_me():
print('cancel_me(): before sleep')

try:
# Wait for 1 hour
await asyncio.sleep(3600)
except asyncio.CancelledError:
print('cancel_me(): cancel sleep')
raise
finally:
print('cancel_me(): after sleep')

async def main():
# Create a "cancel_me" Task
task = asyncio.create_task(cancel_me())

# Wait for 1 second
await asyncio.sleep(1)

task.cancel()
try:
await task
except asyncio.CancelledError:
print("main(): cancel_me is cancelled now")

asyncio.run(main())

# Expected output:
#
# cancel_me(): before sleep
# cancel_me(): cancel sleep
# cancel_me(): after sleep
# main(): cancel_me is cancelled now

.. method:: cancelled()

Return ``True`` if the Task is *cancelled*.

The Task is *cancelled* when the cancellation was requested with
:meth:`cancel` and the wrapped coroutine propagated the
:exc:`CancelledError` exception thrown into it.

.. method:: uncancel()

Decrement the count of cancellation requests to this Task.

Returns the remaining number of cancellation requests.

Note that once execution of a cancelled task completed, further
calls to :meth:`uncancel` are ineffective.

.. versionadded:: 3.11

This method is used by asyncio's internals and isn't expected to be
used by end-user code. In particular, if a Task gets successfully
uncancelled, this allows for elements of structured concurrency like
:ref:`taskgroups` and :func:`asyncio.timeout` to continue running,
isolating cancellation to the respective structured block.
For example::

async def make_request_with_timeout():
try:
async with asyncio.timeout(1):
# Structured block affected by the timeout:
await make_request()
await make_another_request()
except TimeoutError:
log("There was a timeout")
# Outer code not affected by the timeout:
await unrelated_code()

While the block with ``make_request()`` and ``make_another_request()``
might get cancelled due to the timeout, ``unrelated_code()`` should
continue running even in case of the timeout. This is implemented
with :meth:`uncancel`. :class:`TaskGroup` context managers use
:func:`uncancel` in a similar fashion.

.. method:: cancelling()

Return the number of pending cancellation requests to this Task, i.e.,
the number of calls to :meth:`cancel` less the number of
:meth:`uncancel` calls.

Note that if this number is greater than zero but the Task is
still executing, :meth:`cancelled` will still return ``False``.
This is because this number can be lowered by calling :meth:`uncancel`,
which can lead to the task not being cancelled after all if the
cancellation requests go down to zero.

This method is used by asyncio's internals and isn't expected to be
used by end-user code. See :meth:`uncancel` for more details.

.. versionadded:: 3.11
4 changes: 2 additions & 2 deletions Lib/asyncio/tasks.py
View file
Expand Up @@ -243,8 +243,8 @@ def cancelling(self):
def uncancel(self):
"""Decrement the task's count of cancellation requests.
This should be used by tasks that catch CancelledError
and wish to continue indefinitely until they are cancelled again.
This should be called by the party that called `cancel()` on the task
beforehand.
Returns the remaining number of cancellation requests.
"""
Expand Down
128 changes: 124 additions & 4 deletions Lib/test/test_asyncio/test_tasks.py
View file
Expand Up @@ -521,7 +521,7 @@ async def task():
finally:
loop.close()

def test_uncancel(self):
def test_uncancel_basic(self):
loop = asyncio.new_event_loop()

async def task():
Expand All @@ -534,17 +534,137 @@ async def task():
try:
t = self.new_task(loop, task())
loop.run_until_complete(asyncio.sleep(0.01))
self.assertTrue(t.cancel()) # Cancel first sleep

# Cancel first sleep
self.assertTrue(t.cancel())
self.assertIn(" cancelling ", repr(t))
self.assertEqual(t.cancelling(), 1)
self.assertFalse(t.cancelled()) # Task is still not complete
loop.run_until_complete(asyncio.sleep(0.01))
self.assertNotIn(" cancelling ", repr(t)) # after .uncancel()
self.assertTrue(t.cancel()) # Cancel second sleep

# after .uncancel()
self.assertNotIn(" cancelling ", repr(t))
self.assertEqual(t.cancelling(), 0)
self.assertFalse(t.cancelled()) # Task is still not complete

# Cancel second sleep
self.assertTrue(t.cancel())
self.assertEqual(t.cancelling(), 1)
self.assertFalse(t.cancelled()) # Task is still not complete
with self.assertRaises(asyncio.CancelledError):
loop.run_until_complete(t)
self.assertTrue(t.cancelled()) # Finally, task complete
self.assertTrue(t.done())

# uncancel is no longer effective after the task is complete
t.uncancel()
self.assertTrue(t.cancelled())
self.assertTrue(t.done())
finally:
loop.close()

def test_uncancel_structured_blocks(self):
# This test recreates the following high-level structure using uncancel()::
#
# async def make_request_with_timeout():
# try:
# async with asyncio.timeout(1):
# # Structured block affected by the timeout:
# await make_request()
# await make_another_request()
# except TimeoutError:
# pass # There was a timeout
# # Outer code not affected by the timeout:
# await unrelated_code()

loop = asyncio.new_event_loop()

async def make_request_with_timeout(*, sleep: float, timeout: float):
task = asyncio.current_task()
loop = task.get_loop()

timed_out = False
structured_block_finished = False
outer_code_reached = False

def on_timeout():
nonlocal timed_out
timed_out = True
task.cancel()

timeout_handle = loop.call_later(timeout, on_timeout)
try:
try:
# Structured block affected by the timeout
await asyncio.sleep(sleep)
structured_block_finished = True
finally:
timeout_handle.cancel()
if (
timed_out
and task.uncancel() == 0
and sys.exc_info()[0] is asyncio.CancelledError
):
# Note the five rules that are needed here to satisfy proper
# uncancellation:
#
# 1. handle uncancellation in a `finally:` block to allow for
# plain returns;
# 2. our `timed_out` flag is set, meaning that it was our event
# that triggered the need to uncancel the task, regardless of
# what exception is raised;
# 3. we can call `uncancel()` because *we* called `cancel()`
# before;
# 4. we call `uncancel()` but we only continue converting the
# CancelledError to TimeoutError if `uncancel()` caused the
# cancellation request count go down to 0. We need to look
# at the counter vs having a simple boolean flag because our
# code might have been nested (think multiple timeouts). See
# commit 7fce1063b6e5a366f8504e039a8ccdd6944625cd for
# details.
# 5. we only convert CancelledError to TimeoutError; for other
# exceptions raised due to the cancellation (like
# a ConnectionLostError from a database client), simply
# propagate them.
#
# Those checks need to take place in this exact order to make
# sure the `cancelling()` counter always stays in sync.
#
# Additionally, the original stimulus to `cancel()` the task
# needs to be unscheduled to avoid re-cancelling the task later.
# Here we do it by cancelling `timeout_handle` in the `finally:`
# block.
raise TimeoutError
except TimeoutError:
self.assertTrue(timed_out)

# Outer code not affected by the timeout:
outer_code_reached = True
await asyncio.sleep(0)
return timed_out, structured_block_finished, outer_code_reached

# Test which timed out.
t1 = self.new_task(loop, make_request_with_timeout(sleep=10.0, timeout=0.1))
timed_out, structured_block_finished, outer_code_reached = (
loop.run_until_complete(t1)
)
self.assertTrue(timed_out)
self.assertFalse(structured_block_finished) # it was cancelled
self.assertTrue(outer_code_reached) # task got uncancelled after leaving
# the structured block and continued until
# completion
self.assertEqual(t1.cancelling(), 0) # no pending cancellation of the outer task

# Test which did not time out.
t2 = self.new_task(loop, make_request_with_timeout(sleep=0, timeout=10.0))
timed_out, structured_block_finished, outer_code_reached = (
loop.run_until_complete(t2)
)
self.assertFalse(timed_out)
self.assertTrue(structured_block_finished)
self.assertTrue(outer_code_reached)
self.assertEqual(t2.cancelling(), 0)

def test_cancel(self):

def gen():
Expand Down

0 comments on commit f00645d

Please sign in to comment.