If a task is canceled at the right moment, the cancellation is ignored

[abacabadabacaba]

BPO	30048
Nosy	@methane, @1st1
PRs	bpo-30048: asyncio: fix Task.cancel() was ignored. #1097 [3.6] bpo-30048: asyncio: fix Task.cancel() was ignored. #1546 [3.5] bpo-30048: asyncio: fix Task.cancel() was ignored. #1547

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2017-05-12.05:37:59.175>
created_at = <Date 2017-04-12.00:11:24.225>
labels = ['type-bug', 'expert-asyncio']
title = 'If a task is canceled at the right moment, the cancellation is ignored'
updated_at = <Date 2017-05-12.05:37:59.174>
user = 'https://bugs.python.org/abacabadabacaba'

bugs.python.org fields:

activity = <Date 2017-05-12.05:37:59.174>
actor = 'methane'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2017-05-12.05:37:59.175>
closer = 'methane'
components = ['asyncio']
creation = <Date 2017-04-12.00:11:24.225>
creator = 'abacabadabacaba'
dependencies = []
files = []
hgrepos = []
issue_num = 30048
keywords = []
message_count = 11.0
messages = ['291522', '291523', '291528', '291531', '291577', '291578', '291579', '291580', '293492', '293493', '293524']
nosy_count = 3.0
nosy_names = ['methane', 'abacabadabacaba', 'yselivanov']
pr_nums = ['1097', '1546', '1547']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30048'
versions = ['Python 3.5', 'Python 3.6']

[abacabadabacaba]

If I run this code:

    import asyncio as a

    @a.coroutine
    def coro1():
        yield from a.ensure_future(coro2())
        print("Still here")
        yield from a.sleep(1)
        print("Still here 2")

    @a.coroutine
    def coro2():
        yield from a.sleep(1)
        res = task.cancel()
        print("Canceled task:", res)

    loop = a.get_event_loop()
    task = a.ensure_future(coro1())
    loop.run_until_complete(task)

I expect the task to stop shortly after a call to cancel(). It should surely stop when I try to sleep(). But it doesn't. On my machine this prints:

Canceled task: True
Still here
Still here 2

So, cancel() returns True, but the task doesn't seem to be canceled.

[1st1]

Interesting. It doesn't work for both C and Python versions of the Task. I'll take a look in detail when I return from vacation.

[methane]

When task.cancel() called, CancelledError is thrown to coro2.
But coro2 doesn't call yield from after task.cancel().
So the exception is never raised.

If you add yield from after task.cancel(), the script runs expected.
---

$ cat at.py
import asyncio as a

@a.coroutine
def coro1():
    yield from a.ensure_future(coro2())
    print("Still here")
    yield from a.sleep(.1)
    print("Still here 2")

@a.coroutine
def coro2():
    yield from a.sleep(.1)
    res = task.cancel()
    print("Canceled task:", res)
    yield from a.sleep(.1)  # !!! added this line

loop = a.get_event_loop()
task = a.ensure_future(coro1())
loop.run_until_complete(task)

$ ./python.exe at.py
Canceled task: True
Traceback (most recent call last):
  File "at.py", line 19, in <module>
    loop.run_until_complete(task)
  File "/Users/inada-n/work/python/cpython/Lib/asyncio/base_events.py", line 465, in run_until_complete
    return future.result()
concurrent.futures._base.CancelledError

[methane]

This behavior is documented as:

https://docs.python.org/3.6/library/asyncio-task.html#asyncio.Task.cancel

Unlike Future.cancel(), this does not guarantee that the task will be cancelled: the exception might be caught and acted upon, delaying cancellation of the task or preventing cancellation completely. The task may also return a value or raise a different exception.

Immediately after this method is called, cancelled() will not return True (unless the task was already cancelled). A task will be marked as cancelled when the wrapped coroutine terminates with a CancelledError exception (even if cancel() was not called).

I agree that this behavior is somewhat surprising.
But I don't know how can I fix the behavior.

[abacabadabacaba]

The problem is that the task doesn't catch CancelledError, yet it disappears.

[methane]

The problem is that the task doesn't catch CancelledError, yet it disappears.

The problem is CancelledError is not raised, even it's thrown.
Task can't catch exception not raised. See below example which demonstrates how task works.

---

from asyncio import CancelledError

cancelled = False

def coro():
    global cancelled
    print(1)
    yield (1,)
    print("cancel")
    cancelled = True
    #print(2)
    #yield (2,)  # uncomment this line makes cancel success.

c = coro()

while True:
    try:
        if cancelled:
            r = c.throw(CancelledError)
        else:
            r = c.send(None)
    except StopIteration:
        print("end")
        break
    except CancelledError:
        print("cancelled")
        break

[1st1]

In Evgeny's example the 'task' is 'coro1' (not 'coro2'). It has plenty of yield points after being cancelled.

[methane]

In Evgeny's example the 'task' is 'coro1' (not 'coro2'). It has plenty of yield points after being cancelled.

Since coro1 waiting coro2 when cancelling, Task(coro1).cancel() redirect to Task(coro2).cancel().

But I was wrong about "CancelledError is thrown to coro2."
Task(coro2) is cancelled in last step. CancelledError is not thrown to coro2 actually.
So there is chance to detect it.

[methane]

New changeset 991adca by INADA Naoki in branch 'master':
bpo-30048: asyncio: fix Task.cancel() was ignored. (GH-1097)
991adca

[methane]

New changeset 3dc7c52 by INADA Naoki in branch '3.6':
bpo-30048: asyncio: fix Task.cancel() was ignored. (GH-1546)
3dc7c52

[methane]

New changeset 5e94ded by INADA Naoki in branch '3.5':
bpo-30048: asyncio: fix Task.cancel() was ignored. (GH-1547)
5e94ded