gh-105836: Fix asyncio.run_coroutine_threadsafe leaving underlying cancelled asyncio task running

[iaalm]

• This issue caused by _chain_future which supports concurrent.futures.Future and asycio.Future. While asyncio.run_coroutine_threadsafe gives destination a concurrent.futures.Future, the dest_loop is None. Then dest_loop is source_loop is not good enough for the check. We should always the current running loop for the future operating.
• In case of concurrent.futures.Future created by asyncio.run_coroutine_threadsafe, we need to wait for the cancellation before return. That's what I done in _cancel_future_in_loop.

• Issue: asyncio.run_coroutine_threadsafe leaves underlying cancelled asyncio task running #105836

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[gvanrossum]

Thanks!

I tried this with the example from gh-105836. This indeed fixes it, but still requires the await asyncio.sleep(0) after the shutdown() call. (Also note that the example passes if there are two sleep(0) calls.)

Is there a way to do better?

Also, probably that example should be added as a unit test (preferably without long sleeps).

[iaalm]

Hi @gvanrossum,

Added one (no very cool) test case. The new test failed on windows, checking..

Regarding the asyncio.sleep(0), the example in gh-105836 uses Thread.run() which actually run the function in main thread (maybe by typo?). In such cases, because future.cancel() is a sync function running in loop. There's no good way in my opinion to wait for the cancellation in future.cancel(). So one asyncio.sleep(0) is always needed to execute the cancellation work. I think that is also the reason all test in test_asyncio.test_tasks.RunCoroutineThreadsafeTests needs a asyncio.sleep(0) after cancel.
My code change is mainly avoiding another sleep to run the "future.cancel()" itself.

# emulates some worker thread (e.g. APScheduler)
    runner = threading.Thread(target=myapp.start_async_task, args=[thread_event])
    runner.run()

Does that make sense?

[gvanrossum]

Nearly there! Thanks for persevering.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[iaalm]

will investigate why the case fail on windows tomorrow (need to find a windows device😝)

[gvanrossum]

I'd move this import into the test function -- otherwise it could be confused with asyncio.Event.

[gvanrossum]

I'd use import threading followed by target_started = threading.Event() to be super clear about what kind of event this is.

[iaalm]

I am going to apply my style nits and then it should be perfect, right?

Thank you! Things have been hectic on my end. Come back to the PR now.

[gvanrossum]

@iaalm Alas, the Windows tests still fail. Can you look into that? I suspect the timing is different because of IOCP (proactor_loop).

[iaalm]

@iaalm Alas, the Windows tests still fail. Can you look into that? I suspect the timing is different because of IOCP (proactor_loop).

You're right. I tried it on a Windows VM. The best fix I can see is filter out BaseProactorEventLoop._loop_self_reading which seems always there in _ready while counting tasks.
PR updated.

[gvanrossum]

Now the tests pass, I'm approving this.

I just want @kumaraditya303 to have a look as well.

[kumaraditya303]

I have improved the test to not rely on _ready and removed the windows hack, the test passes on this branch while fails on current main:

Running Debug|x64 interpreter...
== CPython 3.15.0a1+ (heads/cancel_in_loop:787c9fc5de0, Dec 6 2025, 20:45:00) [MSC v.1944 64 bit (AMD64)]
== Windows-11-10.0.26200-SP0 little-endian
== Python build: debug
== cwd: D:\cpython\build\test_python_worker_24356æ
== CPU count: 12
== encodings: locale=cp1252 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1952231759
0:00:00 Run 1 test sequentially in a single process
0:00:00 [1/1] test_asyncio.test_tasks
test_run_coroutine_threadsafe_and_cancel (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_and_cancel) ... FAIL

======================================================================
FAIL: test_run_coroutine_threadsafe_and_cancel (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_and_cancel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cpython\Lib\test\test_asyncio\test_tasks.py", line 3704, in test_run_coroutine_threadsafe_and_cancel
    self.assertTrue(task.cancelled())
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 0.014s

FAILED (failures=1)
Task was destroyed but it is pending!
task: <Task cancelling name='Task-2' coro=<sleep() done, defined at D:\cpython\Lib\asyncio\tasks.py:687> wait_for=<Future cancelled> cb=[_chain_future.<locals>._call_set_state() at D:\cpython\Lib\asyncio\futures.py:397]>
test test_asyncio.test_tasks failed
0:00:00 [1/1/1] test_asyncio.test_tasks failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_asyncio.test_tasks

Total duration: 542 ms
Total tests: run=1 (filtered) failures=1
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE

[gvanrossum]

Thanks @kumaraditya303, that looks much better!

[miss-islington-app]

Thanks @iaalm for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-142358 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-142359 is a backport of this pull request to the 3.13 branch.