Thread.join() cannot be interrupted by Ctrl+C on Windows

[james-oldfield-inex]

Bug report

Bug description:

The following short program will always run for 10 seconds on Windows, no matter how quickly you press ctrl+C.

from threading import Thread
import time

t = Thread(target=time.sleep, args=(10,))
t.start()

try:
    print("Joining ...")
    t.join()
    print("Joined without exception")
except KeyboardInterrupt:
    print("Keyboard interrupt")

The program does print "Keyboard interrupt", so the signal handler does actually get run and the exception gets thrown. It just doesn't interrupt the join. Indeed, if the thread function is changed to one that runs forever then this code would wait forever too.

The following expanded version, with a custom signal handler, shows that the signal handler is not run until the join() completes. (As opposed to the possibility that it runs straight away, but then the join() wait resumes before raising the exception. Unlikely I know.)

Version with custom signal handler

import signal
from threading import Thread
import time

start_time = time.monotonic()
def pr(s):
    print(f"{time.monotonic() - start_time:.2f} {s}")

def sig_handler(signal, frame):
    pr("Ctrl-C")
    raise KeyboardInterrupt

signal.signal(signal.SIGINT, sig_handler)

t = Thread(target=time.sleep, args=(10,))
t.start()

try:
    pr("Joining ...")
    t.join()
    pr("Joined without exception")
except KeyboardInterrupt:
    pr("Keyboard interrupt")

If ctrl+c is pressed within 10 seconds then this prints:

0.00 Joining ...
10.00 Ctrl-C
10.00 Keyboard interrupt

Related issues:

• [Windows] KeyboardInterrupt during Thread.join hangs that Thread #66021 (bpo-21822) Sounds similar but is about Thread.join() not returning even after the thread is actually finished. In contrast, this issue is about wanting Thread.join() to complete before the thread is finished.
• Race condition in Thread._wait_for_tstate_lock() #89437 (bpo-45274) This is about the same thing; the above issue was marked as a duplicate of this one.
• threading.Lock.acquire() not interruptible on Windows #74157 (bpo-29971) This looks like it's a similar bug but for a different call (Lock.acquire() rather than Thread.join().
• pycore_condvar.h: remove Windows conditonal variable emulation, use Windows native conditional variable #89464 (bpo-45301 This is referenced from the above issues, but I'm not sure of the exact relationship.

One of the comments on #66021 is about this Thread.join() rather than the real topic of that issue, but they are told "This is a different issue: bpo-29971. Currently, threading.Lock.acquire() cannot be interrupted by CTRL+C." This suggests to me that #74157 is very related to Thread.join() not being interruptable. But it's not exactly the same ... because it's marked as fixed and this issue still persists.

CPython versions tested on:

3.10, 3.12

Operating systems tested on:

Windows

[picnixz]

I think this is unfortunately something that is specific to Windows. On Linux, I would have a hang after calling Ctrl+C because I would then move to thread.shutdown(), which I would once again need to Ctrl+C because this still tries to join it.

As for the signal, IIRC, it's only delivered to the main thread. That's the POSIX standard (and somehow documented at the end of https://docs.python.org/3/library/_thread.html#module-_thread).

I think most of the weird behavior you can see on Windows is actually dependent on the platform itself and not on CPython. Interruptible locks (that are used to implement join()) are not always supported (see https://mail.python.org/pipermail/python-dev/2017-August/148805.html).

cc @ZeroIntensity

[ZeroIntensity]

which I would once again need to Ctrl+C because this still tries to join it.

Hm, this doesn't sound right. If you interrupt threading._shutdown, the interpreter won't attempt to join remaining threads. In fact, I broke this behavior recently and had to fix it.

Interruptible locks (that are used to implement join()) are not always supported

We don't use interruptible locks to implement Thread.join(). Instead, we have a loop with PyEvent_WaitTimed that calls Py_MakePendingCalls every millisecond or something like that. I think that should work on Windows.

I wonder if this is something we fixed with the introduction of free-threading in 3.13. @james-oldfield-inex Does this still occur on 3.13 (or 3.14, since that's getting released today)?

[james-oldfield-inex]

Oops, I've found there was an issue that this is an exact duplicate of:

• threading.Thread.join() cannot be interrupted by a Ctrl-C #41740 (bpo-1167930)

Guido closed that saying "This is because the regular acquire() method on a basic lock cannot be interrupted. That's unlikely to go away, so you'll just have to live with this." But as we've seen in #89464, that has now gone away, so presumably this is now fixable (and presumably this is why one of those comments equated this issue with the acquire() one). So I think that either this issue should stay open, or be merged with #41740 and that be reopened.

---

@ZeroIntensity

We don't use interruptible locks to implement Thread.join().

Hmm, perhaps this is a change since Guido's comment? I guess that's why this didn't magically go away when #89464 was fixed, as at least one person expected. But I suppose it proves it can be done? I'm tempted to go on to ask why you don't use interruptible locks to implement Thread.join() but the reality is I don't know the internals well enough to do anything with that information.

I will try out some other Python versions soon.

---

@picnixz

On Linux, I would have a hang after calling Ctrl+C because I would then move to thread.shutdown(), ... As for the signal, IIRC, it's only delivered to the main thread.

I think you're reading too much into the example code. I was including just minimal code to demonstrate the behaviour. Yes, the toy program would go on to hang anyway if the join() were successfully interrupted. In the real program that this is based upon, after thread.join() is interrupted I would do something application-specific to stop my thread (close the socket it is using) then do another thread.join() which should now return fairly promptly. And I understand that signals can only happen in the main thread (yes that's true on Windows too).

[picnixz]

@ZeroIntensity

Hm, this doesn't sound right. If you interrupt threading._shutdown

Maybe I wasn't clear. I meant

from threading import Thread
import time

def p():
    print(1)
    time.sleep(3)
    print(3)

t = Thread(target=p)
t.start()
t.join()

$ ./python test.py
1
^CTraceback (most recent call last):
  File "/[...]/cpython/test.py", line 48, in <module>
    t.join()
    ~~~~~~^^
  File "/[...]/cpython/Lib/threading.py", line 1125, in join
    self._os_thread_handle.join(timeout)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyboardInterrupt
3

Indeed, if I hit Ctrl+C again I get:

1
^CTraceback (most recent call last):
  File "/[...]/cpython/test.py", line 48, in <module>
    t.join()
    ~~~~~~^^
  File "/[...]/cpython/Lib/threading.py", line 1125, in join
    self._os_thread_handle.join(timeout)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyboardInterrupt
^CException ignored on threading shutdown:
Traceback (most recent call last):
  File "/[...]/cpython/Lib/threading.py", line 1576, in _shutdown
    _thread_shutdown()
    ~~~~~~~~~~~~~~~~^^
KeyboardInterrupt

So yes, _thread_shutdown() doesn't try to join again.

[james-oldfield-inex]

Update:

• I tried on Python 3.13 (installed with Anaconda) and it still has the same bug in the example
• I tried python.exe from Python 3.14rc3 installer from the website and this is now fixed! join() gets interrupted immediately
• I tried python3.14t.exe (free-threaded binary) from Python 3.14rc3 and again it's fixed!

These are all 64 bit binaries by the way.

Is there something plausible in 3.14 that would have fixed this? If so I guess this can just be closed.

[ZeroIntensity]

We made a bunch of changes to _thread in 3.14 for free-threading, so someone probably fixed this along the way. I think we should just close this since it's fixed and probably not something we want to backport to 3.13.

[james-oldfield-inex]

@picnixz About your last post - this is correct behaviour, as these are not daemon threads so Python is documented as waiting for them to exit even after the main script has completed. If you set daemon=True then you'll find the whole program stops immediately when you press ctrl+c (if you're running on a platform/version where Thread.join() is interruptible), which again is correct.

[james-oldfield-inex]

As per @ZeroIntensity closing this issue as fixed in 3.14 (to be released later today - now that is a very fast resolution 😄 )

[kumaraditya303]

For reference this was done in #125546 following #125110.