`PyThreadState_Clear()` should only be called from the same interpreter

[colesbury]

PyThreadState_Clear() includes the following comment:

cpython/Python/pystate.c

Lines 1553 to 1558 in 1e4680c

	/* XXX Conditions we need to enforce:
	* the GIL must be held by the current thread
	* current_fast_get()->interp must match tstate->interp
	* for the main interpreter, current_fast_get() must be the main thread
	*/

We should enforce this, particularly the comment about the matching interpreters. Calling PyThreadState_Clear() from the "wrong" interpreter is unsafe because if any of the PyObjects on the tstate are not NULL, calling their destructors from the wrong thread can lead to memory corruption.

This is also important for the "free threaded" builds because they have free lists associated with the PyThreadState and these will be cleared in PyThreadState_Clear() -- doing this in the wrong interpreter leads to memory corruption.

There are currently two places which call PyThreadState_Clear() from the "wrong" interpreter:

1. interp_create() in _xxsubinterpretersmodule.c. This is pretty easy to fix by setting the thread state before calling clear. `

   cpython/Modules/_xxsubinterpretersmodule.c

   Line 266 in 1e4680c

   PyThreadState_Clear(tstate);

2. new_interpreter() in pylifecycle.c in the error code path. This is trickier because the thread state is not fully initialized.
   

   cpython/Python/pylifecycle.c

   Line 2164 in 1e4680c

   PyThreadState_Clear(tstate);

Related: #101436 (comment)

cc @ericsnowcurrently

Linked PRs

• gh-112723: Call PyThreadState_Clear() from the correct interpreter #112776