TSAN reports multiple data races, SEGV errors, and thread leaks when running the full test suite

[ashm-dev]

Bug report

Bug description:

Description:

When running the full test suite (./python -m test) on a debug build of the main branch configured with ThreadSanitizer (TSAN) and the experimental JIT, the test run terminates with a FAILURE result.

The attached logs show thousands of warnings from TSAN, including:

1. Massive numbers of data race warnings, primarily during the parallel initialization of sub-interpreters.
2. Several DEADLYSIGNAL (SEGV) reports, causing processes to abort.
3. Multiple thread leak warnings related to multiprocessing tests.
4. Failures in several core tests, including test_datetime, test_faulthandler, test_pyrepl, and test.test_multiprocessing_forkserver.test_manager.

Build & Environment

• CPython Version: 3.15.0a1+ (heads/main:7ac94fcb1d0, Oct 16 2025)
• Platform: Linux-6.12.48+deb13-cloud-amd64-x86_64-with-glibc2.41
• Compiler: Clang 19.1.7
• Configuration command:

  CC=clang CXX=clang++ ./configure --with-thread-sanitizer --with-pydebug --enable-experimental-jit=yes

• Build command:

  make -j$(nproc)

• Test command:

  ./python -m test -j$(nproc)

Summary of Failures

The test suite concludes with the following result:

== Tests result: FAILURE ==

4 tests failed:
    test.test_multiprocessing_forkserver.test_manager test_datetime
    test_faulthandler test_pyrepl

Key Issues Reported by ThreadSanitizer

The full logs will be attached to this report, but here are the primary categories of errors observed.

1. Data Races during Sub-interpreter Initialization

Thousands of data race warnings are reported when multiple threads attempt to initialize global state concurrently. This is especially prominent in test_datetime, which fails with an exit code from TSAN.

• Example Data Race Warning:

  ==================
  WARNING: ThreadSanitizer: data race (pid=1111680)
    Write of size 8 at 0x56174f42e490 by thread T2:
      #0 _PyExc_InitTypes /root/cpython/main/Objects/exceptions.c:4428:32
      #1 pycore_init_types /root/cpython/main/Python/pylifecycle.c:732:9
      ...
  
    Previous write of size 8 at 0x56174f42e490 by thread T1:
      #0 _PyExc_InitTypes /root/cpython/main/Objects/exceptions.c:4428:32
      #1 pycore_init_types /root/cpython/main/Python/pylifecycle.c:732:9
      ...
  
    Location is global '_PyExc_ReferenceError' of size 416 at 0x56174f42e300 (python+0xa57490)
  ...
  SUMMARY: ThreadSanitizer: data race /root/cpython/main/Objects/exceptions.c:4428:32 in _PyExc_InitTypes
  ==================
  

• Resulting test_datetime Failure:

  test test_datetime failed -- Traceback (most recent call last):
  ...
  AssertionError: Process return code is 66
  command line: ['/root/cpython/main/python', '-X', 'faulthandler', '-I', '-c', "if True:\n from concurrent.futures import InterpreterPoolExecutor\n\n def func():\n import _datetime\n ..."]
  

2. SEGV Errors and Process Aborts

Multiple tests trigger fatal segmentation faults, which are caught and reported by TSAN.

• Example SEGV Report:

  ThreadSanitizer:DEADLYSIGNAL
  ==1101487==ERROR: ThreadSanitizer: SEGV on unknown address 0x00000010ceaf (pc 0x7f05f669195c bp 0x000000000000 sp 0x7ffde6ad06a0 T1101487)
  ==1101487==The signal is caused by a READ memory access.
      #0 <null> <null> (libc.so.6+0x9495c)
      #1 raise <null> (libc.so.6+0x3fcc1)
      #2 raise <null> (python+0x103503)
      #3 faulthandler_raise_sigsegv /root/cpython/main/./Modules/faulthandler.c:1158:5
      ...
  SUMMARY: ThreadSanitizer: SEGV (/lib/x86_64-linux-gnu/libc.so.6+0x9495c) 
  ==1101487==ABORTING
  

3. Thread Leaks

TSAN reports that threads created during multiprocessing tests are not properly joined, leading to resource leaks.

• Example Thread Leak Warning:

  ==================
  WARNING: ThreadSanitizer: thread leak (pid=1106511)
    Thread T2 'Thread-2 (handl' (tid=1106562, finished) created by thread T1 at:
      #0 pthread_create <null> (python+0xfaa35)
      #1 do_start_joinable_thread /root/cpython/main/Python/thread_pthread.h:281:14
      ...
  SUMMARY: ThreadSanitizer: thread leak (/root/cpython/main/python+0xfaa35) in pthread_create
  ==================
  

CPython versions tested on:

3.15, CPython main branch

Operating systems tested on:

Linux

[ashm-dev]

logs.txt

[StanFromIreland]

In our CI, we only run a subset of tests (--tsan):

TSAN_TESTS = [
    'test_asyncio',
    'test_capi',
    'test_code',
    'test_ctypes',
    'test_concurrent_futures',
    'test_enum',
    'test_functools',
    'test_httpservers',
    'test_imaplib',
    'test_importlib',
    'test_io',
    'test_logging',
    'test_opcache',
    'test_queue',
    'test_signal',
    'test_socket',
    'test_sqlite3',
    'test_ssl',
    'test_syslog',
    'test_thread',
    'test_thread_local_bytecode',
    'test_threadedtempfile',
    'test_threading',
    'test_threading_local',
    'test_threadsignals',
    'test_weakref',
    'test_free_threading',
]

So I consider this a feature.

[colesbury]

1. See Various data races in subinterpreter tests under TSAN #129824
2. We run tests with TSAN_OPTIONS=handle_segv=0. Some test cases that are expected to crash launch processes with TSAN_OPTIONS=handle_segv=0 in case you forget it when running TSAN locally. If some test is missing that, we can add it.
3. Some tests that use daemon threads leak threads. That's expected. We suppress that warning in https://github.com/python/cpython/blob/main/Tools/tsan/suppressions.txt