Add support for thread sanitizer (TSAN) via --with-thread-sanitizer

[colesbury]

Feature or enhancement

GCC and Clang provide ThreadSanitizer, a tool that detects data races. We should add support for thread sanitizer to CPython. Note that Python already supports the memory and address sanitizers.

• Add --with-thread-sanitizer as a configure option
• Define _Py_THREAD_SANITIZER if thread-sanitizer is enabled (see, e.g., _Py_ADDRESS_SANITIZER in pyport.h)
• Move the definition of _Py_NO_SANITIZE_THREAD from obmalloc.c to a place that's more widely available (like pyport.h). We're going to need this in a few places

Eventually, it would be helpful to have a continuous build for the combination of --disable-gil --with-thread-sanitizer. Note that we probably won't want to run all the tests. ThreadSanitizer is slow and also not very useful for single-threaded tests. We should collect a subset of our tests that use threading for the ThreadSanitizer continuous build.

Linked PRs

• gh-112536: Add support for thread sanitizer (TSAN) #112648
• gh-112536: enable tsan in build #113232
• gh-112536: Set up TSAN CI for free-threading #116555
• gh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-thread-sanitizer #116558
• gh-112536: Add --tsan test for reasonable TSAN execution times. #116601
• gh-112536: Add TSAN builds on Github Actions #116872
• gh-112536: Add more TSAN tests #116896
• gh-112536: Add test_threading to TSAN tests #116898
• GH-112536: Add more TSan tests #116911
• [3.12] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116924
• [3.11] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116926
• [3.12] gh-112536: Add --tsan test for reasonable TSAN execution times. (gh-116601) #116929
• [3.12] gh-112536: Add TSAN build on Github Actions (GH-116872) #116953
• gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled #117702
• [3.12] gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled (GH-117702) #117713

[samety]

Hey,
I would like to pick this up.
As a first step tried copying the struct for ASAN to TSAN
Here is a draft PR: #112648

Happy to get some initial feedback on it and then add remaining things like _Py_THREAD_SANITIZER etc.

[samety]

Added --with-thread-sanitizer as a configure option in pr 112648.

Still looking for a good way to enable TSAN for the continuous build, I will make another PR when I find one.

[pitrou]

Do we want to backport these at some point?

[corona10]

@pitrou We can go for the default build. I will submit the PR soon

[corona10]

@pitrou cc @colesbury
By the way, once the backport patches are completed, let's close this issue.
Expanding TSAN test can be tracked from the new issue.
(I prefer to cut off the issues as the checkpoint)

[pitrou]

Ok, closing as done now.