Access violations in C extension modules on Python 3.9.3

[cgohlke]

BPO	43710
Nosy	@terryjreedy, @gpshead, @pfmoore, @tjguk, @ambv, @markshannon, @zware, @zooba
PRs	[3.9] bpo-43710: Change layout of pystate struct to preserve intra-version ABI compatibiliy. #25160 [3.9] bpo-43710: Rollback the 3.9 bpo-42500 fix, it broke the ABI in 3.9.3 #25179

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-04.11:03:35.837>
created_at = <Date 2021-04-03.03:09:40.678>
labels = ['extension-modules', 'release-blocker', 'OS-windows', 'type-crash', '3.9']
title = 'Access violations in C extension modules on Python 3.9.3'
updated_at = <Date 2021-05-03.21:42:14.320>
user = 'https://bugs.python.org/cgohlke'

bugs.python.org fields:

activity = <Date 2021-05-03.21:42:14.320>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-04.11:03:35.837>
closer = 'lukasz.langa'
components = ['Extension Modules', 'Windows']
creation = <Date 2021-04-03.03:09:40.678>
creator = 'cgohlke'
dependencies = []
files = []
hgrepos = []
issue_num = 43710
keywords = ['patch']
message_count = 5.0
messages = ['390116', '390134', '390172', '390181', '390212']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'gregory.p.smith', 'paul.moore', 'tim.golden', 'cgohlke', 'lukasz.langa', 'Mark.Shannon', 'zach.ware', 'steve.dower']
pr_nums = ['25160', '25179']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue43710'
versions = ['Python 3.9']

[cgohlke]

First reported at numpy/numpy#18720

After upgrading to Python 3.9.3, 32-bit on Windows, importing numpy (installed via pip install numpy) crashes:

Microsoft Windows [Version 10.0.19041.906]
C:\Python39-32>python -X dev
Python 3.9.3 (tags/v3.9.3:e723086, Apr  2 2021, 11:01:03) [MSC v.1928 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Windows fatal exception: access violation

Current thread 0x0000105c (most recent call first):
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1116 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1116 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1116 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "C:\Python39-32\lib\site-packages\numpy\random\_pickle.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 790 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "C:\Python39-32\lib\site-packages\numpy\random\__init__.py", line 179 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 790 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1058 in _handle_fromlist
  File "C:\Python39-32\lib\site-packages\numpy\__init__.py", line 156 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 790 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<stdin>", line 1 in <module>

Rebuilding numpy from source against Python 3.9.3 fixes this crash, but many other packages compiled against older versions of Python 3.9.x still crash.

I think this might be due to <8b795ab#diff-7285576bdacf86fe37274d3d0d399c29b4be2959005f60ac0729615d8fca9186%5C%3E.

In bpo-42500: Fix recursion in or after except (GH-23568) (bpo-24501), the public PyThreadState struct changed. This breaks ABI compatibility for code that is directly accessing the PyThreadState struct members, which is not uncommon. E.g. in Cython https://github.com/cython/cython/blob/0.29.x/Cython/Utility/Exceptions.c#L93-L115. This code will fail on 32-bit. On 64-bit it passes because of fortunate struct alignment AFAICT.

[ambv]

Tomorrow evening CEST (around 30 hours from now) I intend to release a hotfix 3.9.4 with Mark's fix.

[gpshead]

I really suggest a simple rollback (revert) of the change rather than a redo-fix for 3.9.4. Unless you like the idea of a possible 3.9.5 if the redo-fix itself has issues that haven't been given due thought in the rush to undo the 3.9.3 mistake.

This leaves more time for a proper fix of bpo-42500 in 3.9 not crafted under duress.

[ambv]

New changeset c7b0fec by Gregory P. Smith in branch '3.9':
[3.9] bpo-43710: Rollback the 3.9 bpo-42500 fix, it broke the ABI in 3.9.3 (bpo-25179)
c7b0fec

[cgohlke]

Thank you for fixing this issue so fast! Python 3.9.4 works well.