Mutlithread XML parsing cause segfault

[mrDoctorWho0]

BPO	17970
Nosy	@amauryfa, @pitrou, @tiran, @alex, @iritkatriel
Files	pyexpat_crash_multithread.py: pyexpat test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-05-13.13:52:38.896>
labels = ['expert-XML', '3.10', '3.9', 'type-crash', '3.11']
title = 'Mutlithread XML parsing cause segfault'
updated_at = <Date 2021-09-07.12:55:23.850>
user = 'https://bugs.python.org/mrDoctorWho0'

bugs.python.org fields:

activity = <Date 2021-09-07.12:55:23.850>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['XML']
creation = <Date 2013-05-13.13:52:38.896>
creator = 'mrDoctorWho0..'
dependencies = []
files = ['30250']
hgrepos = []
issue_num = 17970
keywords = []
message_count = 8.0
messages = ['189131', '189159', '189161', '189162', '189225', '189232', '189533', '401257']
nosy_count = 6.0
nosy_names = ['amaury.forgeotdarc', 'pitrou', 'christian.heimes', 'alex', 'mrDoctorWho0..', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue17970'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[mrDoctorWho0]

Linux i386, Python 2.7.4. Multithread xml parsing via pyexpat cause segmentation fault

[amauryfa]

Expat is not thread-safe at the object level, a single Parser cannot be used from multiple threads.
Pyexpat could add locks to Parser objects.

[alex]

It could also track tids and raise an error if you attempt to use it from multiple threads.

[amauryfa]

But this would break working code which already uses locks correctly (or some kind of pool of cached parsers)

[tiran]

In my opinion it's fine to document Python's XML parser as not thread-safe and leave locking to the user. Any fancy locking or tracking is going to make it slower for users. Any it takes a lot of effort to implement the feature, too. lxml offers a faster XML parser with multi-threading support.

[amauryfa]

In my opinion it's not fine to let Python crash.
The implementation could be similar to the one in bufferedio.c, it's quite lightweight.

[pitrou]

I agree with Amaury, multi-threaded parsing should definitely not crash. Adding a lock should be quite easy. I wonder what would be the effect on performance, if there are lots of backs and forths between expat and Python.

[iritkatriel]

I've reproduced the segfault on 3.11.