list_repr not safe against concurrent mutation

[pitrou]

BPO	30774
Nosy	@rhettinger, @pitrou, @serhiy-storchaka, @iritkatriel
Files	siglist.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-06-26.19:56:58.992>
labels = ['3.10', '3.9', 'type-crash', '3.11']
title = 'list_repr not safe against concurrent mutation'
updated_at = <Date 2021-10-18.10:01:47.948>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2021-10-18.10:01:47.948>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2017-06-26.19:56:58.992>
creator = 'pitrou'
dependencies = []
files = ['46975']
hgrepos = []
issue_num = 30774
keywords = []
message_count = 4.0
messages = ['296936', '296938', '321410', '404148']
nosy_count = 4.0
nosy_names = ['rhettinger', 'pitrou', 'serhiy.storchaka', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue30774'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[pitrou]

list_repr() is careful to fetch the list size on each iteration. However, it seems that the list size may be mutated just after by concurrent code, perhaps in _PyUnicodeWriter_WriteASCIIString() or Py_EnterRecursiveCall().

I witnessed this with a signal handler that shrank a list while printing a list from a loop in the main code path.

gdb excerpt:

Program received signal SIGSEGV, Segmentation fault.
0x00000000004a17d6 in PyObject_Repr (v=<unknown at remote 0x7ffff6044670>) at Objects/object.c:485
485 res = (*v->ob_type->tp_repr)(v);

(gdb) bt
#0 0x00000000004a17d6 in PyObject_Repr (v=<unknown at remote 0x7ffff6044670>) at Objects/object.c:485
#1 0x000000000047eef3 in list_repr (v=0x7ffff68e1d08) at Objects/listobject.c:373
[...]

(gdb) frame 1
#1 0x000000000047eef3 in list_repr (v=0x7ffff68e1d08) at Objects/listobject.c:373
373 s = PyObject_Repr(v->ob_item[i]);
(gdb) p v
$2 = (PyListObject *) 0x7ffff68e1d08
(gdb) p i
$3 = 19
(gdb) p v->ob_item[18]
$12 = <float at remote 0x7ffff6044628>
(gdb) p v->ob_item[19]
$13 = <unknown at remote 0x7ffff6044670>
(gdb) p ((PyVarObject*)(v))->ob_size
$14 = 19

[pitrou]

Attaching reproducer script. I don't know if that is worth fixing.

[serhiy-storchaka]

Is it possible to reproduce this issue without involving signal handlers?

[iritkatriel]

Reproduced on 3.11.