Memory leak in `map_next` in strict mode in case of error

[YuanchengJiang]

Bug report

Bug description:

def pack(*args):
    return args
tuple(map(pack, (1, 999999999999, 3), '\u2603', strict=True))

=================================================================
==2066743==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 32 byte(s) in 1 object(s) allocated from:
    #0 0x76cd190b39c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x5f18b4d0c06b in long_alloc ../Objects/longobject.c:180
    #2 0x5f18b4d0c06b in PyLong_FromLong ../Objects/longobject.c:403
    #3 0x5f18b4b3c73d in parsenumber_raw ../Parser/pegen.c:640
    #4 0x5f18b4b401f9 in parsenumber ../Parser/pegen.c:668
    #5 0x5f18b4b401f9 in _PyPegen_number_token ../Parser/pegen.c:707
    #6 0x5f18b4b5d749 in atom_rule ../Parser/parser.c:15160
    #7 0x5f18b4b62f2f in primary_raw ../Parser/parser.c:14779
    #8 0x5f18b4b62f2f in primary_rule ../Parser/parser.c:14577
    #9 0x5f18b4b63e89 in await_primary_rule ../Parser/parser.c:14531
    #10 0x5f18b4b64de7 in power_rule ../Parser/parser.c:14407
    #11 0x5f18b4b64de7 in factor_rule ../Parser/parser.c:14357
    #12 0x5f18b4b6655f in term_raw ../Parser/parser.c:14198
    #13 0x5f18b4b6655f in term_rule ../Parser/parser.c:13941
    #14 0x5f18b4b68044 in sum_raw ../Parser/parser.c:13894
    #15 0x5f18b4b68044 in sum_rule ../Parser/parser.c:13773
    #16 0x5f18b4b69307 in shift_expr_raw ../Parser/parser.c:13733
    #17 0x5f18b4b69307 in shift_expr_rule ../Parser/parser.c:13593
    #18 0x5f18b4b6a964 in bitwise_and_raw ../Parser/parser.c:13553
    #19 0x5f18b4b6a964 in bitwise_and_rule ../Parser/parser.c:13471
    #20 0x5f18b4b6b594 in bitwise_xor_raw ../Parser/parser.c:13431
    #21 0x5f18b4b6b594 in bitwise_xor_rule ../Parser/parser.c:13349
    #22 0x5f18b4b6c1c4 in bitwise_or_raw ../Parser/parser.c:13309
    #23 0x5f18b4b6c1c4 in bitwise_or_rule ../Parser/parser.c:13227
    #24 0x5f18b4b6ce90 in comparison_rule ../Parser/parser.c:12467
    #25 0x5f18b4b6ce90 in inversion_rule ../Parser/parser.c:12418
    #26 0x5f18b4b70125 in conjunction_rule ../Parser/parser.c:12295
    #27 0x5f18b4b71125 in disjunction_rule ../Parser/parser.c:12207
    #28 0x5f18b4b84084 in expression_rule ../Parser/parser.c:11495
    #29 0x5f18b4b8e017 in named_expression_rule ../Parser/parser.c:12153
    #30 0x5f18b4b8f83b in star_named_expression_rule ../Parser/parser.c:12003
    #31 0x5f18b4b8fc6c in _gather_58_rule ../Parser/parser.c:31182
    #32 0x5f18b4b8fc6c in star_named_expressions_rule ../Parser/parser.c:11913
    #33 0x5f18b4b90adc in _tmp_82_rule ../Parser/parser.c:32764
    #34 0x5f18b4b90adc in tuple_rule ../Parser/parser.c:17366
    #35 0x5f18b4b5dbda in _tmp_66_rule ../Parser/parser.c:31644
    #36 0x5f18b4b5dbda in atom_rule ../Parser/parser.c:15181
    #37 0x5f18b4b62f2f in primary_raw ../Parser/parser.c:14779
    #38 0x5f18b4b62f2f in primary_rule ../Parser/parser.c:14577
    #39 0x5f18b4b63e89 in await_primary_rule ../Parser/parser.c:14531
    #40 0x5f18b4b64de7 in power_rule ../Parser/parser.c:14407
    #41 0x5f18b4b64de7 in factor_rule ../Parser/parser.c:14357
    #42 0x5f18b4b6655f in term_raw ../Parser/parser.c:14198
    #43 0x5f18b4b6655f in term_rule ../Parser/parser.c:13941
    #44 0x5f18b4b68044 in sum_raw ../Parser/parser.c:13894
    #45 0x5f18b4b68044 in sum_rule ../Parser/parser.c:13773
    #46 0x5f18b4b69307 in shift_expr_raw ../Parser/parser.c:13733
    #47 0x5f18b4b69307 in shift_expr_rule ../Parser/parser.c:13593
    #48 0x5f18b4b6a964 in bitwise_and_raw ../Parser/parser.c:13553
    #49 0x5f18b4b6a964 in bitwise_and_rule ../Parser/parser.c:13471

SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-140517: Fix leak in map_next in strict mode #140543
• [3.14] gh-140517: fix leak in map_next in strict mode (GH-140543) #140554
• Add missing NEWS entry for gh-140517 #140560
• [3.14] Add missing NEWS entry for gh-140517 (GH-140560) #140565

[Abhi210]

I was unable to reproduce this bug on the CPython main branch (Ubuntu 25.04). Instead, I got a ValueError:

def pack(*args):
	return args
 tuple(map(pack, (1, 999999999999, 3), '\u2603', strict=True))
 
Traceback (most recent call last):
  File "<python-input-7>", line 3, in <module>
    tuple(map(pack, (1, 999999999999, 3), '\u2603', strict=True))
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: map() argument 2 is shorter than argument 1

Are we missing something here?

[yihong0618]

I was unable to reproduce this bug on the CPython main branch (Ubuntu 25.04). Instead, I got a ValueError:

def pack(*args):
	return args
 tuple(map(pack, (1, 999999999999, 3), '\u2603', strict=True))
 
Traceback (most recent call last):
  File "<python-input-7>", line 3, in <module>
    tuple(map(pack, (1, 999999999999, 3), '\u2603', strict=True))
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: map() argument 2 is shorter than argument 1

Are we missing something here?

you need to compile with with-address-sanitizer

./configure -with-address-sanitizer

[StanFromIreland]

I got a ValueError

The ValueError is to be expected in this repro.

[efimov-mikhail]

I've created another variant of this leak:

pack = lambda x, y: None
t1 = (None, object())
t2 = (None,)
m = map(pack, t1, t2, strict=True)

next(m)
next(m)

IMO, it's slightly simple to understand.

It seems to be that problem is in map_next function from Python/bltinmodule.c.
I'll create a PR.

[kumaraditya303]

This looks like a real leak, looking at the code under check label:

cpython/Python/bltinmodule.c

Lines 1527 to 1562 in fc2e23c

	check:
	if (PyErr_Occurred()) {
	if (!PyErr_ExceptionMatches(PyExc_StopIteration)) {
	// next() on argument i raised an exception (not StopIteration)
	return NULL;
	}
	PyErr_Clear();
	}
	if (i) {
	// ValueError: map() argument 2 is shorter than argument 1
	// ValueError: map() argument 3 is shorter than arguments 1-2
	const char* plural = i == 1 ? " " : "s 1-";
	return PyErr_Format(PyExc_ValueError,
	"map() argument %d is shorter than argument%s%d",
	i + 1, plural, i);
	}
	for (i = 1; i < niters; i++) {
	PyObject *it = PyTuple_GET_ITEM(lz->iters, i);
	PyObject *val = (*Py_TYPE(it)->tp_iternext)(it);
	if (val) {
	Py_DECREF(val);
	const char* plural = i == 1 ? " " : "s 1-";
	return PyErr_Format(PyExc_ValueError,
	"map() argument %d is longer than argument%s%d",
	i + 1, plural, i);
	}
	if (PyErr_Occurred()) {
	if (!PyErr_ExceptionMatches(PyExc_StopIteration)) {
	// next() on argument i raised an exception (not StopIteration)
	return NULL;
	}
	PyErr_Clear();
	}
	// Argument i is exhausted. So far so good...
	}
	// All arguments are exhausted. Success!

It is incorrect to return NULL directly here on line 1531 and similar, it should instead jump to exit label.

[efimov-mikhail]

I've linked a PR.
It seems it was two different leaks: previous values on stack object and stack itself.