Integer overflow & Int values loaded into Bool detected via Libfuzzer & UndefinedBehaviorSanitizer

[swirsz]

BPO	46294
Nosy	@mdickinson, @swirsz
Files	crash-a0d.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-01-07.16:00:03.730>
created_at = <Date 2022-01-07.15:06:13.080>
labels = ['invalid', 'type-crash', '3.11']
title = 'Integer overflow & Int values loaded into Bool detected via Libfuzzer & UndefinedBehaviorSanitizer'
updated_at = <Date 2022-01-09.11:37:39.806>
user = 'https://github.com/swirsz'

bugs.python.org fields:

activity = <Date 2022-01-09.11:37:39.806>
actor = 'mark.dickinson'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-07.16:00:03.730>
closer = 'swirsz'
components = []
creation = <Date 2022-01-07.15:06:13.080>
creator = 'swirsz'
dependencies = []
files = ['50547']
hgrepos = []
issue_num = 46294
keywords = []
message_count = 4.0
messages = ['409973', '409976', '409978', '410140']
nosy_count = 2.0
nosy_names = ['mark.dickinson', 'swirsz']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue46294'
versions = ['Python 3.11']

[swirsz]

Compiling source from github on January 6, 2022, detected via Libfuzzer & UndefinedBehaviorSanitizer:

# ./fuzz_struct_unpack crash-a0d.txt
Running: crash-a0d.txt

/src/cpython3/Modules/_struct.c:509:28: runtime error: load of value 128, which is not a valid value for type '_Bool'

Python/pyhash.c:396:9: runtime error: unsigned integer overflow: 17111126337582519137 + 1455368869671451682 cannot be represented in type 'unsigned long'

Python/pyhash.c:414:5: runtime error: unsigned integer overflow: 6843264283216330929 + 16329705011411640967 cannot be represented in type 'unsigned long'

Python/pyhash.c:417:5: runtime error: unsigned integer overflow: 13747253807228978341 + 10396395245414858527 cannot be represented in type 'unsigned long'

Python/pyhash.c:418:5: runtime error: unsigned integer overflow: 17173606624272818715 + 4069551840979798976 cannot be represented in type 'unsigned long'

Python/pyhash.c:419:5: runtime error: unsigned integer overflow: 12388162105911730119 + 9634611433502982398 cannot be represented in type 'unsigned long'

Objects/longobject.c:288:22: runtime error: unsigned integer overflow: 0 - 18446744073709550595 cannot be represented in type 'unsigned long'

Objects/longobject.c:4872:31: runtime error: unsigned integer overflow: 18446744073709551615 + 1 cannot be represented in type 'unsigned long'

Objects/longobject.c:3124:33: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'

Objects/longobject.c:3130:33: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'

Objects/tupleobject.c:426:21: runtime error: unsigned integer overflow: 219911203979059663 * 14029467366897019727 cannot be represented in type 'unsigned long'

Objects/tupleobject.c:428:13: runtime error: unsigned integer overflow: 14367201699383568926 * 11400714785074694791 cannot be represented in type 'unsigned long'

Objects/tupleobject.c:426:13: runtime error: unsigned integer overflow: 18351143362227076666 + 1497884194698650478 cannot be represented in type 'unsigned long'

Objects/tupleobject.c:432:9: runtime error: unsigned integer overflow: 18406138070188819878 + 2870177450013471924 cannot be represented in type 'unsigned long'

Python/traceback.c:247:86: runtime error: unsigned integer overflow: 18446744073709551615 * 2 cannot be represented in type 'unsigned long'

Objects/frameobject.c:51:72: runtime error: unsigned integer overflow: 18446744073709551615 * 2 cannot be represented in type 'unsigned long'

[mdickinson]

@swirsz: Thanks for the report.

Most of these look like false positives: we're intentionally making use of C's unsigned arithmetic behaviour. Note that these are technically *not* overflows. As the C standard itself says, in C99 §6.2.5, paragraph 9:

A computation involving unsigned operands can never overflow,
because a result that cannot be represented by the resulting
unsigned integer type is reduced modulo the number that is one
greater than the largest value that can be represented by the
resulting type.

.. and we're deliberately depending on exactly that well-defined reduction behaviour.

Would you be able to do a first pass over the results and identify those that might be genuine issues, worthy of further investigation?

[swirsz]

Closing this report. I investigated the remaining issue and it looks like a perfectly valid call to PyBool_FromLong:

/src/cpython3/Modules/_struct.c:509:28: runtime error: load of value 128, which is not a valid value for type '_Bool'

static PyObject *
nu_bool(_structmodulestate *state, const char *p, const formatdef *f)
{
    _Bool x;
    memcpy((char *)&x, p, sizeof x);
    return PyBool_FromLong(x != 0);
}

[mdickinson]

Thanks for checking, Steven. Your report also helped me to notice a minor portability bug (at Objects/longobject.c:288, where the wrong type is used in a cast); a fix is in #74681.