Memory error in operator.mod

[baltsers]

Bug report

We did a fuzzing test on Python3.9.15, a memory error happened.

from operator import *
import operator


def demoFunc(arg1,arg2):
    try:
        ret = operator.mod(arg1, arg2)
    except (AssertionError, AttributeError, ImportError, LookupError, OSError, TypeError, ValueError) as e:
        pass

a= "155%7000000000000ret00000015082063?0000000000000000p1155900000005082063303862299307%+4"
b= "!@#$%^&*9523"
with open('/dev/null', 'r'):
    demoFunc(a,b)

details

Bug info

    #1 0x499964 in PyMem_RawMalloc /root/CpyFuzz/experiments/Python-3.9.15/Objects/obmalloc.c:572:12
    #2 0x499964 in _PyObject_Malloc /root/CpyFuzz/experiments/Python-3.9.15/Objects/obmalloc.c:1645:11
    #3 0x499964 in pymalloc_realloc /root/CpyFuzz/experiments/Python-3.9.15/Objects/obmalloc.c:1988:10
    #4 0x499964 in _PyObject_Realloc /root/CpyFuzz/experiments/Python-3.9.15/Objects/obmalloc.c:2007:9
    #5 0x51b57b in resize_compact /root/CpyFuzz/experiments/Python-3.9.15/Objects/unicodeobject.c:1084:31
    #6 0x4f3746 in _PyUnicodeWriter_PrepareInternal /root/CpyFuzz/experiments/Python-3.9.15/Objects/unicodeobject.c:14011:25
    #7 0x520432 in unicode_format_arg_output /root/CpyFuzz/experiments/Python-3.9.15/Objects/unicodeobject.c:15261:9
    #8 0x520432 in unicode_format_arg /root/CpyFuzz/experiments/Python-3.9.15/Objects/unicodeobject.c:15365:15
    #9 0x520432 in PyUnicode_Format /root/CpyFuzz/experiments/Python-3.9.15/Objects/unicodeobject.c:15438:17
    #10 0x6be614 in binary_op1 /root/CpyFuzz/experiments/Python-3.9.15/Objects/abstract.c:869:13
    #11 0x6bec07 in binary_op /root/CpyFuzz/experiments/Python-3.9.15/Objects/abstract.c:898:24
    #12 0x6bec07 in PyNumber_Remainder /root/CpyFuzz/experiments/Python-3.9.15/Objects/abstract.c:1086:12
    #13 0x6fabbe in cfunction_vectorcall_FASTCALL /root/CpyFuzz/experiments/Python-3.9.15/Objects/methodobject.c:430:24
    #14 0x5635fd in _PyObject_VectorcallTstate /root/CpyFuzz/experiments/Python-3.9.15/./Include/cpython/abstract.h:118:11
    #15 0x5635fd in PyObject_Vectorcall /root/CpyFuzz/experiments/Python-3.9.15/./Include/cpython/abstract.h:127:12
    #16 0x5635fd in call_function /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:5077:13
    #17 0x55d8bf in _PyEval_EvalFrameDefault /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:3489:23
    #18 0x43336f in _PyEval_EvalFrame /root/CpyFuzz/experiments/Python-3.9.15/./Include/internal/pycore_ceval.h:40:12
    #19 0x43336f in function_code_fastcall /root/CpyFuzz/experiments/Python-3.9.15/Objects/call.c:330:24
    #20 0x5635fd in _PyObject_VectorcallTstate /root/CpyFuzz/experiments/Python-3.9.15/./Include/cpython/abstract.h:118:11
    #21 0x5635fd in PyObject_Vectorcall /root/CpyFuzz/experiments/Python-3.9.15/./Include/cpython/abstract.h:127:12
    #22 0x5635fd in call_function /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:5077:13
    #23 0x55d96e in _PyEval_EvalFrameDefault /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:3520:19
    #24 0x565264 in _PyEval_EvalFrame /root/CpyFuzz/experiments/Python-3.9.15/./Include/internal/pycore_ceval.h:40:12
    #25 0x565264 in _PyEval_EvalCode /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:4329:14
    #26 0x554061 in _PyEval_EvalCodeWithName /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:4361:12
    #27 0x554061 in PyEval_EvalCodeEx /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:4377:12
    #28 0x554061 in PyEval_EvalCode /root/CpyFuzz/experiments/Python-3.9.15/Python/ceval.c:828:12
    #29 0x5d2367 in run_eval_code_obj /root/CpyFuzz/experiments/Python-3.9.15/Python/pythonrun.c:1221:9
    #30 0x5d2367 in run_mod /root/CpyFuzz/experiments/Python-3.9.15/Python/pythonrun.c:1242:19
    #31 0x5d2367 in pyrun_file /root/CpyFuzz/experiments/Python-3.9.15/Python/pythonrun.c:1140:15
    #32 0x5cedc6 in pyrun_simple_file /root/CpyFuzz/experiments/Python-3.9.15/Python/pythonrun.c:450:13
    #33 0x5cedc6 in PyRun_SimpleFileExFlags /root/CpyFuzz/experiments/Python-3.9.15/Python/pythonrun.c:483:15
    #34 0x41e1fd in pymain_run_file /root/CpyFuzz/experiments/Python-3.9.15/Modules/main.c:373:15
    #35 0x41e1fd in pymain_run_python /root/CpyFuzz/experiments/Python-3.9.15/Modules/main.c:598:21
    #36 0x41e1fd in Py_RunMain /root/CpyFuzz/experiments/Python-3.9.15/Modules/main.c:677:5
    #37 0x41ea66 in pymain_main /root/CpyFuzz/experiments/Python-3.9.15/Modules/main.c:707:12
    #38 0x41ead2 in Py_BytesMain /root/CpyFuzz/experiments/Python-3.9.15/Modules/main.c:731:12
    #39 0x7f608e374c86 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21c86)

My environment

CPython: 3.9.15
Ubuntu: 18.0.4 (x86 64)

[ronaldoussoren]

This is the same issue as #103687, but using strings instead of byte strings. operator.mod(a, b) is the same as a % b and a contains '%7000000000000r' which will result in trying to allocate a very large amount of memory.

[ronaldoussoren]

@baltsers : All issues you are filing are basically the same: your script tries to allocate a very large amount of memory and fails because of that, either with a MemoryError if the process is constrained or by being killed by the OOM killer when it is not constrained.

This (OOM killer) is not something the Python interpreter can guard against, memory allocations functions (e.g. malloc(3)) will succeed if there's free address space and the proces will crash later on when trying to use that address space when there's not enough available RAM and swap space.