segfault while trying to print error message

[sudipm-mukherjee]

Crash report

What happened?

The segfault is happens when https://gitlab.com/thomasross/mirage/-/blob/master/mirage/__init__.py?ref_type=heads#L56 is executed, But that is only when mirage is built with LTO enabled.
I am pasting the full trace from the coredump.
It seems to me that when LTO is enabled, then PyCMethod_New() fails to load the module because of bad call flags. But then unicode_fromformat_write_cstr() gets a bad pointer for its str argument which is causing the segfault.
PyCMethod_New() can fail due to bad module or any other reason but Python should not crash with a segfault.

The trace from the coredump:

#0  __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:76
#1  0x000000000055d34d in unicode_fromformat_write_cstr (writer=0x7ffefbd1bea0, str=0x1 <error: Cannot access memory at address 0x1>, width=-1, precision=<optimized out>)
    at ../Objects/unicodeobject.c:2769
#2  0x000000000052097c in unicode_fromformat_arg (vargs=0x7ffefbd1be80, f=0x6ca323 "s() method: bad call flags", writer=0x7ffefbd1bea0) at ../Objects/unicodeobject.c:2983
#3  PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at ../Objects/unicodeobject.c:3100
#4  0x000000000055c91b in _PyErr_FormatV (vargs=0x7ffefbd1bf20, format=0x6ca322 "%s() method: bad call flags", exception=0x94f640 <_PyExc_SystemError.lto_priv.0>, 
    tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/errors.c:1106
#5  PyErr_Format (exception=0x94f640 <_PyExc_SystemError.lto_priv.0>, format=0x6ca322 "%s() method: bad call flags") at ../Python/errors.c:1149
#6  0x000000000042a833 in PyCMethod_New (cls=0x0, module=<optimized out>, self=<optimized out>, ml=0x7f424152c040 <xmouse_module>) at ../Objects/methodobject.c:73
#7  _add_methods_to_object (module=module@entry=0x7f423f78c8b0, name=name@entry=0x7f423f77bc70, functions=functions@entry=0x7f424152c020 <methods>) at ../Objects/moduleobject.c:171
#8  0x00000000006068a1 in PyModule_AddFunctions (m=m@entry=0x7f423f78c8b0, functions=0x7f424152c020 <methods>) at ../Objects/moduleobject.c:457
#9  0x0000000000606640 in _PyModule_CreateInitialized (module=0x7f424152c040 <xmouse_module>, module_api_version=<optimized out>) at ../Objects/moduleobject.c:244
#10 0x0000000000634d4f in _PyImport_LoadDynamicModuleWithSpec (fp=0x0, spec=0x7f423f77bc50) at ../Python/importdl.c:169
#11 _imp_create_dynamic_impl (module=<optimized out>, file=<optimized out>, spec=0x7f423f77bc50) at ../Python/import.c:2397
#12 _imp_create_dynamic (module=<optimized out>, args=<optimized out>, nargs=<optimized out>) at ../Python/clinic/import.c.h:446
#13 0x000000000054bdbc in cfunction_vectorcall_FASTCALL (func=0x7f42429a6890, args=0x7f424051d6d8, nargsf=<optimized out>, kwnames=<optimized out>)
    at ../Include/cpython/methodobject.h:52
#14 0x0000000000532929 in do_call_core (use_tracing=<optimized out>, kwdict=0x7f4241b5e040, callargs=0x7f424051d6c0, func=0x7f42429a6890, tstate=<optimized out>)
    at ../Python/ceval.c:7324
#15 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:5376
#16 0x0000000000558a35 in _PyEval_EvalFrame (throwflag=0, frame=0x7f4242dbf4c0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Include/internal/pycore_ceval.h:73
#17 _PyEval_Vector (kwnames=<optimized out>, argcount=2, args=0x7ffefbd1c340, locals=0x0, func=0x7f424296bce0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/ceval.c:6434
#18 _PyFunction_Vectorcall (func=0x7f424296bce0, stack=0x7ffefbd1c340, nargsf=<optimized out>, kwnames=<optimized out>) at ../Objects/call.c:393
#19 0x0000000000543b60 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7ffefbd1c340, callable=0x7f424296bce0, tstate=0xa7ba38 <_PyRuntime+166328>)
    at ../Include/internal/pycore_call.h:92
#20 object_vacall (tstate=0xa7ba38 <_PyRuntime+166328>, base=<optimized out>, callable=0x7f424296bce0, vargs=0x7ffefbd1c3d0) at ../Objects/call.c:819
#21 0x000000000057db3f in PyObject_CallMethodObjArgs (obj=0x0, name=<optimized out>) at ../Objects/call.c:878
#22 0x000000000057d192 in import_find_and_load (abs_name=0x7f424189b7b0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/import.c:1748
#23 PyImport_ImportModuleLevelObject (name=0x7f424189b7b0, globals=<optimized out>, locals=<optimized out>, fromlist=0x950cc0 <_Py_NoneStruct>, level=0) at ../Python/import.c:1847
#24 0x000000000052f52c in import_name (level=0xa533c8 <_PyRuntime+840>, fromlist=0x950cc0 <_Py_NoneStruct>, name=0x7f424189b7b0, frame=<optimized out>, tstate=<optimized out>)
    at ../Python/ceval.c:7424
#25 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3946
#26 0x0000000000608f5f in _PyEval_EvalFrame (throwflag=0, frame=0x7f4242dbf450, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Include/internal/pycore_ceval.h:73
#27 _PyEval_Vector (args=0x0, argcount=0, kwnames=0x0, locals=<optimized out>, func=0x7f4241b60fe0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/ceval.c:6434
#28 PyEval_EvalCode (co=0x27531b0, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:1148
#29 0x000000000061cf14 in builtin_exec_impl (module=<optimized out>, closure=<optimized out>, locals=0x7f424289c700, globals=0x7f424289c700, source=0x27531b0)
    at ../Python/bltinmodule.c:1077
#30 builtin_exec (module=<optimized out>, args=<optimized out>, nargs=2, kwnames=<optimized out>) at ../Python/clinic/bltinmodule.c.h:465
#31 0x0000000000538efb in cfunction_vectorcall_FASTCALL_KEYWORDS (func=0x7f42429a4fe0, args=0x7f4241b3b418, nargsf=<optimized out>, kwnames=<optimized out>)
    at ../Include/cpython/methodobject.h:52
#32 0x0000000000532929 in do_call_core (use_tracing=<optimized out>, kwdict=0x7f4241b0f780, callargs=0x7f4241b3b400, func=0x7f42429a4fe0, tstate=<optimized out>)
    at ../Python/ceval.c:7324
#33 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:5376
--Type <RET> for more, q to quit, c to continue without paging--c
#34 0x0000000000558a35 in _PyEval_EvalFrame (throwflag=0, frame=0x7f4242dbf158, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Include/internal/pycore_ceval.h:73
#35 _PyEval_Vector (kwnames=<optimized out>, argcount=2, args=0x7ffefbd1c9b0, locals=0x0, func=0x7f424296bce0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/ceval.c:6434
#36 _PyFunction_Vectorcall (func=0x7f424296bce0, stack=0x7ffefbd1c9b0, nargsf=<optimized out>, kwnames=<optimized out>) at ../Objects/call.c:393
#37 0x0000000000543b60 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7ffefbd1c9b0, callable=0x7f424296bce0, tstate=0xa7ba38 <_PyRuntime+166328>)
    at ../Include/internal/pycore_call.h:92
#38 object_vacall (tstate=0xa7ba38 <_PyRuntime+166328>, base=<optimized out>, callable=0x7f424296bce0, vargs=0x7ffefbd1ca40) at ../Objects/call.c:819
#39 0x000000000057db3f in PyObject_CallMethodObjArgs (obj=0x0, name=<optimized out>) at ../Objects/call.c:878
#40 0x000000000057d192 in import_find_and_load (abs_name=0x7f42427d1430, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/import.c:1748
#41 PyImport_ImportModuleLevelObject (name=0x7f42427d1430, globals=<optimized out>, locals=<optimized out>, fromlist=0x950cc0 <_Py_NoneStruct>, level=0) at ../Python/import.c:1847
#42 0x0000000000535ffd in import_name (level=0xa533c8 <_PyRuntime+840>, fromlist=0x950cc0 <_Py_NoneStruct>, name=0x7f42427d1430, frame=<optimized out>, tstate=<optimized out>)
    at ../Python/ceval.c:7424
#43 _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3946
#44 0x0000000000608f5f in _PyEval_EvalFrame (throwflag=0, frame=0x7f4242dbf020, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Include/internal/pycore_ceval.h:73
#45 _PyEval_Vector (args=0x0, argcount=0, kwnames=0x0, locals=<optimized out>, func=0x7f42427cc7c0, tstate=0xa7ba38 <_PyRuntime+166328>) at ../Python/ceval.c:6434
#46 PyEval_EvalCode (co=0x7f42429dc030, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:1148
#47 0x00000000006230bb in run_eval_code_obj (tstate=0xa7ba38 <_PyRuntime+166328>, co=0x7f42429dc030, globals=0x7f4242a0ebc0, locals=0x7f4242a0ebc0) at ../Python/pythonrun.c:1710
#48 0x000000000061f7a3 in run_mod (mod=<optimized out>, filename=<optimized out>, globals=0x7f4242a0ebc0, locals=0x7f4242a0ebc0, flags=<optimized out>, arena=<optimized out>)
    at ../Python/pythonrun.c:1731
#49 0x0000000000631bc6 in pyrun_file (fp=fp@entry=0x2518450, filename=filename@entry=0x7f4242a0ee70, start=start@entry=257, globals=globals@entry=0x7f4242a0ebc0, 
    locals=locals@entry=0x7f4242a0ebc0, closeit=closeit@entry=1, flags=0x7ffefbd1cf18) at ../Python/pythonrun.c:1626
#50 0x0000000000631931 in _PyRun_SimpleFileObject (fp=fp@entry=0x2518450, filename=filename@entry=0x7f4242a0ee70, closeit=closeit@entry=1, flags=flags@entry=0x7ffefbd1cf18)
    at ../Python/pythonrun.c:440
#51 0x0000000000631748 in _PyRun_AnyFileObject (fp=0x2518450, filename=filename@entry=0x7f4242a0ee70, closeit=closeit@entry=1, flags=flags@entry=0x7ffefbd1cf18)
    at ../Python/pythonrun.c:79
#52 0x000000000063002b in pymain_run_file_obj (skip_source_first_line=0, filename=0x7f4242a0ee70, program_name=0x7f42429bc530) at ../Modules/main.c:360
#53 pymain_run_file (config=0xa61a80 <_PyRuntime+59904>) at ../Modules/main.c:379
#54 pymain_run_python (exitcode=0x7ffefbd1cf14) at ../Modules/main.c:601
#55 Py_RunMain () at ../Modules/main.c:680
#56 0x00000000005fc4fb in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:734
#57 0x00007f4242aa30d0 in __libc_start_call_main (main=main@entry=0x5fc460 <main>, argc=argc@entry=2, argv=argv@entry=0x7ffefbd1d148) at ../sysdeps/nptl/libc_start_call_main.h:58
#58 0x00007f4242aa3189 in __libc_start_main_impl (main=0x5fc460 <main>, argc=2, argv=0x7ffefbd1d148, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7ffefbd1d138) at ../csu/libc-start.c:360
#59 0x00000000005fc395 in _start ()

The issue can be reproduced, and I will be happy to provide extra logs or test if needed.

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.11.7 (main, Dec 8 2023, 14:22:46) [GCC 13.2.0]

[Eclips4]

Hello!
Could you provide a MRE, please?

[ronaldoussoren]

https://gitlab.com/thomasross/mirage/-/blob/master/mirage/xmouse.c does not have a sentinel value at the end of the array of methods. This cause PyModule_Create to access memory beyond the end of the array.

[sudipm-mukherjee]

https://gitlab.com/thomasross/mirage/-/blob/master/mirage/xmouse.c does not have a sentinel value at the end of the array of methods. This cause PyModule_Create to access memory beyond the end of the array.

Thanks a lot @ronaldoussoren. That was it. I was preparing a simple reproducer of the segfault and I can no longer reproduce the issue after adding the sentinel value .