[i386/i686] Fatal Python error: Segmentation fault while running the tests

[mr-c]

Crash Report

Repeated crashes while running the mypy 1.19.1 test suite on the Debian "i386" ci nodes using Python 3.14.3 (but not Python 3.13.12). This is causing us a problem promoting a newer build of the mypy package for the next stable release of Debian. orjson is not installed (and the problem persists even if it is).

Traceback

Current thread 0xf7ba3700 [python3.14] (most recent call first):                                                                                                                                                                       
  File "/usr/lib/python3/dist-packages/mypyc/test/testutil.py", line 123 in build_ir_for_single_file2
  File "/usr/lib/python3/dist-packages/mypyc/test/testutil.py", line 98 in build_ir_for_single_file
  File "/usr/lib/python3/dist-packages/mypyc/test/test_irbuild.py", line 83 in run_case
  File "/usr/lib/python3/dist-packages/mypy/test/data.py", line 323 in runtest
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 179 in pytest_runtest_call
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 121 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 512 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 245 in <lambda>
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 353 in from_call
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 244 in call_and_report
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 137 in runtestprotocol
  File "/usr/lib/python3/dist-packages/_pytest/runner.py", line 118 in pytest_runtest_protocol
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 121 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 512 in __call__
  File "/usr/lib/python3/dist-packages/xdist/remote.py", line 227 in run_one_test
  File "/usr/lib/python3/dist-packages/xdist/remote.py", line 206 in pytest_runtestloop
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 121 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 512 in __call__
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 372 in _main
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 318 in wrap_session
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 365 in pytest_cmdline_main
  File "/usr/lib/python3/dist-packages/pluggy/_callers.py", line 121 in _multicall
  File "/usr/lib/python3/dist-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3/dist-packages/pluggy/_hooks.py", line 512 in __call__
  File "/usr/lib/python3/dist-packages/xdist/remote.py", line 427 in <module>
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 1291 in executetask
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 341 in run
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 411 in _perform_spawn
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 389 in integrate_as_primary_thread
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 1273 in serve
  File "/usr/lib/python3/dist-packages/execnet/gateway_base.py", line 1806 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>

Current thread's C stack trace (most recent call first):
  Binary file "/usr/bin/python3.14", at _Py_DumpStack+0x36 [0x80adcf4]
  Binary file "/usr/bin/python3.14" [0x81262f9]
  Binary file "linux-gate.so.1", at __kernel_sigreturn+0x0 [0xf7f5f5a0]
  Binary file "/usr/bin/python3.14" [0x81a8042]
  Binary file "/usr/bin/python3.14", at PyErr_Format+0xaa [0x81a6b6a]
  Binary file "/usr/bin/python3.14", at _PyNumber_Index+0x5c [0x81aefbc]
  Binary file "/usr/bin/python3.14", at PyLong_AsInt+0x45 [0x81d5e65]
  Binary file "/usr/bin/python3.14" [0x82b6dd7]
  Binary file "/usr/bin/python3.14", at _PyObject_MakeTpCall+0x23d [0x814f19d]
  Binary file "/usr/bin/python3.14", at PyObject_CallFunction+0x145 [0x814a7d5]
  Binary file "/usr/bin/python3.14" [0x82b6299]
  Binary file "/usr/bin/python3.14", at PyObject_Vectorcall+0x33 [0x81547d3]
  Binary file "/usr/lib/python3/dist-packages/4c842c94c09923bae9e4__mypyc.cpython-314-i386-linux-gnu.so", at CPyDef_mypy___build___add_catch_all_gitignore+0x138 [0xf3c946e8]
  Binary file "/usr/lib/python3/dist-packages/4c842c94c09923bae9e4__mypyc.cpython-314-i386-linux-gnu.so", at CPyDef_mypy___build____build+0x110e [0xf3cd87ce]
  Binary file "/usr/lib/python3/dist-packages/4c842c94c09923bae9e4__mypyc.cpython-314-i386-linux-gnu.so", at CPyDef_mypy___build___build+0x316 [0xf3cda096]
  Binary file "/usr/lib/python3/dist-packages/4c842c94c09923bae9e4__mypyc.cpython-314-i386-linux-gnu.so", at CPyPy_mypy___build___build+0x191 [0xf3cdace1]
  Binary file "/usr/bin/python3.14", at PyObject_Vectorcall+0x33 [0x81547d3]
  Binary file "/usr/bin/python3.14", at _PyEval_EvalFrameDefault+0x6891 [0x8170131]
  Binary file "/usr/bin/python3.14" [0x81a2449]
  Binary file "/usr/bin/python3.14" [0x81559d2]
  Binary file "/usr/bin/python3.14" [0x82352a9]
  Binary file "/usr/bin/python3.14" [0x81b9164]
  Binary file "/usr/bin/python3.14", at _PyEval_EvalFrameDefault+0x4c52 [0x816e4f2]
  Binary file "/usr/bin/python3.14" [0x81a2449]
  Binary file "/usr/bin/python3.14" [0x81559d2]
  Binary file "/usr/bin/python3.14" [0x82352a9]
  Binary file "/usr/bin/python3.14", at _PyObject_MakeTpCall+0x313 [0x814f273]
  Binary file "/usr/bin/python3.14", at PyObject_Vectorcall+0x5b [0x81547fb]
  Binary file "/usr/bin/python3.14", at _PyEval_EvalFrameDefault+0x6891 [0x8170131]
  Binary file "/usr/bin/python3.14" [0x81f9c8e]
  Binary file "/usr/bin/python3.14" [0x81b9127]
  Binary file "/usr/bin/python3.14", at _PyEval_EvalFrameDefault+0x4c52 [0x816e4f2]
  <truncated rest of calls>

To Reproduce

# on a i386/i686 system or virtual machine or docker image or chroot
apt-get install -y mypy pybuild-plugin-autopkgtest; apt-get build-dep -y mypy
git clone https://salsa.debian.org/python-team/packages/mypy.git  # to get a copy of the tests
cd mypy
export DEB_BUILD_OPTIONS=parallel=2
pybuild-autopkgtest

Your Environment

• Mypy version used: 1.19.1
• Python version used: 3.14.3 (no problems with 3.13)
• librt version: python3-librt i386 0.8.1-1
• Operating system and version: Debian "sid" (unstable)

In another error log there is a mention of test_irbuild.py::TestGenOps::irbuild-weakref.test::testWeakrefProxy
or in this log: mypyc/test/test_irbuild.py::TestGenOps::irbuild-classes.test::testGetAttr; see also this log for 'mypyc/test/test_irbuild.py::TestGenOps::irbuild-classes.test::testMypycAttrNativeClassErrors but I'm unsure if these are the real culprits

[mr-c]

I installed the dbg symbol packages (libc6-dbg python3-mypy-dbgsym python3.14-dbg python3.13-dbg) but the tracebacks didn't improve. Here's a sample capture of another failure and the full junit file

/usr/lib/python3/dist-packages/mypyc/test/test_irbuild.py:83: in run_case
    ir = build_ir_for_single_file(testcase.input, options)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/lib/python3/dist-packages/mypyc/test/testutil.py:98: in build_ir_for_single_file
    return build_ir_for_single_file2(input_lines, compiler_options)[0].functions
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/lib/python3/dist-packages/mypyc/test/testutil.py:123: in build_ir_for_single_file2
    result = build.build(sources=[source], options=options, alt_lib_path=test_temp_dir)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mypy/build.py:222: in build
    ???
mypy/build.py:301: in _build
    ???
mypy/build.py:2952: in dispatch
    ???
mypy/build.py:3347: in process_graph
    ???
mypy/build.py:925: in wait_for_done
    ???
mypy/build.py:3516: in process_stale_scc
    ???
mypy/build.py:2508: in write_cache
    ???
mypy/build.py:1609: in write_cache
    ???
mypy/util.py:946: in json_dumps
    ???
/usr/lib/python3.14/json/__init__.py:242: in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
/usr/lib/python3.14/json/encoder.py:202: in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/json/encoder.py:263: in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
E   MemoryError
E   when serializing dict item '.class'
E   when serializing dict item 'type'
E   when serializing dict item 'node'
E   when serializing dict item 'sort'
E   when serializing dict item 'names'
E   when serializing dict item 'node'
E   when serializing dict item 'list'
E   when serializing dict item 'names

pytest-junit.xml

[mr-c]

On Debian, mypy is compiled using gcc version 15.2.0-15, not llvm. Maybe we'll get a more stable binary if we build with MYPYC_OPT_LEVEL=2 instead of the default of 3, so I'll try that next.

[mr-c]

Compiling with MYPYC_OPT_LEVEL=2 for i386 resulted in many tests failing with a MemoryError, culminating in a seg fault and pytest worker node crash.

Sample failure

2362s ___________________________ testAsyncWithMixedReturn ___________________________
2362s [gw0] linux -- Python 3.14.3 /usr/bin/python3.14
2362s data: /tmp/autopkgtest-lxc.rdgnz86s/downtmp/build.nKT/src/mypyc/test-data/run-async.test:751:
2362s /usr/lib/python3/dist-packages/mypyc/test/test_run.py:163: in run_case
2362s     self.run_case_inner(testcase)
2362s /usr/lib/python3/dist-packages/mypyc/test/test_run.py:179: in run_case_inner
2362s     self.run_case_step(testcase, step)
2362s /usr/lib/python3/dist-packages/mypyc/test/test_run.py:269: in run_case_step
2362s     ir, cfiles, _ = emitmodule.compile_modules_to_c(
2362s mypyc/codegen/emitmodule.py:458: in compile_modules_to_c
2362s     ???
2362s mypyc/codegen/emitmodule.py:298: in compile_modules_to_ir
2362s     ???
2362s mypyc/codegen/emitmodule.py:253: in compile_scc_to_ir
2362s     ???
2362s mypyc/transform/refcount.py:72: in insert_ref_count_opcodes
2362s     ???
2362s mypyc/analysis/dataflow.py:369: in analyze_must_defined_regs
2362s     ???
2362s mypyc/analysis/dataflow.py:571: in run_analysis
2362s     ???
2362s E   MemoryError

Full log https://ci.debian.net/packages/m/mypy/testing/i386/69627923/

Copy for posterity: mypy-69627923-log.gz

Next I'm going to turn off compiling mypy with mypyc on i386 for the Debian package