dev-lang/python-2.7.14-r2[pgo] segfaults

[zeule]

The compiled executable segfaults when called during the package build as LD_LIBRARY_PATH=/var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu ./python -E -S -m sysconfig --generate-posix-vars

(gdb) set args E -S -m sysconfig --generate-posix-vars
(gdb) run
Starting program: /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/python E -S -m sysconfig --generate-posix-vars
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff73ee8f2 in PyInstance_NewRaw () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
(gdb) bt
#0  0x00007ffff73ee8f2 in PyInstance_NewRaw () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#1  0x00007ffff73f87df in PyInstance_New () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#2  0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#3  0x00007ffff72bfd7d in PyEval_EvalFrameEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#4  0x00007ffff72d06ea in PyEval_EvalCodeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#5  0x00007ffff72d0d0f in PyEval_EvalCode () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#6  0x00007ffff731dfea in PyImport_ExecCodeModuleEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#7  0x00007ffff7327a8e in load_source_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#8  0x00007ffff72a8ea6 in load_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#9  0x00007ffff72a9dc7 in import_submodule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#10 0x00007ffff72aa858 in load_next () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#11 0x00007ffff72aae6c in PyImport_ImportModuleLevel () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#12 0x00007ffff73485ac in builtin___import__ () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#13 0x00007ffff73528d1 in PyCFunction_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#14 0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#15 0x00007ffff728ca66 in PyEval_CallObjectWithKeywords () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#16 0x00007ffff72c5013 in PyEval_EvalFrameEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#17 0x00007ffff72d06ea in PyEval_EvalCodeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#18 0x00007ffff72d0d0f in PyEval_EvalCode () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#19 0x00007ffff731dfea in PyImport_ExecCodeModuleEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#20 0x00007ffff7327a8e in load_source_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#21 0x00007ffff72a8ea6 in load_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#22 0x00007ffff72a9dc7 in import_submodule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#23 0x00007ffff72aa858 in load_next () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#24 0x00007ffff72aae6c in PyImport_ImportModuleLevel () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#25 0x00007ffff73485ac in builtin___import__ () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#26 0x00007ffff73528d1 in PyCFunction_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#27 0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#28 0x00007ffff728ca66 in PyEval_CallObjectWithKeywords () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#29 0x00007ffff72c5013 in PyEval_EvalFrameEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#30 0x00007ffff72d06ea in PyEval_EvalCodeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#31 0x00007ffff72d0d0f in PyEval_EvalCode () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#32 0x00007ffff731dfea in PyImport_ExecCodeModuleEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#33 0x00007ffff7327a8e in load_source_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#34 0x00007ffff72a8ea6 in load_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#35 0x00007ffff72a9dc7 in import_submodule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#36 0x00007ffff72aa858 in load_next () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#37 0x00007ffff72aae6c in PyImport_ImportModuleLevel () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#38 0x00007ffff73485ac in builtin___import__ () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#39 0x00007ffff73528d1 in PyCFunction_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#40 0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#41 0x00007ffff728ca66 in PyEval_CallObjectWithKeywords () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#42 0x00007ffff72c5013 in PyEval_EvalFrameEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#43 0x00007ffff72d06ea in PyEval_EvalCodeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#44 0x00007ffff72d0d0f in PyEval_EvalCode () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#45 0x00007ffff731dfea in PyImport_ExecCodeModuleEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#46 0x00007ffff7327a8e in load_source_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#47 0x00007ffff72a8ea6 in load_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#48 0x00007ffff72a9dc7 in import_submodule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#49 0x00007ffff72aa858 in load_next () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#50 0x00007ffff72aae6c in PyImport_ImportModuleLevel () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#51 0x00007ffff73485ac in builtin___import__ () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#52 0x00007ffff73528d1 in PyCFunction_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#53 0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#54 0x00007ffff728ca66 in PyEval_CallObjectWithKeywords () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#55 0x00007ffff72c5013 in PyEval_EvalFrameEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#56 0x00007ffff72d06ea in PyEval_EvalCodeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#57 0x00007ffff72d0d0f in PyEval_EvalCode () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#58 0x00007ffff731dfea in PyImport_ExecCodeModuleEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#59 0x00007ffff7327a8e in load_source_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#60 0x00007ffff72a8ea6 in load_module.lto_priv () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#61 0x00007ffff72a9dc7 in import_submodule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#62 0x00007ffff72aa858 in load_next () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#63 0x00007ffff72aae6c in PyImport_ImportModuleLevel () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#64 0x00007ffff73485ac in builtin___import__ () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#65 0x00007ffff73528d1 in PyCFunction_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#66 0x00007ffff742d5bf in PyObject_Call () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#67 0x00007ffff742dc07 in call_function_tail () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#68 0x00007ffff742e51c in PyObject_CallFunction () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#69 0x00007ffff72abd0d in PyImport_Import () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#70 0x00007ffff72ac12f in PyImport_ImportModule () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#71 0x00007ffff72eeb9f in initsite () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
---Type <return> to continue, or q <return> to quit---
#72 0x00007ffff72f3d9d in Py_InitializeEx () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#73 0x00007ffff72f48e2 in Py_Initialize () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#74 0x00007ffff7270989 in Py_Main () from /var/portage/tmp/portage/dev-lang/python-2.7.14-r2/work/x86_64-pc-linux-gnu/libpython2.7.so.1.0
#75 0x0000555555555409 in main ()
(gdb)

[zeule]

build.log

[InBetweenNames]

Interesting -- did Python 2.7 build OK for you before the parallel PGO patch? Or was this your first attempt at building it?

Below is a link to the original Python 2.7 PGO ebuild in this repo (it did PGO serially instead of in parallel):

https://github.com/InBetweenNames/gentooLTO/blob/a7dd670412caded72e4ae807ab12d3a570cc0beb/dev-lang/python/python-2.7.14-r2.ebuild

If you haven't already, could you try building with that one and letting me know if it segfaults?

[zeule]

The first attempt to compile it with PGO I made on Thursday, which is before the parallel PGO change, I believe, and it failed in the very same way. Python 3.6.5-r2 with PGO builds fine,

[InBetweenNames]

Very interesting. Does the problem with 2.7 manifest even when PGO is disabled?

[zeule]

dev-lang/python-2.7.14-r2::lto-overlay[-pgo] installs without the problem.

[InBetweenNames]

What about PGO using -O2 optimization instead of -O3?

[zeule]

Compilation with -O2 yields the same error.

[zwim]

Hello, the same problem here. python-3.x.x compile all fine, but python-2.7.14-r2 fails.

Maybe it is the same bug as described in https://bugs.python.org/issue33374

It seems to be triggered with python gcc 8.

[InBetweenNames]

Nice find! According to the linked thread, it may be fixed in 2.7.15. Anyone willing to test? I'd do it myself, but I can't get it to happen on my system.

[zeule]

Gentoo bug: https://bugs.gentoo.org/654822

[InBetweenNames]

I just updated the ebuild to 2.7.15. Is it still a problem?

[zwim]

python 2.7.15 compiles with pgo.

Thank you

[InBetweenNames]

Great! Going to close this for now, if the problem comes up again just comment and I'll reopen.