oil.ovm segfaults if built with gcc 8.1.1 #107

DennisMitchell · May 17, 2018

Building Oil with gcc 8.1.1 appears to succeed, but running the resulting oil.ovm executable immediately crashes with a segmentation fault. gdb only prints the following.

(gdb) run
Starting program: /opt/osh/build/_bin/oil.ovm 

Program received signal SIGSEGV, Segmentation fault.
0x0000000000421a36 in ?? ()

oil.ovm-debug is not affected. oil.ovm works just fine if built with either gcc 7.3.1 or clang. At least versions 0.3, 0.4, and 0.5 are affected.

Building with gcc 8.1.1 produces the following output.

./configure: Wrote _build/detected-config.sh and _build/detected-config.h
build/compile.sh build-opt _build/oil/ovm-opt _build/oil/module_init.c _build/oil/main_name.c _build/oil/c-module-srcs.txt
/opt/osh/build/Python-2.7.13 /opt/osh/build
In file included from Include/Python.h:78,
                 from Objects/listobject.c:3:
Objects/listobject.c: In function ‘list_resize’:
Include/pymem.h:110:34: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
                                  ^
Include/pymem.h:77:21: note: in definition of macro ‘PyMem_REALLOC’
     : realloc((p), (n) ? (n) : 1))
                     ^
Objects/listobject.c:62:9: note: in expansion of macro ‘PyMem_RESIZE’
         PyMem_RESIZE(items, PyObject *, new_allocated);
         ^~~~~~~~~~~~
Objects/listobject.c: In function ‘list_ass_subscript’:
Objects/listobject.c:2647:41: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
                 PyMem_MALLOC(slicelength*sizeof(PyObject*));
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Objects/listobject.c:2728:41: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
                 PyMem_MALLOC(slicelength*sizeof(PyObject*));
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
In file included from Include/Python.h:78,
                 from Objects/dictobject.c:10:
Objects/dictobject.c: In function ‘dictresize’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Objects/dictobject.c:643:20: note: in expansion of macro ‘PyMem_NEW’
         newtable = PyMem_NEW(PyDictEntry, newsize);
                    ^~~~~~~~~
In file included from Include/Python.h:78,
                 from Objects/setobject.c:7:
Objects/setobject.c: In function ‘set_table_resize’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Objects/setobject.c:316:20: note: in expansion of macro ‘PyMem_NEW’
         newtable = PyMem_NEW(setentry, newsize);
                    ^~~~~~~~~
In file included from Include/Python.h:78,
                 from Objects/structseq.c:4:
Objects/structseq.c: In function ‘PyStructSequence_InitType’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Objects/structseq.c:504:15: note: in expansion of macro ‘PyMem_NEW’
     members = PyMem_NEW(PyMemberDef, n_members-n_unnamed_members+1);
               ^~~~~~~~~
Objects/structseq.c: In function ‘structseq_repr’:
Objects/structseq.c:257:5: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=]
     strncpy(pbuf, typ->tp_name, len);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/structseq.c:255:11: note: length computed here
     len = strlen(typ->tp_name) > TYPE_MAXSIZE ? TYPE_MAXSIZE :
           ^~~~~~~~~~~~~~~~~~~~
In file included from Include/Python.h:78,
                 from Objects/typeobject.c:3:
Objects/typeobject.c: In function ‘pmerge’:
Objects/typeobject.c:1514:44: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
     remain = (int *)PyMem_MALLOC(SIZEOF_INT*to_merge_size);
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
In file included from Include/Python.h:78,
                 from Python/bltinmodule.c:3:
Python/bltinmodule.c: In function ‘builtin_map’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Python/bltinmodule.c:980:17: note: in expansion of macro ‘PyMem_NEW’
     if ((seqs = PyMem_NEW(sequence, n)) == NULL) {
                 ^~~~~~~~~
In file included from Include/Python.h:78,
                 from Python/getargs.c:4:
Python/getargs.c: In function ‘convertsimple’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Python/getargs.c:1133:27: note: in expansion of macro ‘PyMem_NEW’
                 *buffer = PyMem_NEW(char, size + 1);
                           ^~~~~~~~~
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Python/getargs.c:1181:23: note: in expansion of macro ‘PyMem_NEW’
             *buffer = PyMem_NEW(char, size + 1);
                       ^~~~~~~~~
In file included from Include/Python.h:78,
                 from Python/import.c:4:
Python/import.c: In function ‘_PyImport_Init’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Python/import.c:165:15: note: in expansion of macro ‘PyMem_NEW’
     filetab = PyMem_NEW(struct filedescr, countD + countS + 1);
               ^~~~~~~~~
Python/import.c: In function ‘PyImport_ExtendInittab’:
Include/pymem.h:110:34: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
                                  ^
Include/pymem.h:77:21: note: in definition of macro ‘PyMem_REALLOC’
     : realloc((p), (n) ? (n) : 1))
                     ^
Python/import.c:3490:5: note: in expansion of macro ‘PyMem_RESIZE’
     PyMem_RESIZE(p, struct _inittab, i+n+1);
     ^~~~~~~~~~~~
In file included from Include/Python.h:78,
                 from Python/marshal.c:9:
Python/marshal.c: In function ‘r_object’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Python/marshal.c:859:18: note: in expansion of macro ‘PyMem_NEW’
         buffer = PyMem_NEW(char, n);
                  ^~~~~~~~~
Python/marshal.c: In function ‘PyMarshal_WriteLongToFile’:
Python/marshal.c:70:35: warning: ‘wf.ptr’ may be used uninitialized in this function [-Wmaybe-uninitialized]
                       else if ((p)->ptr != (p)->end) *(p)->ptr++ = (c); \
                                   ^~
Python/marshal.c:70:47: warning: ‘wf.end’ may be used uninitialized in this function [-Wmaybe-uninitialized]
                       else if ((p)->ptr != (p)->end) *(p)->ptr++ = (c); \
                                               ^~
Python/marshal.c:77:10: warning: ‘wf.str’ may be used uninitialized in this function [-Wmaybe-uninitialized]
     if (p->str == NULL)
         ~^~~~~
Python/pystrtod.c: In function ‘format_float_short’:
Python/pystrtod.c:1007:13: warning: ‘strncpy’ output truncated before terminating nul copying 3 bytes from a string of the same length [-Wstringop-truncation]
             strncpy(p, "ERR", 3);
             ^~~~~~~~~~~~~~~~~~~~
In file included from Include/Python.h:78,
                 from Modules/posixmodule.c:28:
Modules/posixmodule.c: In function ‘posix_execv’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/posixmodule.c:3185:16: note: in expansion of macro ‘PyMem_NEW’
     argvlist = PyMem_NEW(char *, argc+1);
                ^~~~~~~~~
Modules/posixmodule.c: In function ‘posix_execve’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/posixmodule.c:3261:16: note: in expansion of macro ‘PyMem_NEW’
     argvlist = PyMem_NEW(char *, argc+1);
                ^~~~~~~~~
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/posixmodule.c:3282:15: note: in expansion of macro ‘PyMem_NEW’
     envlist = PyMem_NEW(char *, i + 1);
               ^~~~~~~~~
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/posixmodule.c:3324:13: note: in expansion of macro ‘PyMem_NEW’
         p = PyMem_NEW(char, len);
             ^~~~~~~~~
Modules/posixmodule.c: In function ‘posix_major’:
Modules/posixmodule.c:7077:13: warning: In the GNU C Library, "major" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "major", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "major", you should undefine it after including <sys/types.h>.
     return PyInt_FromLong((long)major(device));
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                                                                                                                     
Modules/posixmodule.c: In function ‘posix_minor’:
Modules/posixmodule.c:7090:13: warning: In the GNU C Library, "minor" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "minor", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "minor", you should undefine it after including <sys/types.h>.
     return PyInt_FromLong((long)minor(device));
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                                                                                                                     
Modules/posixmodule.c: In function ‘posix_makedev’:
Modules/posixmodule.c:7103:13: warning: In the GNU C Library, "makedev" is defined
 by <sys/sysmacros.h>. For historical compatibility, it is
 currently defined by <sys/types.h> as well, but we plan to
 remove this soon. To use "makedev", include <sys/sysmacros.h>
 directly. If you did not intend to use a system-defined macro
 "makedev", you should undefine it after including <sys/types.h>.
     return _PyInt_FromDev(makedev(major, minor));
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                                                                                                                                                                                                                                                                                         
In function ‘read_directory’,
    inlined from ‘zipimporter_init’ at Modules/zipimport.c:133:21:
Modules/zipimport.c:845:9: warning: ‘strncpy’ output may be truncated copying between 0 and 4095 bytes from a string of length 4100 [-Wstringop-truncation]
         strncpy(path + length + 1, name, MAXPATHLEN - length - 1);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from Include/Python.h:78,
                 from Modules/_localemodule.c:12:
Modules/_localemodule.c: In function ‘PyLocale_strcoll’:
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/_localemodule.c:318:11: note: in expansion of macro ‘PyMem_NEW’
     ws1 = PyMem_NEW(wchar_t, len1);
           ^~~~~~~~~
Include/pymem.h:97:30: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
  ( (type *) PyMem_MALLOC((n) * sizeof(type)) ) )
                              ^
Include/pymem.h:75:15: note: in definition of macro ‘PyMem_MALLOC’
     : malloc((n) ? (n) : 1))
               ^
Modules/_localemodule.c:327:11: note: in expansion of macro ‘PyMem_NEW’
     ws2 = PyMem_NEW(wchar_t, len2);
           ^~~~~~~~~
Modules/readline.c: In function ‘flex_complete’:
Modules/readline.c:898:31: warning: passing argument 1 of ‘completion_matches’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
     return completion_matches(text, *on_completion);
                               ^~~~
Modules/readline.c:39:15: note: expected ‘char *’ but argument is of type ‘const char *’
 extern char **completion_matches(char *, rl_compentry_func_t *);
               ^~~~~~~~~~~~~~~~~~
Modules/readline.c: In function ‘call_readline’:
Modules/readline.c:1182:9: warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
         strncpy(p, q, n);
         ^~~~~~~~~~~~~~~~
Modules/readline.c:1160:9: note: length computed here
     n = strlen(p);
         ^~~~~~~~~

real    0m49.615s
user    0m45.869s
sys     0m3.215s
/opt/osh/build
objcopy --only-keep-debug _build/oil/ovm-opt _build/oil/ovm-opt.symbols
#strip -o _build/oil/ovm-opt.stripped --strip-debug _build/oil/ovm-opt _build/oil/ovm-opt.symbols
strip -o _build/oil/ovm-opt.stripped _build/oil/ovm-opt  # What's the difference with debug symbols?
# We need a relative path since it will be _bin/oil.ovm
objcopy --add-gnu-debuglink=_build/oil/ovm-opt.symbols _build/oil/ovm-opt.stripped
cat _build/oil/ovm-opt.stripped _build/oil/bytecode-opy.zip > _bin/oil.ovm
chmod +x _bin/oil.ovm
Installing to /opt/osh/bin/oil.ovm

Thanks for the report. Out of curiosity is gcc 8 the default on the distro you're using?

I think the way to debug this is to run the debug version of the binary, which is

_bin/oil.ovm-dbg

This might work better in the dev tree, not sure about the release tarball.

andychu · May 18, 2018

Thanks for the report. Out of curiosity is gcc 8 the default on the distro you're using?

I think the way to debug this is to run the debug version of the binary, which is

_bin/oil.ovm-dbg

This might work better in the dev tree, not sure about the release tarball.

Yes, gcc 8 is the default on Fedora 28.

I'm not sure what to do with oil.ovm-dbg, as it doesn't segfault. Only oil.ovm is affected.

DennisMitchell · May 18, 2018

Yes, gcc 8 is the default on Fedora 28.

I'm not sure what to do with oil.ovm-dbg, as it doesn't segfault. Only oil.ovm is affected.

OK thanks for the report. I might have to get a Fedora VM to repro this.

The way I bundled part of the Python interpreter is a big hack, and unfortunately it's starting to fray, judging from several related open issues. Though I'm not sure what's going on here as I would expect Python 2.7 to have the same issue with gcc 8.

andychu · May 18, 2018

OK thanks for the report. I might have to get a Fedora VM to repro this.

The way I bundled part of the Python interpreter is a big hack, and unfortunately it's starting to fray, judging from several related open issues. Though I'm not sure what's going on here as I would expect Python 2.7 to have the same issue with gcc 8.

I managed to make oil.ovm-dbg segfault by compiling it with -O3.

Reading symbols from _bin/oil.ovm-dbg...done.
(gdb) run
Starting program: /opt/osh/build/_bin/oil.ovm-dbg 

Program received signal SIGSEGV, Segmentation fault.
0x0000000000420ec5 in PyInstance_NewRaw (klass=klass@entry=0x7ffff7ed3a10, 
    dict=0x7ffff7e73398, dict@entry=0x0) at Objects/classobject.c:544
544         inst->in_dict = dict;
(gdb) list
539             return NULL;
540         }
541         inst->in_weakreflist = NULL;
542         Py_INCREF(klass);
543         inst->in_class = (PyClassObject *)klass;
544         inst->in_dict = dict;
545         _PyObject_GC_TRACK(inst);
546         return (PyObject *)inst;
547     }
548
(gdb) where
#0  0x0000000000420ec5 in PyInstance_NewRaw (klass=klass@entry=0x7ffff7ed3a10, 
    dict=0x7ffff7e73398, dict@entry=0x0) at Objects/classobject.c:544
#1  0x0000000000420fcb in PyInstance_New (klass=0x7ffff7ed3a10, arg=0x7ffff7e69d20, kw=0x0)
    at Objects/classobject.c:561
#2  0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7ed3a10, 
    arg=arg@entry=0x7ffff7e69d20, kw=kw@entry=0x0) at Objects/abstract.c:2547
#3  0x00000000004a9c65 in do_call (nk=<optimized out>, na=<optimized out>, 
    pp_stack=0x7fffffffc978, func=0x7ffff7ed3a10) at Python/ceval.c:4569
#4  call_function (oparg=<optimized out>, pp_stack=0x7fffffffc978) at Python/ceval.c:4374
#5  PyEval_EvalFrameEx (f=f@entry=0x7ffff7f27400, throwflag=throwflag@entry=0)
    at Python/ceval.c:2989
#6  0x00000000004b2163 in PyEval_EvalCodeEx (co=co@entry=0x7ffff7f39030, 
    globals=globals@entry=0x7ffff7e71d70, locals=locals@entry=0x7ffff7e71d70, 
    args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0, defs=0x0, 
    defcount=0, closure=0x0) at Python/ceval.c:3584
#7  0x00000000004b2429 in PyEval_EvalCode (co=co@entry=0x7ffff7f39030, 
    globals=globals@entry=0x7ffff7e71d70, locals=locals@entry=0x7ffff7e71d70)
    at Python/ceval.c:669
#8  0x00000000004bc2ae in PyImport_ExecCodeModuleEx (name=0x7ffff7f63984 "__future__", 
    co=co@entry=0x7ffff7f39030, pathname=<optimized out>) at Python/import.c:735
#9  0x00000000004ef040 in zipimporter_load_module (obj=<optimized out>, 
    args=<optimized out>) at Modules/zipimport.c:364
#10 0x00000000004064b5 in PyObject_Call (kw=0x0, arg=0x7ffff7f651d0, func=0x7ffff7f74fc8)
    at Objects/abstract.c:2547
#11 call_function_tail (callable=0x7ffff7f74fc8, args=0x7ffff7f651d0)
    at Objects/abstract.c:2579
#12 0x000000000040bc3d in PyObject_CallMethod (o=<optimized out>, 
    name=name@entry=0x5254fe "load_module", format=format@entry=0x50ab76 "s")
    at Objects/abstract.c:2654
#13 0x00000000004bd084 in load_module (name=name@entry=0x843934 "__future__", 
    fp=<optimized out>, pathname=pathname@entry=0x844940 "/opt/osh/build/_bin/oil.ovm-dbg", 
    type=<optimized out>, loader=<optimized out>) at Python/import.c:2001
#14 0x00000000004bd3cf in import_submodule (mod=mod@entry=0x778b70 <_Py_NoneStruct>, 
    subname=subname@entry=0x843934 "__future__", 
    fullname=fullname@entry=0x843934 "__future__") at Python/import.c:2743
#15 0x00000000004bd6d6 in load_next (mod=mod@entry=0x7ffff7f1fbe8, 
    altmod=0x778b70 <_Py_NoneStruct>, p_name=p_name@entry=0x7fffffffcd28, 
    buf=buf@entry=0x843930 "bin.__future__", p_buflen=p_buflen@entry=0x7fffffffcd38)
    at Python/import.c:2561
#16 0x00000000004bdc09 in import_module_level (name=<optimized out>, 
    globals=<optimized out>, fromlist=0x7ffff7f65690, level=-1, locals=<optimized out>)
    at Python/import.c:2265
#17 0x00000000004be55b in PyImport_ImportModuleLevel (name=<optimized out>, 
    globals=<optimized out>, locals=<optimized out>, fromlist=<optimized out>, 
    level=<optimized out>) at Python/import.c:2330
#18 0x00000000004a6234 in builtin___import__ (self=<optimized out>, args=<optimized out>, 
    kwds=<optimized out>) at Python/bltinmodule.c:49
#19 0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7fc9fc8,
    arg=arg@entry=0x7ffff7f6ee68, kw=kw@entry=0x0) at Objects/abstract.c:2547
#20 0x00000000004abcd9 in PyEval_CallObjectWithKeywords (kw=0x0, arg=0x7ffff7f6ee68,
    func=0x7ffff7fc9fc8) at Python/ceval.c:4221
#21 PyEval_EvalFrameEx (f=f@entry=0x837a40, throwflag=throwflag@entry=0)
---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:2624
#22 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>,
    globals=globals@entry=0x7ffff7f64d70, locals=locals@entry=0x7ffff7f64d70,
    args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:3584
#23 0x00000000004aead6 in PyEval_EvalCode (locals=0x7ffff7f64d70, globals=0x7ffff7f64d70,
    co=<optimized out>) at Python/ceval.c:5050
#24 exec_statement (locals=0x7ffff7f64d70, globals=0x7ffff7f64d70, prog=<optimized out>,
    f=0x7dd170) at Python/ceval.c:5050
#25 PyEval_EvalFrameEx (f=f@entry=0x7dd170, throwflag=throwflag@entry=0)
    at Python/ceval.c:2106
#26 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>,
    locals=locals@entry=0x0, args=<optimized out>, argcount=argcount@entry=7,
    kws=kws@entry=0x80fac0, kwcount=0, defs=0x7ffff7f24c68, defcount=5, closure=0x0)
    at Python/ceval.c:3584
#27 0x00000000004af23b in fast_function (nk=<optimized out>, na=7, n=<optimized out>,
    pp_stack=0x7fffffffd1c8, func=0x7ffff7e740c8) at Python/ceval.c:4447
#28 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd1c8) at Python/ceval.c:4372
#29 PyEval_EvalFrameEx (f=f@entry=0x80f860, throwflag=throwflag@entry=0)
    at Python/ceval.c:2989
#30 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>,
    locals=locals@entry=0x0, args=args@entry=0x7ffff7ee2188, argcount=<optimized out>,
    kws=kws@entry=0x0, kwcount=0, defs=0x7ffff7f306a8, defcount=1, closure=0x0)
    at Python/ceval.c:3584
#31 0x0000000000439a3b in function_call (func=0x7ffff7e745f0, arg=0x7ffff7ee2170, kw=0x0)
    at Objects/funcobject.c:523
#32 0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7e745f0,
    arg=arg@entry=0x7ffff7ee2170, kw=kw@entry=0x0) at Objects/abstract.c:2547
#33 0x00000000004da34c in RunModule (set_argv0=0, module=<optimized out>)
    at Modules/main.c:194
#34 RunMainFromImporter (
    filename=filename@entry=0x7fffffffe742 "/opt/osh/build/_bin/oil.ovm-dbg")
    at Modules/main.c:233
#35 0x00000000004da549 in Ovm_Main (argc=1, argv=0x7fffffffe4d8) at Modules/main.c:356
#36 0x00007ffff745a1bb in __libc_start_main (main=0x405060 <main>, argc=1,
    argv=0x7fffffffe4d8, init=<optimized out>, fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=0x7fffffffe4c8) at ../csu/libc-start.c:308
#37 0x000000000040509a in _start ()

Please let me know if there's anything I can do to help.

DennisMitchell · May 18, 2018

I managed to make oil.ovm-dbg segfault by compiling it with -O3.

Reading symbols from _bin/oil.ovm-dbg...done.
(gdb) run
Starting program: /opt/osh/build/_bin/oil.ovm-dbg 

Program received signal SIGSEGV, Segmentation fault.
0x0000000000420ec5 in PyInstance_NewRaw (klass=klass@entry=0x7ffff7ed3a10, 
    dict=0x7ffff7e73398, dict@entry=0x0) at Objects/classobject.c:544
544         inst->in_dict = dict;
(gdb) list
539             return NULL;
540         }
541         inst->in_weakreflist = NULL;
542         Py_INCREF(klass);
543         inst->in_class = (PyClassObject *)klass;
544         inst->in_dict = dict;
545         _PyObject_GC_TRACK(inst);
546         return (PyObject *)inst;
547     }
548
(gdb) where
#0  0x0000000000420ec5 in PyInstance_NewRaw (klass=klass@entry=0x7ffff7ed3a10, 
    dict=0x7ffff7e73398, dict@entry=0x0) at Objects/classobject.c:544
#1  0x0000000000420fcb in PyInstance_New (klass=0x7ffff7ed3a10, arg=0x7ffff7e69d20, kw=0x0)
    at Objects/classobject.c:561
#2  0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7ed3a10, 
    arg=arg@entry=0x7ffff7e69d20, kw=kw@entry=0x0) at Objects/abstract.c:2547
#3  0x00000000004a9c65 in do_call (nk=<optimized out>, na=<optimized out>, 
    pp_stack=0x7fffffffc978, func=0x7ffff7ed3a10) at Python/ceval.c:4569
#4  call_function (oparg=<optimized out>, pp_stack=0x7fffffffc978) at Python/ceval.c:4374
#5  PyEval_EvalFrameEx (f=f@entry=0x7ffff7f27400, throwflag=throwflag@entry=0)
    at Python/ceval.c:2989
#6  0x00000000004b2163 in PyEval_EvalCodeEx (co=co@entry=0x7ffff7f39030, 
    globals=globals@entry=0x7ffff7e71d70, locals=locals@entry=0x7ffff7e71d70, 
    args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0, defs=0x0, 
    defcount=0, closure=0x0) at Python/ceval.c:3584
#7  0x00000000004b2429 in PyEval_EvalCode (co=co@entry=0x7ffff7f39030, 
    globals=globals@entry=0x7ffff7e71d70, locals=locals@entry=0x7ffff7e71d70)
    at Python/ceval.c:669
#8  0x00000000004bc2ae in PyImport_ExecCodeModuleEx (name=0x7ffff7f63984 "__future__", 
    co=co@entry=0x7ffff7f39030, pathname=<optimized out>) at Python/import.c:735
#9  0x00000000004ef040 in zipimporter_load_module (obj=<optimized out>, 
    args=<optimized out>) at Modules/zipimport.c:364
#10 0x00000000004064b5 in PyObject_Call (kw=0x0, arg=0x7ffff7f651d0, func=0x7ffff7f74fc8)
    at Objects/abstract.c:2547
#11 call_function_tail (callable=0x7ffff7f74fc8, args=0x7ffff7f651d0)
    at Objects/abstract.c:2579
#12 0x000000000040bc3d in PyObject_CallMethod (o=<optimized out>, 
    name=name@entry=0x5254fe "load_module", format=format@entry=0x50ab76 "s")
    at Objects/abstract.c:2654
#13 0x00000000004bd084 in load_module (name=name@entry=0x843934 "__future__", 
    fp=<optimized out>, pathname=pathname@entry=0x844940 "/opt/osh/build/_bin/oil.ovm-dbg", 
    type=<optimized out>, loader=<optimized out>) at Python/import.c:2001
#14 0x00000000004bd3cf in import_submodule (mod=mod@entry=0x778b70 <_Py_NoneStruct>, 
    subname=subname@entry=0x843934 "__future__", 
    fullname=fullname@entry=0x843934 "__future__") at Python/import.c:2743
#15 0x00000000004bd6d6 in load_next (mod=mod@entry=0x7ffff7f1fbe8, 
    altmod=0x778b70 <_Py_NoneStruct>, p_name=p_name@entry=0x7fffffffcd28, 
    buf=buf@entry=0x843930 "bin.__future__", p_buflen=p_buflen@entry=0x7fffffffcd38)
    at Python/import.c:2561
#16 0x00000000004bdc09 in import_module_level (name=<optimized out>, 
    globals=<optimized out>, fromlist=0x7ffff7f65690, level=-1, locals=<optimized out>)
    at Python/import.c:2265
#17 0x00000000004be55b in PyImport_ImportModuleLevel (name=<optimized out>, 
    globals=<optimized out>, locals=<optimized out>, fromlist=<optimized out>, 
    level=<optimized out>) at Python/import.c:2330
#18 0x00000000004a6234 in builtin___import__ (self=<optimized out>, args=<optimized out>, 
    kwds=<optimized out>) at Python/bltinmodule.c:49
#19 0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7fc9fc8,
    arg=arg@entry=0x7ffff7f6ee68, kw=kw@entry=0x0) at Objects/abstract.c:2547
#20 0x00000000004abcd9 in PyEval_CallObjectWithKeywords (kw=0x0, arg=0x7ffff7f6ee68,
    func=0x7ffff7fc9fc8) at Python/ceval.c:4221
#21 PyEval_EvalFrameEx (f=f@entry=0x837a40, throwflag=throwflag@entry=0)
---Type <return> to continue, or q <return> to quit---
    at Python/ceval.c:2624
#22 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>,
    globals=globals@entry=0x7ffff7f64d70, locals=locals@entry=0x7ffff7f64d70,
    args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0, defs=0x0,
    defcount=0, closure=0x0) at Python/ceval.c:3584
#23 0x00000000004aead6 in PyEval_EvalCode (locals=0x7ffff7f64d70, globals=0x7ffff7f64d70,
    co=<optimized out>) at Python/ceval.c:5050
#24 exec_statement (locals=0x7ffff7f64d70, globals=0x7ffff7f64d70, prog=<optimized out>,
    f=0x7dd170) at Python/ceval.c:5050
#25 PyEval_EvalFrameEx (f=f@entry=0x7dd170, throwflag=throwflag@entry=0)
    at Python/ceval.c:2106
#26 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>,
    locals=locals@entry=0x0, args=<optimized out>, argcount=argcount@entry=7,
    kws=kws@entry=0x80fac0, kwcount=0, defs=0x7ffff7f24c68, defcount=5, closure=0x0)
    at Python/ceval.c:3584
#27 0x00000000004af23b in fast_function (nk=<optimized out>, na=7, n=<optimized out>,
    pp_stack=0x7fffffffd1c8, func=0x7ffff7e740c8) at Python/ceval.c:4447
#28 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd1c8) at Python/ceval.c:4372
#29 PyEval_EvalFrameEx (f=f@entry=0x80f860, throwflag=throwflag@entry=0)
    at Python/ceval.c:2989
#30 0x00000000004b2163 in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>,
    locals=locals@entry=0x0, args=args@entry=0x7ffff7ee2188, argcount=<optimized out>,
    kws=kws@entry=0x0, kwcount=0, defs=0x7ffff7f306a8, defcount=1, closure=0x0)
    at Python/ceval.c:3584
#31 0x0000000000439a3b in function_call (func=0x7ffff7e745f0, arg=0x7ffff7ee2170, kw=0x0)
    at Objects/funcobject.c:523
#32 0x000000000040b83a in PyObject_Call (func=func@entry=0x7ffff7e745f0,
    arg=arg@entry=0x7ffff7ee2170, kw=kw@entry=0x0) at Objects/abstract.c:2547
#33 0x00000000004da34c in RunModule (set_argv0=0, module=<optimized out>)
    at Modules/main.c:194
#34 RunMainFromImporter (
    filename=filename@entry=0x7fffffffe742 "/opt/osh/build/_bin/oil.ovm-dbg")
    at Modules/main.c:233
#35 0x00000000004da549 in Ovm_Main (argc=1, argv=0x7fffffffe4d8) at Modules/main.c:356
#36 0x00007ffff745a1bb in __libc_start_main (main=0x405060 <main>, argc=1,
    argv=0x7fffffffe4d8, init=<optimized out>, fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=0x7fffffffe4c8) at ../csu/libc-start.c:308
#37 0x000000000040509a in _start ()

Please let me know if there's anything I can do to help.

Though I'm not sure what's going on here as I would expect Python 2.7 to have the same issue with gcc 8.

I tried building Python 2.7.13 (and Python 2.7.14) from source, and I am having the same issue. Python 2.7.15 seems unaffected.

DennisMitchell · May 18, 2018

Though I'm not sure what's going on here as I would expect Python 2.7 to have the same issue with gcc 8.

I tried building Python 2.7.13 (and Python 2.7.14) from source, and I am having the same issue. Python 2.7.15 seems unaffected.

OK awesome! That's what I was just thinking in the shower -- maybe this bug was fixed in Python 2.7.14 or 2.7.15. I did almost nothing to CPython besides strip out the parser/bytecode compiler -- there is very little new code.

Hopefully it is a matter of doing a diff -r of the source tree or going through the commit history. I'm traveling now with a slow laptop so I might not get to it for awhile.

Honestly I have been surprised by how much non-portable stuff and #ifdef logic there is in CPython (mostly for speed as far as I can tell). The shell should really be written more toward POSIX and ANSI C which would in theory avoid things like this. (Though I'm saying that without really knowing what the issue is ...)

andychu · May 18, 2018

OK awesome! That's what I was just thinking in the shower -- maybe this bug was fixed in Python 2.7.14 or 2.7.15. I did almost nothing to CPython besides strip out the parser/bytecode compiler -- there is very little new code.

Hopefully it is a matter of doing a diff -r of the source tree or going through the commit history. I'm traveling now with a slow laptop so I might not get to it for awhile.

Honestly I have been surprised by how much non-portable stuff and #ifdef logic there is in CPython (mostly for speed as far as I can tell). The shell should really be written more toward POSIX and ANSI C which would in theory avoid things like this. (Though I'm saying that without really knowing what the issue is ...)

Also, I suppose another way to fix it is just to copy Python 2.7.15 wholesale and reapply the diffs.

I think the history in the Python-2.7.13/ dir is fairly clean but I'm not sure how well it would merge.

andychu · May 18, 2018

Also, I suppose another way to fix it is just to copy Python 2.7.15 wholesale and reapply the diffs.

I think the history in the Python-2.7.13/ dir is fairly clean but I'm not sure how well it would merge.

note: Eli S ran into this too. CentOS, version?

andychu · May 20, 2018

note: Eli S ran into this too. CentOS, version?

Eli here, I ran into the same issue on my Arch installation.
I was running GCC version 8.1.0. After downgrading to GCC 7.3.1, I ran into no issues.
Linux kernel 4.16.8-1.

sxldier · May 20, 2018

Eli here, I ran into the same issue on my Arch installation.
I was running GCC version 8.1.0. After downgrading to GCC 7.3.1, I ran into no issues.
Linux kernel 4.16.8-1.

Thanks, FWIW I will be working on this since it seems pretty common.

andychu · May 20, 2018

Thanks, FWIW I will be working on this since it seems pretty common.

OK I reproduced the issue on Arch Linux, and found links on gcc and Python mailing lists:

https://www.mail-archive.com/gcc-bugs@gcc.gnu.org/msg570352.html

https://mail.python.org/pipermail/python-dev/2018-January/152000.html

Now I have to look at what the patch actually is.

NOTE: Python 2.7.13 and .14 fail -- .15 is fixed. So looking at those release notes.

andychu · Jun 10, 2018

OK I reproduced the issue on Arch Linux, and found links on gcc and Python mailing lists:

https://www.mail-archive.com/gcc-bugs@gcc.gnu.org/msg570352.html

https://mail.python.org/pipermail/python-dev/2018-January/152000.html

Now I have to look at what the patch actually is.

NOTE: Python 2.7.13 and .14 fail -- .15 is fixed. So looking at those release notes.

Gah, no release notes!

python/pythondotorg#1252

Relevant commit from 2012:

python/cpython@e348c8d#diff-fb41bdaf12f733cf6ab8a82677d03adc

This change was for Python 3. Python 2 still had the bug for a long time, and then it looks like they did something slightly different to preserve ABI compatibility.

We don't care about ABI compatibility so we can maybe just do what Python 3 does.

andychu · Jun 10, 2018

Gah, no release notes!

python/pythondotorg#1252

Relevant commit from 2012:

python/cpython@e348c8d#diff-fb41bdaf12f733cf6ab8a82677d03adc

This change was for Python 3. Python 2 still had the bug for a long time, and then it looks like they did something slightly different to preserve ABI compatibility.

We don't care about ABI compatibility so we can maybe just do what Python 3 does.

OK I tested this under Arch on virtualbox, and it no longer segfaults! Unblocked for 0.5.

andychu · Jun 11, 2018

OK I tested this under Arch on virtualbox, and it no longer segfaults! Unblocked for 0.5.

andychu added help wanted high-priority labels Jun 4, 2018

andychu added release-note-0.5 and removed help wanted high-priority labels Jun 11, 2018

andychu closed this Jul 24, 2018

oilshell/oil

oil.ovm segfaults if built with gcc 8.1.1 #107

DennisMitchell commented May 17, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

andychu commented May 20, 2018

This comment has been minimized.

sxldier commented May 20, 2018 •

edited

This comment has been minimized.

andychu commented May 20, 2018

andychu added help wanted high-priority labels Jun 4, 2018

This comment has been minimized.

andychu commented Jun 10, 2018

This comment has been minimized.

andychu commented Jun 10, 2018 •

edited

andychu pushed a commit that referenced this issue Jun 11, 2018

This comment has been minimized.

andychu commented Jun 11, 2018

andychu added release-note-0.5 and removed help wanted high-priority labels Jun 11, 2018

andychu closed this Jul 24, 2018

oilshell/oil

Join GitHub today

oil.ovm segfaults if built with gcc 8.1.1 #107

Comments

DennisMitchell commented May 17, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

DennisMitchell commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

andychu commented May 18, 2018

This comment has been minimized.

andychu commented May 20, 2018

This comment has been minimized.

sxldier commented May 20, 2018 • edited

This comment has been minimized.

andychu commented May 20, 2018

andychu added help wanted high-priority labels Jun 4, 2018

This comment has been minimized.

andychu commented Jun 10, 2018

This comment has been minimized.

andychu commented Jun 10, 2018 • edited

andychu pushed a commit that referenced this issue Jun 11, 2018

This comment has been minimized.

andychu commented Jun 11, 2018

andychu added release-note-0.5 and removed help wanted high-priority labels Jun 11, 2018

andychu closed this Jul 24, 2018

sxldier commented May 20, 2018 •

edited

andychu commented Jun 10, 2018 •

edited