duk cores at start with segfault on FreeBSD 11.1-RELEASE #1752

michael-o · 2017-09-22T21:15:49Z

Compiled 2.1.1 on

FreeBSD mika-ion 11.1-RELEASE-p1 FreeBSD 11.1-RELEASE-p1 #0: Wed Aug  9 11:17:49 UTC 2017     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  i386

with

FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
Target: i386-unknown-freebsd11.1
Thread model: posix
InstalledDir: /usr/bin

duk core immediately at start with

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `./duk -i'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0804c10d in duk_pop_n ()
(gdb) where
#0  0x0804c10d in duk_pop_n ()
#1  0x0805bc0e in duk__parse_stmts ()
#2  0x0805cbdc in duk__parse_stmt ()
#3  0x0805c945 in duk__parse_stmt ()
#4  0x0805bbe6 in duk__parse_stmts ()
#5  0x080582c8 in duk__parse_func_body ()
#6  0x080581bc in duk__parse_func_like_raw ()
#7  0x0805da67 in duk__parse_func_like_fnum ()
#8  0x0805f108 in duk ()
#9  0x0805f150 in duk ()
#10 0x0805ddb5 in duk__exprtop ()
#11 0x00000035 in ?? ()
#12 0x0805bdaa in duk__parse_stmt ()
#13 0x0805bbe6 in duk__parse_stmts ()
#14 0x080582c8 in duk__parse_func_body ()
#15 0x08057ea9 in duk__js_compile_raw ()
#16 0x0804a739 in duk_handle_safe_call ()
#17 0x0804a537 in duk_safe_call ()
#18 0x00000001 in ?? ()
#19 0x00000001 in ?? ()
#20 0x28629000 in ?? ()
#21 0x080578e0 in duk_js_compile ()
#22 0x0804bd9c in duk__do_compile ()
#23 0x0804bcae in duk_compile_raw ()
#24 0x0804bc0f in duk_eval_raw ()
#25 0x0807a8df in create_duktape_heap ()
#26 0x00000001 in ?? ()

The text was updated successfully, but these errors were encountered:

svaarala · 2017-09-22T21:17:11Z

Could you try what happens with -DDUK_USE_SELF_TESTS and -DDUK_USE_ASSERTIONS (given to configure.py) enabled?

svaarala · 2017-09-22T21:19:17Z

There was an earlier FreeBSD m32 issue which is documented here: https://github.com/svaarala/duktape/blob/master/misc/clang_aliasing.c. The output from that test might also be useful.

michael-o · 2017-09-22T21:44:27Z

Done, hopefully I did the right thing here:

[mosipov@mika-ion ~/Projekte/duktape-2.1.1]$ python2.7 tools/configure.py --output-directory /tmp/output --source-directory src-input --config-metadata config  -DDUK_USE_SELF_TESTS -DDUK_USE_ASSERTIONS
$ mv /tmp/output ~/Projekte/duktape-2.1.1/src-freebsd 	
$ make -f Makefile.cmdline
cc -o duk  -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./src-freebsd -DDUK_CMDLINE_PRINTALERT_SUPPORT -I./extras/print-alert -DDUK_CMDLINE_CONSOLE_SUPPORT -I./extras/console -DDUK_CMDLINE_LOGGING_SUPPORT -I./extras/logging -DDUK_CMDLINE_MODULE_SUPPORT -I./extras/module-duktape src-freebsd/duktape.c examples/cmdline/duk_cmdline.c extras/print-alert/duk_print_alert.c extras/console/duk_console.c extras/logging/duk_logging.c extras/module-duktape/duk_module_duktape.c -lm
$ ./duk
*** FATAL ERROR: self test(s) failed
Causing intentional segfault...
Segmentation fault (Speicherabzug geschrieben)
[mosipov@mika-ion ~/Projekte/duktape-2.1.1]$ gdb ./duk ./duk.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)...
Core was generated by `./duk'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0808f5dd in cmdline_fatal_handler ()
(gdb) where
#0  0x0808f5dd in cmdline_fatal_handler ()
#1  0x0804e5a8 in duk_create_heap ()
#2  0x0808f2ba in create_duktape_heap ()
#3  0x00000000 in ?? ()
(gdb)

svaarala · 2017-09-22T21:45:07Z

One more thing worth checking out is that the Duktape source and header files are an exact match. Sometimes if a -I... is missing for duktape.h one may accidentally use a system-installed duktape.h for a different version than the duktape.c being compiled (been there done that).

With debug symbols it would also be nice to see what duk_eval() call is causing the segfault. If there's something obviously wrong, like the packed duk_tval not working due to the aliasing issue, it's most likely the first one.

If the aliasing issue is at fault, -UDUK_USE_PACKED_TVAL (given to configure.py) would help.

svaarala · 2017-09-22T21:45:37Z

Ok, that self test failure means all bets are off - it's most likely the clang aliasing issue mentioned above, so you could try -UDUK_USE_PACKED_TVAL (given to configure.py) and see what happens.

michael-o · 2017-09-22T22:06:52Z

Unsetting DUK_USE_PACKED_TVAL did indeed work:

[mosipov@mika-ion ~/Projekte/duktape-2.1.1]$ ./duk
((o) Duktape 2.1.1 (external)
duk> print("hellp");
hellp
= undefined
duk> alert("toll");
toll
= undefined
duk> new Date();
= "2017-09-22T22:01:43.030Z"
duk> quit
ReferenceError: identifier 'quit' undefined
    at [anon] (src-freebsd/duktape.c:78145) internal
    at global (input:1) preventsyield
duk> = undefined
Cleaning up...

I will retry with Clang 5.0 from ports. Currently, the Duktape FreeBSD port is broken due to this which I need for Gerbera.

Does this issue imply that libduktape.so will also segfault if I do not regenerate with configure.py?

svaarala · 2017-09-22T22:10:00Z

Assuming you're trying to do a system-wide install:

You should run configure.py which generates duk_config.h, duktape.h, and duktape.c.
The libduktape.so library should be compiled from these sources.
The system headers must include the duktape.h and duk_config.h created by configure.py.

After that, the -UDUK_USE_PACKED_TVAL takes effect for any application built and linked against libduktape.so -- provided it is compiled with the correct, matching headers.

svaarala · 2017-09-22T22:11:16Z

If you're not doing a system-wide install, what matters is that any application code calling into libduktape.so is compiled with the exact same headers as the library itself, i.e. the same set of duk_config.h, duktape.c, duktape.h which is written by configure.py.

svaarala · 2017-09-22T22:13:10Z

I will retry with Clang 5.0 from ports. Currently, the Duktape FreeBSD port is broken due to this which I need for Gerbera.

It'll be interesting to hear if this fixes the issue. So far this issue has only happened on FreeBSD, with -m32. Linux Clang with -m32 is fine for example. The aliasing issue (see the link) is quite bizarre, to me it seems like an incorrect optimization. In short, a union with just an 8-byte array works, but if you add a double into the union, even when it is not referenced at all, union assignment copy starts failing because it compiles into a floating point store (even when the double field is not explicitly referenced at all).

michael-o · 2017-09-22T22:22:47Z

I currently do not have a system-wide install because I need to make things run first. libduktape.so is passed directly to Gerbera's CMake build. I'll see how far I get with 5.0 and will also install 6.0-dev. This might be worth reporting either to LLVM or FreeBSD itself. I'll get back to you tomorrow. Thank you!

svaarala · 2017-09-22T22:27:00Z

I've already reported it in 2013: http://lists.llvm.org/pipermail/cfe-users/2013-December/000321.html.

michael-o · 2017-09-22T22:44:36Z

Still worth reporting it to bugs.freebsd.org.

michael-o · 2017-09-23T18:43:05Z

So, I have now clang from base (4.0.0) and 4.0.1 + 5.0.0 from ports:

[mosipov@mika-ion ~/Projekte/duktape/misc]$ clang -Os -m32  -std=c99 -fstrict-aliasing -fomit-frame-pointer clang_aliasing.c
[mosipov@mika-ion ~/Projekte/duktape/misc]$ ./a.out
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f1 ff
[mosipov@mika-ion ~/Projekte/duktape/misc]$ clang -Os -m32  -std=c99 -fstrict-aliasing -fomit-frame-pointer clang_aliasing.c
[mosipov@mika-ion ~/Projekte/duktape/misc]$ ./a.out
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f1 ff
[mosipov@mika-ion ~/Projekte/duktape/misc]$ clang40 -Os -m32  -std=c99 -fstrict-aliasing -fomit-frame-pointer clang_aliasing.c
[mosipov@mika-ion ~/Projekte/duktape/misc]$ ./a.out
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f9 ff
11 22 33 44 00 00 f1 ff
[mosipov@mika-ion ~/Projekte/duktape/misc]$ clang50 -Os -m32  -std=c99 -fstrict-aliasing -fomit-frame-pointer clang_aliasing.c
[mosipov@mika-ion ~/Projekte/duktape/misc]$ ./a.out
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f1 ff
11 22 33 44 00 00 f1 ff

I will report this against FreeBSD base.

michael-o · 2017-09-23T18:46:03Z

Can you please update the wiki for clang 4.0.1 and 5.0.0?

svaarala · 2017-09-23T18:49:42Z

Ok, I'll open a pull for it.

svaarala · 2017-09-23T19:00:32Z

svaarala/duktape-wiki#189

michael-o · 2017-09-23T23:46:04Z

@svaarala I think this is the issue and the fix for it. WDYT?

svaarala · 2017-09-23T23:53:26Z

Looks very likely, I'll link to these to explain the cause a bit more.

Any suggestions if Duktape could do something to (a) reliably detect the issue or (b) work around it?

For (a), disabling packed duk_tval for Clang prior to 5.0 (or whatever the exact version is) seems quite wrong, because this issue does not appear on e.g. Linux clang prior to 5.0 (even with -m32). I'm not sure why, but maybe there's something in the way the compiler is configured/built. So one would ideally want something more precise.

Maybe "FreeBSD && clang < 5.0 && x86" would be a good check? It would mean more memory usage (because duk_tval will be larger) until clang 5.0 but it seems better than a broken build which is detected by self checks only. There's no guarantee the issue wouldn't appear elsewhere too, but this narrow case at least seems broken.

For (b), I don't have a good idea of how to work around it reliably.

fatcerberus · 2017-09-24T00:00:20Z

According to the bug report linked above compiling with-fno-strict-aliasing works around the bug.

svaarala · 2017-09-24T00:01:56Z

@fatcerberus It does, but duk_config.h has no reasonable way of detecting whether strict aliasing is used or not?

svaarala · 2017-09-24T00:02:51Z

So what I'm mainly after, how to work around the issue so that the build comes our working without going through segfaults and diagnosis.

I would also be happy with an #error for FreeBSD + clang < 5.0 + m32 + -fstrict-aliasing, but I don't think one can detect that.

svaarala · 2017-09-24T00:08:51Z

#1764 is a possible automatic workaround.

michael-o · 2017-09-24T09:40:23Z

-fno-strict-aliasing does not seem to work:

[mosipov@mika-ion ~/Projekte/duktape-2.1.1]$ make -f Makefile.cmdline
cc -o duk  -fno-strict-aliasing -Os -pedantic -std=c99 -Wall -fomit-frame-pointer -I./src -DDUK_CMDLINE_PRINTALERT_SUPPORT -I./extras/print-alert -DDUK_CMDLINE_CONSOLE_SUPPORT -I./extras/console -DDUK_CMDLINE_LOGGING_SUPPORT -I./extras/logging -DDUK_CMDLINE_MODULE_SUPPORT -I./extras/module-duktape -DDUK_CMDLINE_FANCY -I./linenoise src/duktape.c examples/cmdline/duk_cmdline.c extras/print-alert/duk_print_alert.c extras/console/duk_console.c extras/logging/duk_logging.c extras/module-duktape/duk_module_duktape.c linenoise/linenoise.c -lm
linenoise/linenoise.c:568:27: warning: must specify at least one argument for '...'
      parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
        lndebug("clear+up");
                          ^
linenoise/linenoise.c:195:9: note: macro 'lndebug' defined here
#define lndebug(fmt, ...)
        ^
linenoise/linenoise.c:574:20: warning: must specify at least one argument for '...'
      parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
    lndebug("clear");
                   ^
linenoise/linenoise.c:195:9: note: macro 'lndebug' defined here
#define lndebug(fmt, ...)
        ^
linenoise/linenoise.c:591:28: warning: must specify at least one argument for '...'
      parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
        lndebug("<newline>");
                           ^
linenoise/linenoise.c:195:9: note: macro 'lndebug' defined here
#define lndebug(fmt, ...)
        ^
linenoise/linenoise.c:619:17: warning: must specify at least one argument for '...'
      parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
    lndebug("\n");
                ^
linenoise/linenoise.c:195:9: note: macro 'lndebug' defined here
#define lndebug(fmt, ...)
        ^
4 warnings generated.
[mosipov@mika-ion ~/Projekte/duktape-2.1.1]$ ./duk
Segmentation fault (Speicherabzug geschrieben)

I will try #1764.

michael-o · 2017-09-24T11:31:44Z

Here is the report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222551

svaarala · 2017-09-24T20:08:25Z

It might be that the root cause is not, after all, correct if the suggested workaround doesn't work.

michael-o · 2017-09-25T12:22:20Z

Feel free to close this issue as it has been decently resolved. Thank you for the precious input. The FreeBSD bug report gained some traction and instruction what to do: use the port.

svaarala · 2017-09-25T12:52:53Z

@michael-o Ok, thanks for taking the time to run all these tests :)

svaarala added the portability label Sep 22, 2017

svaarala mentioned this issue Sep 23, 2017

Portability page updates, FreeBSD clang versions svaarala/duktape-wiki#189

Open

4 tasks

svaarala closed this as completed Sep 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

duk cores at start with segfault on FreeBSD 11.1-RELEASE #1752

duk cores at start with segfault on FreeBSD 11.1-RELEASE #1752

michael-o commented Sep 22, 2017 •

edited

Loading

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017 •

edited

Loading

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017 •

edited

Loading

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017

michael-o commented Sep 23, 2017

michael-o commented Sep 23, 2017

svaarala commented Sep 23, 2017

svaarala commented Sep 23, 2017

michael-o commented Sep 23, 2017

svaarala commented Sep 23, 2017

fatcerberus commented Sep 24, 2017

svaarala commented Sep 24, 2017

svaarala commented Sep 24, 2017 •

edited

Loading

svaarala commented Sep 24, 2017

michael-o commented Sep 24, 2017

michael-o commented Sep 24, 2017

svaarala commented Sep 24, 2017

michael-o commented Sep 25, 2017

svaarala commented Sep 25, 2017

duk cores at start with segfault on FreeBSD 11.1-RELEASE #1752

duk cores at start with segfault on FreeBSD 11.1-RELEASE #1752

Comments

michael-o commented Sep 22, 2017 • edited Loading

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017 • edited Loading

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017 • edited Loading

svaarala commented Sep 22, 2017

michael-o commented Sep 22, 2017

michael-o commented Sep 23, 2017

michael-o commented Sep 23, 2017

svaarala commented Sep 23, 2017

svaarala commented Sep 23, 2017

michael-o commented Sep 23, 2017

svaarala commented Sep 23, 2017

fatcerberus commented Sep 24, 2017

svaarala commented Sep 24, 2017

svaarala commented Sep 24, 2017 • edited Loading

svaarala commented Sep 24, 2017

michael-o commented Sep 24, 2017

michael-o commented Sep 24, 2017

svaarala commented Sep 24, 2017

michael-o commented Sep 25, 2017

svaarala commented Sep 25, 2017

michael-o commented Sep 22, 2017 •

edited

Loading

michael-o commented Sep 22, 2017 •

edited

Loading

michael-o commented Sep 22, 2017 •

edited

Loading

svaarala commented Sep 24, 2017 •

edited

Loading