lib-test test failure

[jryans]

On at least Ubuntu 18.04.2, the lib-test test segfaults, and gdb gives the following stack:

#0  0x00007ffff781ee97 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7820801 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff65dbb05 in add_all_loaded_segments (info=<optimized out>, size=<optimized out>, 
    maybe_lment=0x0) at /home/jryans/Projects/liballocs/src/allocators/static.c:172
#3  0x00007ffff7945f21 in dl_iterate_phdr () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x00007ffff65d1ce1 in __static_allocator_init ()
    at /home/jryans/Projects/liballocs/src/allocators/static.c:35
#5  0x00007ffff65d1ece in global constructors keyed to 65535_1_liballocs_pic.a_0x872a.8401 ()
    at /home/jryans/Projects/liballocs/src/allocators/stack.c:55
#6  0x00007ffff7de5733 in ?? () from /lib64/ld-linux-x86-64.so.2
#7  0x00007ffff7dea1ff in ?? () from /lib64/ld-linux-x86-64.so.2
#8  0x00007ffff79472df in _dl_catch_exception () from /lib/x86_64-linux-gnu/libc.so.6
#9  0x00007ffff7de97ca in ?? () from /lib64/ld-linux-x86-64.so.2
#10 0x00007ffff7bd1f96 in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
#11 0x00007ffff79472df in _dl_catch_exception () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x00007ffff794736f in _dl_catch_error () from /lib/x86_64-linux-gnu/libc.so.6
#13 0x00007ffff7bd2735 in ?? () from /lib/x86_64-linux-gnu/libdl.so.2
#14 0x00007ffff7bd2051 in dlopen () from /lib/x86_64-linux-gnu/libdl.so.2
#15 0x00005555555550b5 in main () at lib-test.c:11

[stephenrkell]

Hmm. Interesting. That test is passing for me, but it is sensitive to things like memory placement.

The failure line (static.c:172) is testing for a NULL info pointer from dl_iterate_phdr, which should never happen. However, sometimes the optimizer merges abort() paths, so the line numbers are not reliable. Can you do

cd ~/Projects/liballocs && make -C src clean && make DEBUG=1

and generate another backtrace? That should rebuild with -O0.

For background: 'lib-test' exists for running unit tests that get baked into a special liballocs-test.so binary. The idea is that the test program dlopens that binary, and the unit tests are run in the constructor logic. This has always seemed ugly to me, but I don't know good ways of running unit tests that would work for liballocs.

[jryans]

Thanks for the context and tips!

Curiously, the test passes when rebuilding with DEBUG=1. Looking a bit further, it appears related to LTO:

If I change the following in src/Makefile, then the optimised build allows the test to pass:

-    FAST_EXTRA_CFLAGS := -O3 -DNDEBUG -flto -ffat-lto-objects
+    FAST_EXTRA_CFLAGS := -O3 -DNDEBUG

This is with gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0.

[stephenrkell]

This is fixed as of 0f044d6... following several changes to how this test works, it is now re-enabled.

It's arguably cheating a bit, because in the new way of things, this particular test isn't run against an optimised build of the main liballocs code... it uses a special build which is always -O0 -DDEBUG -DTEST. So I can't be sure whether there is any LTO issue. But enough has changed that that would morally be a different issue. LTO of liballocs will get more testing via libcrunch, which I will get running in CI soon-ish.