Sample C++ program fails to build with address sanitizer

[chouquette]

Hi,

I'm trying to build VLC using asan, and it seems that all C++ modules are failing to link when building with ASAN. As far as I can see, there's no issue with C modules.
I've reproduce the issue with the attached sample program:
test.zip

It works fine when building the program & lib as C, but it fails when building as C++:

root@compile-vlc-llvm-mingw:/tmp/test# make
/bin/bash ./libtool  --tag=CXX   --mode=compile x86_64-w64-mingw32-g++ -DPACKAGE_NAME=\"test\" -DPACKAGE_TARNAME=\"test\" -DPACKAGE_VERSION=\"0.0.0-dev\" -DPACKAGE_STRING=\"test\ 0.0.0-dev\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"test\" -DVERSION=\"0.0.0-dev\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DLT_OBJDIR=\".libs/\" -I.    -fsanitize=address -g -O2 -MT libtest_la-libtest.lo -MD -MP -MF .deps/libtest_la-libtest.Tpo -c -o libtest_la-libtest.lo `test -f 'libtest.cpp' || echo './'`libtest.cpp
libtool: compile:  x86_64-w64-mingw32-g++ -DPACKAGE_NAME=\"test\" -DPACKAGE_TARNAME=\"test\" -DPACKAGE_VERSION=\"0.0.0-dev\" "-DPACKAGE_STRING=\"test 0.0.0-dev\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"test\" -DVERSION=\"0.0.0-dev\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DLT_OBJDIR=\".libs/\" -I. -fsanitize=address -g -O2 -MT libtest_la-libtest.lo -MD -MP -MF .deps/libtest_la-libtest.Tpo -c libtest.cpp  -DDLL_EXPORT -DPIC -o .libs/libtest_la-libtest.o
mv -f .deps/libtest_la-libtest.Tpo .deps/libtest_la-libtest.Plo
/bin/bash ./libtool  --tag=CXX   --mode=link x86_64-w64-mingw32-g++ -fsanitize=address -g -O2 -no-undefined  -o libtest.la -rpath /usr/local/lib libtest_la-libtest.lo  

*** Warning: Trying to link with static lib archive /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because the file extensions .a of this argument makes me believe
*** that it is just a static archive that I should not use here.

*** Warning: Trying to link with static lib archive /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because the file extensions .a of this argument makes me believe
*** that it is just a static archive that I should not use here.
libtool: link: x86_64-w64-mingw32-g++ -shared -nostdlib /opt/llvm-mingw/x86_64-w64-mingw32/lib/dllcrt2.o /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtbegin.o  .libs/libtest_la-libtest.o   -L/opt/llvm-mingw/x86_64-w64-mingw32/lib -L/opt/llvm-mingw/lib -L/opt/llvm-mingw/x86_64-w64-mingw32/sys-root/mingw/lib -L/opt/llvm-mingw/lib/clang/11.0.0/lib/windows -lc++ -lmingw32 -lmoldname -lmingwex -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lmoldname -lmingwex -lmsvcrt -lkernel32 /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtend.o  -fsanitize=address -g -O2   -o .libs/libtest-0.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libtest.dll.a
lld-link: error: undefined symbol: __asan_init
>>> referenced by libtest.cpp:7
>>>               .libs/libtest_la-libtest.o:(asan.module_ctor)

lld-link: error: undefined symbol: __asan_version_mismatch_check_v8
>>> referenced by libtest.cpp:7
>>>               .libs/libtest_la-libtest.o:(asan.module_ctor)

lld-link: error: undefined symbol: __asan_shadow_memory_dynamic_address
>>> referenced by .libs/libtest_la-libtest.o

lld-link: error: undefined symbol: ___chkstk_ms
>>> referenced by ../crt/pseudo-reloc.c:477
>>>               libmingw32.a(lib64_libmingw32_a-pseudo-reloc.o):(_pei386_runtime_relocator)
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:438: recipe for target 'libtest.la' failed
make: *** [libtest.la] Error 1

The tests were run on the provided docker image and on the VideoLAN one, using a patched libtool, and resulted with the same errors.

Please let me know if I'm missing something or if you'd like me to try something out!

Thanks

[mstorsjo]

Hi!

Awesome to hear that you're trying to use the sanitizers for real! I have a feeling that they haven't been tested all that much so far, as I've had very few reports about them; either everything is working perfectly, or it's just undertested.

Also great that you have a testcase! Unfortunately, it seems to be corrupted (and very creatively so), the configure.ac file in the zip seems to actually be a .tar.gz containing the rest of the files. Do you have a real copy of that file and I could try this out?

In general, this sounds to me like another case of libtool accidentally missing even more libs (when libtool does a test link with -v, parsing out the implicitly linked libs, and then trying to readd them when actually linking in the end) - maybe it doesn't include the -fsanitize=address option when it does its test link (during configure) to pick up the implicit libs? Also I presume that linking C++ libs/executables with asan succeeds if you do it manually without any build system involved - because it does for me at least.

Based on the quoted error log, as you also have cases of missing ___chkstk_ms, it does sound to me like it really isn't generated with the properly patched libtool. Or does it succeed if you skip the sanitizer bits, so the sanitizer stuff somehow makes libtool lose even the basic libclang_rt.builtins library?

But I presume that ___chkstk_ms isn't an issue for you in the real setup where you're testing this on VLC.

Linking with sanitizers adds a bunch of extra args (try linking with -fsanitize=address -v), something like <prefix>/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic-x86_64.dll.a --require-defined __asan_seh_interceptor --whole-archive <prefix>/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic_runtime_thunk-x86_64.a --no-whole-archive, and it's quite plausible that some of these end up clobbered/missed by libtool - unfortunately :-(

[chouquette]

Woops my bad, new archive attached

testcase.zip

Also I presume that linking C
++ libs/executables with asan succeeds if you do it manually without any build system involved - because it does for me at least.

Yeah same here, building a simple executable without autotools/libtool works fine

Based on the quoted error log, as you also have cases of missing ___chkstk_ms, it does sound to me like it really isn't generated with the properly patched libtool. Or does it succeed if you skip the sanitizer bits, so the sanitizer stuff somehow makes libtool lose even the basic libclang_rt.builtins library?

You are correct, and indeed removing the sanitizer bits still fails with the same __chkstk_ms missing symbol error. I don't know where/how I failed to apply the patch, but it now works fine without the sanitizer flags, but mostly it now also works with those.

I'll try to understand what's missing when building VLC and will report back here, thanks for your help!

[chouquette]

Indeed, building a test program with x86_64-w64-mingw32-g++ yields:
"/opt/llvm-mingw/bin/ld.lld" -m i386pep -Bdynamic -o test.exe /opt/llvm-mingw/x86_64-w64-mingw32/lib/crt2.o /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtbegin.o -L/opt/llvm-mingw/x86_64-w64-mingw32/lib -L/opt/llvm-mingw/lib -L/opt/llvm-mingw/x86_64-w64-mingw32/sys-root/mingw/lib -L/opt/llvm-mingw/lib/clang/11.0.0/lib/windows test.o -lc++ -lmingw32 /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lmsvcrt /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic-x86_64.dll.a /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic_runtime_thunk-x86_64.a --require-defined __asan_seh_interceptor --whole-archive /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic_runtime_thunk-x86_64.a --no-whole-archive -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lmsvcrt -lkernel32 /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtend.o

While building a VLC module using libtool yields:
"/opt/llvm-mingw/bin/ld.lld" -m i386pep --shared -Bstatic -e DllMainCRTStartup --enable-auto-image-base -o .libs/libdcp_plugin.dll -L/home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib -L/usr/local/lib -L/home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib64 -L/opt/llvm-mingw/x86_64-w64-mingw32/lib -L/opt/llvm-mingw/lib -L/opt/llvm-mingw/x86_64-w64-mingw32/sys-root/mingw/lib -L/opt/llvm-mingw/lib/clang/11.0.0/lib/windows -L/opt/llvm-mingw/x86_64-w64-mingw32/lib -L/opt/llvm-mingw/lib -L/opt/llvm-mingw/x86_64-w64-mingw32/sys-root/mingw/lib -L/opt/llvm-mingw/lib/clang/11.0.0/lib/windows .libs/libdcp_plugin.dll.def /opt/llvm-mingw/x86_64-w64-mingw32/lib/dllcrt2.o /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtbegin.o ../modules/.libs/module.rc.o access/dcp/.libs/libdcp_plugin_la-dcp.o access/dcp/.libs/libdcp_plugin_la-dcpparser.o access/dcp/.libs/libdcp_plugin_la-dcpdecrypt.o ../compat/.libs/libcompat.a ../src/.libs/libvlccore.dll.a -lole32 /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libiconv.a -lws2_32 -liphlpapi -lssp /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libasdcp.a /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libkumu.a -lpthread -lnettle /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libgmp.a /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libgcrypt.a -lbcrypt /home/chouquette/dev/vlc/contrib/x86_64-w64-mingw32/lib/libgpg-error.a -lc++ -lmingw32 /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lmsvcrt -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lmsvcrt -lkernel32 /opt/llvm-mingw/x86_64-w64-mingw32/lib/crtend.o --nxcompat --no-seh --dynamicbase --high-entropy-va --image-base 0x140000000 --enable-auto-image-base --out-implib .libs/libdcp_plugin.dll.a

So it most likely requires some libtool patching :(

[chouquette]

I'm really surprised /opt/llvm-mingw/lib/clang/11.0.0/lib/windows/libclang_rt.asan_dynamic-x86_64.dll.a doesn't get forwarded by libtool though, it should match | */libclang_rt.*.a

[mstorsjo]

Indeed, that's a bit strange. But getting all the --require-defined and --whole-archive bits through seems quite daunting :-(

[chouquette]

It seems more productive to try to build with meson, and potentially fix some cross compilation issue, than trying to add yet another set of libtool fixes

[mstorsjo]

Yes, I'd pretty much agree with that.

[chouquette]

Ok then I'll close this and will give meson a try, I'll reopen if something less libtool related comes across :)

Thanks again!