Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leak sanitizer is broken with glibc 2.32 #1342

Open
andrew-aladev opened this issue Nov 7, 2020 · 3 comments
Open

Leak sanitizer is broken with glibc 2.32 #1342

andrew-aladev opened this issue Nov 7, 2020 · 3 comments

Comments

@andrew-aladev
Copy link

andrew-aladev commented Nov 7, 2020

Please start a new machine with glibc 2.32 and try to use leak sanitizer (clang version doesn't matter, tried 10 and 11):

echo "int main () { return 0; }" > /tmp/fit.c && clang -fsanitize=leak /tmp/fit.c -o /tmp/fit && LSAN_OPTIONS=verbosity=1:log_threads=1 /tmp/fit

==27314==Installed the sigaction for signal 11
==27314==Installed the sigaction for signal 7
==27314==Installed the sigaction for signal 8
==27315==Processing thread 27314.
==27315==Stack at 0x7ffce203f000-0x7ffce283f000 (SP = 0x7ffce283ce98).
==27315==TLS at 0x7f035204ef40-0x7f035205dbc0.
Tracer caught signal 11: addr=0x7f035204ef40 pc=0x41c401 sp=0x7f0350fffd30
==27314==LeakSanitizer has encountered a fatal error.
==27314==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==27314==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)
@andrew-aladev andrew-aladev changed the title Leak sanitizer broken with glibc 2.32 Leak sanitizer is broken with glibc 2.32 Nov 7, 2020
@kdurnoga
Copy link

I'm facing the same issue on glibc 2.32. I've tried glibc 2.31 and it seems to be working fine. That might have been caused by the TLS-related changes introduced with glibc 2.32 (but that's just my guess).

Anyways, as a temporary workaround you can try running your program with LSAN_OPTIONS=use_tls=0. You should be ok unless your program relies on TLS.

@darksylinc
Copy link

darksylinc commented Aug 5, 2021

I'm encountering the same problem however I'm using libc 2.27 + Kernel 5.13.7; works fine with Kernel 5.11.0

LSAN_OPTIONS=use_tls=0 fixes the problem as well (however I do use TLS, so I'm screwed).

Update: Scratch that. Problem w/ kernel 5.13.7 seems to be intermittent like in #1353; it's now working without the LSAN_OPTIONS hack.
Update 2: I have a VM (Virtualbox, guest Ubuntu 18.04.5) running in the background during these failures (ASAN is ran on the host). Likehood of failure appears to be linked to VM's activity. Higher VM activity leads to higher failure rate. The errors may still appear when the VM is shutdown though. The error has never appeared so far if ran with use_tls=0
Update 3: Runtime length of the process being analyzed by ASAN also seems to be linked with failure rate. Short-lived apps tend to succeed. Long-lived apps (i.e. more than 60 secs) tend to fail.

@clemente0731
Copy link

LSAN_OPTIONS=use_tls=0.
helps me

mqudsi added a commit to mqudsi/fish-shell that referenced this issue Oct 24, 2022
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* google/sanitizers#1342
* google/sanitizers#1409
mqudsi added a commit to mqudsi/fish-shell that referenced this issue Oct 24, 2022
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* google/sanitizers#1342
* google/sanitizers#1409
mqudsi added a commit to mqudsi/fish-shell that referenced this issue Oct 24, 2022
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* google/sanitizers#1342
* google/sanitizers#1409
mqudsi added a commit to mqudsi/fish-shell that referenced this issue Oct 25, 2022
LSAN seems to have an issue with glibc's TLS functionality that causes it to
intermittently crash with SIGSEGV when run virtualized, as it is in our CI.

Relevant GitHub issues:
* google/sanitizers#1342
* google/sanitizers#1409
ax3l added a commit to ECP-WarpX/impactx that referenced this issue Mar 14, 2023
Disable TLS for the leak sanitizer due to an issue in glibc 2.32:
google/sanitizers#1342
d-janus-qnx pushed a commit to qnx/weston that referenced this issue Apr 9, 2024
Without this fix, we have randomly been getting CI failures due to
LeakSanitizer itself crashing after all the tests in a program have
succeeded. This has been happening randomly for a long time, but
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1486
made it very reliably repeatable in the job x86_64-debian-full-build
(and no other job) in the test-subsurface-shot program.

--- Fixture 2 (GL) ok: passed 4, skipped 0, failed 0, total 4
Tracer caught signal 11: addr=0x1b8 pc=0x7f6b3ba640f0 sp=0x7f6b2cc77d10
==489==LeakSanitizer has encountered a fatal error.

I was also able to get a core file after twiddling, but there it ended
up with lsan aborting itself rather than a segfault.

We got some clues that use_tls=0 might work around this, from
google/sanitizers#1342
google/sanitizers#1409
and some other projects that have cargo-culted the same workaround.

Using that cause more false leaks to appear, so they need to be
suppressed. I suppose we are not interested in catching leaks in glib
using code, so I opted to suppress g_malloc0 altogether. Pinpointing it
better might have required much more slower stack tracing.

wl_shm_buffer_begin_access() uses TLS, so no wonder it gets flagged.

ld-*.so is simply uninteresting to us, and it got flagged too.

Since this might have been fixed already in LeakSanitizer upstream, who
knows, leave some notes to revisit this when we upgrade that in CI.

This fix seems to make the branch of
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1486
in my quick testing.

Suggested-by: Derek Foreman <derek.foreman@collabora.com>
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants