Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interesting issue on OpenSUSE, dlmopen stalls #20

Closed
Docteh opened this issue Sep 20, 2022 · 3 comments
Closed

Interesting issue on OpenSUSE, dlmopen stalls #20

Docteh opened this issue Sep 20, 2022 · 3 comments

Comments

@Docteh
Copy link

Docteh commented Sep 20, 2022

On opensuse tumbleweed, from https://get.opensuse.org/tumbleweed/

litrace opensuse issue.txt

strace.txt

The usr/optional/cxx/libstdc++.so.6 is compiled with GCC11, that one loads, but then when checkrt goes to check the local library, the dlmopen never returns.

@al95
Copy link

al95 commented Sep 20, 2022

Same problem on Arch.

I thought it might be an upstream glibc bug, but a simple program that just runs dlmopen(LM_ID_NEWLM, "libstdc++.so.6", RTLD_LAZY); works as expected without hanging has the same issue (see next message).

But it gets weirder.

checkrt checks libgcc_s.so.1 first and then libstdc++.so.6.
However if the order is swapped, meaning swapping lines 374-378 with lines 380-384 (and moving the int var) it segfaults at the strncmp at line 194.

This seems to be caused by the strtab/dynstr search at lines 147-165, where finding either the .strtab section or the .dynstr section stops the looping with a break statement.
This means that if a library contains both .strtab and .dynstr, only one of the addresses will be set, while the other address will stay at the value set at lines 143-144 .

Deleting the two breaks at line 157 and line 163 fixes the segfault and checkrt runs successfully.

However, restoring the order of the library checks, meaning libgcc first and libstdc++ second, still makes dlmopen hang while checking the system libstdc++.

This might be an upstream bug but I'm not sure what checkrt is doing to confuse dlmopen.
Edit: this is definitely a glibc bug, see next message.

This is the complete backtrace:

#0  _dl_debug_initialize (ldbase=ldbase@entry=0, ns=ns@entry=2) at dl-debug.c:78
#1  0x00007ffff7fd5d39 in dl_open_worker_begin (a=a@entry=0x7fffffffda70) at dl-open.c:530
#2  0x00007ffff7ee3dce in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at /usr/src/debug/glibc/elf/dl-error-skeleton.c:208
#3  0x00007ffff7fd54fb in dl_open_worker (a=a@entry=0x7fffffffda70) at dl-open.c:782
#4  0x00007ffff7ee3dce in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at /usr/src/debug/glibc/elf/dl-error-skeleton.c:208
#5  0x00007ffff7fd58ed in _dl_open (file=0x555555557354 "libstdc++.so.6", mode=<optimized out>, caller_dlopen=0x555555555369 <get_libpath+64>, nsid=<optimized out>, argc=1, argv=0x7fffffffe028, env=0x7fffffffe038) at dl-open.c:886
#6  0x00007ffff7e16694 in dlmopen_doit (a=a@entry=0x7fffffffdce0) at dlmopen.c:61
#7  0x00007ffff7ee3dce in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffdc40, operate=<optimized out>, args=<optimized out>) at /usr/src/debug/glibc/elf/dl-error-skeleton.c:208
#8  0x00007ffff7ee3e83 in __GI__dl_catch_error (objname=0x7fffffffdc98, errstring=0x7fffffffdca0, mallocedp=0x7fffffffdc97, operate=<optimized out>, args=<optimized out>) at /usr/src/debug/glibc/elf/dl-error-skeleton.c:227
#9  0x00007ffff7e162ff in _dlerror_run (operate=operate@entry=0x7ffff7e16630 <dlmopen_doit>, args=args@entry=0x7fffffffdce0) at dlerror.c:138
#10 0x00007ffff7e167a6 in dlmopen_implementation (dl_caller=<optimized out>, mode=<optimized out>, file=<optimized out>, nsid=<optimized out>) at dlmopen.c:76
#11 ___dlmopen (nsid=<optimized out>, file=<optimized out>, mode=<optimized out>) at dlmopen.c:86
#12 0x0000555555555369 in get_libpath (lib=0x555555557354 "libstdc++.so.6", verbose=0 '\000') at checkrt.c:70
#13 0x000055555555549d in symbol_version (lib=0x555555557354 "libstdc++.so.6", sym_prefix=0x55555555736f "GLIBCXX_", verbose=0 '\000') at checkrt.c:95
#14 0x000055555555648f in main (argc=1, argv=0x7fffffffe028) at checkrt.c:382

The glibc line numbers are for version 2.36, commit 645d94808aaa90fb1b20a25ff70bb50d9eb1d55b.
It seems to be hanging in the for loop at line 76 of elf/dl-debug.c, but that's all I got for now.

@al95
Copy link

al95 commented Sep 21, 2022

This is definitely a glibc bug.
Calling dlmopen will hang depending on which libraries were previously loaded.
Example: https://gist.github.com/al95/52e0f6cf074a06dfc106dac494355664
On my pc it hangs while loading libpthread.so.0, but changing the libraries or their order makes it hang at a different one.
I'll check if there is already a bug report upstream and if not I'll make one.

EDIT: I bisected the code and this issue was introduced with commit 924e4f3eaa502ce82fccf8537f021a796d158771, which was a fix for another bug.
I'm waiting for my account to be approved so I can report this. Reported.

Regarding the strtab/dynstr bug from my first message, I opened a new issue (#21), considering it's completely unrelated.

@darealshinji
Copy link
Owner

darealshinji commented Apr 26, 2023

Closing this issue since it's a bug in glibc. Make sure your distro backports this patch if it's still affecting you:
https://sourceware.org/pipermail/libc-alpha/2022-September/142199.html

Note: this fix was released in glibc version 2.37 (bug no. 29600):
https://sourceware.org/pipermail/libc-alpha/2023-February/145190.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants