Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Netopeer crashes by get / get-config request #1

Closed
rastislavs opened this issue Apr 26, 2016 · 2 comments
Closed

Netopeer crashes by get / get-config request #1

rastislavs opened this issue Apr 26, 2016 · 2 comments

Comments

@rastislavs
Copy link
Contributor

Netopeer crashes by get / get-config request. Logs + backtrace:

netopeer2-server[17615]: Session 1: received message:
<rpc message-id="101" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <get/>
</rpc>
netopeer2-server[17615]: DICT: inserting "rpc" with collision 
netopeer2-server[17615]: DICT: inserting "message-id" with collision 
netopeer2-server[17615]: DICT: inserting "101" with collision 
netopeer2-server[17615]: DICT: inserting (refcount) "urn:ietf:params:xml:ns:netconf:base:1.0"
netopeer2-server[17615]: DICT: inserting (refcount) "get"
netopeer2-server[17615]: DICT: inserting (refcount) ""
netopeer2-server[17615]: DICT: inserting (refcount) ""

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4fce700 (LWP 17616)]
0x00007ffff7965fd5 in lys_parent (node=0x63f648) at /home/rasto/libraries/libyang/src/tree_schema.c:2854
2854        if (node->parent->nodetype == LYS_AUGMENT) {
(gdb) bt
#0  0x00007ffff7965fd5 in lys_parent (node=0x63f648) at /home/rasto/libraries/libyang/src/tree_schema.c:2854
#1  0x00007ffff795e986 in ly_check_mandatory (data=0x7fffec0017d0, schema=0x640920, status=1) at /home/rasto/libraries/libyang/src/tree_schema.c:594
#2  0x00007ffff7966350 in lyd_check_topmandatory (data=0x7fffec0017d0, ctx=0x6212c0, rpc=0x0, options=528) at /home/rasto/libraries/libyang/src/tree_data.c:88
#3  0x00007ffff79400f1 in lyd_parse_xml (ctx=0x6212c0, root=0x7fffec0008d8, options=528) at /home/rasto/libraries/libyang/src/parser_xml.c:562
#4  0x00007ffff76f27ae in nc_recv_rpc (session=0x670f90, rpc=0x7ffff4fcde88) at /home/rasto/libraries/libnetconf2/src/session_server.c:820
#5  0x00007ffff76f2fa5 in nc_ps_poll (ps=0x670600, timeout=500) at /home/rasto/libraries/libnetconf2/src/session_server.c:1039
#6  0x0000000000404211 in process_loop (arg=0x0) at /home/rasto/libraries/Netopeer2/server/main.c:382
#7  0x00007ffff7bc4182 in start_thread (arg=0x7ffff4fce700) at pthread_create.c:312
#8  0x00007ffff71c847d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

List of modules installed in sysrepo (minimal set required by Netopeer + turing-machine):

Module Name                   | Revision   | Data Owner          | Permissions | Submodules                    | Enabled Features
---------------------------------------------------------------------------------------------------------------------------------
turing-machine                | 2013-12-27 | root:root           | 644         |                               |
ietf-netconf-monitoring       | 2010-10-04 | root:root           | 644         |                               |
ietf-netconf-acm              | 2012-02-22 |                     |             |                               |
ietf-netconf                  | 2011-06-01 | root:root           | 644         |                               |
ietf-netconf-with-defaults    | 2011-06-01 | root:root           | 644         |                               |
@michalvasko
Copy link
Member

Hi,
please try it now again, with the current master branches of all libyang, libnetconf2, and netopeer2-server. It works for me now because we added many bugfixes mainly to libyang, but I may have missed something.

Regards,
Michal

@rastislavs
Copy link
Contributor Author

works correctly now, I'm closing the issue

jktjkt added a commit to jktjkt/Netopeer2 that referenced this issue Jun 15, 2023
As per the docs, the SIGEV_THREAD sigevent option instructs the kernel
to create a thread in this process when that timer expires (in fact this
is done by the C library, and the kernel "just" wakes up that thread
AFAICT). However, this newly created thread appears to have completely
bypassed TSAN, and when the just-created thread hits any TSAN
interceptor which assumes that some per-thread TSAN structures have been
already set up, the application segfaults (for example like this):

 #0  0x00005555556192d4 in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
 CESNET#1  0x00005555556196a4 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) ()
 CESNET#2  0x00005555555d3db5 in malloc ()
 CESNET#3  0x00007ffff7889470 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 CESNET#4  0x00007ffff787ddd4 in start_thread (arg=<optimized out>) at pthread_create.c:444
 CESNET#5  0x00007ffff78ff9b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

That's with the latest clang, or, in another case, with GCC:

 #0  0x00007ffff4e88fc6 in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<__tsan::AP64>, __sanitizer::LargeMmapAllocatorPtrArrayDynamic>::Allocate(__sanitizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocator64<__tsan::AP64> >*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#1  0x00007ffff4e8686a in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
    from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#2  0x00007ffff4e869d8 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#3  0x00007ffff4e4331e in malloc () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#4  0x00007ffff4cab4c0 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 CESNET#5  0x00007ffff4c9fe24 in start_thread (arg=<optimized out>) at pthread_create.c:444
 CESNET#6  0x00007ffff4d219b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

I have no clue on how to fix this properly within TSAN.

Bug: google/sanitizers#1612
jktjkt added a commit to jktjkt/Netopeer2 that referenced this issue Jun 16, 2023
As per the docs, the SIGEV_THREAD sigevent option instructs the kernel
to create a thread in this process when that timer expires (in fact this
is done by the C library, and the kernel "just" wakes up that thread
AFAICT). However, this newly created thread appears to have completely
bypassed TSAN, and when the just-created thread hits any TSAN
interceptor which assumes that some per-thread TSAN structures have been
already set up, the application segfaults (for example like this):

 #0  0x00005555556192d4 in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
 CESNET#1  0x00005555556196a4 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) ()
 CESNET#2  0x00005555555d3db5 in malloc ()
 CESNET#3  0x00007ffff7889470 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 CESNET#4  0x00007ffff787ddd4 in start_thread (arg=<optimized out>) at pthread_create.c:444
 CESNET#5  0x00007ffff78ff9b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

That's with the latest clang, or, in another case, with GCC:

 #0  0x00007ffff4e88fc6 in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<__tsan::AP64>, __sanitizer::LargeMmapAllocatorPtrArrayDynamic>::Allocate(__sanitizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocator64<__tsan::AP64> >*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#1  0x00007ffff4e8686a in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
    from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#2  0x00007ffff4e869d8 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#3  0x00007ffff4e4331e in malloc () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 CESNET#4  0x00007ffff4cab4c0 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 CESNET#5  0x00007ffff4c9fe24 in start_thread (arg=<optimized out>) at pthread_create.c:444
 CESNET#6  0x00007ffff4d219b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

I have no clue on how to fix this properly within TSAN.

Bug: google/sanitizers#1612
michalvasko pushed a commit that referenced this issue Jun 16, 2023
As per the docs, the SIGEV_THREAD sigevent option instructs the kernel
to create a thread in this process when that timer expires (in fact this
is done by the C library, and the kernel "just" wakes up that thread
AFAICT). However, this newly created thread appears to have completely
bypassed TSAN, and when the just-created thread hits any TSAN
interceptor which assumes that some per-thread TSAN structures have been
already set up, the application segfaults (for example like this):

 #0  0x00005555556192d4 in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
 #1  0x00005555556196a4 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) ()
 #2  0x00005555555d3db5 in malloc ()
 #3  0x00007ffff7889470 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 #4  0x00007ffff787ddd4 in start_thread (arg=<optimized out>) at pthread_create.c:444
 #5  0x00007ffff78ff9b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

That's with the latest clang, or, in another case, with GCC:

 #0  0x00007ffff4e88fc6 in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<__tsan::AP64>, __sanitizer::LargeMmapAllocatorPtrArrayDynamic>::Allocate(__sanitizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocator64<__tsan::AP64> >*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 #1  0x00007ffff4e8686a in __tsan::user_alloc_internal(__tsan::ThreadState*, unsigned long, unsigned long, unsigned long, bool) ()
    from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 #2  0x00007ffff4e869d8 in __tsan::user_alloc(__tsan::ThreadState*, unsigned long, unsigned long) () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 #3  0x00007ffff4e4331e in malloc () from /nix/store/ds46sf6mb9xydn0gy97131y1w5hag5s8-gcc-12.2.0-lib/lib/libtsan.so.2
 #4  0x00007ffff4cab4c0 in timer_helper_thread (arg=<optimized out>) at ../sysdeps/unix/sysv/linux/timer_routines.c:88
 #5  0x00007ffff4c9fe24 in start_thread (arg=<optimized out>) at pthread_create.c:444
 #6  0x00007ffff4d219b0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

I have no clue on how to fix this properly within TSAN.

Bug: google/sanitizers#1612
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants