SEGFAULT when using C11 threading primitives

[neildick]

Hi,
I'm playing around with TSAN and the C11 threading primitives in <threads.h>. If I modify one of the example data-race programs I get a segfault in both clang and GCC when I run with TSAN enabled. This is using Debian 10, clang version 7.0.1-8

#include <threads.h>
#include <stdio.h>

int Global;

int Thread1(void *x) {
        Global++;
        thrd_exit(0);
}

int Thread2(void *x) {
        Global--;
        thrd_exit(0);
}

int main() {
        thrd_t t[2];
        thrd_create(&t[0], Thread1, NULL);
        thrd_create(&t[1], Thread2, NULL);
        thrd_join(t[0], NULL);
        thrd_join(t[1], NULL);
}

Compiled with
clang -fsanitize=thread -g -O1 new-main.c -o c11

outputs

neil@debian:~/scratch$ ./c11
ThreadSanitizer:DEADLYSIGNAL
==1549==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00000048609b bp 0x000000000000 sp 0x7f9ea72bf328 T1550)
==1549==The signal is caused by a WRITE memory access.
==1549==Hint: address points to the zero page.
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.

GDB backtrace

Reading symbols from c11...done.
(gdb) r
Starting program: /home/neil/scratch/c11
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff5dff700 (LWP 1558)]

Thread 2 "c11" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff5dff700 (LWP 1558)]
0x000000000048609b in __tsan_func_entry ()
(gdb) bt
#0  0x000000000048609b in __tsan_func_entry ()
#1  0x00000000004aebbb in Thread1 (x=<optimized out>) at new-main.c:6
#2  0x00007ffff7faf0a2 in start_thread (arg=<optimized out>) at pthread_create.c:483     #3  0x00007ffff7d334cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 

GDB Dissassembly

(gdb) disass
Dump of assembler code for function __tsan_func_entry:
   0x0000000000486020 <+0>:     mov    %fs:0xfffffffffffc0480,%rax
   0x0000000000486029 <+9>:     add    $0x1,%rax
   0x000000000048602d <+13>:    mov    %rax,%fs:0xfffffffffffc0480
   0x0000000000486036 <+22>:    mov    %rax,%rcx
   0x0000000000486039 <+25>:    shr    $0x2a,%rcx
   0x000000000048603d <+29>:    and    $0x7,%cl
   0x0000000000486040 <+32>:    mov    $0xffffffffffffc000,%rdx
   0x0000000000486047 <+39>:    shl    %cl,%rdx
   0x000000000048604a <+42>:    not    %rdx
   0x000000000048604d <+45>:    movabs $0x3ffffffffff,%rcx
   0x0000000000486057 <+55>:    and    %rax,%rcx
   0x000000000048605a <+58>:    and    %rdx,%rcx
   0x000000000048605d <+61>:    test   $0x1fff,%eax
   0x0000000000486062 <+66>:    je     0x4860ac <__tsan_func_entry+140>
   0x0000000000486064 <+68>:    shr    $0x32,%rax
   0x0000000000486068 <+72>:    and    $0x1fff,%eax
   0x000000000048606d <+77>:    imul   $0x1130000,%rax,%rax
   0x0000000000486074 <+84>:    movabs $0x600000000000,%rdx
   0x000000000048607e <+94>:    or     %rax,%rdx
   0x0000000000486081 <+97>:    movabs $0x2000000000000000,%rax
   0x000000000048608b <+107>:   or     %rdi,%rax
   0x000000000048608e <+110>:   mov    %rax,(%rdx,%rcx,8)
   0x0000000000486092 <+114>:   mov    %fs:0xfffffffffffc0540,%rax
=> 0x000000000048609b <+123>:   mov    %rdi,(%rax)
   0x000000000048609e <+126>:   add    $0x8,%rax
   0x00000000004860a2 <+130>:   mov    %rax,%fs:0xfffffffffffc0540
   0x00000000004860ab <+139>:   retq
   0x00000000004860ac <+140>:   sub    $0x400,%rsp
   0x00000000004860b3 <+147>:   callq  0x490720 <__tsan_trace_switch_thunk>
   0x00000000004860b8 <+152>:   add    $0x400,%rsp
   0x00000000004860bf <+159>:   jmp    0x486064 <__tsan_func_entry+68>
End of assembler dump.

[kcc]

Clang 7 is very old, the master is at 11.0.0 right now.
On master your test case works for me.
If you can reproduce the problem with clang master, please reopen.

[rhoot]

I'm seeing this exact thing both with clang 11.0.0 and gcc 10.2.1. With clang it always happens up, but with gcc it only shows up if my thread does work.

$ clang --version
clang version 11.0.0 (Fedora 11.0.0-2.fc33)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

#include <threads.h>

static int s_foo;

static int thread(void* p)
{
	s_foo = 1; // Without this it won't SIGSEGV with gcc
	return 0;
}

int main(int argc, char** argv)
{
	thrd_t thrd;
	thrd_create(&thrd, &thread, NULL);
	thrd_join(thrd, NULL);
}

Compiled with: clang -g -O0 -fsanitize=thread -o test test.c

Outputs:

ThreadSanitizer:DEADLYSIGNAL
==69890==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0000004bf3fc bp 0x7fbadcebf250 sp 0x7fbadcebf238 T69891)
==69890==The signal is caused by a WRITE memory access.
==69890==Hint: address points to the zero page.
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.

Exact same backtrace and instruction as the original report. But with symbols:

(gdb) bt
#0  0x00007ffff7509df0 in __tsan::FuncEntry (pc=140737340572874, thr=0x7ffff50bfac0) at ../../../../libsanitizer/tsan/tsan_rtl.cpp:1025
#1  __tsan_func_entry (pc=0x7ffff73104ca <start_thread+442>) at ../../../../libsanitizer/tsan/tsan_interface_inl.h:103
#2  0x000000000040119f in thread (p=0x0) at test.c:7
#3  0x00007ffff73104ca in start_thread () from /lib64/libpthread.so.0
#4  0x00007ffff723db53 in clone () from /lib64/libc.so.6

(gdb) list
1020	  DCHECK_LT(thr->shadow_stack_pos, thr->shadow_stack_end);
1021	#else
1022	  if (thr->shadow_stack_pos == thr->shadow_stack_end)
1023	    GrowShadowStack(thr);
1024	#endif
1025	  thr->shadow_stack_pos[0] = pc;
1026	  thr->shadow_stack_pos++;
1027	}
1028	
1029	ALWAYS_INLINE USED

(gdb) p thr->shadow_stack_pos
$2 = (__sanitizer::uptr *) 0x0

[dvyukov]

Sanitizers don't have interceptors for C threading primitives, and somehow they don't call pthread functions (or call via internal uninterceptable symbols). If that's the case, I guess sanitizers would need interceptors for C threading primitives. However, we somehow need to make sure we don't intercept both C and pthread primitives for the same function.

[tavplubix]

Hi, exactly the same segfault is reproducing in our CI with Clang 13.0.1 and it does not seem to be related to C11 threading primitives: ClickHouse/ClickHouse#36547 (comment)