Init-order checking is dlopen-hostile

[ramosian-glider]

Originally reported on Google Code with ID 178

Currently, when we call dlopen(), __asan_before_dynamic_init()
poisons all the dynamically initialized globals (except those in the current module).
Some of them may be concurrently accessed by other threads in the program. Thus, we
have data races on shadow memory and false positives:

$ cat tmp/init-order/dlopen/a.cc
#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

int foo() {
  return 42;
}
int global = foo();

void *global_poller(void *arg) {
  fprintf(stderr, "global poller started!\n");
  while (true) {
    if (global != 42) {
      break;
    }
  }
  fprintf(stderr, "global poller exiting!\n");
  return 0;
}

int main() {
  pthread_t p;
  pthread_create(&p, 0, global_poller, 0);
  fprintf(stderr, "global poller created!\n");
  if (0 == dlopen("b.so", RTLD_NOW)) {
    fprintf(stderr, "%s\n", dlerror());
  }
  pthread_join(p, 0);
  return 0;
}

$ cat tmp/init-order/dlopen/b.cc 
#include <stdio.h>
#include <unistd.h>

int slow_init() {
  fprintf(stderr, "slow init starting!\n");
  sleep(1);
  fprintf(stderr, "slow init exiting!\n");
  return 42;
}

int slowly_init_glob = slow_init();

$ ./bin/clang++ -fsanitize=address tmp/init-order/dlopen/b.cc -fPIC -shared -o b.so
$ ./bin/clang++ -fsanitize=address tmp/init-order/dlopen/a.cc
$ ASAN_OPTIONS=check_initialization_order=true LD_LIBRARY_PATH=. ./a.out 
global poller created!
slow init starting!
global poller started!
=================================================================
==12341==ERROR: AddressSanitizer: initialization-order-fiasco
<....>
0x00000105f500 is located 0 bytes inside of global variable 'global' from 'tmp/init-order/dlopen/a.cc'

Reported by samsonov@google.com on 2013-04-15 12:49:15

[ramosian-glider]

LLVM revision r179843 should fix this testcase. It changes the default init-order checker
mode to the one that _accepts_ access to already initialized globals. Note, that this
change is required for, e.g. testing LLVM itself: when a certain global is initialized,
it adds itself to a global registry, assuming it can be safely accessed during the
following initialization. I've seen at least one other project that uses the same pattern.

Still, we may lose a lot of potential bugs in this way :(
My suggestion is to use "strict-init-order" flag for codebases that are known to not
use such machinery, and fight with described dlopen() issue by some of:
1) intercept pthread_create() and/or dlopen() and stop init-order checking when they
are called.
2) add an interface function __asan_stop_init_order_checking(). It can be called at
the beginning of main().
3) use nifty hacks: if we've dlopened() a .so, find out where its globals are located,
and poison only them during module initialization (ew-w-w).

Reported by samsonov@google.com on 2013-04-19 08:56:58

[ramosian-glider]

#1.a is probably the simplest (stop once we have more than 1 thread). Although I can
imagine a situation where a thread is started from a module initializer. Grrr. 

Reported by konstantin.s.serebryany on 2013-04-19 14:08:34

[ramosian-glider]

I've implemented #1.a in LLVM r180106.

Reported by samsonov@google.com on 2013-04-23 13:59:40

[ramosian-glider]

can we close this? 

Reported by konstantin.s.serebryany on 2014-01-29 13:24:25

[ramosian-glider]

Reported by samsonov@google.com on 2014-01-29 14:54:25

• Status changed: Fixed

[ramosian-glider]

FTR the problem description provided here is a bit misleading. strict init-order checker
is *both* multithreading- and dlopen()-hostile: we can end up with false positives
both if use dlopen() in a single-threaded program, and if we call spawn new threads
during initialization, but don't call dlopen(). That is, we need to disable init-order
checker both in pthread_create interceptor (was done in r180106) and in dlopen interceptor
(this is addressed in r230288).

Reported by samsonov@google.com on 2015-02-24 00:41:48

[ramosian-glider]

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:13:41

• Labels added: ProjectAddressSanitizer