private loader for safe system library use by DR and clients

[derekbruening]

From derek.br...@gmail.com on June 06, 2009 16:39:03

private loader that isolates private copy of libraries for use by DR and
client has many advantages:

• can have DR and client use same libc
• have entire, modern, distro-appropriate libc at disposal
• transparent
• do same thing on Windows and Linux
  on Windows, 2nd copy of kernel32 calls into single system ntdll copy
• lets us inject earlier
• can use for more than just libc:
  • libstdc++
  • symbol resolution using libelf/libdwarf in-process
  • other utility libs: anything "well-behaved" that can handle 2nd copy

the main alternative of static libc is problematic:

• difficult to find all-BSD-license libc
• mini-libc (just string routines) both hard to isolate and not
  sufficient for clients
• requiring users to build static PIC libc on their own from
  distro's sources too onerous
• want libstdc++, libelf, etc. as well

challenges of private loader:

• system calls
  • on Windows, we can redirect the dup kernel32's calls to ntdll to
    go through DR instead
  • on Linux, could redirect list of libc syscall wrappers to DR.
    for other embedded syscalls: leave it up to user to ensure not using
    lib routine that calls syscall that trips up DR?
• libraries that have global assumptions: absolute addresses, IPC, etc.
  we just won't support those

would solve:
issue #30/PR 200203: support for standard libraries
issue #46/PR 206369: libc independence
PR 352425: [linux] transparency: hide from dl_iterate_phdr()
issue #44/PR 243532: [API] symbol table lookup support

should enable:
issue #47/PR 204554: early injection on linux
PR 204587: earliest injection

still problematic:
issue #36: GLIBC_2.7 from sscanf
!HAVE_PROC_MAPS uses dl_iterate_phdr: but have alternative of VSI (PR 398083)

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=157

[derekbruening]

From qin.zhao@gmail.com on November 05, 2009 10:26:58

I am not sure a private load may solve the problem, or it may lead to other new problems
For example, malloc use brk and mmap to request memory from operating system.
If we use our loader, there are two malloc functions, one used by app and the other
used by DR and its client.
The two malloc may request memory using brk by turns, it is highly possible that they
interfere with each other and cause state corrupt.

[derekbruening]

From derek.br...@gmail.com on November 05, 2009 11:21:41

from my original notes, I pasted most of this above already:

challenges of private loader:

• global resources: main one is the brk: best to redirect malloc/free
  to DR's, just like we're doing w/ --wrap today
• system calls
  • on Windows, we can redirect the dup kernel32's calls to ntdll to
    go through DR instead
  • on Linux, could redirect list of libc syscall wrappers to DR.
    for other embedded syscalls: leave it up to user to ensure not using
    lib routine that calls syscall that trips up DR?
• libraries that have global assumptions: absolute addresses, IPC, etc.
  we just won't support those

[derekbruening]

From derek.br...@gmail.com on November 29, 2009 19:51:13

issue #157 : private Windows loader for safe system library use by DR and clients

This is a pretty big Windows-specific feature that attempts to provide a
solid initial implementation. The loader itself is fairly straightforward
but providing full isolation on Windows, with so many globally shared
resources pointed at by the PEB along with global assumptions, is not easy,
and this commit does not yet solve all the issues. I filed other cases to
cover future work.

This commit adds a fairly complete loader:

• locates by searching first client paths and then system paths
• loads and relocates
• processes imports
• handles forwarded exports
• calls entry points for process and thread attach and detach
• ref count and ordered list for proper unloading
• redirect Rtl*Heap, but for now only the core alloc/free routines
• redirect callbacks that kernel32 sets up for ntdll routines
• for FlsAlloc callbacks, using new code to run private library
  callbacks natively after app code call* targets them
• redirect some kernel32 routines that later we should replace
  w/ redirecting ntdll routines
• updated docs on use of libraries by clients

Testing:

• enabled stl_test on windows, tested on calc on xp, 2k, and vista
• tested stl_test with /MDd (with hardcoded SxS dir) => private
  msvcp80d.dll and msvcr80d.dll in addition to kernel32.dll
• tested kjrupnow's extended stl_test test on calc on xp
• tested use of dbghelp.dll for online symbol lookup, which brings in
  5 dependent libraries, on calc on xp, 2k, and vista

There are some improvements and features I'm not supporting for now which I
filed as separate Issues. I don't expect these to be a problem for initial
use, and we may never need these, but just in case I put the use of the
private loader under an on-by-default runtime option -private_loader so the
regular loader can be used if desired. issue #235 : redirect more of ntdll for more transparent private libraries:

• in particular, redirect Ldr*
• redirect more Rtl*Heap routines + redirect for other than peb->ProcessHeap
• redirect Rtl routines that call Rtl*Heap routines directly?
  see notes inside redirect_RtlReAllocateHeap()
• we'll redirect any additional routines as transparency issues come up
• handle a duplicate user32: for now giving up and using app's user32 under
  the assumption that clients using user32 are only doing so b/c it was
  pulled in for one or two small routines (e.g., on win2k
  dbghelp->version->lz32->user32) that don't allocate (much) memory or take
  other global actions. a duplicate user32 registers callbacks
  (KiUserCallbackDispatcher called USER32!__fnINOUTLPPOINT5 on calc) that
  the app then executes, which cause problems.
• try to load duplicate ntdll: has some complexities but may end up being
  the simplest solution to a lot of these issues
• add tests of clients (xref issue build: convert suite/ to cmake and ctest #65 ) that use dbghelp.dll and other libs issue support for advanced private Windows loader features #233 : advanced private Windows loader features:
• import by ordinal
• delay-load dlls
• bound imports
• import hint
• TLS (though expect only in .exe not .dll)
• read .manifest files and locate SxS C++ (msvcp) and C (msvcr) runtime libs
  (for now if clients compile statically (/MT) we're ok) issue nested try/except support #232 : nested try/except:
• then we can check readability of everything more easily: today
  not checking everything in the name of performance

This loader completely solves on Windows: issue #30 : support clients using standard libraries: STL in particular

This loader is a step toward: issue #234 : earliest Windows injection:

• use bootstrap loader w/ manual syscalls or ntdll binding to load DR
  itself with this private loader at very first APC point issue symbol table lookup support for clients #44 : symbol table lookup support for clients
• just need to add a platform-independent API and add link support for
  dbghelp.dll along with documentation on where to get it

Leaving this Issue open for the Linux loader. For the Windows loader all
future work is covered by other Issues.

[derekbruening]

From rnk@google.com on October 25, 2011 19:41:55

I'm closing this because Linux private loader is on by default now, and any issues that we encounter with it will fall under separate bugs.

Status: Fixed