Rename all global symbols to avoid conflicts when used as a static library

[derekbruening]

Xref #975. We're using DR as a static library more and more these days, and the current solution for hiding its global symbols, running objcopy --localize-hidden, is a fragile post-link step that conflicts with other build steps such as lld's ICF and which can break when used in complex build configurations.

This issue is a proposal to rename all of DR's global symbols (i.e., everything not file-static) to dramatically reduce the risk of symbol conflicts when statically linked into large applications.

We would leave alone all of the symbols exported as part of the API. Unfortunately these have a variety of prefixes but at least they have prefixes:

$ nm lib64/release/libdynamorio_static.a | grep ' T ' | awk '{print $3}' | perl -p -e 's/^([a-zA-Z0-9]+)_.*/\1/' | sort | uniq -c
     10 decode
      5 disassemble
    311 dr
      3 dynamorio
      3 get
    184 instr
     35 instrlist
     93 opnd
     21 proc
     26 reg
      1 set
      1 __wrap_calloc
      1 __wrap_free
      1 __wrap_malloc
      1 __wrap_realloc
$ nm lib64/release/libdynamorio_static.a | grep ' T ' | awk '{print $3}' | grep '^[gs]et'
get_application_short_name
get_register_name
get_x86_mode
set_x86_mode

Moving on to the non-API routines, there are thousands of them, including functions and variables.
Some thoughts on renaming rules:

"dr_": a simple prefix, except today it means "exported for API" and we have
several instances of "dr_foo" exported and "foo" internally: e.g., internal "safe_read" and interface "dr_safe_read". So how do we rename the internal one?

"dri_": where "i" means internal.

Sthg specific to the module? "drinterp_xxx", "drvm_xxx", "drxl8_xxx", etc.

"dr__" or "_dr": these violate the C/C++ standards which say no double underscores anywhere, no leading single underscore plus capital letter anywhere, and no leading underscore in global symbols.

Could do a single underscore suffix: arch_init_(), clean_call_opt_exit_(),
add_dynamo_vm_area_(), executable_areas_. More of a minimal change.
It would mean "global but not in API". Could combine it with a "dr_" prefix.

[derekbruening]

We're de-prioritizing any large step of renaming all symbols, in favor of incremental tackling of likely-to-conflict symbols: mem{set,cpy,move} (xref #3315), perhaps a few others. For mem*, since we can't easily rename the implicit ones, it may be safe to have a statically-linked DR use compiler-provided versions. These should be re-entrant, and shared DR mostly just wants to avoid imports, rather than having a worry about transparency from non-re-entrant functions.

[derekbruening]

As mentioned for #3315 which removed memcpy and memset from static DR: there is a downside to having static DR use libc functions. They can be intercepted, such as by sanitizer tools. There are other interoperability issues with such tools, though, so we live with the potential problem of
the app overriding DR's memory function as the lesser of two evils, assuming symbol conflicts are more prevalent and serious. We can also try to rename DR's internal uses of functions like these to avoid interception.

[derekbruening]

If we remove the objcopy --localize-hidden step, we hit duplicate symbol errors:

[3/20] Linking CXX executable clients/bin64/opcode_mix_launcher
FAILED: clients/bin64/opcode_mix_launcher 
: && /usr/bin/c++   -Wl,--hash-style=both  -rdynamic -Wl,--as-needed  clients/drcachesim/CMakeFiles/opcode_mix_launcher.dir/tools/opcode_mix_launcher.cpp.o  -o clients/bin64/opcode_mix_launcher  clients/lib64/debug/libdrmemtrace_analyzer.a clients/lib64/debug/libdrmemtrace_opcode_mix.a clients/lib64/debug/libdrmemtrace_raw2trace.a ext/lib64/debug/libdrcovlib_static.a lib64/libdrfrontendlib.a lib64/debug/libdynamorio_static.a -lz clients/lib64/debug/libdirectory_iterator.a lib64/libdrfrontendlib.a lib64/libdrmemfuncs.a -lpthread ext/lib64/debug/libdrutil_static.a ext/lib64/debug/libdrx_static.a ext/lib64/debug/libdrreg_static.a ext/lib64/debug/libdrcontainers.a ext/lib64/debug/libdrmgr_static.a lib64/debug/libdynamorio.so lib64/libdrhelper.a -ldl && /usr/bin/objcopy --only-keep-debug clients/bin64/opcode_mix_launcher clients/bin64/opcode_mix_launcher.debug && /usr/bin/objcopy --add-gnu-debuglink=clients/bin64/opcode_mix_launcher.debug clients/bin64/opcode_mix_launcher && /usr/bin/strip -g -x clients/bin64/opcode_mix_launcher && :
lib64/debug/libdynamorio_static.a(libdynamorio_static.o): In function `open_syscall':
/home/bruening/dr/git/src/core/unix/os.c:4129: multiple definition of `open_syscall'
lib64/libdrfrontendlib.a(os.c.o):/home/bruening/dr/git/src/core/unix/os.c:4129: first defined here
lib64/debug/libdynamorio_static.a(libdynamorio_static.o): In function `os_open':
/home/bruening/dr/git/src/core/unix/os.c:4188: multiple definition of `os_open'
lib64/libdrfrontendlib.a(os.c.o):/home/bruening/dr/git/src/core/unix/os.c:4188: first defined here

So it looks like we may need to solve #1409 and separate out a library with module_get_platform() and os_read().

[derekbruening]

For renames: the plan is to rename only single-word or likely-to-conflict double-word symbols (where underscores divide words), and as mentioned above to not attempt to rename the world.

"d_r_" is another candidate prefix for renames.

[derekbruening]

The one-word global symbol names in the core library (the ones most likely to conflict) after removing memcpy and memset (this is with a local change to create a _nohide version for testing purposes):

$ nm lib64/debug/libdynamorio_static_nohide.a | egrep ' [A-TV-Z] [^_\.]+$'
000000000011f88d T crc32
000000000000fce0 D debugRegister
00000000001f3dff T decode
0000000000202099 T disassemble
0000000000082ba4 T dispatch
0000000000227852 T emulate
000000000000b9a0 D extensions
0000000000000008 C initstack
00000000001e1860 T loginst
00000000001e1a0a T logopnd
00000000001e1bbd T logtrace
000000000022ba6d T mangle
000000000011fb17 T MD5Final
000000000011f8f0 T MD5Init
000000000011f932 T MD5Update
00000000001b7f67 T memcmp
00000000001b7dab T memmove
000000000009f6d7 T notify
0000000000068ef8 R regparms
0000000000293136 T stackdump
0000000000001830 B stats
00000000001b8077 T strcasecmp
00000000001b7c26 T strchr
00000000001b7e2d T strcmp
00000000001b7bbf T strlen
00000000001b7d1f T strncat
00000000001b7ec1 T strncmp
00000000001b7ca4 T strncpy
00000000001b7c61 T strrchr
00000000001b7ff5 T strstr
00000000001b8115 T strtoul
00000000001b8057 T tolower
00000000001b7bf1 T wcslen

For the string routines, we have two choices: one is to mirror memcpy and make a "strfuncs" library that others can omit. The other is to mirror snprintf which is renamed to our_snprintf. The latter seems the better choice. It may be better long-term to explicitly call the our_str* (or _d_r_str*, however named) or we can minimize code churn by renamed via macro as is done for snprintf (partly b/c of Windows names complicating things).

As part of that we'll have to remove <string.h> from all core files, to avoid macros and intrinsics from bypassing our calls.

[derekbruening]

Another complication: what about __strncpy_chk and the other _chk versions,
which gcc emits calls to? Presumably if DR never calls the plain strncpy
gcc will never emit those _chk calls.

[derekbruening]

I'm going with d_r_ as the prefix for "global internal". See other ideas above, summarized here on initstack:

d_r_initstack
dri_initstack
dy_initstack
d_r_initstack_
__dr__initstack (technically reserved b/c contains double underscore)
_dr_initstack (technically reserved as global b/c starts w/ underscore)
dr__initstack (technically reserved b/c contains double underscore)
initstack_dr_