re-add support for building as a static library #975
Comments
|
From rnk@google.com on November 26, 2012 11:12:23 I've started working on this for Linux. I'd like to punt making it work on Windows for now. Status: Started |
|
From rnk@google.com on November 26, 2012 12:38:55 For plain DR, the static library support seems pretty straightforward.
We have to make some decisions about client support, though. Clients could always be linked statically, in which case I see two ways forward:
My preference is to implement 1 now since it's the least work and our existing clients are all compatible with it. Later, when we have a use case for linking in two clients and deciding which one to use, we can implement another strategy, or go back to being a shared library. Another concern is that we have lots of checks for clients based on client_lib being empty or not. We could do something like setting client_lib to ";0;", in which case DR may do something close to the right thing. We'd have to make it not add the exe to dynamo_areas. |
|
From bruen...@google.com on November 26, 2012 14:07:57
this may be too simplistic: there are a bunch of tests of whether in DR. though mostly what I'm thinking of are DR-owned memory tests, and this is just the DR library. we should go through all the tests and make sure they're ok.
No, they're not: DR marks .data as read-only at init time. This will break the app. |
|
From rnk@google.com on November 27, 2012 08:27:20 Another problem: we leak hidden symbols to the application. For example, our versions or __errno_location and strcpy will preempt the ones the app expects in libc. Maybe before we create the archive, we can use "ld -r" to create a single object file (dynamorio.o), and then use some linker options to hide non-exported symbols. |
|
From bruen...@google.com on November 27, 2012 09:13:53
No, let's design a cross-platform solution here. So long as we avoid any ELF-specific design decisions and implementation hacks that rely on toolchain-specific tricks, it shouldn't be much more work to make it work on Windows. |
|
From rnk@google.com on December 05, 2012 11:21:22 To avoid leaking symbols into the application, it looks like we can use a combination of "ld -r" and "objcopy --localize-hidden" in a post build hook on dynamorio. I'm not sure what the equivalent is for Windows. |
|
From rnk@google.com on December 05, 2012 11:21:40 I meant to link the stackoverflow article I found: http://stackoverflow.com/questions/393980/restricting-symbols-in-a-linux-static-library |
|
From bruen...@google.com on December 06, 2012 08:34:05 for taking over on windows if we assume it's compiled with MSVC we can use .CRT$XCU |
|
From bruen...@google.com on August 02, 2013 07:14:35 some notes from a discussion that never made their way into this issue: what about private libs? do we need separate static copy or just interp vs yes, we wrap malloc. can wrap via linker flag: but that affects whole app.
Owner: peter.goodman |
|
We're re-reviving this! My proposal is to throw out the top-level STATIC_LIBRARY build type and instead have a side-by-side dynamorio_static library target that we create in every build, just like we do with the extension libraries. This will make it easier to maintain and test. |
|
Since it's a pain to set DR runtime options, we should just commit to -code_api being on by default for dynamorio_static. |
|
I'm leaning toward combining proposals 1 and 2 above: we enable -code_api and assume that client code could be run at any time, even if there's no dr_init, but we also look for and call dr_init and dr_client_main (aside: should we drop dr_init for static to reduce collision risk with the app?) |
|
Further issues:
|
|
I wish github had an auto-link feature other than on closing an issue. Here is what I've done: commit 4f70937 commit 8db6180 commit c268019 |
|
Other missing pieces here are documenting how to use dynamorio_static (incl init-time module differences) and adding an auto-takeover test (and implementing |
|
For the client options: above we just said that the app could pass them to dr_client_main. Except it's not easy to do so for clients written separately from the app w/o some convention on a global var or env var to pass the string, and then somebody has to parse it: so better for instrument.c to centralize the parsing. The code inside DR that parses options has a fixed buffer size. There could be tons of options added to our tracer? Maybe better as exported symbol from rest of app instead of env var? So DR would use dlsym to look it up, for static? I'm going with something simple for now, and we can change it later if we need to. Leaning toward just using the existing -client_lib and ignoring the path, b/c app might want to set DR ops too (-disable_traces or sthg) and can do that in same env var. |
|
For the app setting DYNAMORIO_OPTIONS, we have a problem: on UNIX we cache environ in our constructor. If the app then calls setenv() and DYNAMORIO_OPTIONS was not already in the environment, it will make a copy of the env block, leaving our our_environ cached pointer pointing to the originals and thus not seeing the app's change. The plan is to eliminate the cache for static builds. |
|
To help avoid malloc mid-run, yet allow it at init and exit, as we're doing with drmemtrace (#2006), I'm adding a feature where a client library can request that the private loader, in debug build, complain if malloc & co. are called outside of init or exit. |
|
To elaborate: drmemtrace's use of C++ and droption incurs malloc and free calls at init time (libstdc++ lib init, droption static initializers, droption_parser_t::parse_argv) and exit time (droption). We would have to rewrite the tracer in C and not use droption to avoid these. |
Adds a feature where a client library can request that the private loader complain if malloc & co. are called at any time other than process init or exit, to help clients that want to support being linked statically with the app. Fixes drcachesim to use placement new for its offline custom module data allocations. Issue: #975, #2006
|
I want to avoid changing the hot redirect_malloc() path so I'm using a var decl to request these static import checks. Yet, the checks fire in drcachesim with -use_physical, so we need a way to disable them dynanically. We can't switch from the var decl to pure runtime b/c client code runs too |
Adds a feature where a client library can request that the private loader complain if malloc & co. are called at any time other than process init or exit, to help clients that want to support being linked statically with the app. Because it has to be early, the feature is triggered by a variable declaration DR_DISALLOW_UNSAFE_STATIC. It can be overridden dynamically by a new API routine dr_allow_unsafe_static_behavior(). Fixes drcachesim to use placement new for its offline custom module data allocations. Issue: #975, #2006
Adds support for calling dr_app_setup();dr_app_stop_and_cleanup() with no start in between. This is useful to use DR as a decode/encode library when it's statically linked and also used for instrumentation, as that setup precludes using drdecodelib, which relies on redirecting heap allocation via name redirection. Add a test to api.static_noclient. Issue: #975
Adds support for calling dr_app_setup();dr_app_stop_and_cleanup() with no start in between. This is useful to use DR as a decode/encode library when it's statically linked and also used for instrumentation, as that setup precludes using drdecodelib, which relies on redirecting heap allocation via name redirection. Adds a test to api.static_noclient. Unfortunately this hits #2040 on Windows and we disable it there. Issue: #975
The static DR libs are quite large and an enclosing project (such as Dr. Memory) may not want to include them. We add a new variable DO_DR_INSTALL_STATIC_DR here to control this behavior. Issue: #975
Adds dr_standalone_exit() with support for subsequent DR use, including full attach. This is a better solution than setup;detach support added in de99d45 for the use case of performing DR decode/encode with DR statically linked (which precludes using drdecodelib), followed by a separate use of DR for instrumentation. Updates the test of this use case in api.static_noclient. Issue: #975
Adds dr_standalone_exit() with support for subsequent DR use, including full attach. This is a better solution than setup;detach support added in de99d45 for the use case of performing DR decode/encode with DR statically linked (which precludes using drdecodelib), followed by a separate use of DR for instrumentation. Updates the test of this use case in api.static_noclient. Issue: #975
Resets a flag used to auto-initialize when heap routines are called without an explicit dr_standalone_init(), enable drdecode and other use cases. Without this reset, a prior dr_standalone_exit() can cause a subsequent crash when using implicit-initialization interfaces. Issue: #975
Resets a flag used to auto-initialize when heap routines are called without an explicit dr_standalone_init(), enable drdecode and other use cases. Without this reset, a prior dr_standalone_exit() can cause a subsequent crash when using implicit-initialization interfaces. Issue: #975
A std::unordered_set, even using dr_allocator_t, raises transparency risks when statically linked on Windows (from lock functions and other non-allocator resources). We thus create our own resource-isolated class to track GPR register inclusion for drmemtrace elision optimizations. Adds unit tests of the set class to burst_traceopts. Further tested manually by ensuring the same number of elided addresses is seen as with std::unordered_set in the burst_traceopts test: [drmemtrace]: Reconstructed 622 elided addresses. Even further tested by ensuring that raw2trace using the new set correctly operates on a raw file that was created using std::unordered_set. Issue: #975, #4403
A std::unordered_set, even using dr_allocator_t, raises transparency risks when statically linked on Windows (from lock functions and other non-allocator resources). We thus create our own resource-isolated class to track GPR register inclusion for drmemtrace elision optimizations. Adds unit tests of the set class, which are linked into burst_traceopts. Further tested manually by ensuring the same number of elided addresses is seen as with std::unordered_set in the burst_traceopts test: [drmemtrace]: Reconstructed 622 elided addresses. Even further tested by ensuring that raw2trace using the new set correctly operates on a raw file that was created using std::unordered_set. Issue: #975, #4403
From bruen...@google.com on November 13, 2012 14:24:59
long, long ago in a galaxy far, far away, DR supported being built as a static library. we now want to revive that support.
issues we need to address include:
Original issue: http://code.google.com/p/dynamorio/issues/detail?id=975
The text was updated successfully, but these errors were encountered: