Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Weak linking CMake module #47
Adapted from vtk#1511.
Adds a new experimental CMake module:
Reviewers, please note the major differences in this PR when compared to vtk#1511.
Also, includes a fix for when testing the
referenced this pull request
Jul 6, 2016
The goal of the original cmake file (in SimpleITK) was NOT to link again libpython on Linux and OSX as per distutils and Pypa recommendations. This sounds like you are trying to still link against the library?
I am not sure what the goal of this is now...
This I think is how I began down this issue by naively linking against the shared library:
Perhaps it would be work testing on the manylinux1 docker image too?
The logic change was on purpose, but clearly, it was a mistake if everything I read is true.
So, my main confusion stems from the fact that I could not replicate this behavior on Linux -- the linker always refuses to leave the symbols undefined. Is there a linker flag I should be using?
This sounds right to me, but I'm not sure what the point of bringing this up is? Neither of those scenarios are being tested in the test project. Should they be? Besides this tolerance for undefined symbols that I have yet to observe, is there anything we should be checking for that we're not?
What is your platform and what exactly are you trying to do to test if "undefined" symbols are allowed? The original CMake file seems to pass on all my linux (elf binaries) systems I have tests .
I may have been mistaken about what this CMake code and the test project was trying to do. And I don't know why the test is constructed this way as opposed to other permutations of executable, linked library and runtime loaded library.
I also don't know what problem or issue the HAS_SYMBOL_DEDUPE is trying to address.
I thought I was understanding the linker process pretty well now, but I am guess I have more to read.
I haven't actually tried that particular test code, but I was more referring to building actual Python extension modules. When trying to build a simple extension module, the link fails with undefined symbol errors on Linux. That is- when I don't link to any libpython.
The test project in this PR is based on my experiments here. There is a shared lib, L; a module lib, M; and the executable, E. E is linked against L. E also dynamically loads M. And M, itself uses code from L, but the main question is whether M should be linked against L. This test was supposed to emulate a typical example of building a python extension module. As an aside, I haven't been able to build M without linking it against L on Linux, I run into the exact same problems as I did with actual Python extension modules.
I figure besides library version mismatches, the only other potential source of problems with linking for both the module and the executable is that it may result in code duplication. If the library had some shared state, we should be able to observe the effects of such duplication by inspecting the shared state from the perspectives of E and M, and then comparing. If there is code duplication, the counting routines from each would have their own counters instead of sharing. If the loader coalesces the duplicate symbols, then they would share, which is the desired behavior. This is what I check for under the "SYMBOL_DEDUPING" name.
I'm going to check to see if I can get your simpler test code to compile and link. If so, then there's clearly something about how linking on Linux works that I'm not privy to.
Another specific case where not linking solved a problem was on OS X. With binaries of SimpleITK compiled against the system Python 2.7 with explicit linking, when these would create a load error when run on Anaconda or home brew. However, if the "-undefined dynamic_lookup" was used and not linked against libpython, then the binaries would work!
SimpleITK is not linking against libpython, on every linux system I have tested RH5,RH6,RH7, Cent OS5, Some Debian, Ubuntu 12, Ubuntu 14....
I'd be interested in a minimal example and the link line used. I'm thinking of starting tests with gcc command lines here.
Well, I've finally gotten extension modules to build and link on Linux as you and others describe -- without linking.
I have no idea what I'm doing differently, but I'm now a believer.
I want to try to incorporate this into my weak-linking-demo and see how that changes things. Then, I'll come back to this and make the necessary corrections.
Thanks for keeping me honest.