Using shared lib Conan packages is broken on OSX

[jondo2010]

I'm not sure what version of OSX was in mind when the documentation was written, but on my OSX machine running El Capitan, the Getting Started tutorial does not work at the point of linking to the shared library versions of Poco and OpenSSL.

This is because the shared library install name, and RPATH approach talked about in the documentation will not generally work. The only way the example will work as currently set up, is if you execute it from the bin/ folder. This doesn't work, for example:

$ ./bin/timer
dyld: Library not loaded: libPocoUtil.43.dylib
  Referenced from: /Users/johughes/conantest/build/./bin/timer
  Reason: image not found

Shared libraries on OSX have an "install name" that must match what is referenced in the binary.

Looking at libPocoCrypto:

otool -L bin/libPocoCrypto.43.dylib
bin/libPocoCrypto.43.dylib:
        libPocoCrypto.43.dylib (compatibility version 43.0.0, current version 43.0.0)
...

The install name is "libPocoCrypto.43.dylib" which means that the dynamic linker will ONLY ever be able to find it in the PWD of the parent process, regardless of where the actual binary is located.

[memsharded]

Thanks very much for your report.
Not very sure about the issue (my main OS is not OSX). To make sure:

• The dylibs have been copied to the local bin folder, where the executable has been created
• If you move to the bin folder, like cd bin and then ./timer does it work?
  If it works, I'd say the behavior is consistent. The problem with RPATH is that they don't work if you change from machine to machine where the paths will be different. So if you want to reuse binaries, it has to be disregarded, and use other paths (rather than RPATH) to locate the libraries.
  Maybe @lasote can give more insights.

[lasote]

I agree with @jondo2010 except a couple of things:

• I don't know where is the "RPATH approach talked about in the documentation" that doesn't fit with the reality. Please help me understand it to fix it.

• I usually set the install_name in the OSX packages, to "./" clearing the absolute paths so they can be reused in different machines. So no rpaths are involved, only the DYLD_LIBRARY_PATH paths, but of course can be errors in some packages. What you showed with Poco libraries of the getting started sounds correct and consistent to me. But again, those are just recipes, is the package creator choice how to handle the rpaths, Conan doesn't force any special use.

[jondo2010]

@memsharded Yes, the dylibs have been copied to the local bin folder using [import] in the conanfile. Running cd bin && ./timer does work.

It's simply that this setup is super less than ideal, to the point of being not useful. I have to cd to the bin folder to run my binary every time?

With regards to the docs, @lasote, at the bottom of http://docs.conan.io/en/latest/manage_deps/conanfile_txt.html where it talks about rpaths:

So, for OSX, conan requires dylibs to be built having an rpath with only the name of the required library (just the name, without path).

I would argue that a more robust approach is as follows:

1. When shared libs on OSX are installed in .conan, the install names are modified to the absolute location.
2. When shared libs on OSX are copied with [import], the install names are modified to be @rpath/libname.dylib.

This would give the consumer much better flexibility in how their binaries link to the shared libraries. For case 1, they would add the absolute path to the folder containing the library to the binaries' RPATH (e.g. ~/.conan/data/Boost/..).
For case 2, where the .dylib sits in the local bin folder, simply adding an RPATH of @executable_path/. will work. The bin folder is completely relocatable in this case.

The install_namess can be modified using https://pypi.python.org/pypi/macholib

As a further example, let's take a look at how a homebrew-installed lib does it:

$ otool -L /usr/local/lib/libboost_log.dylib
/usr/local/lib/libboost_log.dylib:
        /usr/local/opt/boost/lib/libboost_log.dylib (compatibility version 0.0.0, current version 0.0.0)
        @loader_path/libboost_date_time.dylib (compatibility version 0.0.0, current version 0.0.0)
        @loader_path/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
        @loader_path/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
        @loader_path/libboost_regex.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)

libboost_log.dylib is given an install_name of the absolute path where it is located, and all other dependent libs from the same package are linked relative using @loader_path.

[lasote]

@jondo2010 thanks for your long explanation. Yes, modify dynamically the rpaths is something we thought about but we didn't get too much feedback about it (so thanks again).

I think we should at least:

• clarify the docs to explain another approaches and how to do it.

• try the python lib you mentioned and think again about rpaths wildcards convenience.

[jcar87]

I agree with @jondo2010 as well, I think that the libraries on macOS should have @rpath prepended to them to avoid issues.

One of the potential pitfalls of naming the libraries with: libsomething.dylib instead of @rpath/libsomething.dylib is that executables linked against them may end up loading the library from a different location (/usr/lib, /usr/local/lib) rather than the one intended. If a conanfile.py maintainer fails to "import" them, the test_package may be building and running successfully but actually loading a different library at run-time, just out of pure coincidence. This will typically happen if on that system, macports or homebrew are installed.

Not only that, but conan's convention as mentioned in the documentation,
The only limitation of this convention is that dylibs have to be copied to the folder of our executable, just like dll files in windows.
is not particularly a light limitation. This means that the convention of placing binaries in/bin and libraries in /lib will no work without calls to install_name_tool. And same goes if the application is an actual OSX bundle (where libraries are to be placed in the /Frameworks directory).

Most libraries that provide precompiled binaries (Qt5 being the biggest example, but also the pre-compiled OSX tbb binaries) do provide them with "@rpath" prepended, and these place nicely in all scenarios. Things like macdeployqt or cmake's own bundle fix up will struggle finding the libraries at the correct locations.

[lasote]

Thanks @jcar87.

To clarify your use case, you build your shared libraries with "@rpath" prefix, then, Are you importing the libraries near the executable and adjusting DYLD_LIBRARY_PATH to locate them in a subdirectory? Would you need to alter the rpaths with an external tool?

Thanks

[jcar87]

Hi @lasote,
No, I don't alter DYLD_LIBRARY_PATH. I know that on Linux it is not uncommon for "standalone" packages to have a "launcher" script that alters the library path to point to subdirectories relative to the script/executable.

On Mac it's more common to alter the executable itself and set the rpath to something relative to the executable, e.g. @executable_path/../lib (if the "package" is of the bin/lib form), or @executable_path/../../Frameworks if the package is a OSX app bundle.

The alternative, if all the imported libraries (through conan) have "naked" install names, would be to actually set the rpath or modify the install names of ALL libraries. I think the "libraries on the same directory" only works for the same working directory. When launching an application from an app bundle, the working directory is the ".app" dir, I think, and the executable is two levels down.

I am trying to gather all information on this, as this is all due to different behaviour of the dynamic loader on Linux and macOS. It doesn't help that "@rpath" wasn't a thing prior to 10.5 either.

[lasote]

It makes sense. Thanks!

[lasote]

So, the actions to perform in conan could be:

1. Try the https://pypi.python.org/pypi/macholib and probably analyze alternatives for linux. Provide a conan tool to alter the executables/libraries rpaths.
2. Improve the docs with:
   • Better explanation about the rpaths in OSX and how to proceed (not only the current "clear the rpaths and copy to current")
   • An example of how to create the shared libraries in OSX with the @rpath prefis and how to generate some consumer executables declaring the rpath (importing the libraries to a custom directory)

I'm going to add this to the next 0.26 milestone.

[jcar87]

I was trying to come up with a better explanation of how OSX works and turns out Kitware already did it for me some time ago:
https://blog.kitware.com/upcoming-in-cmake-2-8-12-osx-rpath-support/

Also, one of the comments there summarizes the situation quite well. I'm quoting:

For install names, we have these options:

1. full path/filename
2. filename only
3. @rpath/filename
4. @loader_path/filename
5. @executable_path/filename

Option 1 is really only used by OSX system libraries, where they are guaranteed to be found at a certain location.
Option 2 is used by any library where you expect it to be installed in one of the following directories: $(HOME)/lib, /usr/local/lib, /lib, /usr/lib (this is the default fallback library path, unless overridden by the environment variable DYLD_LIBRARY_PATH)
Option 3 is what's in style these days, and cmake usually handles these cases quite well, I think by default since a certain version.

Options 3 and 4 used to be common place before apple introduced the "@rpath" prefix in 10.5.

As for Conan, we have to take into account a few things.

• Currently the Conan documentation states that all dylib files are to be placed alongside the executable. But there is nothing in the documentation of dyld to suggest that this will actually work. However, the manpages for dlopen do suggest that the current working directory is in the search path and that's maybe what is happening. Note that because of this limitation, any application linked with conan-delivered libraries with naked install names will only run if all files (executable and libraries) are in the same folder, and the libraries are in the working directory, or if those libraries are moved to any of the directories in the fall-back search path (or if DYLD_LIBRARY_PATH is set correctly, but this is not very mac friendly and can actually cause problems loading libraries-that-depend-on-other-libraries).

• Another very potential issue of using naked install names is that if I have download a package with, let's say, "libz.dylib" from conan, and my executables are linked against it, unless I explicitly guarantee that the conan one can be found by the dynamic loader search path, it can very well be loading /usr/lib/libz.dylib from the fallback path. That file does exist and is installed by Apple, and it is potentially a different version that the conan one. It can get to the point where the application runs, loads the wrong library, and crashes inexplicably, and these scenarios are really hard to diagnose (unlike ldd on Linux, otool -L doesn't tell you where it will be loading libraries from).

• CMake produces binaries with different install names / rpaths in two different scenarios: "make" and "make install". This is what cmake calls the "build tree" and the "install tree". The build tree is usually "polluted" with full, hardcoded paths to all library dependencies, to guarantee that developers will always be able to run their stuff. Build tree is not meant nor guaranteed to be relocatable. The install tree is cleaner, both on Linux (no paths to the local build machines), and on OSX. I've seen many conan recipes that are built using CMake but do not use the install tree (they don't build the install target), and do not override CMAKE_SKIP_RPATH or CMAKE_BUILD_WITH_INSTALL_RPATH (the recipes that patch the root CMakeLists to call conan_basic_setup would be safe). It is possible that libraries deployed that way still keep references to hardcoded full paths of the local build machines. On Linux this isn't a big issue, but on Mac this is bad. Even if the install name was fine (@rpath/libName.dylib), if there is any binary with a RPATH that points to a fixed location that eventually gets included in a signed OSX App bundle, it will fail the gatekeeper validation. And the only way to diagnose that problem is by reading the system log on the machine where it fails.

• The ideal way, which is option 2, is to prepend @rpath to the install names of libraries, but you will also need the executables to set their RPATH to the paths where those libraries will be found. So this places the 'problem' at the executable level.
  In CMake you can build like this:
  CMAKE_MACOSX_RPATH = ON (to prepend @rpath to the install name)
  CMAKE_INSTALL_RPATH = "/path/to/libs"
  The second will add that directory to the executables as well. An absolute path wouldn't make sense for relocatable libraries, but a relative path would, e.g. @executable_path/../lib. Which should take care of it.

However, some libraries that use CMake already do this a different way (e.g. OpenCV, their install names on mac are /lib/opencv_module.dylib, which makes them impossible to use... !