Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to have @rpath install_name instead of an absolute path #2121

Open
SoapGentoo opened this issue Jul 25, 2017 · 14 comments

Comments

@SoapGentoo
Copy link
Member

commented Jul 25, 2017

Description

Meson's RPATH and install_name handling are currently broken on macOS. Here's a toy example. Let foo be the library (aka the provider) and let bar be the executable (aka the consumer). For foo we have:
foo.cpp:

#include <iostream>

void foo_func()
{
	std::cout << "Hello World!\n";
}

foo.hpp:

void foo_func();

meson.build:

project('libfoo', 'cpp')

foo_lib = library('foo', 'foo.cpp', install : true)
install_headers('foo.hpp', subdir : 'foo')

import('pkgconfig').generate(
  libraries : foo_lib,
  version : '1.0',
  name : 'foo',
  filebase : 'foo',
  subdirs : 'foo',
  description : 'A foo library')

For bar we have:
bar.cpp:

#include <foo.hpp>

int main()
{
	foo_func();
	return 0;
}

meson.build:

project('barapp', 'cpp')

foo_dep = dependency('foo')

# build_rpath is used here for test dependencies
executable('bar', 'bar.cpp', dependencies : foo_dep, build_rpath : foo_dep.get_pkgconfig_variable('libdir'))

How to reproduce

in foo (the rm -r build at the end is crucial to reproduce the issue):

meson.py --prefix /Users/Soap/Desktop/PREFIX/ build . && ninja -C build -v install && rm -r build

in bar:

export PKG_CONFIG_PATH=/Users/Soap/Desktop/PREFIX/lib/pkgconfig
meson.py --prefix /Users/Soap/Desktop/PREFIX/ build . && ninja -C build -v
./build/bar

which then greets you with a message:

dyld: Library not loaded: /Users/Soap/Desktop/foo/build/libfoo.dylib
  Referenced from: /Users/Soap/Desktop/bar/./build/bar
  Reason: image not found

The actual problem

The culprit is how meson builds libraries on macOS. The linking line is currently:

c++  -o libfoo.dylib 'foo@sha/foo.cpp.o' -shared -install_name /Users/Soap/Desktop/foo/build/libfoo.dylib '-Wl,-rpath,$ORIGIN/'

which is incorrect. MacOS has a peculiar and very unusual way of linking, namely that consumers of a library inherit a provider's install_name. If you run otool -l on the library you get

          cmd LC_ID_DYLIB
      cmdsize 72
         name /Users/Soap/Desktop/foo/build/libfoo.dylib (offset 24)

where the name is clearly wrong, as it uses the builddir name. As an absolute minimum this should have been /Users/Soap/Desktop/PREFIX/lib/libfoo.dylib (but hold on, you still don't want this). This is why, when not running rm -r build in foo, the linking seems to work, because bar is still using the build dir lib, not the installed one.

Furthermore, another peculiarity of macOS is the fact that for RPATH of consumers to work, the provider has to play along. This is different from Linux, where RPATH/RUNPATH are purely a consumer problem, the provider does not have to know about consumers' RPATH behaviour. Thus, in order for RPATH to work for consumers, the install_name has to be @rpath/<libname>.dylib. In a quick and hacky way, doing this in foo:

c++  -o libfoo.dylib 'foo@sha/foo.cpp.o' -shared -install_name @rpath/libfoo.dylib '-Wl,-rpath,$ORIGIN/'

And then in bar (notice the missing colon : after -Wl,-rpath,):

c++  -o bar 'bar@exe/bar.cpp.o' -L/Users/Soap/Desktop/PREFIX/lib -lfoo -Wl,-rpath,/Users/Soap/Desktop/PREFIX/lib

and then ./bar gives me

Hello World!

Conclusion

This issue is caused by macOS' unconventional linking behaviour, and the fact that library providers have to be RPATH aware, which is unduly burdensome and complicates things. I believe the following to be the best line of operation:

  1. all (shared) libraries would use -install_name @rpath/<libname>.dylib unconditionally, purely to be consumed in an RPATH-capable way. While using an absolute install_name would make things a bit easier in the short run, this is short-sighted and a path full of pain. There are two reasons to avoid absolute install_names:
    1. reproducible builds, as this would code path/layout-dependent information into libraries
    2. it makes libraries non-relocatable, whereas relative RPATH-based builds are relocatable.
  2. all consumers (that is, other libraries and executables) should specify RPATHs in a relative way, where @loader_path is the proper analogue to $ORIGIN on Linux.

References

conda/conda-build#279
https://www.mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html
https://wincent.com/wiki/%40executable_path%2C_%40load_path_and_%40rpath
http://jorgen.tjer.no/post/2014/05/20/dt-rpath-ld-and-at-rpath-dyld/

@mojca mojca referenced this issue Nov 11, 2017
6 of 10 tasks complete
@mojca

This comment has been minimized.

Copy link

commented Nov 11, 2017

Getting this fixed is of critical importance for the ability to even consider using meson in our package manager on Mac. I would argue that meson should provide a configuration option to tell whether the final build should use relative or absolute paths. Not something that software developer themselves should worry about, but something that the people building software could specify (depending on whether they are building a bundled app or perhaps a package manager where location of files is fixed).

@ilovezfs

This comment has been minimized.

Copy link

commented Nov 28, 2017

This also impacts gobject-introspection files, which end up with the build path hard coded since they're taking their cues from the LC_ID_DYLIB. So even if you fix up the dylibs manually after installation, you still have broken gobject-introspection files.

@mojca

This comment has been minimized.

Copy link

commented Nov 28, 2017

See #2577 and #2618.

@siriobalmelli

This comment has been minimized.

Copy link

commented Feb 16, 2018

I'm running into this as well - did a precise write-up and opened a separate issue so as not to hijack this thread: #3077

@ryandesign

This comment has been minimized.

Copy link

commented Mar 8, 2018

While using an absolute install_name would make things a bit easier in the short run, this is short-sighted and a path full of pain.

Using an absolute install_name is what every other build system normally does on macOS. It works great. It's not relocatable, but those who want relocatable binaries should run install_name_tool afterward to fix things up. What meson is currently doing does not work. Please change it. We are starting to use meson for some ports in MacPorts, and each port has to individually fix up the install_name in each of the port's libraries and binaries. We don't want to do that, and we shouldn't have to.

@mojca

This comment has been minimized.

Copy link

commented Mar 13, 2018

I cannot find the exact location, but someone asked me how this issues is handled in CMake.

Please take a look at a hello-world example that I have put on https://github.com/mojca/test-meson-libraries.

In short: CMake would call install_name_tool again during make install.

@jpakkane

This comment has been minimized.

Copy link
Member

commented Apr 2, 2018

Based on the sample code, everyone would need to set an install_name property on all their targets. We can certainly add that but it is a bit imperfect. Is there a way to make this work correct automatically (say, 95% of the time)?

@jpakkane

This comment has been minimized.

Copy link
Member

commented Apr 2, 2018

As an example we could have a new option, say, b_installname which could be set to either preserve or absolute. For the latter value, if a target does not have install_name property set, its install_name will be changed to an absolute path. Otherwise the value specified (or the default value) is used.

@SoapGentoo

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2018

@jpakkane both CMake and libtool get it right? Set a temporary install_name and/or RPATH during build, and relink with correct absolute install_name during install?

@jpakkane

This comment has been minimized.

Copy link
Member

commented Apr 2, 2018

Not relinking, direct manipulation by install_name_tool but basically yes.

@SoapGentoo

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2018

@jpakkane this is somewhat related too: I've noticed that Meson doesn't remove RPATHs when installing, leaving the RPATHs in the binaries

@nirbheek nirbheek changed the title RPATH + install_name broken on macOS Add an option to have @rpath install_name instead of an absolute path Jun 18, 2018
@nirbheek nirbheek self-assigned this Jun 18, 2018
@nirbheek nirbheek added this to the meson-next milestone Jun 18, 2018
@RJVB

This comment has been minimized.

Copy link

commented Feb 8, 2019

Using an absolute install_name is what every other build system normally does on macOS.

Isn't that what you get anyway if you don't use the install_name option but just let the toolchain figure things out?

@smancill

This comment has been minimized.

Copy link

commented Feb 18, 2019

Using an absolute install_name is what every other build system normally does on macOS.

Well, no. CMake uses @rpath as the install_name directory by default since CMake 3.0.

https://cmake.org/cmake/help/latest/policy/CMP0042.html

@RJVB

This comment has been minimized.

Copy link

commented Feb 18, 2019

And the MacPorts CMake PortGroup sets -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.