Use patchelf to set RPATH

[sjackman]

Nicolas Hillegeer wrote...

Hey!

I just tried to post an issue to the issue tracker but it seems it won't budge, so I'll just paste my findings over to you:

NOTE: this issue might be double-posted, I think there was some issue with github so it didn't appear for me...

I really wanted ag, the silver searcher and I really wanted it through homebrew because I like pretty colours and upgrades, and dependency building, and rainbows.

I basically installed with

git clone git://github.com/Homebrew/linuxbrew.git
sudo mv linuxbrew /opt/homebrew
sudo chown `whoami`:`whoami` /opt/homebrew -R
edit [.zshrc/.bashrc/...] and add /opt/homebrew/bin in front of $PATH

So, that left me with three tiny hacks to get it to work:

1. Setup more environment variables in .zshrc, which basically comes down to

export PATH="/opt/homebrew/bin:$PATH"                  # already done earlier
export HOMEBREW_PREFIX="/opt/homebrew"
export HOMEBREW_REPOSITORY="/opt/homebrew"
export HOMEBREW_CELAR="/opt/homebrew/Cellar"
export HOMEBREW_CACHE="/opt/homebrew/hbcache"

2. to fix brew doctor, it seems to assume mktemp is in/usr/bin/mktemp, on debian it's in /bin/mktemp

Success!!! ag installs, drags in pcre kicking and screaming, and I am the proud father of a new compiled silver searcher.

But wait... it says that it can't find libpcre.so.1 when it runs. But I do have it, it got installed to /opt/homebrew/lib as part of the dependency resolving phase. What do we do?

3. The first reflex would be to add /opt/homebrew/lib to LD_LIBRARY_PATH. Which would probably cause weird troubles for users down the road. There are some articles out there detailing the pitfalls of LD_LIBRARY_PATH. Especially on a linux system such as debian, I just wouldn't do it.

But, is there a way to change the LD_LIBRARY_PATH for just one executable? Well yea, we could write a wrapper script! There are a lot of packages that do that. And it's bad, but not quite as evil as changing LD_LIBRARY_PATH. So, I could hack homebrew to have it generate wrappers instead of symlinks.

But that's so dirty...

And I'm so lazy...

And I don't know ruby...

So I bust out google-fu and find that in fact, there is a solution that works for many unices. Basically, every executable has the possibility to have a RPATH and a RUNPATH. It's basically exactly what we want, the runtime linker will look first in RPATH, then in the normal path and last but not least in RUNPATH. At least that's my understanding.

But I find that you can set this variable at link-time, with a switch to ld or gcc. :(

That would mean hacking homebrew again.

I google once more before giving up and I find the venerable patchelf. Which... patches elfs to have a different RUNPATH. Exactly what we need.

cd /opt/homebrew/bin
patchelf --set-rpath '$ORIGIN/../lib' ag                   # $ORIGIN refers to the current directory, don't evaluate it, use the single quotes

Does it work? Why yes, it works. Now I can run homebrew on my debian wheezy. Nice.

[aktau]

Ok forgot to mention that I also had to adjust ENV.rb to look for gcc and g++ instead of gcc-4.2 and g++-4.2. Seems to work nicely.

[sjackman]

@aktau The compiler name issue (gcc/g++ vs gcc-4.2/g++-4.2) is now fixed.

[sjackman]

@aktau Hi Nicolas,

Homebrew is usually installed in /usr/local, and on OS X /usr/local/lib is in the default library search path. When I install Homebrew in my home directory, I set LD_LIBRARY_PATH to $HOME/local/lib.

On Linux, /usr/local/lib is not in the default library search path. One simple solution is to add the Homebrew library path to the ld.so config file, /etc/ld.so.conf.

Cheers,
Shaun

[aktau]

Actually, we could also just set it as a flag to gcc if and when that compiler is used, to obviate the need for patchelf:

LDFLAGS += -Wl,-rpath,/path/to/homebrew/lib

Actually I'm not entirely sure if the -Wl, is necessary when it's put in LDFLAGS...

[sjackman]

This issue affects the master branch of Homebrew just as well as Linuxbrew. Would you open an issue at http://github.com/mxcl/homebrew? It would be good to see it solved upstream.

[wildmichael]

The problem discussed here is IMHO actually a bit more complex, and not yet resolved (despite the referenced issue #9 being marked as closed). Quite many libraries are keg-only, and hence dependent binaries must include ${HOMEBREW_PREFIX}/opt/<dep_pkg>/lib in their RPATH. Take for example the fontconfig package, which depends on the freetype package, which in turn depends on libpng. All of them are keg-only. Linking the programs in fontconfig (without any modifications) fails, because libpng16.so.16, required by libfreetype.so.6, cannot be found.

The hackish solution is to update every single formula to add -Wl,-rpath,... flags to LDFLAGS for every single dependency, which is frankly quite unmaintainable. However, I don't see how this could be handled automagically right now.

[mistydemeo]

The hackish solution is to update every single formula to add -Wl,-rpath,... flags to LDFLAGS for every single dependency, which is frankly quite unmaintainable.

Does linuxbrew use superenv? If so, it wouldn't be difficult to add to flags automatically based on the existing code there.

[wildmichael]

Well, of course {home,linux}brew might just do it based on the listed dependencies. Would that be safe?

[sjackman]

@mistydemeo Linuxbrew does not currently use superenv. It would require a few patches to Library/ENV to get it going, I think.

@themiwi How does this work on Mac OS?

[wildmichael]

@sjackman It's been a while I used Mac OS, but if I remember correctly, each dylib contains an embedded path (the so-called install_name, something like the SONAME on Linux, but usually it is absolute), and this is then used when linking. Hence, (almost) every binary references its libraries via an absolute path, and thus the issue does not exist. I think you can dump this info using otool -L <somefile> and can be modified using the install_name_tool.