brew does not specify that GCC must prefer custom libraries for building (example: mpfr)

[aktau]

I'm on debian wheezy

I have a libmpfr installed through apt-get, and develop a lot with standard debian dev packages, which is why I can't put /my/homebrew/lib in LD_LIBRARY_PATH. However this becomes problematic while building with homebrew, as it now prefers the system paths to the homebrew paths, therefore including the old libmpfr in my case. A stopgap solution that works for me is:

$ LD_LIBRARY_PATH=/my/homebrew/lib brew install mpfr

Which compiles and tests just fine. I'm not sure what the best way is to go about fixing this, so I'm not suggesting anything yet.

I don't really think it would be a good solution to just put the homebrew libs folder at the front of the library search path system-wide, as it could wreak havoc on other development activities. This doesn't just go for debian, but pretty much any distro with a package manager, I believe.

Any comments?

[aktau]

I'd like to add that after solving the two issues I posted 2 hours ago, I successfully built gcc-4.8, nice!

[sjackman]

The only three solutions that come to mind are

• Add /my/homebrew/lib to /etc/ld.so.conf
• Add /my/homebrew/lib to LD_LIBRARY_PATH
• Set the RPATH of each executable compiled by brew to /my/homebrew/lib

This post seems relevant:
http://blog.tremily.us/posts/rpath/

[MikeMcQuaid]

I'd recommend the second in the short term and third in the long term.

[aktau]

Actually the thing with the RPATH is for running executables and I already made a post about that (in which I mention using patchelf to set the rpath after compilation because the linker might not support setting rpath while linking in an earlier issue: ).

See here: #7

This question pertains only to building, in which it is sufficient to just prepend to the LD_LIBRARY_PATH variable only for the building phase. I have this vague recollection that homebrew uses a "clean" environment for building so it would be sufficient to just inject the LD_LIBRARY_PATH with the HOMEBREW_LIB folder and Bob's your uncle, no need to mess with the standard library path.

Changing LD_LIBRARY_PATH systemwide may be good for developers who use projects in-house and have full control over everything installed on the PC's, but for someone who needs their (other) stuff to run on normal debian/ubuntu/fedora/..., this is not such a great thing.

[sjackman]

It shouldn't be necessary to set LD_LIBRARY_PATH when building Homebrew packages unless you are you using compiler tools (i.e. binutils and GCC) built by Homebrew. Are you?

[aktau]

It shouldn't be necessary to set LD_LIBRARY_PATH when building Homebrew packages unless you are you using compiler tools (i.e. binutils and GCC) built by Homebrew. Are you?

No, as a matter of fact I was busy trying to build gcc-4.8 with the stock debian wheezy gcc-4.7. To build gcc-4.8 you need other libraries like (mpfr afaik) and those need yet others. Thankfully, homebrew builds all necessary dependencies. Unthankfully, it doesn't know (by default) where to look for the libraries it just built.

Unless you start tampering with LD_LIBRARY_PATH or similar that is. Luckily for us in the build phase it's easy to just prepend to LD_LIBRARY_PATH for that run. Which is exactly what I did with my commandline which made the gcc-4.8 compile work:

# mpfr is a requirement of gcc, building it separately
$ LD_LIBRARY_PATH=/my/homebrew/lib brew install mpfr

# alternatively (but messes up the loader path of the current shell session)
$ export LD_LIBRARY_PATH=/my/homebrew/lib:$PATH
$ brew install mpfr

So to summarize:

For running applications in a safe way that doesn't interfere with the standard OS loading path we still need an agreed upon solution. One that I like a lot is setting the RPATH on every executable produced by homebrew. It's a pretty clean solution.

For building applications it's much easier, the LD_LIBRARY_PATH can be altered without fear of messing up the systemwide path, which is what I did to build gcc-4.8.

What do you think? If we solve these things it will become far easier to just install and use homebrew for linux for quite a wide range of software, which I'm eager to see happen...

[sjackman]

For what it's worth, it seems that it's only necessary to set LD_LIBRARY_PATH for mpfr because of the make check in the formula: https://github.com/mxcl/homebrew/blob/master/Library/Formula/mpfr.rb#L43

If you remove this line, it compiles fine without setting LD_LIBRARY_PATH. Nonetheless, I think I agree with you that brew install should set LD_LIBRARY_PATH to $HOMEBREW_PREFIX/lib.

Cheers,
Shaun

[aktau]

For what it's worth, it seems that it's only necessary to set LD_LIBRARY_PATH for mpfr because of the make check in the formula: https://github.com/mxcl/homebrew/blob/master/Library/Formula/mpfr.rb#L43

That's... unexpected (to me). I really didn't think most of these things would work if they don't even know where $HOMEBREW_PREFIX/lib is.

If you remove this line, it compiles fine without setting LD_LIBRARY_PATH. Nonetheless, I think I agree with you that brew install should set LD_LIBRARY_PATH to $HOMEBREW_PREFIX/lib.

Great!

[sjackman]

Here's another article on RPATH:
http://www.cmake.org/Wiki/CMake_RPATH_handling

[aktau]

Nice, I'm starting to think more and more that the rpath line of thought was a good one. Though it might be that relying on a specific compiler-chain (gcc) or build-tool (cmake, which not every project uses ofcourse) supporting setting the rpath is a bit fragile, so it might make sense to evaluate possibilities of setting the rpath after the fact. What do you reckon?

I'm also wondering how it might interact with scripts that try to package homebrew-built apps (https://github.com/tmm1/brew2deb) into packages (.deb, .rpm, ...). I'm guessing at the very least that rpath can't hurt. If it's not found, it won't we used, otherwise it will. Can't make the situation worse.

[sjackman]

Ideally the RPATH would be set at link time, but making this happen for all the different build environments could be challenging. It's probably easier to patch the executable after installation, as you suggest. On OS X, the executable can be modified like so:

"install_name_tool -rpath ... -add_rpath ... -delete_rpath ..." to change RPATHs

See http://www.cmake.org/Wiki/CMake_RPATH_handling

[aktau]

On OS X, the executable can be modified like so: ..

Awesome! But is it necessary on OSX? I recall using homebrew on my macbook all the time and it works great. Or has the installer secretly changed something LD_LIBRARY_PATH like?

It seems indeed more general to change the RPATH after the fact, but perhaps the LLVM linker also supports it, who knows. That would be at least the two big build environments. At any rate, for ELF binaries there appear to be these two (well, three...) options:

A nice article about it: http://blog.tremily.us/posts/rpath/

• link time (build-tool dependent): LDFLAGS=-Wl,-rpath=$HOMEBREW_PREFIX/lib,--enable-new-dtags ./configure …
• after build (build-toold independent): chrpath or patchelf

I've had success with patchelf as mentioned in my other issue. It has no big dependencies and can run from everywhere so we could make a homebrew formula for patchelf and have it be installed before anything else. What do you think?

EDIT: a distinct disadvantage of chrpath is that it doesn't allow one enlarge the rpath. So if there wasn't an rpath to begin with you can't add one. So chrpath is out.