macos: use DYLD_FALLBACK_LIBRARY_PATH

[tgamblin]

Closes #9221. (or should we just merge that first?)
Fixes #16084. @DiegoMagdaleno

DYLD_LIBRARY_PATH can frequently break builtin macOS software when pointed at Spack libraries. This is because it takes higher precedence than the default library search paths, which are used by system software.

DYLD_FALLBACK_LIBRARY_PATH, on the other hand, takes lower precedence. At first glance, this might seem bad, because the software installed by Spack in an environment needs to find its libraries, and it should not use the defaults. However, Spack's isntallations are always RPATH'd, so they do not have this problem.

DYLD_FALLBACK_LIBRARY_PATH is thus useful for things built in an environment that need to use Spack's libraries, that don't set their RPATHs correctly for whatever reason. We now prefer it to DYLD_LIBRARY_PATH in modules and in environments because it helps a little bit, and it is much less intrusive.

I think we should probably also do this for LD_LIBRARY_PATH on Linux, as it can interfere in siilar ways. People do want to build against their environments, though, and it's much more common to use LD_LIBRARY_PATH there, so I'm torn. Thoughts?

@goxberry @adamjstewart @sethrj @gartung

[adamjstewart]

should we just merge that first?

Probably

[goxberry]

Closes #9221. (or should we just merge that first?)

Probably. I suspect similar issues would arise on other BSD-derived systems, and merging #9221 would fix issues on those platforms as well.

I think we should probably also do this for LD_LIBRARY_PATH on Linux, as it can interfere in similar ways. People do want to build against their environments, though, and it's much more common to use LD_LIBRARY_PATH there, so I'm torn. Thoughts?

Have people run into a similar issue on Linux systems? I'm tempted to say the Linux behavior should be left alone.

The macOS behavior descends from BSD, not Linux. I think it's important to keep this platform distinction in mind, because there may not be equivalent features on these two classes of platforms, and if there aren't, it makes no sense trying to pigeonhole Linux to behave like BSD or vice versa -- doing so would be a waste of time.

I couldn't find an ld environment variable equivalent in Linux to dyld's DYLD_FALLBACK_LIBRARY_PATH in BSDs -- the search order precedence for dyld in BSDs is different from that of ld -- which is why I don't think there is an equivalent feature in Linux, and why I think the current LD_LIBRARY_PATH behavior for Linux should stay as-is.

For anyone interested, here are links to man pages for dyld and ld (in that order) that include search precedence order:

• https://www.manpagez.com/man/1/dyld/ (note: the precedence order is buried in the text)
• http://man7.org/linux/man-pages/man8/ld.so.8.html (here, the precedence order is in a nice, easy-to-read list)

[DiegoMagdaleno]

@goxberry I don't think *BSDs suffer from this issue, since it doesn't look like BSD, uses DYLD, according to Apple's own BSD differences. However, I could be wrong, I don't think this issue affects anything else than Darwin.

I think *BSD and Linux behavior should be left as it is, since it doesn't look like Linux or BSD users report similar issues.

[adamjstewart]

I think @hartzell and @svenevs have reported similar problems with setting LD_LIBRARY_PATH on Linux. Maybe they can chime in.

[tgamblin]

I think @hartzell and @svenevs have reported similar problems with setting LD_LIBRARY_PATH on Linux. Maybe they can chime in.

Maybe I should start a new issue for this, as it's kind of a tangent for this PR. LD_LIBRARY_PATH is useful for people building in the environment, but it has the same risks for system software that DYLD_LIBRARY_PATH does on mac. As @goxberry mentions, there is no analog of DYLD_FALLBACK_LIBRARY_PATH in glibc's ld.so -- it's a dyld thing. I kind of wish there was, because that leaves linux people having to choose between spack libs and system libs, when DYLD_FALLBACK_LIBRARY_PATH gives you a decent way to use both, for builds that do not RPATH. So your best bet on Linux is to properly RPATH anything you're building in an environment, and not to rely on LD_LIBRARY_PATH. So I'm inclined to disable it and let the user set it, rather than setting it and letting the user disable it. Would be interesting to hear what people think.

[goxberry]

@goxberry I don't think *BSDs suffer from this issue, since it doesn't look like BSD, uses DYLD, according to Apple's own BSD differences. However, I could be wrong, I don't think this issue affects anything else than Darwin.

@DiegoMagdaleno You're right, this issue is limited to Darwin/macOS, and not any other BSDs.