Forwarding DYLD_LIBRARY_PATH #1161

Closed
bitemyapp opened this Issue Oct 14, 2015 · 33 comments

Projects

None yet

10 participants

@bitemyapp
Contributor

https://www.reddit.com/r/haskell/comments/3ooxu4/library_not_loaded_libmariadb2dylib_os_x/

Ah, I got the solution myself. It seems that I needed to do a

export DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/

to add it to the search paths for dylds and it seems stack isn't forwarding env variables like DYLD_LIBRARY_PATH. I noticed because I tried to run the hsc2hs generated executable manually and it worked as expected.

Is this intended behaviour for stack?

@snoyberg
Contributor

Just saw this issue after responding on Reddit. I'm not aware of anything Stack is doing to make DYLD_LIBRARY_PATH be handled specially, perhaps it's the OS itself doing something funny?

@alexbiehl
Contributor
Spawning children processes of processes restricted by System Integrity Protection, such as by
launching a helper process in a bundle with NSTask or calling the exec(2) command, resets the Mach
special ports of that child process. Any dynamic linker (dyld) environment variables, such as
DYLD_LIBRARY_PATH, are purged when launching protected processes.

Source: https://forums.developer.apple.com/thread/9233

@snoyberg
Contributor

Well, that explains that. If someone has an idea for how to resolve this issue, please send a PR, but my lack of OS X experience will prevent me from getting involved.

@snoyberg snoyberg added this to the P3: Optional milestone Oct 15, 2015
@borsboom
Contributor

Is this specific to Stack, or does the same thing happen with cabal-install? I just wonder if a workaround is even possible, because it would presumably require getting the subprocess to set DYLD_LIBRARY_PATH itself. But if cabal-install doesn't have this problem then it should be fixable in Stack. Could it be related to the fact that ghc is actually a shell script wrapper rather than a binary?

I'm not a Mac developer, just a Un*x developer that happens run OS X on the desktop, so I don't have much understanding of System Integrity Protection at this point. I'm avoiding upgrading to El Capitan because I don't want to break my own development environment, but probably I should bite the bullet so I can look at these issues...

@DemiMarie

Sounds like the only solution is to avoid going through a shell script wrapper, or distributing a custom shell.

@alexbiehl
Contributor

I can have a look at it when I'm back home. The idea is:

I was able to compile with regular libmysqlclient (without setting DYLD_LIBRARY_PATH) maybe it was in PATH or some other variable so that hsc2hs were able to find it nonetheless. The same principle should apply to libmariadb. Maybe then I can find what differed there.

@borsboom
Contributor

@alexbiehl Have you had a chance to look into this? I'm also still curious whether this is unique to Stack or whether cabal-install has the same problem.

@YPares
Collaborator
YPares commented Nov 20, 2015

@borsboom After doing some tests on OSX, I can say it's not specific to stack. cabal-install has the same problem.
When trying to build https://github.com/tweag/HaskellR/tree/master/inline-r for instance:

$ cabal configure --extra-lib-dirs=/nix/store/hjr3jryjq1kgk1lgc7ikwnqh0xwlc5yx-R-3.2.2/lib/R/lib
$ cabal build

it causes the error (during the hsc2hs phase since it says dist/build/Foreign/R/Type_hsc_make failed), whereas (after cleaning)

$ export DYLD_LIBRARY_PATH=/nix/store/hjr3jryjq1kgk1lgc7ikwnqh0xwlc5yx-R-3.2.2/lib/R/lib
$ cabal configure
$ cabal build

works. So it's not stack-specific, cabal does the same.

@mgsloan
Collaborator
mgsloan commented Nov 21, 2015

Sounds like we should close this one out and file a bug with Cabal (if one doesn't already exist).

@borsboom borsboom closed this Nov 22, 2015
@borsboom borsboom reopened this Nov 23, 2015
@borsboom
Contributor

@YPares: unless I am misunderstanding the issue, it actually looks like this problem is Stack-specific based on your results with cabal-install. As I understand it, Stack doesn't even work if you do set DYLD_LIBRARY_PATH, since El Capitan's System Integrity Protection prevents the environment variable from being passed through. Did you perform your test on El Capitan?

@YPares
Collaborator
YPares commented Nov 23, 2015

@borsboom Yes, I'm on El Capitan.
And for my part it worked when I set DYLD_LIBRARY_PATH. Maybe I misunderstood the issue then, as @bitemyapp first post seemed to indicate that setting DYLD_LIBRARY_PATH fixed that. (I'm also pretty new to OSX development myself)

@borsboom
Contributor

I've upgraded a system to El Capitan and was able to reproduce @bitemyapp's issue. Here's a quick example:

On El Capitan (note the first command outputs an empty line):

$ DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $DYLD_LIBRARY_PATH'

$ FOO=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $FOO'
/usr/local/lib/mariadb/

vs. Yosemite (both commands output the path):

$ DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $DYLD_LIBRARY_PATH'
/usr/local/lib/mariadb/
$ FOO=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $FOO'
/usr/local/lib/mariadb/

@YPares: can you try both commands on your El Capitan setup? And can you confirm that you didn't disable System Integrity Protection?

@borsboom
Contributor

aside: leaving out Stack entirely, just the following exhibits the same behaviour:

DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ sh -c 'echo $DYLD_LIBRARY_PATH'
@borsboom
Contributor

After I disabled System Integrity Protection, El Capitan behaves like Yosemite with respect to DYLD_LIBRARY_PATH. So I think that is going to have to be our recommended course of action to work around this issue.

@borsboom borsboom added a commit that referenced this issue Nov 26, 2015
@borsboom borsboom doc: FAQ entries for #1161 and #563 77e1ac9
@borsboom borsboom added a commit that referenced this issue Nov 26, 2015
@borsboom borsboom doc: FAQ entries for #1161 and #563 a0eecd2
@borsboom
Contributor

I've added a FAQ entry for this and linked to it from the OS X installation instructions.

@borsboom borsboom closed this Nov 26, 2015
@Kevin-Lee

@borsboom Isn't there any better way than disabling System Integrity Protection? I mean like the one mentioned here.

@sam2000
sam2000 commented Jan 25, 2016

Hi, I am a newbie so sorry if this is a silly question. Disabling a security feature like this should be fine as a workaround I think. But would since the issue is now 'closed' does this mean this will not be 'fixed' in a way that would make it work on OSX without having to mess with OS security? Because I am just getting started with haskell and yesod but I am hoping to use a Mac mini as a production server for Yesod eventually and and I am worried I may have to do something like this on the production server to get Yesod to run?

@cbarrett

Telling users to disable SEP to use Stack is really disappointing. Indeed, as @Kevin-Lee's link shows, it's only binaries in system locations that aren't allowed to be affected by DYLD_* environment variables. Hopefully Stack can resolve this.

@mgsloan
Collaborator
mgsloan commented Jan 26, 2016

Note that this isn't directly a problem with stack, but something that appears to affect all uses of cabal / ghc. While maybe there's a workaround that stack could attempt, the "proper" solution is likely a change outside of stack itself.

@sam2000
sam2000 commented Jan 26, 2016

Thanks mgsloan, In that case I guess someone has to report this to the cabal/ghc people I guess.

I have sent the following message to the Haskell libraries mailing list. Please let me know if there is a better place to report this issue.

Hi,
There was a discussion in github about stack (see link below) where the conclusion seems to be pointing towards a bug in Cabal on Mac OSX EL Capitan related to forwarding DYLD_LIBRARY_PATH

#1161 (comment)

At the moment the suggested workaround is to disable System Integrity Protection which sounds a bit scary to me.

So I thought this might be a good idea to reported to someone in a Cabal team.

At the moment I am using an Ubuntu Virtual Machine to avoid having to mess with SIP on my mac. Hope some clever/kind person from the Haskell community would be able to fix this soon.

Thanks

Sam

@cbarrett

@mgsloan ah, good point. Thank you, I'll keep investigating there.

@sam2000
sam2000 commented Jan 27, 2016

@cbarrett Thanks so much for investigating this. Please could you post a link if a bug is registered against cabal or GHC so I can keep track of any developments. I am hoping I don't have to use the VM for too long

@mgsloan
Collaborator
mgsloan commented Feb 17, 2016

There is now another discussion about this (see #1799). Mac folks please try to help us come up with a hack that allows things to work without disabling SIP.

@sam2000 Also, you will be able to run binaries produced by haskell. This is just an issue with passing environment variables when invoking a process during the build.

@borsboom
Contributor

(copied from #1799 (comment))

I think this may be happening at the GHC level as well. ghc is actually a shell script wrapper, and since /bin/sh is a protected binary DYLD_LIBRARY_PATH will get stripped from the environment at as soon as the compiler is invoked. It may be possible to work around by copying /bin/sh elsewhere and re-writing the shell script wrapper to use the new path. Note: this shell script wrapper is included with GHC, so this would more properly be fixed upstream.

@borsboom borsboom reopened this Feb 18, 2016
@borsboom borsboom modified the milestone: P2: Should, P3: Optional Feb 18, 2016
@borsboom
Contributor

FYI: I've been doing some more testing, and I can now reproduce the issue with stack, cabal-install, and directly using runghc Setup.lhs.

@borsboom
Contributor

And now I have workaround that worked in my case (more-or-less what I mentioned in #1161 (comment)). First I copied /bin/sh to an another location in my home directory, then I edited <GHC-PATH>/bin/hsc2hs (which is a shell script) and replaced the path in #!/bin/sh with the new path. This fixed it for all three cases (stack, cabal-install, and directly using runghc Setup.lhs). I suspect that some of the other wrappers (such as ghc and ghci at least) would also need to be modified to handle loading dynamic libraries in Template Haskell and the REPL, so probably best to do this to all of them.

Potentially Stack could apply this sort of workaround itself. stack setup could copy /bin/sh somewhere into ~/.stack/programs and scan the GHC bin directory for any shell scripts and replace their shebangs to the new PATH. To work around it with a system-installed GHC would be a bit more involved -- it would have to copy the shell scripts elsewhere before rewriting the shebangs and then modify PATH so that the rewritten scripts are preferred over those installed in the system.

@borsboom
Contributor

Opened a GHC issue to discuss solutions: https://ghc.haskell.org/trac/ghc/ticket/11617

@bitemyapp
Contributor

Thank you @borsboom! All the work to get this sorted is much appreciated ๐Ÿ˜„

@borsboom borsboom removed the help wanted label Feb 20, 2016
@sam2000
sam2000 commented Feb 20, 2016

What wonderful news! I recently wanted to give yesod a try but when I realised that just setting it up required stack which means messing with OSX security stuff I gave up on it! Now what you have done gives me hope! Thank you so much Emanuel Borsboom! You are a star!

Sent from my iPhone

On 20 Feb 2016, at 22:06, Emanuel Borsboom notifications@github.com wrote:

And now I have workaround that worked in my case (more-or-less what I mentioned in #1161 (comment)). First I copied /bin/sh to an another location in my home directory, then I edited GHC-PATH/bin/hsc2hs (which is a shell script) and replaced the path in #!/bin/sh with the new path. This fixed it for all three cases (stack, cabal-install, and directly using runghc Setup.lhs).

Potentially Stack could apply this sort of workaround itself. stack setup could copy /bin/sh somewhere into ~/.stack/programs and scan the GHC bin directory for any shell scripts and replace their shebangs to the new PATH. To work around it with a system-installed GHC would be a bit more involved -- it would have to copy the shell scripts elsewhere before rewriting the shebangs and then modify PATH so that the rewritten scripts are preferred over those installed in the system.

This all seems pretty doable

โ€”
Reply to this email directly or view it on GitHub.

@borsboom
Contributor

@sam2000 Chances are Yesod will work out of the box, without making any changes to SIP, anyway. This is only necessary for the rare case when you have to set DYLD_LIBRARY_PATH, and I had to manually move a some libraries out of the location where Homebrew installs them in order to reproduce this.

@sam2000
sam2000 commented Feb 21, 2016

@EmanuelBorsboom in that case I will try this on my Mac. Thanks again!

Sent from my iPhone

On 20 Feb 2016, at 23:40, Emanuel Borsboom notifications@github.com wrote:

@sam2000 Chances are Yesod will work out of the box, without making any changes to SIP, anyway. This is only necessary for the rare case when you have to set DYLD_LIBRARY_PATH, and I had to manually move a some libraries out of the location where Homebrew installs them in order to reproduce this.

โ€”
Reply to this email directly or view it on GitHub.

@borsboom
Contributor

Gershom recently made this comment on the GHC issue:

Having done a little investigating, it seems to me that the real answer is "you shouldn't need to set DYLD_LIBRARY_PATH" . Indeed I don't think there are any normal workflows on OS X that should require this. The issue would only arise when linking to an external lib that inserts a path on DYLD_LIBRARY_PATH, and such libs are now effectively basically broken throughout the OS X ecosystem.

Here are a few articles that recommend against setting it (ever) as taken from a thread discussing this issue on Oracle bindings for node:

โ€‹https://blogs.oracle.com/ali/entry/avoiding_ld_library_path_the
โ€‹http://linuxmafia.com/faq/Admin/ld-lib-path.html
We may want to add a FAQ page for manual workarounds, but my gut says "don't fix, and recommend to those who encounter this that they should reconsider why they need DYLD_LIBRARY_PATH to begin with".

That said, I'm not one of the people "feeling the pain" here, so opinions may vary.

I've never been bitten by this problem (I've always installed libraries using Homebrew, which puts them in /usr/local/lib where they belong), so I can't really offer any particular opinion there. If you do have this problem, and you disagree with Gershom's assessment, please comment on the GHC issue.

Any ideas for workarounds that don't require DYLD_LIBRARY_PATH also appreciated. For example, maybe we can document an easy way to specify the location of libraries via the command-line rather than via the environment variable.

@dfalster dfalster referenced this issue in richfitz/RedisHeartbeat Apr 7, 2016
Open

problem installing RedisHeartbeat #5

@borsboom
Contributor

Since the discussion here seems to have stalled without any feedback from anyone still having problems, I'm closing this issue. We can re-open later if necessary.

@mgsloan mgsloan closed this Jun 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment