New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forwarding DYLD_LIBRARY_PATH #1161

Closed
bitemyapp opened this Issue Oct 14, 2015 · 35 comments

Comments

Projects
None yet
@bitemyapp
Contributor

bitemyapp commented Oct 14, 2015

https://www.reddit.com/r/haskell/comments/3ooxu4/library_not_loaded_libmariadb2dylib_os_x/

Ah, I got the solution myself. It seems that I needed to do a

export DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/

to add it to the search paths for dylds and it seems stack isn't forwarding env variables like DYLD_LIBRARY_PATH. I noticed because I tried to run the hsc2hs generated executable manually and it worked as expected.

Is this intended behaviour for stack?

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Oct 15, 2015

Just saw this issue after responding on Reddit. I'm not aware of anything Stack is doing to make DYLD_LIBRARY_PATH be handled specially, perhaps it's the OS itself doing something funny?

@alexbiehl

This comment has been minimized.

Contributor

alexbiehl commented Oct 15, 2015

Spawning children processes of processes restricted by System Integrity Protection, such as by
launching a helper process in a bundle with NSTask or calling the exec(2) command, resets the Mach
special ports of that child process. Any dynamic linker (dyld) environment variables, such as
DYLD_LIBRARY_PATH, are purged when launching protected processes.

Source: https://forums.developer.apple.com/thread/9233

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Oct 15, 2015

Well, that explains that. If someone has an idea for how to resolve this issue, please send a PR, but my lack of OS X experience will prevent me from getting involved.

@snoyberg snoyberg added this to the P3: Optional milestone Oct 15, 2015

@borsboom

This comment has been minimized.

Contributor

borsboom commented Oct 16, 2015

Is this specific to Stack, or does the same thing happen with cabal-install? I just wonder if a workaround is even possible, because it would presumably require getting the subprocess to set DYLD_LIBRARY_PATH itself. But if cabal-install doesn't have this problem then it should be fixable in Stack. Could it be related to the fact that ghc is actually a shell script wrapper rather than a binary?

I'm not a Mac developer, just a Un*x developer that happens run OS X on the desktop, so I don't have much understanding of System Integrity Protection at this point. I'm avoiding upgrading to El Capitan because I don't want to break my own development environment, but probably I should bite the bullet so I can look at these issues...

@DemiMarie

This comment has been minimized.

DemiMarie commented Oct 20, 2015

Sounds like the only solution is to avoid going through a shell script wrapper, or distributing a custom shell.

@alexbiehl

This comment has been minimized.

Contributor

alexbiehl commented Oct 20, 2015

I can have a look at it when I'm back home. The idea is:

I was able to compile with regular libmysqlclient (without setting DYLD_LIBRARY_PATH) maybe it was in PATH or some other variable so that hsc2hs were able to find it nonetheless. The same principle should apply to libmariadb. Maybe then I can find what differed there.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Oct 25, 2015

@alexbiehl Have you had a chance to look into this? I'm also still curious whether this is unique to Stack or whether cabal-install has the same problem.

@YPares

This comment has been minimized.

Collaborator

YPares commented Nov 20, 2015

@borsboom After doing some tests on OSX, I can say it's not specific to stack. cabal-install has the same problem.
When trying to build https://github.com/tweag/HaskellR/tree/master/inline-r for instance:

$ cabal configure --extra-lib-dirs=/nix/store/hjr3jryjq1kgk1lgc7ikwnqh0xwlc5yx-R-3.2.2/lib/R/lib
$ cabal build

it causes the error (during the hsc2hs phase since it says dist/build/Foreign/R/Type_hsc_make failed), whereas (after cleaning)

$ export DYLD_LIBRARY_PATH=/nix/store/hjr3jryjq1kgk1lgc7ikwnqh0xwlc5yx-R-3.2.2/lib/R/lib
$ cabal configure
$ cabal build

works. So it's not stack-specific, cabal does the same.

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Nov 21, 2015

Sounds like we should close this one out and file a bug with Cabal (if one doesn't already exist).

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 23, 2015

@YPares: unless I am misunderstanding the issue, it actually looks like this problem is Stack-specific based on your results with cabal-install. As I understand it, Stack doesn't even work if you do set DYLD_LIBRARY_PATH, since El Capitan's System Integrity Protection prevents the environment variable from being passed through. Did you perform your test on El Capitan?

@YPares

This comment has been minimized.

Collaborator

YPares commented Nov 23, 2015

@borsboom Yes, I'm on El Capitan.
And for my part it worked when I set DYLD_LIBRARY_PATH. Maybe I misunderstood the issue then, as @bitemyapp first post seemed to indicate that setting DYLD_LIBRARY_PATH fixed that. (I'm also pretty new to OSX development myself)

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 25, 2015

I've upgraded a system to El Capitan and was able to reproduce @bitemyapp's issue. Here's a quick example:

On El Capitan (note the first command outputs an empty line):

$ DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $DYLD_LIBRARY_PATH'

$ FOO=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $FOO'
/usr/local/lib/mariadb/

vs. Yosemite (both commands output the path):

$ DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $DYLD_LIBRARY_PATH'
/usr/local/lib/mariadb/
$ FOO=/usr/local/lib/mariadb/ stack exec -- sh -c 'echo $FOO'
/usr/local/lib/mariadb/

@YPares: can you try both commands on your El Capitan setup? And can you confirm that you didn't disable System Integrity Protection?

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 25, 2015

aside: leaving out Stack entirely, just the following exhibits the same behaviour:

DYLD_LIBRARY_PATH=/usr/local/lib/mariadb/ sh -c 'echo $DYLD_LIBRARY_PATH'
@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 25, 2015

After I disabled System Integrity Protection, El Capitan behaves like Yosemite with respect to DYLD_LIBRARY_PATH. So I think that is going to have to be our recommended course of action to work around this issue.

borsboom added a commit that referenced this issue Nov 26, 2015

borsboom added a commit that referenced this issue Nov 26, 2015

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 26, 2015

I've added a FAQ entry for this and linked to it from the OS X installation instructions.

@Kevin-Lee

This comment has been minimized.

Kevin-Lee commented Jan 21, 2016

@borsboom Isn't there any better way than disabling System Integrity Protection? I mean like the one mentioned here.

@sam2000

This comment has been minimized.

sam2000 commented Jan 25, 2016

Hi, I am a newbie so sorry if this is a silly question. Disabling a security feature like this should be fine as a workaround I think. But would since the issue is now 'closed' does this mean this will not be 'fixed' in a way that would make it work on OSX without having to mess with OS security? Because I am just getting started with haskell and yesod but I am hoping to use a Mac mini as a production server for Yesod eventually and and I am worried I may have to do something like this on the production server to get Yesod to run?

@cbarrett

This comment has been minimized.

cbarrett commented Jan 26, 2016

Telling users to disable SEP to use Stack is really disappointing. Indeed, as @Kevin-Lee's link shows, it's only binaries in system locations that aren't allowed to be affected by DYLD_* environment variables. Hopefully Stack can resolve this.

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Jan 26, 2016

Note that this isn't directly a problem with stack, but something that appears to affect all uses of cabal / ghc. While maybe there's a workaround that stack could attempt, the "proper" solution is likely a change outside of stack itself.

@sam2000

This comment has been minimized.

sam2000 commented Jan 26, 2016

Thanks mgsloan, In that case I guess someone has to report this to the cabal/ghc people I guess.

I have sent the following message to the Haskell libraries mailing list. Please let me know if there is a better place to report this issue.

Hi,
There was a discussion in github about stack (see link below) where the conclusion seems to be pointing towards a bug in Cabal on Mac OSX EL Capitan related to forwarding DYLD_LIBRARY_PATH

#1161 (comment)

At the moment the suggested workaround is to disable System Integrity Protection which sounds a bit scary to me.

So I thought this might be a good idea to reported to someone in a Cabal team.

At the moment I am using an Ubuntu Virtual Machine to avoid having to mess with SIP on my mac. Hope some clever/kind person from the Haskell community would be able to fix this soon.

Thanks

Sam

@cbarrett

This comment has been minimized.

cbarrett commented Jan 27, 2016

@mgsloan ah, good point. Thank you, I'll keep investigating there.

@sam2000

This comment has been minimized.

sam2000 commented Jan 27, 2016

@cbarrett Thanks so much for investigating this. Please could you post a link if a bug is registered against cabal or GHC so I can keep track of any developments. I am hoping I don't have to use the VM for too long

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Feb 17, 2016

There is now another discussion about this (see #1799). Mac folks please try to help us come up with a hack that allows things to work without disabling SIP.

@sam2000 Also, you will be able to run binaries produced by haskell. This is just an issue with passing environment variables when invoking a process during the build.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Feb 18, 2016

(copied from #1799 (comment))

I think this may be happening at the GHC level as well. ghc is actually a shell script wrapper, and since /bin/sh is a protected binary DYLD_LIBRARY_PATH will get stripped from the environment at as soon as the compiler is invoked. It may be possible to work around by copying /bin/sh elsewhere and re-writing the shell script wrapper to use the new path. Note: this shell script wrapper is included with GHC, so this would more properly be fixed upstream.

@borsboom borsboom reopened this Feb 18, 2016

@borsboom borsboom modified the milestones: P2: Should, P3: Optional Feb 18, 2016

@borsboom

This comment has been minimized.

Contributor

borsboom commented Feb 20, 2016

FYI: I've been doing some more testing, and I can now reproduce the issue with stack, cabal-install, and directly using runghc Setup.lhs.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Feb 20, 2016

And now I have workaround that worked in my case (more-or-less what I mentioned in #1161 (comment)). First I copied /bin/sh to an another location in my home directory, then I edited <GHC-PATH>/bin/hsc2hs (which is a shell script) and replaced the path in #!/bin/sh with the new path. This fixed it for all three cases (stack, cabal-install, and directly using runghc Setup.lhs). I suspect that some of the other wrappers (such as ghc and ghci at least) would also need to be modified to handle loading dynamic libraries in Template Haskell and the REPL, so probably best to do this to all of them.

Potentially Stack could apply this sort of workaround itself. stack setup could copy /bin/sh somewhere into ~/.stack/programs and scan the GHC bin directory for any shell scripts and replace their shebangs to the new PATH. To work around it with a system-installed GHC would be a bit more involved -- it would have to copy the shell scripts elsewhere before rewriting the shebangs and then modify PATH so that the rewritten scripts are preferred over those installed in the system.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Feb 20, 2016

Opened a GHC issue to discuss solutions: https://ghc.haskell.org/trac/ghc/ticket/11617

@bitemyapp

This comment has been minimized.

Contributor

bitemyapp commented Feb 20, 2016

Thank you @borsboom! All the work to get this sorted is much appreciated 😄

@borsboom borsboom removed the help wanted label Feb 20, 2016

@sam2000

This comment has been minimized.

sam2000 commented Feb 20, 2016

What wonderful news! I recently wanted to give yesod a try but when I realised that just setting it up required stack which means messing with OSX security stuff I gave up on it! Now what you have done gives me hope! Thank you so much Emanuel Borsboom! You are a star!

Sent from my iPhone

On 20 Feb 2016, at 22:06, Emanuel Borsboom notifications@github.com wrote:

And now I have workaround that worked in my case (more-or-less what I mentioned in #1161 (comment)). First I copied /bin/sh to an another location in my home directory, then I edited GHC-PATH/bin/hsc2hs (which is a shell script) and replaced the path in #!/bin/sh with the new path. This fixed it for all three cases (stack, cabal-install, and directly using runghc Setup.lhs).

Potentially Stack could apply this sort of workaround itself. stack setup could copy /bin/sh somewhere into ~/.stack/programs and scan the GHC bin directory for any shell scripts and replace their shebangs to the new PATH. To work around it with a system-installed GHC would be a bit more involved -- it would have to copy the shell scripts elsewhere before rewriting the shebangs and then modify PATH so that the rewritten scripts are preferred over those installed in the system.

This all seems pretty doable


Reply to this email directly or view it on GitHub.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Feb 20, 2016

@sam2000 Chances are Yesod will work out of the box, without making any changes to SIP, anyway. This is only necessary for the rare case when you have to set DYLD_LIBRARY_PATH, and I had to manually move a some libraries out of the location where Homebrew installs them in order to reproduce this.

@sam2000

This comment has been minimized.

sam2000 commented Feb 21, 2016

@EmanuelBorsboom in that case I will try this on my Mac. Thanks again!

Sent from my iPhone

On 20 Feb 2016, at 23:40, Emanuel Borsboom notifications@github.com wrote:

@sam2000 Chances are Yesod will work out of the box, without making any changes to SIP, anyway. This is only necessary for the rare case when you have to set DYLD_LIBRARY_PATH, and I had to manually move a some libraries out of the location where Homebrew installs them in order to reproduce this.


Reply to this email directly or view it on GitHub.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Mar 31, 2016

Gershom recently made this comment on the GHC issue:

Having done a little investigating, it seems to me that the real answer is "you shouldn't need to set DYLD_LIBRARY_PATH" . Indeed I don't think there are any normal workflows on OS X that should require this. The issue would only arise when linking to an external lib that inserts a path on DYLD_LIBRARY_PATH, and such libs are now effectively basically broken throughout the OS X ecosystem.

Here are a few articles that recommend against setting it (ever) as taken from a thread discussing this issue on Oracle bindings for node:

https://blogs.oracle.com/ali/entry/avoiding_ld_library_path_the
http://linuxmafia.com/faq/Admin/ld-lib-path.html
We may want to add a FAQ page for manual workarounds, but my gut says "don't fix, and recommend to those who encounter this that they should reconsider why they need DYLD_LIBRARY_PATH to begin with".

That said, I'm not one of the people "feeling the pain" here, so opinions may vary.

I've never been bitten by this problem (I've always installed libraries using Homebrew, which puts them in /usr/local/lib where they belong), so I can't really offer any particular opinion there. If you do have this problem, and you disagree with Gershom's assessment, please comment on the GHC issue.

Any ideas for workarounds that don't require DYLD_LIBRARY_PATH also appreciated. For example, maybe we can document an easy way to specify the location of libraries via the command-line rather than via the environment variable.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Jun 23, 2016

Since the discussion here seems to have stalled without any feedback from anyone still having problems, I'm closing this issue. We can re-open later if necessary.

@leohaskell

This comment has been minimized.

leohaskell commented Nov 6, 2017

Is it possible for stack to copy the sh from which sh and modify the scripts in stack path | grep compiler-bin | cut -d' ' -f2 to use the copied executable?
Doing that manually solves the issue.

@leohaskell

This comment has been minimized.

leohaskell commented Nov 6, 2017

To save some people that manual work, I'm providing the script I'm using.

NOTE: the script must be ran from a stack project or you can pass --stack-yaml <path-to-stack-yaml> to it (so that it can find the appropriate directory containing the GHC scripts for that project).

If you're okay with that behaviour, I can try to implement it in the stack setup step for macOSes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment