make LibClang work in GHCI on both osx and linux

[ghorn]

As discussed in b98dbf9, apparently LibClang only works in ghci on osx if it is built statically. I don't understand that, but if it works, it works. On linux, it only works in ghci if it's build shared. I used to use --configure-option=--enable-llvm-shared=LLVMSHAREDLIB but now that llvm is a submodule i don't think that works anymore.

[chetant]

I can confirm that b2710b9 is building on 32-bit linux (Ubuntu 12.04.2)

[ghorn]

It's building for me also on 64 bit debian, it just doesn't work in ghci

[ghorn]

Sorry, I should have said "works in ghci" above instead of "works", i'll edit it

[sethfowler]

I'm puzzled, as the docs seem to suggest this should work. I'll install Ubuntu in a VM tonight and debug this.

[sethfowler]

Sorry this is taking so long. I can't believe how hard it is to get the Haskell Platform to work on Ubuntu 13.04. Totally broken.

[sethfowler]

Sigh. So indeed this doesn't work. The root problem is this ghc bug:

http://hackage.haskell.org/trac/ghc/ticket/3333

It looks like the developers have decided not to fix this since they're planning to switch to using the system linker soon instead of their own custom linker. This is probably for the best, but it means that it will be a while until statically linking C++ objects works on all platforms. (Looks like at least until ghc 7.8.)

[sethfowler]

Actually I see now that ghci doesn't work on OS X either; my previous testing method was just flawed.

@ghorn, what you're saying is that if LibClang links to LLVM/libclang dynamically, then ghci worked for you in the past?

I can see where that might work, since the problem is that ghci's linker won't resolve weak symbols and __dso_handle, which are needed for C++. Probably by linking dynamically against only the interface exposed by libclang itself, which is a C interface, the issue is avoided.

If we go dynamic, then linking is considerably more complex on OS X and I need to wrestle with it more. There's some useful information here:

https://blogs.oracle.com/dipol/entry/dynamic_libraries_rpath_and_mac

I have spent hours on this approach previously before switching to the current static library approach, so in my opinion we should not delay the release over getting ghci working, especially since the work involved in fixing this will be moot once GHC 7.8 lands.

[sethfowler]

What would probably simplify this quite a bit on OS X is if libHSLibClang was built dynamically. Unfortunately, doing that requires that all the dependencies be rebuilt dynamically too, which is a bad user experience, and I am not sure that dynamic Haskell libraries are entirely stable.

[sethfowler]

Actually, one thing that might make things easier on OS X is just to set the install name of libclang.dylib to the absolute path where we end up installing it. That may break some things (you'll have to do cabal install before cabal test, for example, although we don't actually use cabal test right now) but it might be a pragmatic solution to the problem.

[ghorn]

Before the llvm submodule feature, I was building LibClang as normal (statically), and just using the --configure-option=--enable-llvm-shared=libLLVM.so or something like that, and LibClang was working in GHCI. I probably also had to point LD_LIBRARY_PATH at that lib too, I don't remember. I never tested this on OSX, only linux, but I would expect it to work the same on OSX.

Is the problem that when we build LibClang, we have to use rpath to point it at the libLLVM.so which we also build, but the install location of libLLVM.so is somewhere in ~/.ghc/../../.. and difficult to determine?

How does building libHSLibClang dynamically help things? I would hope that the user would not have to build all of their dependencies a different way than preferred in order to use LibClang.

[sethfowler]

AFAICT, the problem is that only dynamic libraries can specify an rpath relative to themselves. Because libHSLibClang is static, any rpaths end up being relative to whatever links to it. That is, if foo links to libHSLibClang.a, rpaths are resolved relative to foo. This is even weirder on OS X; linking does not work the same way as Linux there, because the paths are embedded in the library and not the thing linking to the library. I've never had to deal with this at a low level before so I might've gotten some details wrong, but that's my understanding right now.

[ghorn]

Oh man, this doesn't sound good. If you can't find a hack to get this working, I'm ok with accepting broken ghci until ghc 7.8 magically fixes everything.

[sethfowler]

I have an idea for a hack that might work, but I don't have time to try it out for a couple of days. I'd really this to work because if ghci doesn't work, Template Haskell also doesn't work, and that means I can't conveniently use things like Data.Lens in programs that use LibClang.

[chetant]

Unless its a quick and easy hack, I'd hold off. Ghc 7.8 would most certainly take care of this. If you'd like, you can build and test against (HEAD)[https://github.com/ghc/ghc] to verify for now, perhaps even work on it. I'll try that this weekend if I get a chance.

[chetant]

Update on my end: I finally got time to work on this, and I can confirm that LibClang in ghci does work with ghc-HEAD. The work I did to make it release(hackage) friendly is in the llvm3.4_ghc7.8 branch. I've tested the build out in OS X 10.5.8 and Ubuntu 12.04 and can confirm that it builds, installs and is usable.

Some changes I did there vs the llvm3.4 branch:

1. Built the static lib monolithic, so clang and llvm do not have to be linked in separately. No more keeping the build lib.
2. Made the install process more cabal-like. For eg: instead of the make -j8 being baked in, it now can be passed as an arg. However a long standing bug in cabal means instead of cabal configure --make-option=-j8 you now need to configure as so: runhaskell Setup.hs configure --make-option=-j8 --user the --user needed only for user level install

Issues still pending:

LibClang can now run on ghci (ghc-7.7+ only), but needs to be explicitly told where the libclang and LLVM-3.4svn libs are located, as well as force their load. so ghci -L<lib_path> -lclang -lLLVM-3.4svn get the job done. This however means end users using cabal install will not be able to do this. The primary issue is clang is not added as a dependency. There are also issues in OS X with rpath not being set correctly. Both need either link-time hooks in cabal/ghc, or post processing the so/dylib to manually add that in.

[chetant]

update: the rpath issue and the clang dependency issues are both resolved in linux in the llvm3.4_ghc7.8 (perhaps on OS X as well, will test the changes on a mac when I get a chance).

[sethfowler]

That sounds very good! Thanks for working on this. I can't wait to try it out!

My only concern is related to the make -j8 change. I expect that in many cases users will install LibClang indirectly by installing an application or library that depends on LibClang. In that case they will not pass a --make-option argument in, and will get whatever default behavior we decide on. Given how expensive it is to build LLVM, we should probably make parallel builds the default. You may have already done this; I haven't checked.

[chetant]

That's true, originally I used GHC.Conc.getNumProcessors to configure make, but there's a bug where in ghci/ghc compiled program it shows the correct number of cores, but in cabal install it shows 1.
Will be investigating that later.
If you know of a workaround to get a good count for the -j parm, let me know.

[sethfowler]

Ah, that's unfortunate that we can't trivially detect this information from the Haskell side. (Maybe cabal install is not running with the threaded runtime? That may have changed in newer versions since it can apparently run jobs in parallel now.) I'm not aware of a great workaround if we can't programmatically get the number of cores. =(

I'd imagine that most people installing this will have either 2 or 4 hyperthreaded cores. In the worst case, -j4 will probably work alright for both of those cases. But it'd be nice if there was a more principled solution.

[sethfowler]

@ghorn @chetant I just confirmed that with GHC 7.8, it's no problem to use LibClang in GHCi on OS X.

Can anyone confirm that it works for them on linux? I don't have a linux machine handy, unfortunately.

If so, we can close this bug. I'm pretty sure it's now fixed.

[ghorn]

I haven't use LibClang for a few months, If you give me a hello world script I'll build LibClang and ensure the script runs tomorrow

[sethfowler]

@ghorn That would be a great help!

You don't even need a script to test. It's enough to install LibClang, run GHCi and type this at the GHCi prompt:

:m + Clang.TranslationUnit Clang.Debug Clang.String
withCreateIndex False False $ \_ -> getVersion >>= unpack

It should return the value "clang version 3.4".

On OS X this will only work with GHC 7.8.x, but I'm not sure about linux.

[chetant]

A few days ago I built and tested the head against ghc 7.8.1 in Ubuntu 13.04
I could get ghci working with the tests, but encountered problems
compiling. The errors pointed to stdc++ and pthreads not linking correctly.
Didn't get a chance to attempt to fix it yet.
@ghorn you can use any of the tests, if they load in ghci and you can
execute :main then its all good
On Apr 14, 2014 5:45 PM, "Seth Fowler" notifications@github.com wrote:

@ghorn https://github.com/ghorn That would be a great help!

You don't even need a script to test. It's enough to install LibClang, run
GHCi and type this at the GHCi prompt:

:m + Clang.TranslationUnit Clang.Debug Clang.String
withCreateIndex False False $ _ -> getVersion >>= unpack

It should return the value "clang version 3.4".

On OS X this will only work with GHC 7.8.x, but I'm not sure about linux.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/33#issuecomment-40427324
.

[ghorn]

ghorn@deb-kerman:~$ ghci
GHCi, version 7.8.2: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Prelude> :m + Clang.TranslationUnit Clang.Debug Clang.String
Prelude Clang.TranslationUnit Clang.Debug Clang.String> withCreateIndex False False $ \_ -> getVersion >>= unpack
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package array-0.5.0.0 ... linking ... done.
Loading package deepseq-1.3.0.2 ... linking ... done.
Loading package containers-0.5.5.1 ... linking ... done.
Loading package bytestring-0.10.4.0 ... linking ... done.
Loading package filepath-1.3.0.2 ... linking ... done.
Loading package pretty-1.1.1.1 ... linking ... done.
Loading package greencard-3.0.4 ... linking ... done.
Loading package text-1.1.0.1 ... linking ... done.
Loading package hashable-1.2.1.0 ... linking ... done.
Loading package mtl-2.1.3.1 ... linking ... done.
Loading package exceptions-0.5 ... linking ... done.
Loading package base-unicode-symbols-0.2.2.4 ... linking ... done.
Loading package transformers-base-0.4.1 ... linking ... done.
Loading package monad-control-0.3.2.3 ... linking ... done.
Loading package lifted-base-0.2.2.1 ... linking ... done.
Loading package mmorph-1.0.2 ... linking ... done.
Loading package resourcet-1.1.1 ... linking ... done.
Loading package old-locale-1.0.0.6 ... linking ... done.
Loading package time-1.4.2 ... linking ... done.
Loading package primitive-0.5.2.1 ... linking ... done.
Loading package vector-0.10.9.1 ... linking ... done.
Loading package LibClang-3.4.0 ... linking ... done.
"clang version 3.4 "
Prelude Clang.TranslationUnit Clang.Debug Clang.String> 

the hello world works

[ghorn]

@chetant I think all of the tests are outdated. At least the ones in the test folder. I couldn't get any to typecheck, and the modules are outdated

[sethfowler]

Yeah, I broke them in the past few days, unfortunately. =) I moved some of the types around between modules.

I'm going to move some of the types around again today, and when I get that done I'll get the tests updated, too.

[sethfowler]

@ghorn Thanks so much for checking that LibClang works for you in GHCi, by the way! It's been tricky to keep OS X and linux working simultaneously. Having more people testing really helps.

[ghorn]

No problem! Ping me if you need more