Fix the Windows build

[RyanGlScott]

There were two sneaky issues preventing accelerate from building on Windows:

1. There was a mysterious error about ProcessId being out of scope in Data.Array.Accelerate.Debug. As it turns out, ProcessId was only being imported if ACCELERATE_DEBUG was on, and ProcessId wasn't being used within an #if ACCELERATE_DEBUG ... #endif.

2. Even more mysteriously, rotateDefault was failing to typecheck. This led me to discover that the BitSize CULong instance was using the size of CULLong, not CULong! This matters for Windows, since the size of CULong is 4 bytes, and the size of CULLong is 8 bytes.

[tmcdonell]

Well spotted, thanks Ryan!

[tmcdonell]

By the way, did you perchance try to build accelerate-llvm on windows?

[RyanGlScott]

No, but I can try that next :)

[tmcdonell]

Cool. Let me know how it goes (:

[RyanGlScott]

Well, that went disastrously :)

The proximal cause the failure I got when trying to build accelerate-llvm is:

$ cabal build
Building accelerate-llvm-1.0.0.0...
Preprocessing library accelerate-llvm-1.0.0.0...
ghc.exe: C:\Users\RyanGlScott\AppData\Roaming\cabal\x86_64-windows-ghc-8.0.2\llvm-general-3.8.0.0-KHGbcTTSuxFL0az0gf3S53\HSllvm-general-3.8.0.0-KHGbcTTSuxFL0az0gf3S53.o: unknown symbol `LLVMIsMultithreaded'

[25 of 52] Compiling Data.Range.Range ( Data\Range\Range.hs, dist\build\Data\Range\Range.o )

<no location info>: error:
    ghc.exe: panic! (the 'impossible' happened)
  (GHC version 8.0.2 for x86_64-unknown-mingw32):
        Dynamic linker not initialised

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

[26 of 52] Compiling Data.Array.Accelerate.LLVM.Util ( Data\Array\Accelerate\LLVM\Util.hs, dist\build\Data\Array\Accelerate\LLVM\Util.o )

<no location info>: error:
    ghc.exe: unable to load package `llvm-general-3.8.0.0'
[27 of 52] Compiling Data.Array.Accelerate.LLVM.CodeGen.Ptr ( Data\Array\Accelerate\LLVM\CodeGen\Ptr.hs, dist\build\Data\Array\Accelerate\LLVM\CodeGen\Ptr.o )

<no location info>: error:
    ghc.exe: panic! (the 'impossible' happened)
  (GHC version 8.0.2 for x86_64-unknown-mingw32):
        Dynamic linker not initialised

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

[28 of 52] Compiling Data.Array.Accelerate.LLVM.CodeGen.Monad[boot] ( Data\Array\Accelerate\LLVM\CodeGen\Monad.hs-boot, dist\build\Data\Array\Accelerate\LLVM\CodeGen\Monad.o-boot )
[29 of 52] Compiling Data.Array.Accelerate.LLVM.CodeGen.IR ( Data\Array\Accelerate\LLVM\CodeGen\IR.hs, dist\build\Data\Array\Accelerate\LLVM\CodeGen\IR.o )

<no location info>: error:
    ghc.exe: panic! (the 'impossible' happened)
  (GHC version 8.0.2 for x86_64-unknown-mingw32):
        Dynamic linker not initialised

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

This definitely seems like GHC's fault, though. I don't know what causes the GHC panic in particular, but that unknown symbol `LLVMIsMultithreaded' error occurs whenever you try to link again LLVM in GHCi (i.e., the runtime linker). And since accelerate-llvm uses Template Haskell, it inevitably tugs on llvm-general (and thus LLVM) in the runtime linker.

I've opened GHC Trac #13093 to track this GHC bug.

[tmcdonell]

You got much further than I expected! I though the mess of C++ that is llvm-general was sure to cause problems.

Try installing llvm-general with the flag -fshared-llvm? (You'll need your llvm install to have the shared library, but that might be the default?).

[RyanGlScott]

Unfortunately, it looks like the MinGW-w64 build of llvm-3.8 that I have is statically linked.

[RyanGlScott]

FWIW, this error is also reproducible on OS X, so it might not be a Windows-exclusive bug for once ;) https://ghc.haskell.org/trac/ghc/ticket/13137

[awson]

I have my custom shared libraries build of llvm on MinGW-w64 and llvm-general and accelerate-llvm work just fine with it.

Thus -fshared-llvm requirement is important.

[RyanGlScott]

@awson, thanks for the tip! After installing mingw-w64-x86_64-llvm-3.9.1-2 from MSYS2 (which includes dynamic versions of the LLVM libs), I was able build llvm-general with -fshared-llvm, and then I was able run the accelerate-examples on Windows successfully!

Yay!

It wasn't totally trivial, though. Aside from needing to use GitHub repos for most the dependencies, I had to make a small patch to llvm-general in order to make it work with mingw-w64-x86_64-llvm-3.9.1-2:

diff --git a/llvm-general/Setup.hs b/llvm-general/Setup.hs
index 37553d6..0686465 100644
--- a/llvm-general/Setup.hs
+++ b/llvm-general/Setup.hs
@@ -128,7 +128,7 @@ main = do
       [llvmVersion] <- liftM lines $ llvmConfig "--version"
       let sharedLib = case llvmVersion of
                         "3.2" -> "LLVM-3.2svn"
-                        x -> "LLVM-" ++ x
+                        x -> "LLVM"
       staticLibs <- liftM (map (fromJust . stripPrefix "-l") . words) $ llvmConfig "--libs"
       systemLibs <- liftM (map (fromJust . stripPrefix "-l") . words) $ llvmConfig "--system-libs"

This is because for some bizarre reason, the dynamically linked LLVM library is called libLLVM instead of, say, libLLVM-3.9 (like on Linux). I'm pretty convinced this is a bug in the MinGW-w64 build of LLVM, however, so I've filed a bug here.

[tmcdonell]

Whoo! Thanks for checking this works @RyanGlScott.

RE dependencies, I'm just waiting on a GHC-8 compatible release of llvm-general to upload everything to Hackage.

[mati865]

This is because for some bizarre reason, the dynamically linked LLVM library is called libLLVM instead of, say, libLLVM-3.9 (like on Linux). I'm pretty convinced this is a bug in the MinGW-w64 build of LLVM, however, so I've filed a bug here.

@RyanGlScott, LLVM upstream decided (or made mistake) to name libLLVM as LLVM.dll on Windows and look for LLVM-3.9.dll in llvm-config.
Right now llvm-config will be patched to search for LLVM.dll:

$ llvm-config --link-shared --libs
-lLLVM.dll

You can report it to upstream and MSYS2 will follow their solution.

[RyanGlScott]

Good to know, @mati865. Once https://github.com/Alexpux/MINGW-packages/issues/2107 is fixed, my plan is to submit a patch to llvm-general that detects the LLVM shared library name by using llvm-config instead of hardcoding "LLVM" ++ <version number>, as it currently does. That way it should work for all OSes without trouble.

[RyanGlScott]

Once bscarlet/llvm-general#203 is merged into llvm-general, installing it with -fshared-llvm should be as painless as possible.

However, I'm not sure if llvm-general is actively maintained...

[tmcdonell]

Yeah, I have been wondering that as well \:

[RyanGlScott]

In any case, my patch did get merged into llvm-hs, a fork of llvm-general which is set to be released at the same time as LLVM 4.0 (which should be out Real Soon Now).

[mchakravarty]

That is interesting. If llvm-hs gains momentum, we should seriously consider switching to it.