haskell: pkgsStatic fixes

[rnhmjoj]

Motivation for this change

Fixing common build failures in pkgsStatic.haskellPackages

See the commit messages for details.

Things done

• built static GHCs 9.0.2 and 9.2.1
• built pkgsStatic.haskellPackages.{QuickCheck,pretty-simple} (previously failing)
• built and tested pkgsStatic.haskellPackages.pandoc

[sternenseemann]

• I'd drop d8dcb588aed526a632c3060c1b091908c7cf31df, since it changes the meaning of haskell.packages.ghc* / haskellPackages between pkgs and pkgsStatic which I find problematic. Especially since the compiler build itself is not affected, this doesn't seem necessary?
• Can you add a package to the staticHaskellPackages jobset in pkgs/top-level/release-haskell.nix which doesn't build without -fexternal-dynamic-refs? This would help keeping tabs on it.

[rnhmjoj]

Especially since the compiler build itself is not affected, this doesn't seem necessary?

I see your point, though I'm not sure how useful the default compiler is, given most packages fails to build. Ideally, we should fix the linking errors, but avoiding gmp is the only way I've found so far.

Can you add a package to the staticHaskellPackages jobset in pkgs/top-level/release-haskell.nix which doesn't build without -fexternal-dynamic-refs? This would help keeping tabs on it.

Done.

[cdepillabout]

Would you be able to add a comment here why -fexternal-dynamic-refs is needed for enableRelocatedStaticLibs?

From the GHC manual: https://downloads.haskell.org/ghc/latest/docs/html/users_guide/phases.html#ghc-flag--fexternal-dynamic-refs

When generating code, assume that entities imported from a different module might be dynamically linked. This flag is enabled automatically by -dynamic.

It is not clear to me why this would be necessary if we are statically linking things. (Although I don't have a good understanding of static linking, so maybe it should be obvious.)

[rnhmjoj]

I have to admit I'm baffled by this too and was kind of expecting the haskell maintainers would know better.
Basically, I'm implementing the solution proposed in this blog post: https://www.tweag.io/blog/2020-09-30-bazel-static-haskell/.

[cdepillabout]

Thanks for the link to the blog post. It says:

When evaluating template Haskell splices, GHC will execute compiled expressions in its built-in bytecode interpreter and this code has to be compatible with the RTS of GHC itself. In short, a GHC configured with a dynamic RTS will not be able to load static Haskell libraries to evaluate template Haskell splices.

It then goes on to say:

We need to build GHC with a static RTS and to make sure that Haskell code is compiled as position independent code so that it can be loaded into a running GHC for template Haskell splices.

Both of these statements make sense of course, but they don't seem to explain why -fexternal-dynamic-refs is necessary.

[sternenseemann]

The GHC man page reads:

Generate code for linking against dynamic libraries

I assume this means that the RTS will have some kind of dynamic library loading facilities even of static (Haskell?) archives, so the byte code interpreter / whatever used by TH will actually work. -fPIC also serves this purpose.

[cdepillabout]

Maybe I'm misinterpreting, but it seems that this change allows building Haskell packages from pkgsStatic.haskellPackages.* that use Template Haskell? This seems like quite a large improvement, and would potentially be game-changing for a lot of projects.

[rnhmjoj]

allows building Haskell packages from pkgsStatic.haskellPackages.* that use Template Haskell?

Yes, correct.

[cdepillabout]

Aside from the above comment suggestions, this LGTM!

I tried nix-build -A pkgsStatic.pretty-simple and it appears to build and run correctly!

[cdepillabout]

@sternenseemann Does this look alright to merge to you?

[sternenseemann]

I see your point, though I'm not sure how useful the default compiler is, given most packages fails to build. Ideally, we should fix the linking errors, but avoiding gmp is the only way I've found so far.

Have you tried actually building anything that uses FFI in TH? I guess that would need a dynamically linked library as well?

I'm still opposed to changing the meaning of haskell.packages.ghc*, but I would be fine with changing the meaning of top level haskellPackages as a compromise, i.e.:

  haskellPackages = dontRecurseIntoAttrs (
    if stdenv.hostPlatform.isStatic
    then haskell.packages.native-bignum.ghc902
    else haskell.packages.ghc902
  );

  /* … */
  ghc = targetPackages.haskellPackages.ghc or (
    if stdenv.targetPlatform.isStatic
    then haskell.compiler.native-bignum.ghc902
    else haskell.compiler.ghc902
  );

@sternenseemann Does this look alright to merge to you?

FWIW, eval is failing. Edit: unrelated.

[rnhmjoj]

but I would be fine with changing the meaning of top level haskellPackages as a compromise,

Looks great to me! Thanks for the suggestion.

[cdepillabout]

I would be fine with changing the meaning of top level haskellPackages as a compromise

I agree this was a good suggestion. I think the current state of this PR makes a lot of sense, having pkgsStatic.haskellPackages explicitly pointing at pkgsStatic.haskell.packages.native-bignum.ghc-902.

Have you tried actually building anything that uses FFI in TH? I guess that would need a dynamically linked library as well?

That's a good question. Do you happen to know a package that uses FFI in TH off the top of your head? I gave it some thought, but I couldn't come up with anything. (I did try compiling pkgsStatic.haskellPackages.zlib, which does successfully compile, but I imagine it is unlikely to use TH.)

Although even if this doesn't work, I don't feel like it should hold up merging this PR, since this PR currently improves the status quo.

[rnhmjoj]

So, can we merge this?

[sternenseemann]

Probably, I'll try to have a final look soon. A small request: Could you use haskellPackages.mkDerivation over haskell-generic-builder in the commit message? I'd like to have correspondence to an attribute where possible. Rebasing on haskell-updates would be useful anyways, so CI wouldn't be broken by an unrelated problem.

[rnhmjoj]

Commit message updated and rebased.

[rnhmjoj]

@sternenseemann Thank you for reviewing!

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/statically-linking-haskell-fails-to-link-libffi/18317/2

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/statically-linking-haskell-fails-to-link-libffi/18317/3