Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully static Haskell executables - overview issue #43795

Open
48 of 60 tasks
nh2 opened this issue Jul 19, 2018 · 186 comments
Open
48 of 60 tasks

Fully static Haskell executables - overview issue #43795

nh2 opened this issue Jul 19, 2018 · 186 comments

Comments

@nh2
Copy link
Contributor

@nh2 nh2 commented Jul 19, 2018

If you just want to build static Haskell executables right now, follow these instructions.

This issue collects/links all issues and ongoing work to make fully-static building of Haskell executables an excellently-working feature in nixpkgs.

I will update here regularly the progress we are making, and sub-goals we encounter on the way that need to be fulfilled to tick off the parent goal.

Obviously contributions to this list are welcome.

Why is static linking desirable?

Static linking means your executable depends on as few things running on the target system as possible.

For Linux, it means that an executable you build will work on any Linux distribution, with essentially infinite forwards-compatibility (because Linux in general does not change its system call interface).

Statically linked executables do not depend on the target system's libc.

Statically linked executable are incredibly easy to deploy (many Go executables are statically linked, which gave them the reputation of being easy to deploy in the devops world).

Statically linked executables can start faster than dynamically linked ones where dynamic linking has to be performed at startup (of course the startup time depends on the number of libraries to link and the amount of symbols contained within).

Statically linked executables are smaller (up to 4x in our mesurements), mostly because GHC's -split-sections has way more effects with them (for unknown reason for now).

Who is working on or contributing to this

Feel free to add yourself!

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

Related:

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

CC @fosskers from here

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

CC @vaibhavsagar whose blog post got me into working on this

@nh2 nh2 mentioned this issue Jul 19, 2018
9 tasks
@puffnfresh
Copy link
Contributor

@puffnfresh puffnfresh commented Jul 19, 2018

I know NixOS is careful about licenses and distribution. I'm thinking about libgmp, which is LGPL:

https://www.gnu.org/licenses/gpl-faq.en.html#LGPLStaticVsDynamic

(1) If you statically link against an LGPL'd library, you must also provide your application in an object (not necessarily source) format, so that a user has the opportunity to modify the library and relink the application.

(2) If you dynamically link against an LGPL'd library already present on the user's computer, you need not convey the library's source. On the other hand, if you yourself convey the executable LGPL'd library along with your application, whether linked with statically or dynamically, you must also convey the library's sources, in one of the ways for which the LGPL provides.

Can we ensure either one of these if we enable caching of static Haskell executables?

@Gabriel439
Copy link
Contributor

@Gabriel439 Gabriel439 commented Jul 19, 2018

@puffnfresh: Or you could use a version of GHC with simple-integer to avoid the GMP dependency

@puffnfresh
Copy link
Contributor

@puffnfresh puffnfresh commented Jul 19, 2018

@Gabriel439 yeah, is that what we'll do for redistributing static Haskell executables?

@vaibhavsagar
Copy link
Member

@vaibhavsagar vaibhavsagar commented Jul 19, 2018

This is great, thanks so much for working on this and creating this issue @nh2!

@dezgeg
Copy link
Contributor

@dezgeg dezgeg commented Jul 19, 2018

Before doing more of this #43524 -style fixing of individual package builds, could someone give a try of https://github.com/endrazine/wcc (which has been packaged in #43014) in converting shared objects to static objects?

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

@puffnfresh Every Haskell package already has a license field (upstream and it's available as a field in nixpkgs).

The best solution seems to be to traverse the licenses of all things linked and ensure that Hydra does not build statically linked exes where one of the dependencies is LGPL or stronger (unless the final project itself is also LGPL or stronger anyway).

For executables blocked that way, a fallback should happen where it's checked if the issue goes away if integer-simple was used instead; then Hydra can build it that way.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

Before doing more of this #43524 -style fixing of individual package builds, could someone give a try of https://github.com/endrazine/wcc (which has been packaged in #43014) in converting shared objects to static objects?

@dezgeg It sounds interesting and may be worth a try but there may be the independent questions whether I want "binary black magic" (as wcc calls it) in my production binaries.

Nix is already pretty custom in its build process, I wouldn't want an upstream library author reject my bug report about a segfault with "well you use totally custom tools, if you just used our normally built static libs we would be more eager to look into that". It's a good feature if nix's builds still essentially are a series of "normal" build steps (compiler and linker invocations).

I haven't looked into wcc in detail so I cannot judge its safety, but my intuition tells me I'd rather sponsor Hydra some build servers and HDs than turn dynamic libs into static ones and have the risk of unexpected behaviour.

@jgm
Copy link

@jgm jgm commented Jul 19, 2018

We already provide fully static builds of pandoc with each release, using docker and alpine.
The build process can be found in the linux/ directory in pandoc's repository.

@fosskers
Copy link

@fosskers fosskers commented Jul 19, 2018

Thank you for CCing me into this. Could Aura be added to the list of projects above? I haven't yet been able to decipher the instructions for testing it myself, apologies.

@domenkozar
Copy link
Member

@domenkozar domenkozar commented Jul 19, 2018

I just want to thank you @nh2 this is excellent work :)

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 19, 2018

@fosskers Does aura have a nix derivation in hackage-packages that I could use as a base?

Currently most executables I'm trying are already in nixpkgs and then I just override them with statify function to build statically.

I haven't yet been able to decipher the instructions for testing it myself, apologies.

Conceptually very simple, for example for dhall I just added 1 line here:

https://github.com/nh2/static-haskell-nix/blob/ef283274ce193f713082591dd462f4bd3fb4dd1f/survey/default.nix#L98

and then built with

NIX_PATH=nixpkgs=https://github.com/nh2/nixpkgs/archive/925aac04f4ca58aceb83beef18cb7dae0715421b.tar.gz nix-build --no-link survey/default.nix -A working

For some packages it works immediately, for others I have to make some small overrides of their dependencies (see a bit further up in the file), usually to give static libraries for system dependencies.

@Infinisil
Copy link
Member

@Infinisil Infinisil commented Jul 19, 2018

Having discussed it on IRC, here's a way of building all Haskell executables:
https://gist.github.com/Infinisil/3bdb01689b5f84b71f8538f467159692

Just nix-build that file. This includes broken packages by default, because if you want to see which new packages your change breaks, you need to know which ones were broken already (so you should build this once before your change and once after, then compare).

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 20, 2018

Having discussed it on IRC, here's a way of building all Haskell executables

I have incorporated (a large part of) that into https://github.com/nh2/static-haskell-nix/blob/09d0eaa605111ea516dfaa0e7341a71ff1a63042/survey/default.nix#L47-L124 now.

So now we can build survey/default.nix -A allStackageExecutables and see (with --keep-going) how many of those build.

If some binary doesn't build because of dependent libraries making problems, those libraries are supposed to be patched here.

Contributors are welcome to help make as many of those build as possible.

@fosskers
Copy link

@fosskers fosskers commented Jul 20, 2018

@nh2 Aura isn't yet connected to Nix infrastructure in any way. I suppose I'll pull up my old notes about getting a project set up with Nix and try to build Aura that way at first.

@dezgeg
Copy link
Contributor

@dezgeg dezgeg commented Jul 20, 2018

It sounds interesting and may be worth a try but there may be the independent questions whether I want "binary black magic" (as wcc calls it) in my production binaries.

Nix is already pretty custom in its build process, I wouldn't want an upstream library author reject my bug report about a segfault with "well you use totally custom tools, if you just used our normally built static libs we would be more eager to look into that". It's a good feature if nix's builds still essentially are a series of "normal" build steps (compiler and linker invocations).

I understand the concern, yes.

my intuition tells me I'd rather sponsor Hydra some build servers and HDs than turn dynamic libs into static ones and have the risk of unexpected behaviour.

The problem isn't the disk space or resource usage but rather the effort of fixing packages one-by-one to support static linking and the cost of maintaining that. But maybe Haskell packages don't use too many native dependencies.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 20, 2018

I have given a first run at building all executables on Stackage, statically.

https://github.com/nh2/static-haskell-nix/blob/09d0eaa605111ea516dfaa0e7341a71ff1a63042/survey/default.nix#L257-L259

See this post for full build outputs.

It took around 3 hours to get there (I built with 5 machines).

The final status line, [2/961/963 built (49 failed), 4245 copied (19215.1 MiB), 210.7 MiB DL] already gives us some information on the success.

The program didn't terminate; right now it's

  • stuck on building darcs-2.14.1
  • dist/build/test-courier/test-courier from the courier package is stuck on a 100% CPU loop for the last 11 hours

Insights:

  • 6 of the 49 failed executables failed because of #43849
    • this can be fixed by either putting Cabal = Cabal_patched; into the haskellPackagesWithLibsReadyForStaticLinking overrides (I would like to avoid it that unpatched Cabal can be used for most packages)
    • or putting useFixedCabal on all the dependencies used in Setup.hs of those 6 packages (as described in the linked ticket, it can be very cumbersome to figure out which are those dependencies)
  • 15 failed because of cannot find -l*, so static libs are missing for those libraries. These are:
    • The libraries in question are:
      • bz2
      • crypto
      • curl
      • expat
      • girepository-1.0
      • glib-2.0
      • gobject-2.0
      • mpfr
      • nettle
      • pcre
      • pq
      • ssl
      • xml2
    • That's not a lot, fixes for this should be done in this section similar to the other libs we already have overridden there.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 20, 2018

But maybe Haskell packages don't use too many native dependencies.

@dezgeg Yes, I think that is accurate.

From what I posted a just above, it looks like most of Stackage's executables will be buildable as long as a set of 15 native libs (13 above and sqlite and lzma which I already have done) are overridden with static support.

@matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented Jul 20, 2018

Great work!

My rule of thumb for static vs shared is:

  • If you're targeting something outside of a Nix store - build static
  • If you're targeting inside of a Nix store - build shared

We pretty much need to support both in Nixpkgs. I can see some of the benefits of always building statically but I think the advantages to shared linking is much greater.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 20, 2018

I've found and PRd a fix for another cabal issue that needs to be merged to make static linking reasonable:

haskell/cabal#5451

This passes --ld-option through to GHC so that we can specify extra options in configureFlags that are needed only for static linking.

I've also added it to the overview in the issue description.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 20, 2018

My rule of thumb for static vs shared is:

If you're targeting something outside of a Nix store - build static
If you're targeting inside of a Nix store - build shared

This makes sense.

We pretty much need to support both in Nixpkgs

Yes. One of my goals is that nixpkgs becomes the building environment which makes it really easy to build any program so that it works on any Linux distribution, forever.

Other Linux distributions make it really hard to build things statically.

I can see some of the benefits of always building statically but I think the advantages to shared linking is much greater.

I guess I agree in general but there are some exceptions / other points, like

  • Haskell libraries are already statically linked today in nixpkgs and because
    • this makes for much better dead-code elimination (this may also apply to other languages)
    • I've seen dynamically-linked Haskell programs with 100s of .so dependencies take up to 2 seconds to show their --help text
  • Nix(OS) users don't really benefit from the "dependencies can be upgraded cheaply" idea (e.g. patching a libc vulnerability takes only a tiny libc update on other distros) because with nix all downstream dependencies are rebuilt and re-shipped anyway even for the smallest change.
  • So they benefit only from the "dependencies can be shared" idea of dynamic linking.

I'd find it very cool if somebody could build a typical NixOS with only static exes and compare what the size difference (and perhaps resident memory difference) is.

Gabriel439 pushed a commit to dhall-lang/dhall-haskell that referenced this issue Jul 21, 2018
This adds a new `dhall-static` target that builds a fully static `dhall`
executable that can be run on any Linux machine (i.e. it is a relocatable
executable that is completely dependency free).  That in turns implies
that even though it is built with Nix it doesn't require that the user
installs Nix to run it (i.e. no dependency on the `/nix/store` or a
Nix installer).  Just copy the standalone executable to any Linux  machine
and it's good to go.

This based on the following work of @nh2:

* NixOS/nixpkgs#43795
* dhall-lang/dhall-lang#192 (comment)

This also bumps the version of `nixpkgs` used for the normal (non-static)
Dhall build to be the closest revision on `nixpkgs` `master` as the one
used by @nh2 in his work.  Once that work is merged into `nixpkgs` `master`
then both builds can use the same revision from `nixpkgs` `master`.
@srid
Copy link
Contributor

@srid srid commented Sep 29, 2020

I have static binaries building on Linux using pkgsMusl: srid/neuron#417

However, on macOS pkgsMusl is unsupported. So I tried pkgsStatic. But this doesn't work (even on Linux):

  • It doesn't know about pkgs.zlib.static (so we just remove this configure flag?)
  • lua fails to build due to linking issues with libreadliine.a; See srid/neuron#417 (comment)
  • I think I saw some gmp build issues

Does anyone know the actual process for building static Haskell binaries on Linux and macOS using nixpkgs?

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Sep 30, 2020

@srid OSX does not support statically linked binaries.

Apple does not support statically linked binaries on Mac OS X.
A statically linked binary assumes binary compatibility at the kernel system call interface, and we do not make any guarantees on that front.

@srid
Copy link
Contributor

@srid srid commented Sep 30, 2020

@nh2 How do people distribute binaries of Haskell app for macOS normally? In particular, dhall has macOS binaries: https://github.com/dhall-lang/dhall-haskell/releases

Is this something we can officially support in nixpkgs?

@matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented Sep 30, 2020

pkgsStatic works for some things on macOS. It does partially static binaries, so the only thing it links to is /usr/lib/libSystem.B.dylib. Not surprised that GHC wouldn't work with it, but I bet it's possible to get it working with some changes.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Sep 30, 2020

How do people distribute binaries of Haskell app for macOS normally?

@srid I'm not very expert on OSX, but the same Apple page says

Rather, we strive to ensure binary compatibility in each dynamically linked system library and framework.

so I suspect people will link the Haskell libs in statically, but the OSX system libraries/frameworks dynamically.

The page further goes on with

If your project absolutely must create a statically linked binary, you can get the Csu (C startup) module from Darwin and try building crt0.o for yourself. Obviously, we won't support such an endeavor.

but I have no clue whether that works in practice.

@anka-213
Copy link
Contributor

@anka-213 anka-213 commented Oct 2, 2020

Regarding the licensing issues, I don't think it should be any real issue. Afaik, everything on Hackage is open source with a GPL compatible license. The only result is that the static binaries won't be MIT/BSD licensed anymore. But that shouldn't matter much, especially since the source and everything is still available elsewhere.

@expipiplus1
Copy link
Contributor

@expipiplus1 expipiplus1 commented Nov 9, 2020

I have fixed two issues with cross musl on nixpkgs (just applying patches from other people!)

My nixpkgs branch is here, https://github.com/expipiplus1/nixpkgs/tree/joe-haskell-cross. I've cross compiled a simple Haskell program (https://github.com/expipiplus1/quoth-the-enterprise, a little more complex than hello world) against pkgsMusl and pkgsStatic. haskellPackages required a few changes for correct static linking and cross compilation (https://github.com/expipiplus1/quoth-the-enterprise/blob/5cf1e84f0f6181ae4991eccd546b0f0c2de27ca9/static.nix#L12-L23). A PR which cross builds this package statically is here: expipiplus1/quoth-the-enterprise#3

The fix for nh2/static-haskell-nix#99 and #85924 was just applying @bgamari's patch to use a binary ghc which links against ncurses6 instead of 5 (expipiplus1@43788d3).

Although there was some talk about the efficacy of the binutils patch (https://sourceware.org/bugzilla/show_bug.cgi?id=16177#c9 and expipiplus1@14efdb2) the resulting binary seems to run without issue on the target platform. There's still something not quite right about the patched binutils though, as ncurses fails to build its c++ demo and needs to be tweaked to not build it (expipiplus1@d732d4c). The ncurses patch isn't required.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Nov 23, 2020

This is a static GTK app written in C built with meson.

For some more reproducibility, I just checked that this still works, and pushed the following branches:

  • static-haskell-nix branch c-static-gtk3-apps (see here) which has as nixpkgs submodule my branch gtk3-static
  • nixpkgs branch gtk3-static-and-rustc where I also had some WIP rustc changes put on top.

I also made the demo app way cooler, it now has a button that actually does something to show that it can do more than rendering empty windows:

image

Exe to try it yourselves.

There's still a lot more to do, e.g. many of my GTK upstream PRs were merged and are probably in 20.09 so I need to rebase on top of that.

@FRidh FRidh removed this from the 20.09 milestone Dec 20, 2020
@FRidh FRidh added this to the 21.03 milestone Dec 20, 2020
@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 7, 2021

Looks like static GTK executables are now dead (for C, and otherwise):

A change to the build system broke my fix, and a PR to fix it again was rejected with

I still don't want to support a static build

This is disappointing, especially because the fix would be trivial.

Looks like developers who would like to build static executables should no longer choose GTK.

Or we need to patch the build system in nixpkgs.

@Gabriel439
Copy link
Contributor

@Gabriel439 Gabriel439 commented May 7, 2021

@nh2: I guess it depends on how easy it would be for Nixpkgs to maintain the patch. If there was a good faith attempt to upstream the patch that was rejected by the maintainer then from my point of view I think it's fine for the patch to live within Nixpkgs. However, I'm not familiar with what is the official Nixpkgs policy on maintaining patches, though.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 7, 2021

It's a 7-line patch now proposed by one of the meson maintainers; I already linked it in my post above.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 7, 2021

@573
Copy link
Contributor

@573 573 commented Jun 14, 2021

I am trying to do the cross platform static dance and knee deep in I am stuck telling qemu-aarch64-static about the location of the dynamic linker (ld-linux-aarch64.so.1) for the generated binary:

# file minimal.nix, usage: nix-build --argstr system aarch64-linux minimal.nix -A hsStatic
# copy pasta from https://github.com/nh2/static-haskell-nix/issues/73#issue-549316223
# synthesize with
# https://github.com/nix-things-mobile/postgrest/blob/main/nix/static-haskell-package.nix
let
  compiler = "ghc884";

  nixpkgs = builtins.fetchTarball {
    url    = "https://github.com/nixos/nixpkgs/archive/8ed718c299934a929b8de238062c209ed9f8c994.tar.gz";
    sha256 = "0vkf308zib3x6k1h1r2mm5psck7ck96716cfdr21yrs41314qcbz";
  };

  static-haskell-nix = builtins.fetchTarball {
    url    = "https://github.com/nh2/static-haskell-nix/archive/749707fc90b781c3e653e67917a7d571fe82ae7b.tar.gz";

    sha256 = "155spda2lww378bhx68w6dxwqd5y6s9kin3qbgl2m23r3vmk3m3w";
  };

allOverlays =
    import nix/overlays;

overlays =
    [
      allOverlays.coreutils
      allOverlays.libuv
    ];

# Evaluated expression of the Nixpkgs repository.
  pkgs =
    import nixpkgs { inherit overlays; };

  patches =
    pkgs.callPackage nix/patches { };

  patchedNixpkgs =
    patches.applyPatches "patched-nixpkgs"
      nixpkgs
      [
        patches.nixpkgs-openssl-split-runtime-dependencies-of-static-builds
        #patches.nixpkgs-gdb-fix-libintl
      ];

  normalPkgs = import patchedNixpkgs {
    system = "aarch64-linux";

    config = { };

    inherit overlays;
  };

  # Each version of GHC needs a specific version of Cabal.
  defaultCabalPackageVersionComingWithGhc =
    {
      ghc884 = "Cabal_3_2_1_0";
    }."${compiler}";

  staticHaskell = import "${static-haskell-nix}/survey/default.nix" {
    inherit compiler normalPkgs defaultCabalPackageVersionComingWithGhc;
  };

lib =
    pkgs.haskell.lib;
in # staticHaskell.haskellPackages.hoogle
rec {

  # Static executable.
  hsStatic =
    lib.justStaticExecutables (lib.dontCheck (staticHaskell.haskellPackages.hledger));
}
# file nix/overlays/default.nix
{
  coreutils = import ./coreutils.nix;
  libuv = import ./libuv.nix;
}
# file nix/overlays/libuv.nix
self: super: {
  libuv = super.libuv.overrideAttrs (oldAttrs: {
    doCheck = false;
    doInstallCheck = false;
    checkTarget = false;
  });
}
# file nix/overlays/coreutils.nix
self: super: {
  coreutils = super.coreutils.overrideAttrs (oldAttrs: {
    doCheck = false;
    doInstallCheck = false;
    checkTarget = false;
  });
}
# file nix/patches/nixpkgs-openssl-split-runtime-dependencies-of-static-builds.patch
diff --git a/pkgs/development/libraries/openssl/default.nix b/pkgs/development/libraries/openssl/default.nix
index 2a586f9e9ef..20914b80a09 100644
--- a/pkgs/development/libraries/openssl/default.nix
+++ b/pkgs/development/libraries/openssl/default.nix
@@ -50,9 +50,22 @@ let
       substituteInPlace crypto/async/arch/async_posix.h \
         --replace '!defined(__ANDROID__) && !defined(__OpenBSD__)' \
                   '!defined(__ANDROID__) && !defined(__OpenBSD__) && 0'
+    '' + optionalString static
+    # On static builds, the ENGINESDIR will be empty, but its path will be
+    # compiled into the library. In order to minimize the runtime dependencies
+    # of packages that statically link openssl, we move it into the OPENSSLDIR,
+    # which will be separated into the 'etc' output.
+    ''
+      substituteInPlace Configurations/unix-Makefile.tmpl \
+        --replace 'ENGINESDIR=$(libdir)/engines-{- $sover_dirname -}' \
+                  'ENGINESDIR=$(OPENSSLDIR)/engines-{- $sover_dirname -}'
     '';
 
-    outputs = [ "bin" "dev" "out" "man" ] ++ optional withDocs "doc";
+    outputs =
+      [ "bin" "dev" "out" "man" ]
+      ++ optional withDocs "doc"
+      # Separate output for the runtime dependencies of the static build.
+      ++ optional static "etc";
     setOutputFlags = false;
     separateDebugInfo = !(stdenv.hostPlatform.useLLVM or false) && stdenv.cc.isGNU;
 
@@ -96,7 +109,17 @@ let
     configureFlags = [
       "shared" # "shared" builds both shared and static libraries
       "--libdir=lib"
-      "--openssldir=etc/ssl"
+      (if !static then
+         "--openssldir=etc/ssl"
+       else
+         # Separate the OPENSSLDIR into its own output, as its path will be
+         # compiled into 'libcrypto.a'. This makes it a runtime dependency of
+         # any package that statically links openssl, so we want to keep that
+         # output minimal. We need to prepend '/.' to the path in order to make
+         # it appear absolute before variable expansion, the 'prefix' would be
+         # prepended to it otherwise.
+         "--openssldir=/.$(etc)/etc/ssl"
+      )
     ] ++ lib.optionals withCryptodev [
       "-DHAVE_CRYPTODEV"
       "-DUSE_CRYPTODEV_DIGESTS"
@@ -126,6 +149,9 @@ let
       if [ -n "$(echo $out/lib/*.so $out/lib/*.dylib $out/lib/*.dll)" ]; then
           rm "$out/lib/"*.a
       fi
+
+      # 'etc' is a separate output on static builds only.
+      etc=$out
     '' + lib.optionalString (!stdenv.hostPlatform.isWindows)
       # Fix bin/c_rehash's perl interpreter line
       #
@@ -147,14 +173,15 @@ let
       mv $out/include $dev/
 
       # remove dependency on Perl at runtime
-      rm -r $out/etc/ssl/misc
+      rm -r $etc/etc/ssl/misc
 
-      rmdir $out/etc/ssl/{certs,private}
+      rmdir $etc/etc/ssl/{certs,private}
     '';
 
     postFixup = lib.optionalString (!stdenv.hostPlatform.isWindows) ''
-      # Check to make sure the main output doesn't depend on perl
-      if grep -r '${buildPackages.perl}' $out; then
+      # Check to make sure the main output and the static runtime dependencies
+      # don't depend on perl
+      if grep -r '${buildPackages.perl}' $out $etc; then
         echo "Found an erroneous dependency on perl ^^^" >&2
         exit 1
       fi

full log

DESTDIR=
prefix=/nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2
exec_prefix=/nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2
bindir=/nix/store/fzyasmc9pg3fd37di6k7p08xaa3nslrl-ncurses-6.2-dev/bin
top_srcdir=..
srcdir=.
datadir=/nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2/share
ticdir=/nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2/share/terminfo
source=terminfo.tmp
cross_compiling=no
/nix/store/ghlgamwp1v91y8jfkrlrkambf5p27qaw-bash-4.4-p23/bin/bash ./run_tic.sh
** Building terminfo database, please wait...
Running tic to install /nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2/share/terminfo ...

You may see messages regarding extended capabilities, e.g., AX.
These are extended terminal capabilities which are compiled
using
tic -x
If you have ncurses 4.2 applications, you should read the INSTALL
document, and install the terminfo without the -x option.

qemu-aarch64-static: Could not open '/lib/ld-linux-aarch64.so.1': No such file or directory
? tic could not build /nix/store/k6xp0rxpjlnlzgvjdxkh6y88myscrrb9-ncurses-6.2/share/terminfo
make[1]: *** [Makefile:93: install.data] Error 1
make[1]: Leaving directory '/build/ncurses-6.2/misc'
make: *** [Makefile:126: install] Error 2

What could possibly be done about this ?

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jun 14, 2021

@573 combining static with cross compilation is beyond my experience.

Do you plan to attend remote ZuriHac this weekend (registration link? I'm thinking of making static-haskell-nix a project there, and maybe we can find some people with more cross experience to have a look together.

@domenkozar
Copy link
Member

@domenkozar domenkozar commented Jun 15, 2021

I wrote a cross-compilation tutorial (review welcome!) that should teach how to cross-compile statically as well: nix-dot-dev/nix.dev#160

@573
Copy link
Contributor

@573 573 commented Jun 15, 2021

@nh2 thanks for the invitation greatly appreciated !
I will see if I can find a spot at least I registered maybe we can chat at another channel regarding the best timing ?
Looking forward to some new insights.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jun 15, 2021

@573 Since ZuriHac is remote, there will be no limits on the available spots.

The schedule is at https://zfoh.ch/zurihac2021/ -- I will attend some of the talks and I'll host the GHC panel discussion, but in between I'll hang out in the #static-haskell-nix channel. (There will likely also be other nix related channels that we may want to join to get help :) )

@573
Copy link
Contributor

@573 573 commented Jun 17, 2021

Regarding comment above Interesting enough I can still build ncurses using

NIX_PATH=nixpkgs=http://nixos.org/channels/nixos-unstable/nixexprs.tar.xz nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform-musl.pkgsStatic.ncurses

@domenkozar
Copy link
Member

@domenkozar domenkozar commented Jun 28, 2021

@nh2 what's the status after ZuriHac? I'd really like to use this for cachix/cachix#374

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jun 30, 2021

@nh2 what's the status after ZuriHac? I'd really like to use this for cachix/cachix#374

@domenkozar Currently in nixpkgs nixos-21.05 (and I suspect master too), trying to build a musl-enabled GHC segfaults. So there isn't currently a working ghc in pkgsMusl. We looked into the issue on ZuriHac but have not figured it out yet.

We probably have to bisect on nixpkgs what broke the pkgsMusl GHCs in order to get further.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 2, 2021

I bisected the ghc865Binary segfault to a musl point release upgrade: #118731 (comment)

I haven't found out why the 1.2.1 -> 1.2.2 upgrade creates the segfault; the release notes don't suggests anything.

Would appreciate anybody ot chime in who can help fix it!

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 4, 2021

I sank my weekend into this, and disentangled the 3 problems that made this tough, with upcoming solutions:

  1. GHC bootstrap binaries segfault on musl (link)
  2. check_match: Assertion `version->filename == NULL error via mismatching versions of ncurses in GHC bindists (link)
  3. ghc-8.6.5-binary.nix compilation fails with recompile with -fPIE (link)

Full writeup with more links: #129247

The path is now clear to get musl-based GHC building again, and thus static executables updated to current nixpkgs / GHC 8.10.

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 7, 2021

The ghc-8.10 branch is now in good shape: 332 executables from Stackage are now building statically.

nh2/static-haskell-nix#98 (comment)

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 7, 2021

Ticking some done items off the big list in the issue description:

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jul 23, 2021

Update:

  • musl-based GHCs segfaulting is fully fixed.
  • GHC 8.10 is now fully working, and the default.
  • stack2nix now supports Stack 2 and current Stackage LTS, thanks to @qrilka.
  • static-haskell-nix is back to nixos-unstable since nh2/static-haskell-nix#107.

So static building is in pretty good shape again!

You can read a more detailed update on my OpenCollective backer page that sponsors the CI server:

https://opencollective.com/static-haskell-nix/updates/improvements-summer-2021-ghc-8-10-stack-2-current-lts-support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet