Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.
Sign upFully static Haskell executables - overview issue #43795
Comments
nh2
referenced this issue
Jul 19, 2018
Closed
How to build statically linked Haskell binaries? #6855
This comment has been minimized.
This comment has been minimized.
|
Related:
|
This comment has been minimized.
This comment has been minimized.
Gabriel439
referenced this issue
Jul 19, 2018
Open
Create packages for `dhall` and `dhall-json` for various Linux distros #192
This comment has been minimized.
This comment has been minimized.
|
CC @vaibhavsagar whose blog post got me into working on this |
This comment has been minimized.
This comment has been minimized.
|
I know NixOS is careful about licenses and distribution. I'm thinking about libgmp, which is LGPL: https://www.gnu.org/licenses/gpl-faq.en.html#LGPLStaticVsDynamic
Can we ensure either one of these if we enable caching of static Haskell executables? |
This comment has been minimized.
This comment has been minimized.
|
@puffnfresh: Or you could use a version of GHC with |
This comment has been minimized.
This comment has been minimized.
|
@Gabriel439 yeah, is that what we'll do for redistributing static Haskell executables? |
This comment has been minimized.
This comment has been minimized.
|
This is great, thanks so much for working on this and creating this issue @nh2! |
This comment has been minimized.
This comment has been minimized.
|
Before doing more of this #43524 -style fixing of individual package builds, could someone give a try of https://github.com/endrazine/wcc (which has been packaged in #43014) in converting shared objects to static objects? |
This comment has been minimized.
This comment has been minimized.
|
@puffnfresh Every Haskell package already has a The best solution seems to be to traverse the licenses of all things linked and ensure that Hydra does not build statically linked exes where one of the dependencies is LGPL or stronger (unless the final project itself is also LGPL or stronger anyway). For executables blocked that way, a fallback should happen where it's checked if the issue goes away if |
This comment has been minimized.
This comment has been minimized.
@dezgeg It sounds interesting and may be worth a try but there may be the independent questions whether I want "binary black magic" (as wcc calls it) in my production binaries. Nix is already pretty custom in its build process, I wouldn't want an upstream library author reject my bug report about a segfault with "well you use totally custom tools, if you just used our normally built static libs we would be more eager to look into that". It's a good feature if nix's builds still essentially are a series of "normal" build steps (compiler and linker invocations). I haven't looked into wcc in detail so I cannot judge its safety, but my intuition tells me I'd rather sponsor Hydra some build servers and HDs than turn dynamic libs into static ones and have the risk of unexpected behaviour. |
This comment has been minimized.
This comment has been minimized.
jgm
commented
Jul 19, 2018
|
We already provide fully static builds of pandoc with each release, using docker and alpine. |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jul 19, 2018
•
|
Thank you for CCing me into this. Could Aura be added to the list of projects above? I haven't yet been able to decipher the instructions for testing it myself, apologies. |
This comment has been minimized.
This comment has been minimized.
|
I just want to thank you @nh2 this is excellent work :) |
This comment has been minimized.
This comment has been minimized.
|
@fosskers Does Currently most executables I'm trying are already in nixpkgs and then I just override them with
Conceptually very simple, for example for and then built with
For some packages it works immediately, for others I have to make some small overrides of their dependencies (see a bit further up in the file), usually to give static libraries for system dependencies. |
This comment has been minimized.
This comment has been minimized.
|
Having discussed it on IRC, here's a way of building all Haskell executables: Just nix-build that file. This includes broken packages by default, because if you want to see which new packages your change breaks, you need to know which ones were broken already (so you should build this once before your change and once after, then compare). |
This comment has been minimized.
This comment has been minimized.
I have incorporated (a large part of) that into https://github.com/nh2/static-haskell-nix/blob/09d0eaa605111ea516dfaa0e7341a71ff1a63042/survey/default.nix#L47-L124 now. So now we can build If some binary doesn't build because of dependent libraries making problems, those libraries are supposed to be patched here. Contributors are welcome to help make as many of those build as possible. |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jul 20, 2018
•
|
@nh2 Aura isn't yet connected to Nix infrastructure in any way. I suppose I'll pull up my old notes about getting a project set up with Nix and try to build Aura that way at first. |
This comment has been minimized.
This comment has been minimized.
I understand the concern, yes.
The problem isn't the disk space or resource usage but rather the effort of fixing packages one-by-one to support static linking and the cost of maintaining that. But maybe Haskell packages don't use too many native dependencies. |
nh2
referenced this issue
Jul 20, 2018
Open
generic-haskell-builder: Overridden Cabal package to be used by Setup.hs is ignored #43849
This comment has been minimized.
This comment has been minimized.
|
I have given a first run at building all executables on Stackage, statically. See this post for full build outputs. It took around 3 hours to get there (I built with 5 machines). The final status line, The program didn't terminate; right now it's
Insights:
|
This comment has been minimized.
This comment has been minimized.
@dezgeg Yes, I think that is accurate. From what I posted a just above, it looks like most of Stackage's executables will be buildable as long as a set of 15 native libs (13 above and |
This comment has been minimized.
This comment has been minimized.
|
Great work! My rule of thumb for static vs shared is:
We pretty much need to support both in Nixpkgs. I can see some of the benefits of always building statically but I think the advantages to shared linking is much greater. |
This comment has been minimized.
This comment has been minimized.
|
I've found and PRd a fix for another cabal issue that needs to be merged to make static linking reasonable: This passes I've also added it to the overview in the issue description. |
This comment has been minimized.
This comment has been minimized.
This makes sense.
Yes. One of my goals is that nixpkgs becomes the building environment which makes it really easy to build any program so that it works on any Linux distribution, forever. Other Linux distributions make it really hard to build things statically.
I guess I agree in general but there are some exceptions / other points, like
I'd find it very cool if somebody could build a typical NixOS with only static exes and compare what the size difference (and perhaps resident memory difference) is. |
Gabriel439
added a commit
to dhall-lang/dhall-haskell
that referenced
this issue
Jul 21, 2018
matthewbauer
added this to the 19.09 milestone
Apr 16, 2019
This comment has been minimized.
This comment has been minimized.
|
Update: I experimented with 2 big branches of improvements: Using
|
This comment has been minimized.
This comment has been minimized.
|
I've now built
Looks like |
This comment has been minimized.
This comment has been minimized.
|
Update: I've now pushed the big update mentioned above to static-haskell-nix
Concrete example for stack being built with the new # Builds a static `stack` executable from a stack source dir.
#
# Usage:
#
# $(nix-build --no-link -A run-stack2nix-and-static-build-script)
let
upstreamNixpkgs = import ../nixpkgs {};
static-stack2nix-builder = import ../static-stack2nix-builder/default.nix {
cabalPackageName = "stack";
normalPkgs = upstreamNixpkgs;
compiler = "ghc822"; # matching stack.yaml
hackageSnapshot = "2019-05-08T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
stack2nix-stack-project-dir = /path/to/stack/source; # where stack.yaml is
};
in
static-stack2nix-builder |
This comment has been minimized.
This comment has been minimized.
When doing this, I found multiple big bugs in various Haskell packages that occur on
|
This comment has been minimized.
This comment has been minimized.
|
I have now added an example how to build It's a bit more verbose than what I posted above, but easier to understand and debug. I've also mentioend that in the README now so it can be easily found. |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
May 24, 2019
|
Thank you! |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
May 24, 2019
•
|
I'm using this to build Aura, and I'm getting the following near the end of the build:
It doesn't seem to give me the option to not skip the patch. Have you ever seen this before? I'm using |
This comment has been minimized.
This comment has been minimized.
|
@fosskers You'll have to tell me exactly what input files and commit of |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
May 24, 2019
•
|
I used the exact version of it supplied in the link above. So, Here's my full # Run using:
#
# $(nix-build --no-link -A fullBuildScript)
{
stack2nix-output-path ? "custom-stack2nix-output.nix",
}:
let
cabalPackageName = "aura";
compiler = "ghc865"; # matching stack.yaml
# Pin nixpkgs version.
pkgs = import (fetchTarball https://github.com/nh2/nixpkgs/archive/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72.tar.gz) {};
# Pin static-haskell-nix version.
static-haskell-nix = fetchTarball https://github.com/nh2/static-haskell-nix/archive/de8346b794031a8104840e2e17193a15e26174a0.tar.gz;
stack2nix-script = import "${static-haskell-nix}/static-stack2nix-builder/stack2nix-script.nix" {
inherit pkgs;
stack-project-dir = toString ./.; # where stack.yaml is
hackageSnapshot = "2019-05-24T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
};
static-stack2nix-builder = import "${static-haskell-nix}/static-stack2nix-builder/default.nix" {
normalPkgs = pkgs;
inherit cabalPackageName compiler stack2nix-output-path;
# disableOptimization = true; # for compile speed
};
# Full invocation, including pinning `nix` version itself.
fullBuildScript = pkgs.writeScript "stack2nix-and-build-script.sh" ''
#!/usr/bin/env bash
set -eu -o pipefail
STACK2NIX_OUTPUT_PATH=$(${stack2nix-script})
${pkgs.nix}/bin/nix-build --no-link -A static_package --argstr stack2nix-output-path "$STACK2NIX_OUTPUT_PATH" $@
'';
in
{
static_package = static-stack2nix-builder.static_package;
inherit fullBuildScript;
# For debugging:
inherit stack2nix-script;
inherit static-stack2nix-builder;
} |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jun 12, 2019
|
I've updated to the latest |
This was referenced Jun 12, 2019
This comment has been minimized.
This comment has been minimized.
@fosskers Please always use exact commits otherwise I have to guess which one you used. The reason for the build failure is because of nixpkgs/pkgs/development/haskell-modules/configuration-nix.nix Lines 473 to 474 in a2d7e9b The LTS you use has The right solution here is to kick out that patch, given that you're using a newer version that doesn't need it. Here is an example of how you can do that, also demonstrating how in general one can override further Haskell packages in the returned package set: # Run using:
#
# $(nix-build --no-link -A fullBuildScript)
{
stack2nix-output-path ? "custom-stack2nix-output.nix",
}:
let
cabalPackageName = "aura";
compiler = "ghc865"; # matching stack.yaml
# Pin nixpkgs version.
pkgs = import (fetchTarball https://github.com/nh2/nixpkgs/archive/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72.tar.gz) {};
# Pin static-haskell-nix version.
static-haskell-nix = fetchTarball https://github.com/nh2/static-haskell-nix/archive/1d37d9a83e570eceef9c7dad5c89557f8179a076.tar.gz;
stack2nix-script = import "${static-haskell-nix}/static-stack2nix-builder/stack2nix-script.nix" {
inherit pkgs;
stack-project-dir = toString ./.; # where stack.yaml is
hackageSnapshot = "2019-05-24T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
};
static-stack2nix-builder = import "${static-haskell-nix}/static-stack2nix-builder/default.nix" {
normalPkgs = pkgs;
inherit cabalPackageName compiler stack2nix-output-path;
# disableOptimization = true; # for compile speed
};
# Full invocation, including pinning `nix` version itself.
fullBuildScript = pkgs.writeScript "stack2nix-and-build-script.sh" ''
#!/usr/bin/env bash
set -eu -o pipefail
STACK2NIX_OUTPUT_PATH=$(${stack2nix-script})
set -x
${pkgs.nix}/bin/nix-build --no-link -A static_package --argstr stack2nix-output-path "$STACK2NIX_OUTPUT_PATH" "$@"
'';
static_haskellPackages = static-stack2nix-builder.haskell-static-nix_output.haskellPackages;
# Override some packages in the snapshot.
static_haskellPackages_with_fixes = static_haskellPackages.override (old: {
overrides = pkgs.lib.composeExtensions (old.overrides or (_: _: {})) (self: super: {
# Remove nixpkgs-specific patch that no longer applies to >= 0.16
# See https://github.com/NixOS/nixpkgs/blob/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72/pkgs/development/haskell-modules/configuration-nix.nix#L473-L474
servant-client-core = pkgs.haskell.lib.overrideCabal super.servant-client-core (old: {
patches = with pkgs.lib;
filter (p: !(hasSuffix "streamBody.patch" p)) (old.patches or []);
});
});
});
in
{
static_package = static_haskellPackages_with_fixes."${cabalPackageName}";
inherit fullBuildScript;
# For debugging:
inherit stack2nix-script;
inherit static-stack2nix-builder;
}I have PR'd that on aurapm/aura#536, to pin the exact version for which this worked for me. |
This comment has been minimized.
This comment has been minimized.
|
Note for other nixpkgs maintainers and people building new (Haskell) infrastructure (like @angerman): (I had to make this PR nh2/static-haskell-nix#26 to make that overriding possible, before there was a small bug where the By now I think it is bad practice to have special functions like |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jun 12, 2019
|
Thank you @nh2 ! |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jun 12, 2019
|
Huge reduction in binary size: 28mb -> 8.4mb |
This comment has been minimized.
This comment has been minimized.
|
I've now made an OpenCollective page for static-haskell-nix to try and crowd-fund a ~28 EUR/month dedicated build server. @nmattia had already contributed a CircleCI config, but Circle's free build machine is pretty slow and it cannot point out as cleanly which packages failed as e.g. Hydra can. |
This comment has been minimized.
This comment has been minimized.
|
Added some backing (and hoping that eventually all this will become available upstream and not being its own big project) |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jun 18, 2019
|
Take my money. |
This comment has been minimized.
This comment has been minimized.
350€/year? Can you get a reasonably potent machine for that? |
This comment has been minimized.
This comment has been minimized.
|
@nh2 I'll ping you as soon as we release Hercules CI agent with cachix support and you can wire that up :) |
This comment has been minimized.
This comment has been minimized.
fosskers
commented
Jun 20, 2019
|
Yo @nh2 we hit the budget. |
This comment has been minimized.
This comment has been minimized.
|
Wow, what a great response, I didn't expect that to go so quickly. Time to order the server! @nomeata @fosskers @domenkozar Thanks!
@nomeata For sure, but we'll have to see to what extent Hydra / cache.nixos.org want to build and host multiple ghc builds (e.g. with and without
@Profpatsch Gotta start lean :) I just ordered a dedicated Xeon E3-1245V2 with 16 GB RAM and 2x3TB disk for 30,77 €/month from Hetzner. |
This comment has been minimized.
This comment has been minimized.
|
Some semi-related news: Google is writing new libc targeted at static linking: https://lists.llvm.org/pipermail/llvm-dev/2019-June/133269.html
Looks like the claims that static linking is dead on GNU systems are becoming more and more untrue (so I've posted my own answer there now |
nh2 commentedJul 19, 2018
•
edited
If you just want to build static Haskell executables right now, follow these instructions.
This issue collects/links all issues and ongoing work to make fully-static building of Haskell executables an excellently-working feature in nixpkgs.
I will update here regularly the progress we are making, and sub-goals we encounter on the way that need to be fulfilled to tick off the parent goal.
Obviously contributions to this list are welcome.
stackbased projects with their ownresolvershould be really easystackbased example should be added (here)AllMost Haskell executables in nixpkgs succeed to build staticallysurveydirectory of https://github.com/nh2/static-haskell-nixnixpkgsStackageconfiguration-hackage2nix.yamlto be able to filter for Stackage packages (thanks @domenkozar for the tip)stackstackbuilds withcrossSystem = { config = "x86_64-unknown-linux-musl"; };stackbuilds withpkgsMuslstackbuilds with its own Stackage snapshot viastack2nixstack1.9 official release uses this toolchain for its static buildintray-web-server(which includes, amongst others,servant) (CC @NorfairKing)dhall(CC @Gabriel439)cachix(CC @domenkozar)xmonadlibgmp.so.3errorlibgmperror is gone since #43795 (comment)pandoc(CC @jgm)hsluatest case segfault might mean some functionality will crashdarcslibcurlwhich itself has many dependencies, and had to fix a cabal issue for it, but it's working now.auracrossSystem = { config = "x86_64-unknown-linux-musl"; };support is working (done by @dtzWill)pkgsMuslsupport has landed in nixpkgs master (done by @dtzWill)masterArgument list too longwhen linkingCabal--enable-executable-staticfeature is available in nixpkgshaskellPackageswhere all exes are statically linkedCabalrelease is available that includes itCabal--ld-optionGHC passthrough feature is available in nixpkgs.afiles) only at the linker stageCabalrelease is available that includes it.afiles)ncurses.aandtinfofix has been merged tostaging(done by (@shlevy).aandtinfofix has been merged tomaster(this is a mass-rebuild PR) (merged)staginghas landed inmaster(only.afix, nottinfopart, because it's not a mass-rebuild PRpkgsStatic- a fully static overlaypkgsStaticto nixpkgsmaster(@matthewbauer)static-haskell-nixon top of it (move system dependency overrides that add static libs there)pkgsStaticcan be used as a base.pkgsStatic(#61575)static-haskell-nix's Haskell functionality over as well-ffunction-sectionscan make final executables even smaller (results)integer-simpleinstead ofinteger-gmp, aslibgmpis the biggest LGPL dependency of all Haskell programs (see here)integer-opensslto be ready for use (CC @ch1bo)Why is static linking desirable?
Static linking means your executable depends on as few things running on the target system as possible.
For Linux, it means that an executable you build will work on any Linux distribution, with essentially infinite forwards-compatibility (because Linux in general does not change its system call interface).
Statically linked executables do not depend on the target system's
libc.Statically linked executable are incredibly easy to deploy (many Go executables are statically linked, which gave them the reputation of being easy to deploy in the devops world).
Statically linked executables can start faster than dynamically linked ones where dynamic linking has to be performed at startup (of course the startup time depends on the number of libraries to link and the amount of symbols contained within).
Statically linked executables are smaller (up to 4x in our mesurements), mostly because GHC's
-split-sectionshas way more effects with them (for unknown reason for now).Who is working on or contributing to this
Feel free to add yourself!