Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully static Haskell executables - overview issue #43795

Open
nh2 opened this issue Jul 19, 2018 · 106 comments

Comments

Projects
None yet
@nh2
Copy link
Contributor

commented Jul 19, 2018

If you just want to build static Haskell executables right now, follow these instructions.

This issue collects/links all issues and ongoing work to make fully-static building of Haskell executables an excellently-working feature in nixpkgs.

I will update here regularly the progress we are making, and sub-goals we encounter on the way that need to be fulfilled to tick off the parent goal.

Obviously contributions to this list are welcome.

Why is static linking desirable?

Static linking means your executable depends on as few things running on the target system as possible.

For Linux, it means that an executable you build will work on any Linux distribution, with essentially infinite forwards-compatibility (because Linux in general does not change its system call interface).

Statically linked executables do not depend on the target system's libc.

Statically linked executable are incredibly easy to deploy (many Go executables are statically linked, which gave them the reputation of being easy to deploy in the devops world).

Statically linked executables can start faster than dynamically linked ones where dynamic linking has to be performed at startup (of course the startup time depends on the number of libraries to link and the amount of symbols contained within).

Statically linked executables are smaller (up to 4x in our mesurements), mostly because GHC's -split-sections has way more effects with them (for unknown reason for now).

Who is working on or contributing to this

Feel free to add yourself!

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

Related:

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

CC @fosskers from here

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

CC @vaibhavsagar whose blog post got me into working on this

@nh2 nh2 referenced this issue Jul 19, 2018

Closed

Feature/no more haskell libs #43713

0 of 9 tasks complete
@puffnfresh

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

I know NixOS is careful about licenses and distribution. I'm thinking about libgmp, which is LGPL:

https://www.gnu.org/licenses/gpl-faq.en.html#LGPLStaticVsDynamic

(1) If you statically link against an LGPL'd library, you must also provide your application in an object (not necessarily source) format, so that a user has the opportunity to modify the library and relink the application.

(2) If you dynamically link against an LGPL'd library already present on the user's computer, you need not convey the library's source. On the other hand, if you yourself convey the executable LGPL'd library along with your application, whether linked with statically or dynamically, you must also convey the library's sources, in one of the ways for which the LGPL provides.

Can we ensure either one of these if we enable caching of static Haskell executables?

@Gabriel439

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

@puffnfresh: Or you could use a version of GHC with simple-integer to avoid the GMP dependency

@puffnfresh

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

@Gabriel439 yeah, is that what we'll do for redistributing static Haskell executables?

@vaibhavsagar

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

This is great, thanks so much for working on this and creating this issue @nh2!

@dezgeg

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

Before doing more of this #43524 -style fixing of individual package builds, could someone give a try of https://github.com/endrazine/wcc (which has been packaged in #43014) in converting shared objects to static objects?

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

@puffnfresh Every Haskell package already has a license field (upstream and it's available as a field in nixpkgs).

The best solution seems to be to traverse the licenses of all things linked and ensure that Hydra does not build statically linked exes where one of the dependencies is LGPL or stronger (unless the final project itself is also LGPL or stronger anyway).

For executables blocked that way, a fallback should happen where it's checked if the issue goes away if integer-simple was used instead; then Hydra can build it that way.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

Before doing more of this #43524 -style fixing of individual package builds, could someone give a try of https://github.com/endrazine/wcc (which has been packaged in #43014) in converting shared objects to static objects?

@dezgeg It sounds interesting and may be worth a try but there may be the independent questions whether I want "binary black magic" (as wcc calls it) in my production binaries.

Nix is already pretty custom in its build process, I wouldn't want an upstream library author reject my bug report about a segfault with "well you use totally custom tools, if you just used our normally built static libs we would be more eager to look into that". It's a good feature if nix's builds still essentially are a series of "normal" build steps (compiler and linker invocations).

I haven't looked into wcc in detail so I cannot judge its safety, but my intuition tells me I'd rather sponsor Hydra some build servers and HDs than turn dynamic libs into static ones and have the risk of unexpected behaviour.

@jgm

This comment has been minimized.

Copy link

commented Jul 19, 2018

We already provide fully static builds of pandoc with each release, using docker and alpine.
The build process can be found in the linux/ directory in pandoc's repository.

@fosskers

This comment has been minimized.

Copy link

commented Jul 19, 2018

Thank you for CCing me into this. Could Aura be added to the list of projects above? I haven't yet been able to decipher the instructions for testing it myself, apologies.

@domenkozar

This comment has been minimized.

Copy link
Member

commented Jul 19, 2018

I just want to thank you @nh2 this is excellent work :)

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 19, 2018

@fosskers Does aura have a nix derivation in hackage-packages that I could use as a base?

Currently most executables I'm trying are already in nixpkgs and then I just override them with statify function to build statically.

I haven't yet been able to decipher the instructions for testing it myself, apologies.

Conceptually very simple, for example for dhall I just added 1 line here:

https://github.com/nh2/static-haskell-nix/blob/ef283274ce193f713082591dd462f4bd3fb4dd1f/survey/default.nix#L98

and then built with

NIX_PATH=nixpkgs=https://github.com/nh2/nixpkgs/archive/925aac04f4ca58aceb83beef18cb7dae0715421b.tar.gz nix-build --no-link survey/default.nix -A working

For some packages it works immediately, for others I have to make some small overrides of their dependencies (see a bit further up in the file), usually to give static libraries for system dependencies.

@Infinisil

This comment has been minimized.

Copy link
Member

commented Jul 19, 2018

Having discussed it on IRC, here's a way of building all Haskell executables:
https://gist.github.com/Infinisil/3bdb01689b5f84b71f8538f467159692

Just nix-build that file. This includes broken packages by default, because if you want to see which new packages your change breaks, you need to know which ones were broken already (so you should build this once before your change and once after, then compare).

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 20, 2018

Having discussed it on IRC, here's a way of building all Haskell executables

I have incorporated (a large part of) that into https://github.com/nh2/static-haskell-nix/blob/09d0eaa605111ea516dfaa0e7341a71ff1a63042/survey/default.nix#L47-L124 now.

So now we can build survey/default.nix -A allStackageExecutables and see (with --keep-going) how many of those build.

If some binary doesn't build because of dependent libraries making problems, those libraries are supposed to be patched here.

Contributors are welcome to help make as many of those build as possible.

@fosskers

This comment has been minimized.

Copy link

commented Jul 20, 2018

@nh2 Aura isn't yet connected to Nix infrastructure in any way. I suppose I'll pull up my old notes about getting a project set up with Nix and try to build Aura that way at first.

@dezgeg

This comment has been minimized.

Copy link
Contributor

commented Jul 20, 2018

It sounds interesting and may be worth a try but there may be the independent questions whether I want "binary black magic" (as wcc calls it) in my production binaries.

Nix is already pretty custom in its build process, I wouldn't want an upstream library author reject my bug report about a segfault with "well you use totally custom tools, if you just used our normally built static libs we would be more eager to look into that". It's a good feature if nix's builds still essentially are a series of "normal" build steps (compiler and linker invocations).

I understand the concern, yes.

my intuition tells me I'd rather sponsor Hydra some build servers and HDs than turn dynamic libs into static ones and have the risk of unexpected behaviour.

The problem isn't the disk space or resource usage but rather the effort of fixing packages one-by-one to support static linking and the cost of maintaining that. But maybe Haskell packages don't use too many native dependencies.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 20, 2018

I have given a first run at building all executables on Stackage, statically.

https://github.com/nh2/static-haskell-nix/blob/09d0eaa605111ea516dfaa0e7341a71ff1a63042/survey/default.nix#L257-L259

See this post for full build outputs.

It took around 3 hours to get there (I built with 5 machines).

The final status line, [2/961/963 built (49 failed), 4245 copied (19215.1 MiB), 210.7 MiB DL] already gives us some information on the success.

The program didn't terminate; right now it's

  • stuck on building darcs-2.14.1
  • dist/build/test-courier/test-courier from the courier package is stuck on a 100% CPU loop for the last 11 hours

Insights:

  • 6 of the 49 failed executables failed because of #43849
    • this can be fixed by either putting Cabal = Cabal_patched; into the haskellPackagesWithLibsReadyForStaticLinking overrides (I would like to avoid it that unpatched Cabal can be used for most packages)
    • or putting useFixedCabal on all the dependencies used in Setup.hs of those 6 packages (as described in the linked ticket, it can be very cumbersome to figure out which are those dependencies)
  • 15 failed because of cannot find -l*, so static libs are missing for those libraries. These are:
    • The libraries in question are:
      • bz2
      • crypto
      • curl
      • expat
      • girepository-1.0
      • glib-2.0
      • gobject-2.0
      • mpfr
      • nettle
      • pcre
      • pq
      • ssl
      • xml2
    • That's not a lot, fixes for this should be done in this section similar to the other libs we already have overridden there.
@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 20, 2018

But maybe Haskell packages don't use too many native dependencies.

@dezgeg Yes, I think that is accurate.

From what I posted a just above, it looks like most of Stackage's executables will be buildable as long as a set of 15 native libs (13 above and sqlite and lzma which I already have done) are overridden with static support.

@matthewbauer

This comment has been minimized.

Copy link
Member

commented Jul 20, 2018

Great work!

My rule of thumb for static vs shared is:

  • If you're targeting something outside of a Nix store - build static
  • If you're targeting inside of a Nix store - build shared

We pretty much need to support both in Nixpkgs. I can see some of the benefits of always building statically but I think the advantages to shared linking is much greater.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 20, 2018

I've found and PRd a fix for another cabal issue that needs to be merged to make static linking reasonable:

haskell/cabal#5451

This passes --ld-option through to GHC so that we can specify extra options in configureFlags that are needed only for static linking.

I've also added it to the overview in the issue description.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jul 20, 2018

My rule of thumb for static vs shared is:

If you're targeting something outside of a Nix store - build static
If you're targeting inside of a Nix store - build shared

This makes sense.

We pretty much need to support both in Nixpkgs

Yes. One of my goals is that nixpkgs becomes the building environment which makes it really easy to build any program so that it works on any Linux distribution, forever.

Other Linux distributions make it really hard to build things statically.

I can see some of the benefits of always building statically but I think the advantages to shared linking is much greater.

I guess I agree in general but there are some exceptions / other points, like

  • Haskell libraries are already statically linked today in nixpkgs and because
    • this makes for much better dead-code elimination (this may also apply to other languages)
    • I've seen dynamically-linked Haskell programs with 100s of .so dependencies take up to 2 seconds to show their --help text
  • Nix(OS) users don't really benefit from the "dependencies can be upgraded cheaply" idea (e.g. patching a libc vulnerability takes only a tiny libc update on other distros) because with nix all downstream dependencies are rebuilt and re-shipped anyway even for the smallest change.
  • So they benefit only from the "dependencies can be shared" idea of dynamic linking.

I'd find it very cool if somebody could build a typical NixOS with only static exes and compare what the size difference (and perhaps resident memory difference) is.

@nh2 nh2 referenced this issue Jul 21, 2018

Merged

Optional static libraries for krb5 and openssl #43870

2 of 9 tasks complete

Gabriel439 added a commit to dhall-lang/dhall-haskell that referenced this issue Jul 21, 2018

Build a completely static Dhall executable in CI
This adds a new `dhall-static` target that builds a fully static `dhall`
executable that can be run on any Linux machine (i.e. it is a relocatable
executable that is completely dependency free).  That in turns implies
that even though it is built with Nix it doesn't require that the user
installs Nix to run it (i.e. no dependency on the `/nix/store` or a
Nix installer).  Just copy the standalone executable to any Linux  machine
and it's good to go.

This based on the following work of @nh2:

* NixOS/nixpkgs#43795
* dhall-lang/dhall-lang#192 (comment)

This also bumps the version of `nixpkgs` used for the normal (non-static)
Dhall build to be the closest revision on `nixpkgs` `master` as the one
used by @nh2 in his work.  Once that work is merged into `nixpkgs` `master`
then both builds can use the same revision from `nixpkgs` `master`.
@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 18, 2019

Update: I experimented with 2 big branches of improvements: Using pkgsStatitc (instead of to pkgsMusl), and refactoring static-haskell-nix to use overlays properly. As well as upgrading to latest nixpkgs master.

  • Using pkgsMusl somewhat works, but I had to disable its property that it disables dynamic libs (.so files completely), because otherwise TemplateHaskell doesn't work. I filed #61575 for this.
  • Using pkgsStatic should make cross compilation of static Haskell exes easier, once the other parts for that are ready (GHC and Cabal, as worked on by e.g. @Ericson2314).
  • Unfortunately pkgsStatic creates ~3x larger Haskell executables currently, so some dead code elimination / symbol stripping doesn't seem to work.
  • However, the nixpkgs upgrade (or something else I did? GHC or C toolchain upgrades?) made some normal pkgsMusl based executables significantly smaller:
    • cachix went from 14 MB to 11 MB
    • dhall went from 14 MB to 9.1 MB
    • shellcheck went from 5.7 MB to 5.6 MB
    • stack did not change
    • xmonad went from 3.2 MB to 3.1 MB
    • pandoc regressed from 39 MB to 44 MB (this may also be new functionality)
@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2019

I've now built stack 2.1 and all dependencies, once with -O0 and once with normal optimisations:

25M	/nix/store/10n8c6529yzmf3adzb9532yvgq9xkas8-stack-2.1.0.1/bin/stack
30M	/nix/store/sab07rs5i838kpnrija9i357awrvr9qx-stack-2.1.0.1/bin/stack

Looks like -O does add a lot to the binary size.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2019

Update: I've now pushed the big update mentioned above to static-haskell-nix master (main commit nh2/static-haskell-nix@04029bc), including many fixes.

  • Many older hacks were removed because my nixpkgs PRs were merged that fix them.
  • GHC 8.6.* compatibility.
  • Many of the nasty *_static overrides went away because of the switch to overlays.
  • The project should now be much better to understand.
  • The project should now be a lot easier to upstream into nixpkgs.
  • I've filed #61575 for a first part to upstream.
  • A flag to compile everything with -O0 is now available for faster building.
  • The set of non-working Stackage executables shrunk from 37 -> 29 many.
  • A new wrapper static-stack2nix-builder was added to make building stack based probjects much easier.

Concrete example for stack being built with the new static-stack2nix-builder; slightly simplified, all that is needed is:

# Builds a static `stack` executable from a stack source dir.
#
# Usage:
#
#     $(nix-build --no-link -A run-stack2nix-and-static-build-script)
let
  upstreamNixpkgs = import ../nixpkgs {};

  static-stack2nix-builder = import ../static-stack2nix-builder/default.nix {
    cabalPackageName = "stack";
    normalPkgs = upstreamNixpkgs;
    compiler = "ghc822"; # matching stack.yaml
    hackageSnapshot = "2019-05-08T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
    stack2nix-stack-project-dir = /path/to/stack/source; # where stack.yaml is
  };
in
  static-stack2nix-builder
@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2019

A flag to compile everything with -O0 is now available for faster building.

When doing this, I found multiple big bugs in various Haskell packages that occur on -O0 or in ghci:

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 23, 2019

I have now added an example how to build stack-based projects.

It's a bit more verbose than what I posted above, but easier to understand and debug.

I've also mentioend that in the README now so it can be easily found.

@fosskers

This comment has been minimized.

Copy link

commented May 24, 2019

Thank you!

@fosskers

This comment has been minimized.

Copy link

commented May 24, 2019

I'm using this to build Aura, and I'm getting the following near the end of the build:

patching sources
Replace Cabal file with edited version from mirror://hackage/servant-client-core-0.16/revision/1.cabal.
applying patch /nix/store/9p5ps7vk03y8hkplaj3xfx1spksfijmr-servant-client-core-streamBody.patch
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/src/Servant/Client/Core/Internal/HasClient.hs b/src/Servant/Client/Core/Internal/HasClient.hs
|index 712007006..6be92ec6d 100644
|--- a/src/Servant/Client/Core/Internal/HasClient.hs
|+++ b/src/Servant/Client/Core/Internal/HasClient.hs
--------------------------
File to patch: 
Skip this patch? [y] 
Skipping patch.
4 out of 4 hunks ignored
builder for '/nix/store/8hddf1wvafir8rq4221zijjw8jw0wvc0-servant-client-core-0.16.drv' failed with exit code 1

It doesn't seem to give me the option to not skip the patch. Have you ever seen this before?

I'm using hackageSnapshot = "2019-05-23T00:00:00Z";.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 24, 2019

@fosskers You'll have to tell me exactly what input files and commit of static-haskell-nix you're using so I can reproduce it.

@fosskers

This comment has been minimized.

Copy link

commented May 24, 2019

I used the exact version of it supplied in the link above. So, b06f8979bfaa27dc4ce76cbeaa393e0c28b5baef.

Here's my full default.nix which I used to try and build Aura's master branch. I think that I only changed cabalPackageName, compiler, and hackageSnapshot:

# Run using:
#
#     $(nix-build --no-link -A fullBuildScript)
{
  stack2nix-output-path ? "custom-stack2nix-output.nix",
}:
let
  cabalPackageName = "aura";
  compiler = "ghc865"; # matching stack.yaml

  # Pin nixpkgs version.
  pkgs = import (fetchTarball https://github.com/nh2/nixpkgs/archive/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72.tar.gz) {};

  # Pin static-haskell-nix version.
  static-haskell-nix = fetchTarball https://github.com/nh2/static-haskell-nix/archive/de8346b794031a8104840e2e17193a15e26174a0.tar.gz;

  stack2nix-script = import "${static-haskell-nix}/static-stack2nix-builder/stack2nix-script.nix" {
    inherit pkgs;
    stack-project-dir = toString ./.; # where stack.yaml is
    hackageSnapshot = "2019-05-24T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
  };

  static-stack2nix-builder = import "${static-haskell-nix}/static-stack2nix-builder/default.nix" {
    normalPkgs = pkgs;
    inherit cabalPackageName compiler stack2nix-output-path;
    # disableOptimization = true; # for compile speed
  };

  # Full invocation, including pinning `nix` version itself.
  fullBuildScript = pkgs.writeScript "stack2nix-and-build-script.sh" ''
    #!/usr/bin/env bash
    set -eu -o pipefail
    STACK2NIX_OUTPUT_PATH=$(${stack2nix-script})
    ${pkgs.nix}/bin/nix-build --no-link -A static_package --argstr stack2nix-output-path "$STACK2NIX_OUTPUT_PATH" $@
  '';

in
  {
    static_package = static-stack2nix-builder.static_package;
    inherit fullBuildScript;
    # For debugging:
    inherit stack2nix-script;
    inherit static-stack2nix-builder;
  }
@fosskers

This comment has been minimized.

Copy link

commented Jun 12, 2019

I've updated to the latest HEAD of your repo, but I'm still getting the same issue w.r.t. servant-client-core.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 12, 2019

I used to try and build Aura's master branch

@fosskers Please always use exact commits otherwise I have to guess which one you used.

The reason for the build failure is because of

# https://github.com/haskell-servant/servant/pull/1128
servant-client-core = appendPatch super.servant-client-core ./patches/servant-client-core-streamBody.patch;

The LTS you use has servant-client-core 0.16, where this patch no longer applies.

The right solution here is to kick out that patch, given that you're using a newer version that doesn't need it.

Here is an example of how you can do that, also demonstrating how in general one can override further Haskell packages in the returned package set:

# Run using:
#
#     $(nix-build --no-link -A fullBuildScript)
{
  stack2nix-output-path ? "custom-stack2nix-output.nix",
}:
let
  cabalPackageName = "aura";
  compiler = "ghc865"; # matching stack.yaml

  # Pin nixpkgs version.
  pkgs = import (fetchTarball https://github.com/nh2/nixpkgs/archive/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72.tar.gz) {};

  # Pin static-haskell-nix version.
  static-haskell-nix = fetchTarball https://github.com/nh2/static-haskell-nix/archive/1d37d9a83e570eceef9c7dad5c89557f8179a076.tar.gz;

  stack2nix-script = import "${static-haskell-nix}/static-stack2nix-builder/stack2nix-script.nix" {
    inherit pkgs;
    stack-project-dir = toString ./.; # where stack.yaml is
    hackageSnapshot = "2019-05-24T00:00:00Z"; # pins e.g. extra-deps without hashes or revisions
  };

  static-stack2nix-builder = import "${static-haskell-nix}/static-stack2nix-builder/default.nix" {
    normalPkgs = pkgs;
    inherit cabalPackageName compiler stack2nix-output-path;
    # disableOptimization = true; # for compile speed
  };

  # Full invocation, including pinning `nix` version itself.
  fullBuildScript = pkgs.writeScript "stack2nix-and-build-script.sh" ''
    #!/usr/bin/env bash
    set -eu -o pipefail
    STACK2NIX_OUTPUT_PATH=$(${stack2nix-script})
    set -x
    ${pkgs.nix}/bin/nix-build --no-link -A static_package --argstr stack2nix-output-path "$STACK2NIX_OUTPUT_PATH" "$@"
  '';

  static_haskellPackages = static-stack2nix-builder.haskell-static-nix_output.haskellPackages;

  # Override some packages in the snapshot.
  static_haskellPackages_with_fixes = static_haskellPackages.override (old: {
    overrides = pkgs.lib.composeExtensions (old.overrides or (_: _: {})) (self: super: {

      # Remove nixpkgs-specific patch that no longer applies to >= 0.16
      # See https://github.com/NixOS/nixpkgs/blob/a2d7e9b875e8ba7fd15b989cf2d80be4e183dc72/pkgs/development/haskell-modules/configuration-nix.nix#L473-L474
      servant-client-core = pkgs.haskell.lib.overrideCabal super.servant-client-core (old: {
        patches = with pkgs.lib;
          filter (p: !(hasSuffix "streamBody.patch" p)) (old.patches or []);
      });

    });
  });
in
  {
    static_package = static_haskellPackages_with_fixes."${cabalPackageName}";
    inherit fullBuildScript;
    # For debugging:
    inherit stack2nix-script;
    inherit static-stack2nix-builder;
  }

I have PR'd that on aurapm/aura#536, to pin the exact version for which this worked for me.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 12, 2019

Note for other nixpkgs maintainers and people building new (Haskell) infrastructure (like @angerman):

(I had to make this PR nh2/static-haskell-nix#26 to make that overriding possible, before there was a small bug where the .override attribute was thought to be a Haskell package.)

By now I think it is bad practice to have special functions like .override in the same attrset as the Haskell packages, and that nixpkgs should move away from that practice in the future, because all those special cases make transforming package sets hard.

@fosskers

This comment has been minimized.

Copy link

commented Jun 12, 2019

Thank you @nh2 !

@fosskers

This comment has been minimized.

Copy link

commented Jun 12, 2019

Huge reduction in binary size: 28mb -> 8.4mb

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 18, 2019

I've now made an OpenCollective page for static-haskell-nix to try and crowd-fund a ~28 EUR/month dedicated build server.

@nmattia had already contributed a CircleCI config, but Circle's free build machine is pretty slow and it cannot point out as cleanly which packages failed as e.g. Hydra can.

@nomeata

This comment has been minimized.

Copy link
Contributor

commented Jun 18, 2019

Added some backing (and hoping that eventually all this will become available upstream and not being its own big project)

@fosskers

This comment has been minimized.

Copy link

commented Jun 18, 2019

Take my money.

@Profpatsch

This comment has been minimized.

Copy link
Member

commented Jun 18, 2019

I've now made an OpenCollective page for static-haskell-nix to try and crowd-fund a ~28 EUR/month dedicated build server.

350€/year? Can you get a reasonably potent machine for that?
I’m sure there’s enough companies interested in that feature that you can go up to 1000€/year and we chip in some money.

@domenkozar

This comment has been minimized.

Copy link
Member

commented Jun 19, 2019

@nh2 I'll ping you as soon as we release Hercules CI agent with cachix support and you can wire that up :)

@fosskers

This comment has been minimized.

Copy link

commented Jun 20, 2019

Yo @nh2 we hit the budget.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 21, 2019

Wow, what a great response, I didn't expect that to go so quickly. Time to order the server!

@nomeata @fosskers @domenkozar Thanks!

and hoping that eventually all this will become available upstream and not being its own big project

@nomeata For sure, but we'll have to see to what extent Hydra / cache.nixos.org want to build and host multiple ghc builds (e.g. with and without integer-simple), and so on, so having some infrastructure that guarantees those being built seems like a good idea.

350€/year? Can you get a reasonably potent machine for that?
I’m sure there’s enough companies interested in that feature that you can go up to 1000€/year and we chip in some money.

@Profpatsch Gotta start lean :) I just ordered a dedicated Xeon E3-1245V2 with 16 GB RAM and 2x3TB disk for 30,77 €/month from Hetzner.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 26, 2019

Some semi-related news: Google is writing new libc targeted at static linking:

https://lists.llvm.org/pipermail/llvm-dev/2019-June/133269.html

support static non-PIE and static-PIE linking

There are also few areas which we do not intend to invest in at this point:
Implement dynamic loading and linking support.

Looks like the claims that static linking is dead on GNU systems are becoming more and more untrue (so I've posted my own answer there now 🙂).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.