nixpkgs is increasing with each Haskell LTS release for 400+ Kb #14897

Closed
danbst opened this Issue Apr 22, 2016 · 17 comments
@danbst
Contributor
danbst commented Apr 22, 2016

nixpkgs is increasing with each Haskell LTS release for 400+Kb and now takes 50Mb out of 128 Mb of checkout of nixpkgs. Are people aware of this? What are the policies on repo size?

Issue should be tagged as closure-size, I think (nixpkgs checkout resides in my /nix/store)

@abbradar
Member

I remember there was a discussion about having only revision and checksum of a Hackage snapshot inside Nixpkgs and building hackage-packages.nix on Hydra instead (and using Nix's import-from-derivation). Was there some sort of conclusion reached?

@danbst
Contributor
danbst commented Apr 22, 2016

My concern is not about hackage, but files like https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-lts-1.5.nix
That's around of 9k lines "library" = dontDistribute super."library"

  1. Maybe dontDistribute should be default and LTS configuration should include only those to distribute?
  2. LTS releases should be based on on some basis LTS config using Nix merge capabilities
@copumpkin
Member

@abbradar I think we need one more nix feature for that to work well (NixOS/nix#52), but that feature needs work and someone to work on it.

@ericsagnes
Contributor

Related ML thread: Publish All of Hackage

@vcunat
Member
vcunat commented Apr 23, 2016

I thought importing e.g. from fetchurl works fine, only you have that consequence of building the imported stuff during evaluation already.

@domenkozar
Member

This is becoming more and more annoying, nixpkgs is 100MB while haskell is 10MB and LTS packages are 40MB.

@domenkozar
Member
domenkozar commented May 30, 2016 edited

Could builtins.fetchTarball be used here? Then we could have a separate job for haskell packages and people could still get LTS packages.

We would need to be very careful introducing new fetchTarball calls, but I think this could be an exception.

cc @peti

@copumpkin
Member
copumpkin commented May 30, 2016 edited

I don't think we need fetchTarball here. It doesn't seem terrible for a particular nixpkgs version to be tied to a particular haskellPackages version, so updating the ref and sha256 every so often (with e.g., import (fetchFromGitHub { owner = "peti"; repo = "haskellPackages"; ... })) doesn't seem too bad.

Ideally, I'd like there to be a hard rule that fetchTarball appear nowhere in nixpkgs.

@vcunat
Member
vcunat commented May 30, 2016

We would want to use immutable references here. Apart from that, AFAIK the main difference is whether you need to compile curl, unzip, etc. during evalutation (e.g. --dry-run).

@peti peti self-assigned this Jun 5, 2016
@peti peti added this to the 16.09 milestone Jun 5, 2016
@peti
Member
peti commented Jun 5, 2016

I'll remove almost all Stackage-related package sets from Nixpkgs soon. Give me a few more days, then I'll have a posting for ready nix-dev that explains my plans.

@peti
Member
peti commented Jun 8, 2016

http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

tl;dr: once the LTS 7.x package set comes out, I intend to make the following changes in "master":

  • All haskell.packages.lts.* package sets will disappear.
  • haskellPackages will loosely follow the most recent LTS release,
@peti peti added a commit to peti/nixpkgs that referenced this issue Jul 19, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first stop towards dropping Stackage support. We keep LTS
6.x around because I don't want to downgrade our default compiler to GHC
7.x again, but once LTS 7.x comes out we'll switch our main package set
to that and drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant: NixOS#16130.
f467ce4
@peti peti added a commit to peti/nixpkgs that referenced this issue Jul 20, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first stop towards dropping Stackage support. We keep LTS
6.x around because I don't want to downgrade our default compiler to GHC
7.x again, but once LTS 7.x comes out we'll switch our main package set
to that and drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant: NixOS#16130.
ca08713
@peti peti added a commit to peti/nixpkgs that referenced this issue Jul 20, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first stop towards dropping Stackage support. We keep LTS
6.x around because I don't want to downgrade our default compiler to GHC
7.x again, but once LTS 7.x comes out we'll switch our main package set
to that and drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant: NixOS#16130.
5e3babd
@peti peti added a commit to peti/nixpkgs that referenced this issue Jul 20, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant:

 - NixOS#16130
 - commercialhaskell/stack#2259
4ca35c4
@peti peti added a commit to peti/nixpkgs that referenced this issue Jul 21, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant:

 - NixOS#16130
 - commercialhaskell/stack#2259
641fc0e
@peti peti added a commit that closed this issue Jul 21, 2016
@peti peti haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes #14897.

Also relevant:

 - #16130
 - commercialhaskell/stack#2259
641fc0e
@peti peti closed this in 641fc0e Jul 21, 2016
@peterhoeg peterhoeg added a commit to peterhoeg/nixpkgs that referenced this issue Jul 22, 2016
@peti @peterhoeg peti + peterhoeg haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant:

 - NixOS#16130
 - commercialhaskell/stack#2259
e1fd0e4
@vrthra vrthra added a commit to vrthra/nixpkgs that referenced this issue Jul 24, 2016
@peti @vrthra peti + vrthra haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant:

 - NixOS#16130
 - commercialhaskell/stack#2259
3a398d3
@kampfschlaefer kampfschlaefer added a commit to kampfschlaefer/nixpkgs that referenced this issue Jul 26, 2016
@peti @kampfschlaefer peti + kampfschlaefer haskell: remove all but the latest LTS package sets (version 6.7)
This is the first step towards dropping Stackage support. We keep LTS 6.x
around because I don't want to downgrade our default compiler to GHC 7.x,
but once LTS 7.x comes out we'll switch our main package set to that and
drop Nightly.

More details are at:

  http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

Closes NixOS#14897.

Also relevant:

 - NixOS#16130
 - commercialhaskell/stack#2259
142a748
@domenkozar
Member

@peti can I ask you to write a changelog entry for this PR?

@peti
Member
peti commented Sep 17, 2016

Yeah, you are right. The Haskell situation needs some documentation. I'll see what I can do. I'll be traveling next week, so I might not get to it until the week after that.

@domenkozar
Member

@peti could we document these changes today before the release? :)

@domenkozar
Member

@peti I've been talking to various people in Haskell community and this brings major pain to them. Could we keep more than just latest LTS package set?

This would give them a timespan of for example one year to migrate. Currently when they update nixpkgs, they get a new TLS set and they're forced to also upgrade at the same time.

I think keeping latest or a few latest sets would yield very little change to nixpkgs size.

@domenkozar domenkozar reopened this Sep 30, 2016
@peti
Member
peti commented Sep 30, 2016

@domenkozar, I'll add an entry to the release notes in a few minutes.

About offering LTS package sets again: I don't want to do it. My life as a maintainer for Haskell packages has become much easier since we're following the latest LTS release only. I already threw all the obsolete code out of my generator tools and everything is so much simpler and nicer now! I realize it's inconvenient for some, but I just don't want to bother worrying about ancient LTS package sets anymore, especially not in a situation where we don't have the bandwidth to do test builds for them. If someone else wants to maintain that stuff, I am all for it, but I don't want to any more.

@peti peti closed this in 6e785be Sep 30, 2016
@peterhoeg peterhoeg added a commit to peterhoeg/nixpkgs that referenced this issue Oct 2, 2016
@peti @peterhoeg peti + peterhoeg Document removal of LTS Haskell package sets in 16.09 release notes.
This patch closes NixOS#14897.
0ff5555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment