Remove huge packages

[matthewbauer]

Motivation for this change

I'm going through some of the larger files in the Nixpkgs repo. Some of these may still be used by people, but their data files are taking up a lot of space.

Removes:

• codimd (680K)
• remarkjs (broken, 116K)
• ue4 (unfree, broken, 468K)

Also:

• Gets rid of herqq's gcc6.patch (unused, 56K)
• Remove data files from pharo vm (72K)

Let me know if anyone is relying on these to work. This gets about 1.5M from the nixpkgs which is a about 1% of total repo space.

[Ma27]

I know from some people that they use codimd with Nix (including myself on my personal setup), so I'm 👎 at least for codimd.

cc @WilliButz

[WilliButz]

I agree with @Ma27 on this, having a Nix setup makes it a lot more comfortable to set up and maintain stuff like this.

As most of codimd's size comes from the generated node dependencies, maybe another approach would be more sensible. Instead of just removing the whole thing, has there ever been any attempt to move out the generated node-packages and then reference them via fetchurl or something similar? I know that this is not the optimal solution but it would certainly reduce the size of nixpkgs :)

[vcunat]

Importing from a derviation would have its own issues, like tools not counting on builds happening during evaluation. In particular, we probably don't want that to happen on Hydra.

[vcunat]

/cc other maintainers: @rickynils, @puffnfresh.

[veprbl]

cc @ivan regarding ue4

[bhipple]

For large generated package sets that are only used by a few users, or for large proprietary packages that we may not want to dedicate community resources to, something like @Mic92's NUR seems ideal:
https://discourse.nixos.org/t/introducing-nur-the-nix-user-repostory-share-all-your-nix-expressions/431

It lacks some of the convenience and discoverability of having all of the packages right there in NixPkgs, and as @vcunat mentions import-from-derivation brings a host of issues, but I could see this someday providing frequently updated channels for some of the huge, auto-generated package sets, such as the emacs, texlive, or nodejs ones.

For instance, texlive by itself is 2.1M, nearly all of which gets regenerated when we do an annual update. emacs has 6.1M of melpa packages as well, which get updated more frequently than the texlive packages.

[ivan]

I don't use ue4 and don't plan to fix it right now. It looks like it was added in #13110 but it's odd that it ever worked - I thought the entire repo was always set to Private since its inception.

[Mic92]

@Ma27 codimd should be integrated into our existing node-packages set so it can reuse existing definitions.

[danbst]

@vcunat if Hydra is main objection for IFD, than we can explicitly mark "no hydra" for IFD packages. And disable nixos test as well. We only have to assure, that meta is accessible without IFD, and this can be done with mkDerivation wrapper.

While for compiled languages (like Haskell) I see benefits of using Hydra builds VS compiling myself, for interpreted languages like JS/Python build cache isn't much required.

[danbst]

@vcunat actually, I can answer myself. Major downside for IFD here is that IFD plays bad with GC, so if you don't add GC root for each IFD you will redownload everything from scratch after nix-collect-garbage.

[7c6f434c]

The GC pinning sounds feasible on the package level. I.e. when doing IFD we could pass the reference to that same derivation, then the package could optionally pin its own definition.

[globin]

I'd prefer keeping codimd, as I know quite a few people using it.

[oxij]

👍 on removing unused patches and etc, I'm sure there are a bunch more of these there. But I don't understand what the reasoning behind removing packages (or moving them to a separate repo) is. Even for the broken ones. Say you moved nixified MELPA to NUR or whatnot. The cons I see (lower discoverability; the resulting pressure to never change anything in `stdenv` and other commonly shared infra; things like OfBorg and `nix-review` losing their effectiveness; and so on), but what are the pros (except saving mere 6 megs of space from the working tree, which is almost nothing anyway and your git history will have it all saved forever anyway; if anything, you will use _more_ space by moving things between repos; and note that you don't evaluate these things if you don't use them anyway)? If something is bothering Hydra - let's disable it. If different people want different sets (like I do) let's merge something like #45841 and get happy. I feel fine with removing broken packages only when they stop _evaluating_ (which rarely ever happens now thanks to OfBorg) and there was a long period with no interest in fixing them before that. Otherwise, removing stuff only makes it harder to rediscover and fix later. Removing working stuff should be a complete no-no, IMHO. If anything, I would love nixpkgs to provide nix recipes for everything, even some obscure not really UNIXy stuff. E.g., I'd very much like FDroid to be using nixpkgs, because while they say they build things from source, in fact, very often they don't: a lot of build scripts there do non-pure things like `wget`, and replicating those builds is hard. I say, let's add more shit nobody wants to build by hand to nixpkgs. Removing things that worked before or almost work now is counterproductive, IMHO.