[RFC 0164] Nim library lockfiles

[ehmry]

This RFC describes a refactoring of Nim packages within Nixpkgs.

Rendered

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfc-0164-nim-library-lockfiles/34140/1

[pbsds]

This could be solved by making buildNimPackage set a default passthru.updateScript

[FRidh]

Good to see this proposal. Given the impact I don't think there is a need to have an rfc at this place, and that instead having a regular issue in Nixpkgs is sufficient, potentially announced via discourse. It would still be an rfc, but without the process that is involved here.

[AndersonTorres]

I like the idea, but I believe it does not need an RFC. A team would suffice.

[ehmry]

This is an issue of policy rather than a request to move some files around in Nixpkgs. I want to know what the long term goals are for languages like this and what the policies are for checking non-nix files into Nixpkgs.

I could have already merged everything I've proposed but I expect that I would get zero review before merging and some complaints afterwards.

Should I merge first like we've done with the other RFCs?

[alyssais]

Here's a concern I have in general with lockfile-based packaging in Nixpkgs, but I don't know if it applies to Nim:

If there's a security vulnerability in curl, we update our curl package, and then thousands of packages that use our libcurl all have the vulnerability patched. If there's a security vulnerability in a Rust library, we have to update every leaf package individually to pick up the new dependency, and this would usually mean patching the lockfile. In practice, nobody does this.

How easy would it be to update a Nim library across the board, if it was found to have a vulnerability or annoying bug?

[ehmry]

@DavHau recommended that we pass the libraries selected by lockfiles though a function that applies overrides according to version constraints, which normally would be used to add buildInputs or add some compiler flags. The same mechanism could also detect libraries with known vulnerabilities and propagate a breakage.

I should prototype something to do this in a language agnostic way before undrafting this PR.

[alyssais]

Yeah, that sounds like a good idea, and what we already do for Ruby. Would it allow straight up overriding the source to point to a newer version? I don't like how reliant we are on upstreams to update their lockfiles.

[ehmry]

I've implemented lockfile overriding by creating an attrset of overlays that take the lockfile data as a parameter, so that should be powerful enough to implement version constraints. - https://github.com/ehmry/nixpkgs/blob/nim-annotations/pkgs/top-level/nim-annotations.nix