[RFC 0144] Versioned Flake References #144
Conversation
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/rfc-0144-versioned-flake-references/26502/1 |
|
I'm not sure it makes sense to have RFCs for modifications to flakes, when flakes itself is still experimental and pending an RFC? (Previously) It might be better to just discuss with the Nix developers. |
|
It was only a matter of time. |
Unlike the first flakes RFC, this one has a reasonable scope. I feel like this RFC could provide a good centerpoint for the discussion of such a feature. @figsoda do you plan to implement this feature yourself? An issue in the repo could serve a complementary role, such as for discussion of implementation work for example. We have an RFC label :) @Profpatsch it's not clear to me what you are referring to. Could you elaborate? |
Well, another question is which of the assumptions about flakes might become complicated issues in the next attempt to perform stabilisation through RFC. (The current plan might avoid all of them; but RFCs often get large rewrites during the discussion) |
|
I'd rather add some |
no, since I don't know C++ |
|
I don't really have an opinion whether that change could be a good one for Nix or not. However, since this looks like it could be easily implemented outside of Nix, I'd strongly suggest doing so first so that everyone can judge the usefulness. It would be much faster for a first version, and that would make the case much stronger. Especially since the main concern right now with Flakes is getting them stabilized, which goes against adding extra features on top of them in the short term. |
|
I can work on a proof of concept when I get time |
|
How would this be implemented outside of nix? Doesn't it change the schema of the |
|
I like the idea, but think versions should be in |
I can write a preprocessor that expands the placeholders
How would nix be able to find the version without fetching every single tag? |
| A new Nix command, `nix flake upgrade`, will edit `flake.nix` and upgrade all | ||
| the version placeholders in the inputs. A new flag, `--upgrade-input` will |
There was a problem hiding this comment.
Upgrading flake.nix in-place is not really an option since we don't have the ability to do that. But I also don't see why it's necessary. nix flake update can simply update to the latest versions that are consistent with the version placeholders. At most, we want a flag to nix flake update to allow/disallow major upgrades that wouldn't be semantically compatible.
There was a problem hiding this comment.
If a flag were to temporarily ignore semantic compatibility, wouldn't that introduce some weird behaviour? Wouldn't the update behaviour on subsequent uses (without the flag) be to downgrade to a compatible version, thus defeating the purpose of initially using the flag? The idea here is to permanently alter the semver to accept the latest version as a convenience (so the user doesn't need to track down the latest tags of every input). I can't see any way of doing that (without potentially confusing behaviour) but inplace changes to flake.nix.
Alternatively, we could have a command/flag which would report the latest tags of the inputs (similar to how update reports lock file changes). This way there's no need to edit flake.nix inplace and the user gets the necessary information at once instead of playing tag whack-a-mole. It's less powerful but still works to make version tracking much less annoying which, I think, is the ulterior motive of this RFC.
| - Not adding the feature, manually update the flake references | ||
|
|
||
| # Unresolved questions | ||
| [unresolved]: #unresolved-questions |
There was a problem hiding this comment.
Not discussed here is the algorithm for selecting versions. In conjunction with follows/overrides, finding a feasible solution could become expensive, e.g. in
inputs.foo.url = "github:bla/foo/{>1.2,<1.5}";
inputs.bar.url = "github:bla/bar/{^1}";
inputs.bar.inputs.foo.follows = "foo";
Nix would need to find a version of bar whose foo input has constraints compatible with the ones given here.
I wouldn't want to have a SAT solver in Nix...
There was a problem hiding this comment.
My thought was that bar's requirement for foo would be ignored/overridden, so only {>1.2,<1.5} would be used, and version requirements would always be combined with and
maybe I should clarify that in detailed design
There was a problem hiding this comment.
I wouldn't want to have a SAT solver in Nix...
We should consider having one outside of Nix, making lockfile generation pluggable.
Perhaps the version constraints should be in a separate attribute that's open for extension?
In fact, we could consider the lock reading and fetching to be part of Nix, and the Flake CLI to be one implementation of the broader Nix package management idea. Dream2nix is also in the business of lock file generation. Perhaps the role of Nix should be limited to the integration of lock file generators, and not actually implementing them. We don't even need a dumb lock file generator like we currently have because that one could just as well live in a flake that's already been locked.
Nix should focus on its core competencies that it does very well.
- building immutable store paths
- bootstrapping other software
Anything that's built on top of that functionality is going to feel as good as native, plus have the benefit of pinning, patching, and decentralized development.
Flakes is oddly centralized for a package manager built on Nix...
There was a problem hiding this comment.
Late to the party, but let me just write this down just in case. One way to avoid the need for SAT-solving is to allow only semver open (^1.2.3) version requirements. That way, a greedy algorithm which just always picks the latest semver-compatible version works.
Moreover, in practice, in Cargo:
- 99% of constraints are in fact just
^1.2.3(it was genius design decision to makeversion="1.2.3"be a syntactic sugar forversion="^1.2.3") - people often try to use non-semver open constraints, most of the time that's a wrong thing to do, and leads to problem down the line (eg, their build works, but builds of folks depending on them gridlocks due to unsatisfiable constraints)
- at some point we even considered deprecating non-
^constraints, but didn't do that, because there are occasionally good reasons for them
So, perhaps a safe move wold be to allow only ^1.2.3 constraints, (and using only implicit 1.2.3 syntax). If that turns out to be expressive enough, that would be a massive simplification. If not, fancier constrains can be added later.
| [motivation]: #motivation | ||
|
|
||
| This will allow Nix libraries to be versioned without requiring their users | ||
| to manually update them. Some package managers (e.g. cargo) for more |
There was a problem hiding this comment.
Here you cite cargo, but the specification they employ to do those machine-driven updates is very dumb - by design. Cargo can update patch version, but minor version has restrictions and major is almost impossible.
It looks so full of restrictions that it is easier to edit by hand.
There was a problem hiding this comment.
Could you elaborate on what "full of restrictions" means? I don't think attacking a reference I made is a good explanation.
There was a problem hiding this comment.
The example tool you gave, Cargo, is purposefully very limited. This design decision conveys the idea that major and minor versions should not be upgraded by a machine, but by a human being.
But your specification is too complex (maybe a consequence of Nix flakes being too complex), and it feels problematic to me.
There was a problem hiding this comment.
This design decision conveys the idea that major and minor versions should not be upgraded by a machine, but by a human being.
The whole point of defining a set of rules for the version specification is to make them upgradable by a machine, hence the strict rules on what version bumps can introduce breaking changes. This is so libraries can follow a predictable pattern, instead of their own preference, so updating can be automated.
Your response still doesn't explain what is limited, and I don't see why Cargo being supposedly limited has anything to do with my proposal being problematic. The purpose of this proposal is to give library authors the freedom to version their library, instead of the current situation of users following the main branch and library authors not having a good communicable way to introduce breaking changes.
There was a problem hiding this comment.
Your response still doesn't explain what is limited
I said, and I will say again: Cargo (or more precisely cargo.toml) is limited. Being limited, it is easy to understand and implement.
Your proposal is too wide. It introduces more syntatical and semantical rules - implying more burden for the compiler writer (of a language with no formal specification yet, by the way).
Further,
give library authors the freedom to version their library
Goes against
strict rules on what version bumps can introduce breaking changes.
The design decision should find the equilibrium between these two points.
There was a problem hiding this comment.
Cargo (or more precisely cargo.toml) is limited.
If you are talking about the expressiveness of Cargo.toml vs flake.nix, I agree. However, I don't think the difference applies to this specific feature.
It introduces more syntatical and semantical rules.
Cargo defines much more rules in this aspect and also more strictly enforces it, as it will fail if the version is absent or not a valid semver.
Implying more burden for the compiler writer
I agree.
freedom goes against strict rules
It doesn't. They are talking about completely different things. This feature will always be opt-in. Library authors can still not version their libraries, or define their own versioning scheme - they just wouldn't be benefited by the change. The only way tooling like this can happen is if people following a predictable set of rules when versioning their libraries, which this proposal will allow by defining strict rules.
|
Overall, to me this RFC looks like a solution to a non-existent problem. |
It comes down to how we define the scope of Flakes. Is it a small extension to a functional build system, or is it a package manager for a highly distributed ecosystem of packages similar to language-specific package managers? If you use Nix for a traditional snapshot-style distribution like Nixpkgs or NixOS, your needs are covered by the functionality already provided. If you use many flakes, to the point where either the libraries used in your expressions, or the many semi-independent components that constitute your build, all coming from independently versioned flakes, then the problem domain starts to look a lot like that of the language-specific package managers. Bumping to latest doesn't work, and tracking all constraints manually becomes a pain. EDIT: Continuing from scope to design, continue reading here |
lheckemann
added
the
status: open for nominations
|
This RFC has not acquired enough shepherds. This typically shows lack of interest from the community. In order to progress a full shepherd team is required. Consider trying to raise interest by posting in Discourse, talking in Matrix or reaching out to people that you know. If not enough shepherds can be found in the next month we will close this RFC until we can find enough interested participants. The PR can be reopened at any time if more shepherd nominations are made. |
|
This RFC is being closed due to lack interest. If enough shepherds are found this issue can be reopened. If you don't have permission to reopen please open an issue for the NixOS RFC Steering Committee linking to this PR. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/announcing-flakestry-dev-new-registry-for-flakes/34583/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/announcing-flakestry-dev-new-registry-for-flakes/34583/7 |
Summary
Flake references can have a special placeholder that will specify the version
requirement of the flake, which
nix flake updatecan use to update the pin tothe latest version compatible with the version specification. A new Nix
command,
nix flake upgradewill upgrade the version requirement inflake.nixto the latest possible version without regard to compatibility.Rendered