Make sense of how we do/don't install things

[roberth]

Problem

We're at a junction.
The Nix team has discussed spinning out commands that are considered essential by many. This could stimulate innovation or cause unnecessary fragmentation that just makes the disparate tools hard to relate and interoperate.

Proposal

• Put some effort into categorizing the various ways in which a package or store path can be used. This will be a valuable reference point for anything that builds on top of Nix.
• Consider how NixOS/nix should deal with each method of installation. How does it relate to the other method? (usually a trade-off between ease of use and solving compatibility problems) What is a minimal way in which the Nix CLI could interact with this type of installation? Should it be deferred to another project? What would such a project need from Nix? In other words, design interfaces between expressions and the CLI, and/or between the CLI and users, and/or between the CLI and other projects.

Prior art

• Elaborate comment rethinking the CLI per usage method

• A spectrum of (non-)installation method ranging from easy-to-use to most compatible

  Spectrum of installation methods

  None: nix run, nix-build

  Env: nix shell

  Combined env: nix shell with multiple packages through buildEnv

  Nestable combined env: nix shell on buildEnv, aware of nesting of nix shells so that there's only a single env

  Ad hoc mutable user profile: nix-env, nix profile

  Transient combined env from file: nix develop (nix-shell without -p)

  Stateless mutable user profile: buildEnv-like, managed in a mutable profile, e.g. home-manager

  Stateless mutable system profile: buildEnv-like

  FHS environment (not sure where this fits actually)

  Stateless image

  ---

  So this is probably not a spectrum, although I think it kind of starts out that way. Perhaps a comparison table would be a better fit?

• This issue could be considered as a generalization Split nix-shell logic #777, except it wouldn't be a single decision like that and the outcome could be different.

• ... please add more ...

Priorities

Add 👍 to issues you find important.

[Ericson2314]

Also, we can consider both spinning commands out of Nix, or "unspinning" the "core" out of Nix.

E.g. we can sidestep some of these concerns by saying Nix is the compat or somewhat-batteries-included layer.

To bad the name, Nix/Niets, is so good for the innermost core which isn't sufficient to use on its own!

[roberth]

I don't think we have a single core; I'd go with the existing library names or concepts and perhaps derive specifications from them.

A lot could be said about that, but the goal of this issue is to clarify and lead in Nix's original application domain, which is software deployment, rather than just a build tool library (or whatever would be the components that the Nix team would scope down to hypothetically).

Or perhaps decide not to lead; not even at the conceptual level, and completely ignore how users use the core concepts (such as derivations and expressions) to achieve their goals, let a bunch chaos, bugs and duplication happen, which in my opinion means we fail to serve our user base, if this responsibility is not covered by another group who looks into this kind of stuff.

Such a separation of concerns would make sense, and I think we could have two teams on this repo.
Of course we'll have cross cutting concerns and a big interface to share, so good communication will be important, but a lot of the problems we try to solve are usually either primarily in the bowels or primarily in the actual CLI area which actually affects UX most.
It'd scale better than growing a single team to have more members.

[roberth]

(although the most effective scaling we can do rn is making the existing members spend more time; perhaps have a fulltime member)

[fricklerhandwerk]

We discussed related issues again with @tomberek and others at Ocean Sprint, and arrived at a the conclusion we already had at some point in the maintainer meetings: we should split the team in at least two - library/framework, backend/frontend, store/{language,cli}, whatever you call them. @tomberek's argument is to use Conway's law to our advantage by nudging the architecture to follow the organisational structure, thereby reducing cognitive load for component maintainers to make better progress on each.

One thing I tend to repeat myself about is our still underdeveloped routine of saying "no" to lower-priority inquiries in order to focus on finishing things we started. This is because we collectively seem to want too much at once, at least way more than we can realistically handle.
I think an organisational split as proposed above would help framing problems in ways that makes them more tractable. We just would have to watch out that we have a pathway for contributions we have to decline. This plays into the broader issue of having places where to put work done with different levels of commitment, see the governance discussion about official projects.

All this apart from just having more time to work on deep issues that are hard to untangle with only an hour or two here and there.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flakey-profile-declarative-nix-profiles-as-flakes/35163/3