doc/manual: define {local,remote} store, binary cache, substituter

[ghost]

Nix veterans intuitively know what the following terms mean. They are used in several places in the nix documentation, but never defined:

• local store
• remote store
• binary cache
• substituter

In particular, I found the last two terms to be confusingly similar. Let's give definitions for them.

This PR contains my attempt at giving a definition for each of these four terms, and mentioning the important-but-non-obvious fact that binary caches are kept in a different on-disk format than stores. Not realizing this was one of the biggest obstacles back when I first started using nix -- I stumbled over this for several hours before realizing you can't just swap them back and forth.

If my definitions are wrong or just poorly worded please replace them.

[edolstra]

This probably should go somewhere else since the manual is going to become a pure reference manual (and the "package management" section is outdated and will probably be removed).

[ghost]

the manual is going to become a pure reference manual

Isn't that what a reference manual is for? Providing definitions for product-specific jargon? I wouldn't know where else to look for such information.

$ rg -e '((local|remote) store)|substituter|binary cache' doc/manual/ | wc -l
115

[toraritte]

This is great and it should probably also be put in the Glossary.
@fricklerhandwerk

[ghost]

This is great and it should probably also be put in the Glossary.

Wow, I don't know how I overlooked that.

Yes, that is exactly where this stuff belongs.

I will implement your feedback and then make a commit to move it (otherwise I can't use github's "commit suggestion" button)

[ghost]

4de95f7 tries to at least acknowledge the question "so why do we need binary caches with a totally different on-disk format?" If a better/longer answer to this question materializes later we can delete this text. It's the other thing that drove me slightly crazy when I was first learning Nix.

It still seems a bit weird that we have two totally incompatible disk layouts for Nix stores, but I guess that's the only way to serve from S3 since it can't run a nix-serve daemon.

[ghost]

By the way, I've tried to match the existing formatting by hard-wrapping at 78 columns, but perhaps we shouldn't be hard-wrapping markdown text? It makes the diffs really hard to read since github doesn't do wdiff. Unlike code, editors can visually rewrap markdown pretty well.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/summer-of-nix-documentation-stream/20351/4

[fricklerhandwerk]

@amjoseph-nixpkgs I agree on stopping the hard-wrapping, and have a pull request for the nix.dev contribution guide, which is the central place to coordinate documentation work and should (in the long run) aggregate best practices and set examples for other documentation resources to follow - something that should be easily visible but the PR against nixos.org is yet to be merged.

I suggest to apply that to new changes, and would be happy to see a PR against the contribution section. @edolstra objections?

Other than my comment on the footnote this looks good.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/summer-of-nix-documentation-stream/20351/2

[kjeremy]

How does this reconcile the fact that the binary-caches option in nix.conf is deprecated in favor of substituters when they represent two distinct ideas?

[Ericson2314]

I like documenting these things in reference form, but this text is not right.

Here's what is going on.

1. There is a Store interface (in store-api.hh). A few related interfaces extend it in various ways (e.g. LocalFSStore, in local-fs-store.hh, which means a Store that exposes stuff via the OS filesystem). There are various implementations of these interfaces.

2. By convention many commands / thing one can do can take a "main store", and also "other stores", in various capacities. "Substitors" are just stores used in a read-only way to get stuff we don't have in the main store.

3. "Binary cache" is/was another name for substitutor. In the implementation, it is also used for a family of stores which use compressed nars, narinfo files, etc. as an implementation detail. These implementation details are not very important.

So the thing to emphasize is the abstract interface of stores, versus the local store everyone is used to which has many "bonus features". "substitutor" sounds like an intrinsic property of a store, but it isn't, it is instead a term about how the store is used. It's thus somewhat a misnomer --- a classic example of natural language words literally referring to objects when the actual concept it notes is a relation between objects. "binary cache" is probably just too overloaded at this point and we should drop it. I would be fine renaming everything that is internally called BinaryCache to NarinfoStore or similar, for a fresh term without baggage.

[fricklerhandwerk]

@Ericson2314 Then what about we consistently use substituter (is that even properly translated from Latin into English? shouldn't it be substitutor?) and collapse the notes here into one?

[ghost]

3. "Binary cache" is/was another name for substitutor.

I got this terminology from libstore, which definitely uses the term "binary cache" (and BinaryCacheStore) in a way which does not encompass all substitut{o,e}rs.

In fact, a BinaryCacheStore isn't even a RemoteStore. Neither extends the other.

This distinction isn't just internal to the codebase; it has leaked out onto user-facing surfaces. I'm just trying to document the current situation, not advocate for it.

[ghost]

How does this reconcile the fact that the binary-caches option in nix.conf is deprecated in favor of substituters when they represent two distinct ideas?

All binary caches are substituters but not vice versa. Adding the more-general option substituters made the more specific option binary-caches unnecessary.

[ghost]

Boy I wish github would let me change peoples' non-repliable comments into repliable comments. It's so much easier to read things with at least rudimentary threading...

[ghost]

3. "Binary cache" is/was another name for substitutor.

This distinction isn't just internal to the codebase; it has leaked out onto user-facing surfaces.

Here's the one I stumbled over the same day I first tried nix:

nix/src/nix/copy.md

Line 11 in 4823067

Note the `file://` - without this, the destination is a chroot

nix/src/nix/copy.md

Line 12 in 4823067

store, not a binary cache.

Depending on whether you point nix copy at a file://-url or a bare path, it produces results in one of two totally different (and non-interchangeable) formats.

Also, I will add an entry for "chroot store" since:

$ rg -i chroot.store | wc -l
11

[edolstra]

"Binary cache" is/was another name for substitutor.

That has never been the case. From the very start (37f7098), it was the name for a particular kind of substituter, namely the one using .narinfo etc. It replaced the manifest-based substituter (scripts/download-using-manifests.pl.in). The confusion between the option names binary-caches / substituters probably comes from the fact that originally, binary-caches was an option of the binary cache substituter and so could only contain binary cache URLs. When substituters were generalized into store types (with the Perl -> C++ migration), the binary-caches option could be generalized into a list of store URLs.

I don't think calling it the "narinfo store" would be more descriptive to end users. "Binary cache" is not a great name but it is what is used everywhere now.

[fricklerhandwerk]

This distinction isn't just internal to the codebase; it has leaked out onto user-facing surfaces.

From the perspective of trying to present a sane interface in documentation, I think this is a bug. Such things should not be part of the manual, but documented in an issue.