Track the dependencies of derivation outputs #4267

…sed ones Fix an overlook of NixOS#4056

This requires adding `nix` to its own closure which is a bit unfortunate, but as it is optional (the test will be disabled if `OUTER_NIX` is unset) it shouldn't be too much of an issue. (Ideally this should go in another derivation so that we can build Nix and run the test independently, but as the tests are running in the same derivation as the build it's a bit complicated to do so).

For each “known” derivation output, store: - its output id - its output path This comes with a set of needed changes: - New `drv-output-info` module declaring the types needed for describing these mappings - New `Store::registerDrvOutput` method registering all the needed informations about a derivation output (also replaces `LocalStore::linkDeriverToPath`) - new `Store::queryDrvOutputInfo` method to retrieve the informations for a derivations This introcudes some redundancy on the remote-store side between `wopQueryDerivationOutputMap` and `wopQueryDrvOutputInfo`. However we might need to keep both (regardless of backwards compat) because we sometimes need to get some infos for all the outputs of a derivation (where `wopQueryDerivationOutputMap` is handy), but all the stores can't implement it − because listing all the outputs of a derivation isn't really possible for binary caches where the server doesn't allow to list a directory.

Copying a derivation output is equivalent to copy the output path, except that it will also link the output to the path (with `linkDeriverToPath` on the remote store)

Commands that accept store paths or installables as input can now take a derivation path with outputs. Like when the input is a nix expr, the derivation will be realised if needed.

Generalises `StorePathsCommand` to allow manipulating `Buildables` rather than just store paths.

Mostly dummy atm as it just computes the closure of the output path, but might be extended later

Add a new table for tracking the derivation output mappings. We used to hijack the `DerivationOutputs` table for that, but (despite its name), it isn't a really good fit: - Its entries depend on the drv being a valid path, making it play badly with garbage collection and preventing us to copy a drv output without copying the whole drv closure too; - It dosen't guaranty that the output path exists; By using a different table, we can experiment with a different schema better suited for tracking the output mappings of CA derivations. (incidentally, this also fixes NixOS#4138)

That way we can `nix copy --from` a store that doesn't have the derivation but just its outputs

That way we don't need to resolve it again to know its output paths. In particular, this means that we don't need to keep its whole build-time closure

`buildPaths` can be called even for stores where it's not defined in case it's bound to be a no-op. The “no-op detection” mechanism was only detecting the case wher `buildPaths` was called on a set of (non-drv) paths that were already present on the store. This commit extends this mechanism to also detect the case where `buildPaths` is called on a set of derivation outputs which are already built on the store.

To prepare for the upcoming DrvOutputSubstitutionGoal

Requires a slight update to the test infra to work properly

Currently never used, nor set but will be useful shortly

We need to topologically sort them like we do for the store paths, otherwise the registration will fail because of an incomplete closure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Track the dependencies of derivation outputs #4267

Track the dependencies of derivation outputs #4267

Commits on Dec 1, 2020