Export & Compare NixOS configurations.

[nbp]

One problem raised by multiple users, almost since the beginning of the module system, is that there is no way to reproduce a configuration once it has been compiled, and it is hard to distinguish the differences between 2 NixOS generations.

To address these problems have to handle the following issues:

• Distinguish between external / internal modules.
• Make a copy of all external modules used during the generation of the configuration.
• Make these modules reusable even after the relocation in the Nix store.
• Catch failures caused by the strictness of the evaluation of mandatory default values.
• Generate a configuration file which summarizes the all user-defined option definitions.
• Provide a CLI to diff the user-defined option definitions of different generations.

[offlinehacker]

+1 That would be usefull :)
On Mar 30, 2015 10:21 PM, "Nicolas B. Pierron" notifications@github.com
wrote:

One problem raised by multiple users, almost since the beginning of the
module system, is that there is no way to reproduce a configuration once it
has been compiled, and it is hard to distinguish the differences between 2
NixOS generations.

To address these problems have to handle the following issues:

• Distinguish between external / internal modules.
• Make a copy of all external modules used during the generation of
  the configuration.
• Make these modules reusable even after the relocation in the Nix
  store.
• Catch failures caused by the strictness of the evaluation of
  mandatory default values.
• Generate a configuration file which summarizes the all user-defined
  option definitions.
• Provide a CLI to diff the user-defined option definitions of
  different generations.

—
Reply to this email directly or view it on GitHub
#7092.

[jagajaga]

👍

[copumpkin]

👍 👍 👍 👍 👍

[copumpkin]

Perhaps we could just get away with a primop for "evaluation-time closure"? Then you'd have a configuration.nix include a field for its own evaluation-time closure and call it a day. I'm probably skimming over all sorts of hard problems in there though.

[copumpkin]

Thinking about this some more, is there any reason this is any different from any other derivation? It would generally be nice to retain (in a secure way) source that generates a given derivation (including the system one) for auditing and reproducibility purposes. If we structure it properly, the channel or git working directory you're using could just become another derivation so that we don't need to copy everything every time.

A quick sketch:

• /nix/store/fjkslafjakl-system/ contains a NixOS system and a reference to configuration closure and channels in effect (as store paths)
• /nix/store/jklajfklajkfljskl-configuration-closure contains a copy of the .nix sources for configuration
• /nix/store/klfjsakfljawklfjkl-channel-foo/ contains one channel you had configured when you built the system

Now if I want to reproduce the full config for a given system derivation, I can look at its reference to the configuration and the channels it used (even if they were local dirty git working directories!) and get all the nix source that was used. If we then get deterministic system tarballs, you should be able to go back and ensure that rebuilding the configuration with the same channels produces the same output.

There's nothing particularly special about the configuration derivation though, and it seems like e.g., my openssl derivation should be able to get the same treatment, with the same auditing/verification benefits.

[nbp]

There's nothing particularly special about the configuration derivation though, and it seems like e.g., my openssl derivation should be able to get the same treatment, with the same auditing/verification benefits.

I agree, but I was wondering if we really want to keep all the files involved in building a package / system, knowing that most of them are already store in a vcs somewhere.

Also, independently of keeping all the source files, I do think that there is some interest to keep (or be able to generate) a configuration file which summarize all (even internal) definitions used to build a system.

[copumpkin]

I realize that the files are probably in source control somewhere, but the key thing we gain with what I describe is linkage: we know for sure which revision (or even a dirty uncommitted revision) went into a given derivation. Someone could then take an arbitrary derivation in the store, look at e.g., nix-support/nix-sources (just a random name I made up) and either see whether it's what was intended, or attempt to rebuild the .nix files referenced and see if they end up with a meaningfully different derivation (a fancier version of nix-build --check for the paranoid).

Perhaps just a flag somewhere could specify if we want to keep them around? Not everyone cares about rigorous auditability, but a security-conscious person or organization will be willing to take the (small!) space hit to store a few megabytes of .nix files.

[copumpkin]

I realize multiple .nix source layouts could result in the same derivation but don't think it's too concerning, if they result in the same output. It would of course be nice to be able to store all of them but it doesn't seem like the common case, and in general it seems fine to just store whichever inputs first created a given derivation if it's a lot easier.

[copumpkin]

It looks like the feature reported as broken in #7974 actually gets us one step closer to the goal of this ticket.

[magnetophon]

+1 for this.
See also https://github.com/matthiasbeyer/nixos-scripts/issues/8

[copumpkin]

See also NixOS/nix#553 (comment).

[Profpatsch]

(triage) any new leads?

[puffnfresh]

There's a nix-diff tool which can do comparisons between derivations. This blog post shows it being used to compare 2 systems:

http://www.haskellforall.com/2017/11/compare-nix-derivations-using-nix-diff.html

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[magnetophon]

still important to me

[Profpatsch]

Are there any ways we can take this forward? Simple first steps?

[magnetophon]

I guess the export part would be a first step.
Right now I use:

  system.extraSystemBuilderCmds = ''
    ln -s ${./.} $out/full-config
  '';

because the last time I checked,

system.copySystemConfiguration = true;

only copies one file, not any imports it uses.

It would be nice if all the files that are imported, but not other ones, would be copied by this option.

[stale]

I marked this as stale due to inactivity. → More info

[RyanGibb]

Are there any ways we can take this forward? Simple first steps?

@Profpatsch Flakes solved this for me. NixOS records the git revision of the configuration repo [0].

[0] https://www.tweag.io/blog/2020-07-31-nixos-flakes/#hermetic-evaluation