Read-only etc

[Mathnerd314]

For purity, /etc should consist only of automatically-generated files and symlinks into the nix store, and be managed only by the NixOS tools. There should thus be no reason for a user to edit these files. To enforce this policy, /etc should be remounted read-only after late boot, similar to /nix/store.

This conflicts with the currently-common practice of editing /etc/nixos/configuration.nix. I propose to deprecate this file and instead require specifying the configuration file as an argument to nixos-rebuild. /etc/nixos/configuration.nix can be removed or changed to a symlink into the nix store pointing to the config file in use and its dependencies (including e.g. the nixpkgs tree)

[CMCDragonkai]

One problem is that a lot of packages have designated config files that are not automatically generated by NixOS.

For example, there's the etc/mdadm.conf for mdadm and the modprobe.d/zfs.conf for ZFS.

We would need to add those functions to the those packages or else there would be no way to configure them.

[bluescreen303]

There's also dynamically changing stuff like /etc/resolv.conf and /etc/wpa_supplicant.conf if managed through wpa_gui.

[wmertens]

Volatile files can be symlinks to /run.
The only exception is sudoers which has to be a real file, but it is not
volatile 😊

To follow the sidetrack, I would love it if all files used to derive the
system were also in the store, perhaps rewritten in some canonical form.
That way a system derivation is completely self-contained and imperative
changes to the configuration become possible.

[Mathnerd314]

sudoers, mdadm, and zfs are all things that NixOS can generate, so I don't think those would be too much of a problem. wpa_gui and friends have options for moving the config file, probably to /run or /var. resolv.conf appears to be dynamically-generated by resolvconf and so should also move to /run. So I guess there will be problems but not too many.

[Profpatsch]

(triage) does anybody want to implement this?

[abbradar]

I do, see #16978 (comment). Alas, I was very busy lately and also will depart on a two-week trip in several days.

[lukateras]

Writable /etc is useful for recovery, e.g. when DNS server is broken, and adding networking.nameservers to config causes dependency on packages that have not been downloaded yet. The only way I see out of this situation is by manually editing /etc/resolv.conf, but do tell me if there's something I'm not aware of.

[copumpkin]

On the nameservers issue, I think there's a class of NixOS config that should be handled before other config: #6830

[joachifm]

@yegortimoshenko symlinks to /run can be used in simple cases like resolv.conf.

[abbradar]

In fact we already solve this with symlinks for dynamic files, for example:

lrwxrwxrwx 5 root root 13 Jan  1  1970 /etc/static/cups -> /var/lib/cups

(and yes, I know that I was suposed to work on this, shame on me ;_;)

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[ajs124]

still important to me

[7c6f434c]

still important to me

At some point I started an implementation which was sufficient for some NixOS tests: https://github.com/7c6f434c/nixpkgs/tree/readonly-etc but it did not pay off enough in case of VM tests, and I do not use NixOS mainline boot sequence on my laptop. (I do have /etc which is a symlink into /var/current-system/global/etc on my actual system)

[stale]

I marked this as stale due to inactivity. → More info

[xaverdh]

In fact we already solve this with symlinks for dynamic files, for example:

lrwxrwxrwx 5 root root 13 Jan  1  1970 /etc/static/cups -> /var/lib/cups

(and yes, I know that I was suposed to work on this, shame on me ;_;)

So maybe we could start by turning /etc/{passwd,shadow,group,subuid,subgid} into symlinks to files in /var/lib/nixos/ ?
This would allow people with mutable users / passwords to keep their setup, but also allow for a fully static etc.
The next step would then be an option which turns /etc into a symlink to the store.

[xaverdh]

Ah this will not work well, because some things expect them not to be symlinks I guess.. even bind mounts don't work with e.g. the passwd command, because it first creates passwd+ and a copy of passwd at passwd-. Is there some documentation on how this works?

[wmertens]

OP only take about /etc/nix*. Perhaps we can already move those to /nix/var/nix/etc ?

[7c6f434c]

Ah this will *not* work well, because some things expect them not to be symlinks I guess.. even bind mounts don't work with e.g. the `passwd` command, because it first creates `passwd+` and a copy of `passwd` at `passwd-`. Is there some documentation on how this works?

Actually, writing them like this is only necessary for just a few management tools, so rebuilding shadow to point to /var/auth instead of /etc just for setuid-wrappers and maybe general PATH, while letting everything use normal shadow and get the symlinks is an option.

[xaverdh]

Ah this will not work well, because some things expect them not to be symlinks I guess.. even bind mounts don't work with e.g. the passwd command, because it first creates passwd+ and a copy of passwd at passwd-. Is there some documentation on how this works?
Actually, writing them like this is only necessary for just a few management tools, so rebuilding shadow to point to /var/auth instead of /etc just for setuid-wrappers and maybe general PATH, while letting everything use normal shadow and get the symlinks is an option.

ah passwd has a chroot option, so it could also be wrapped to chroot into a different location in /var..

[7c6f434c]

ah `passwd` has a `chroot` option, so it could also be wrapped to chroot into a different location in /var..

Note that patching the default location will also work with setuid installation, but using chroot options is true-root-only. Funnily enough, you need more than just the auth files inside the chroot (I use this, and I ended up bind-mounting the basic tools into chroot, too — not that surprising given PAM loading DSOs with modules).

[stale]

I marked this as stale due to inactivity. → More info

[buckley310]

Still interested

[lucasew]

Is impermanence what you are looking for?

[Mathnerd314]

It's certainly related. To phrase the issue in terms of that work, this is making sure that all the persistent state is stored in /var instead of /etc. So based on the example in the README it looks like /etc/NetworkManager/system-connections, /etc/machine-id, and /etc/nix/id_rsa should all be symlinks to /var.

[buckley310]

I like impermanence, but it's designed for the "make a mess, then delete it" strategy, like docker. The way i see it, NixOS generally tries to avoid making a mess in the first place.

[Guanran928]

Does system.etc.overlay.mutable introduced in #270727 resolves this?

I personally haven't tried it yet, because it does not work with secret management tools like sops-nix.
However, based on the description, it seems to be the solution this issue requires.

[Mathnerd314]

It indeed seems to satisfy the requirements. We could leave this open as a tracking issue for defaulting to
system.etc.overlay.enable = true, system.etc.overlay.mutable = false, or just close it and let people file new bugs.

[Guanran928]

We could leave this open as a tracking issue for defaulting to
system.etc.overlay.enable = true, system.etc.overlay.mutable = false

IMO we should open a new issue for that.

[nyabinary]

We could leave this open as a tracking issue for defaulting to
system.etc.overlay.enable = true, system.etc.overlay.mutable = false

IMO we should open a new issue for that.

did anyone open it?