Nix should not `mount -o remount,rw`

Can anybody explain why this is here?

nix/src/libstore/local-store.cc

Lines 576 to 578 in 516e7dd

	if (stat.f_flag & ST_RDONLY) {
	if (mount(0, realStoreDir.get().c_str(), "none", MS_REMOUNT | MS_BIND, 0) == -1)
	throw SysError("remounting %1% writable", realStoreDir);

I traced it back to b9124a5. It looks like the original codepath would check if /nix/store was a --bind mount and act only in that specific situation.

The comment says:

nix/src/libstore/local-store.cc

Lines 565 to 566 in 516e7dd

	/* To improve purity, users may want to make the Nix store a read-only
	bind mount. So make the Nix store writable for this process. */

Note that this codepath gets hit not only for the /nix/store store, but also when you use nix copy --from /mnt/somewhere/nix/store. For example when recovering from a mangled system and trying to copy off whatever can be salvaged. Why would the --from store need to be remounted read-write for that to happen?

Some filesystems, in particular btrfs, will react to serious problems by switching the filesystem to readonly mode, and interpret any subsequent mount -o remount,rw as a "I know what I'm doing" signal by the root user. I don't think it's a good idea for nix-daemon to signal this unilaterally. Remounting read-write without doing an fsck first is usually a bad decision, and bad decisions should be left up to the humans :)

I think this behavior (remount read-write without an fsck) is one of the two reasons why Nixfolk have such strange (and strong!) opinions about btrfs. The other reason is explained in #7126 (comment). I'd hate to see the same thing happen with bcachefs.

[edolstra]

Remounting is necessary because /nix/store is a read-only bind mount on NixOS. This is to avoid accidental changes to the Nix store (in particular by the root user). However, maybe the remounting could be done only when needed, i.e. the first time we do an addToStore or build.

Remounting read-write without doing an fsck first is usually a bad decision

We're not remounting the underlying filesystem (/nix/store typically is not its own filesystem), only changing the permissions on the bind mount. So I don't see why an fsck would be needed.

[thufschmitt]

The message of cc63db1 seems to indicate that since we pass MS_REMOUNT | MS_BIND, it will fail if the mount isn't a bind-mount – so we indeed never try to remount a read-only file-system.

@amjoseph-nixpkgs do you have a way to replicate the issue (Nix remounting a ro filesystem that's not ”just” a bind-mount, not the corruption)?

[Ericson2314]

We have a read only option on the local store now too that should disable this (if it doesn't already). That means if someone uses the right query param on the form store, it should be a workaround.

[joergdw]

I have the same issue as the author. When I want to copy nix-closures from a chroot-store that is mounted read-only, the whole process crashes: error: remounting /mnt/builder_cache/nix/store writable: Operation not permitted

(I use here a builder in a multistage-build-process.)

Generally spoken, all scenarios where you just want to copy without modification would benefit from doing that re-mount only when trying to write on it, see comment by @edolstra .