Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign up/nix will not be writable on macOS Catalina #2925
Comments
This comment has been minimized.
This comment has been minimized.
|
Realistically, this may force us to drop support for macOS. Using a different store location for macOS would require a separate binary cache and a separate Hydra instance, and there would be no guarantee that the new location wouldn't break in the future. I did notice in the talk that it's still possible to make the system volume writable, though not in a persistent way. |
This comment has been minimized.
This comment has been minimized.
|
Speaking for myself: I would be quite sad if macOS support would go away. Yes, you can jump through hoops and get a writable system volume, but only until the next reboot, so this is not a useful option. |
This comment has been minimized.
This comment has been minimized.
|
@edolstra Before canceling support, we can ask the community if it's willing to host its own binary cache and Hydra instance, just as we've enlisted a host of volunteers to keep nixpkgs at near parity on Darwin. Having the answer be, "This doesn't easily fit our model so we won't support Mac" would be an unfortunate response. |
This comment has been minimized.
This comment has been minimized.
|
I think no such decision should be made lightly, but it is definitely a drastic change on Apple’s part. |
This comment has been minimized.
This comment has been minimized.
Maybe you can use this to create a |
This comment has been minimized.
This comment has been minimized.
|
That was my first try to get it temporarily working again. But
If that error can be safely ignored, I think this would be a workaround worth exploring. The way I understand things, anything you place in the root folder would persist until the next system update. |
This comment has been minimized.
This comment has been minimized.
|
There's no real reason for the nix store to be on the system volume, with apfs it's easy to create a separate volume and mount it to |
This comment has been minimized.
This comment has been minimized.
|
@mroi You can set |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@edolstra Thanks for the tip, this is a useful workaround. But future Nix-users on macOS still need to find a reliable way to inject this symlink into the read-only root directory. This probably involves some advanced steps like rebooting into recovery mode after every system upgrade. @LnL7 The same would be true for |
This comment has been minimized.
This comment has been minimized.
|
Sure but if Apple doesn't want to provide some sort of solution for one of these cases there will be no alternative to changing the default prefix. Which is arguably is not worth it for a secondary platform that's continuously moving further away from UNIX. Is there an overview of the full volume layout somewhere? |
This comment has been minimized.
This comment has been minimized.
|
@mroi How have you been testing this? In my testing, I was able to install nix, but I assumed it was due to the comment in the presentation that "system volume is writeable in the developer preview". I'd also like to help test, but was able to get nix installed:
|
This comment has been minimized.
This comment has been minimized.
|
It looks like you can still access the |
This comment has been minimized.
This comment has been minimized.
|
|
This comment has been minimized.
This comment has been minimized.
|
I wonder if that will continue to work in the later builds, when they plan to enforce the read-only aspect. |
This comment has been minimized.
This comment has been minimized.
|
Apple says in the WWDC talk, that the first beta seeds mount the system volume read-write, but this is only temporary. A later seed will turn it read-only. You can create a file |
This comment has been minimized.
This comment has been minimized.
|
But there is good news. I talked to an Apple engineer on Twitter and they are working on a solution to create symlinks in the root directory even when the system volume is mounted read-only. I guess we’ll just wait and see how this turns out. For now, symlinking |
This comment has been minimized.
This comment has been minimized.
|
@grahamc remember our discussion about symlink |
This comment has been minimized.
This comment has been minimized.
|
This appears to be fixed in 10.15 Beta 3! I can create a /nix directory and use it as a store, while /System, /bin, /sbin, etc. are still read only. Users upgrading from previous beta versions can do:
then start the daemon if you are in multi user:
New installs should work out of the box. |
This comment has been minimized.
This comment has been minimized.
|
|
This comment has been minimized.
This comment has been minimized.
Will |
This comment has been minimized.
This comment has been minimized.
If you disable SIP, yes. |
This comment has been minimized.
This comment has been minimized.
Ugh, I don't think we want to recommend that. Are any exceptions for Nix possible? I saw that Fink's To be clear, changing the root directory of Nix from |
This comment has been minimized.
This comment has been minimized.
|
We're working on something that will allow you to create either a symlink or empty directory at
|
This comment has been minimized.
This comment has been minimized.
Sounds reasonable! The main concern with (1) is that realpath(3) will still report /System/Volumes/Data/nix when pwd=/nix (#2926). That can get |
This comment has been minimized.
This comment has been minimized.
|
If we were to mount Nix from a separate volume, we could use the opportunity to use the case-sensitive version of APFS. This would remove the need to fix weirdo packages relying on files that just differ in case. |
This comment has been minimized.
This comment has been minimized.
|
I never object to a change which spreads the case-sensitivity gospel. |
This comment has been minimized.
This comment has been minimized.
I have been doing this with great success (although with hfs+ case sensitive) with an external ssd for a while now. The primary motivation being, that I can take my nix store from my desktop with me when I use my laptop while walking about. |
This comment has been minimized.
This comment has been minimized.
It's a one-time hurdle, at least. Using nix works as well as it did before; it's just the initial installation that people are working to adapt to Catalina. |
This comment has been minimized.
This comment has been minimized.
Maybe this disk image autmount project of mine can help a little bit along the way. |
This comment has been minimized.
This comment has been minimized.
|
Minor update with plist file to go into /Library/LaunchDaemons/ to remount on startup nix-volume.txt -> nix-volume.sh nix-volume.txt Seems to work. Still investigating possible performance implications. |
This comment has been minimized.
This comment has been minimized.
|
@nycnewman This approach means the disk isn't mounted prior to user login session setup. We already have that problem with an encrypted Nix partition, but an unencrypted partition is mounted prior to GUI app restoration, and there's a workaround using a login hook for forcing the encrypted partition to mount first. The DMG approach doesn't seem to support this at all, which is rather unfortunate as it causes problems with GUI app restoration depends on stuff installed from Nix (such as restoring Terminal.app when your login shell is from Nix). |
This comment has been minimized.
This comment has been minimized.
|
@lilyball Finding mixed messages regarding Login Hooks. Many references suggest this is deprecated and to use LaunchDaemons. If you have examples elsewhere, happy to try. |
This comment has been minimized.
This comment has been minimized.
|
The login hook appears to execute before the GUI session is initialized (or rather, before GUI apps are launched). I haven't tried it myself but that's what I infer based on people using it to force-mount the encrypted partition. AFAIK LaunchDaemons do not block GUI app initialization. |
This comment has been minimized.
This comment has been minimized.
Apple states login hook deprecation officially. However, adding a login script using configuration profiles is not deprecated. |
This comment has been minimized.
This comment has been minimized.
|
I think this issue has already been beaten to death and as a disclaimer, I haven't read all prior comments. I was planning on evaluating nix to use as the base to solve cross-platform packaging issues. But this issue seems like a real blocker as it would make the initial setup process harder not easier, when building on top of nix. :( I can't understand why the nix store isn't just moved to /usr/local on all platforms. AFAIK that is the intended location for non-distribution-managed software on most Unix-like platforms. |
This comment has been minimized.
This comment has been minimized.
That would conform to the Filesystem Hierarchy Standard: |
This comment has been minimized.
This comment has been minimized.
|
IMHO By my reading, it seems like nix would qualify has "Add-on application software packages" in the Filesystem Hierarchy Standard . It doesn't seem like nix is registered as a LANANA provider, but it could be? |
This comment has been minimized.
This comment has been minimized.
|
Changing the store path is certainly possible, but at a cost: build products have the store path embedded in them, so artifacts that use something other than /nix aren't compatible with existing stores that do use /nix (including the upstream binary caches). Migrating an existing store is not as simple as |
This comment has been minimized.
This comment has been minimized.
Except for NixOS... |
This comment has been minimized.
This comment has been minimized.
I agree. |
This comment has been minimized.
This comment has been minimized.
The problem with moving the nix store at all is that it breaks all the running systems out in the world. Moving the store on a system effectively amounts to installing a new OS. This is because absolute store paths are baked into the content of most packages. So... if you That said, it's definitely a shame that |
This comment has been minimized.
This comment has been minimized.
|
The problem with moving the nix store _at all_ is that it breaks all the running systems out in the world. Moving the store on a system effectively amounts to installing a new OS. This is because absolute store paths are baked into the content of most packages.
I guess a good upper bound is reinstalling NixOS from an existing configuration.nix (and the partitions and bootloader are already sorted out in advance). That's much better than «install a new OS» if a new OS is unrelated to the current one.
For anything declaratively managed and built against a branch/channel that are currently updated, the move could be made comparable to an stdenv update in the branch/channel. Of course, imperatively managed packages and also old pinned versions would be a worse headache than that.
A part of the incentive difference might be that the move makes little sense on NixOS (there /nix is a core part of the system, not something naturally fitting /opt), and will provide no benefits on NixOS, but will still bring all the annoyances in case of imperative package management or pinning old builds. So from the point of view of NixOS users such a change is likely to be a net loss, and a majority of the core developers (who would have to expend effort to make such a move smooth) use NixOS.
|
This comment has been minimized.
This comment has been minimized.
|
I use Nix on MacOS on my work computer (and NixOS at home), but I think changing |
This comment has been minimized.
This comment has been minimized.
|
@emptyflask The particular workaround you linked to is the symlink approach, which appears to break a lot of packages. I strongly recommend using the separate volume approach instead (documented many times in this thread). |
This comment has been minimized.
This comment has been minimized.
|
I’ll wait until one of approaches would be implemented in the installer |
This comment has been minimized.
This comment has been minimized.
Yeah, that's my take on this too. I better wait this out until a solution has been implemented that I can evaluate. |
This comment has been minimized.
This comment has been minimized.
|
I've deployed the solution in my gist (configuring a volume) in a corporate environment. Personally, I'd call it something already implemented, and available for evaluation. |
This comment has been minimized.
This comment has been minimized.
|
I've been running with the separate volume approach for a while now with zero problems (though I keep the volume unencrypted so it mounts prior to GUI initialization; I don't use it to build work source, just for tooling support). As I understand it the only real holdup with getting this into the installer is testing it on a wide variety of preconditions. |
This comment has been minimized.
This comment has been minimized.
I have a separate volume mounted with an entry in synthetic.conf to /nix, /nix is still a link to Volumes/NixStore (NixStore is the name of the volume). Is there a way to mount the volume directly to /nix? |
This comment has been minimized.
This comment has been minimized.
|
@truh The instructions are higher up in this thread, but basically, your /etc/synthetic.conf should have a line just consisting of With these instructions, if your volume is encrypted then GUI apps will start to restore before it finishes mounting (this is a problem if Terminal.app is one of these apps and you use a Nix-installed shell as your login shell). If your volume is unencrypted then it will mount prior to the GUI session logging in. That said, I assume this caveat applies to your current setup as well (where you're symlinking to /Volumes/NixStore). |
This comment has been minimized.
This comment has been minimized.
|
This worked for me: #2925 (comment) |
This comment has been minimized.
This comment has been minimized.
|
Another data point: Fedora Core OS also locks down / and doesn't provide a work around, thus forcing nix users to work via a chroot. |
This comment has been minimized.
This comment has been minimized.
Maybe we should make a separate ticket for this. But it looks like you can make it work with some overlays: https://www.mikenowak.org/working-around-read-file-systems-coreos-overlay/ (replacing /opt with /nix) |
This comment has been minimized.
This comment has been minimized.
|
My understanding is that at the very least you would need a mountpoint. My current plan is to provide a login shell that has a chroot with everything in place. |
This is not a short term bug, but it will become an issue when macOS Catalina is released this fall. macOS is now split across two volumes (system and data) with a read-only system volume. This means that
/nixwill no longer be writable.Some more information can be found in the related WWDC talk and some session notes people took from a Q&A.
Summary: the system volume, which is mounted at
/will become non-writable. Some directories that need to be writable are connected via firmlinks (an Apple invention) to the data volume./nixis not among these locations, so with the release of macOS Catalina, this location is no longer an option.I see two possible solutions:
/nixas a firmlink to a writable location. I think this has limited success and Nix would then depend on Apple to not drop this link in a future release./usr/localand/opt, so we could move to/usr/local/nixor/opt/nix. I would hope that these locations are common enough so that Apple would not drop them in the future.I wanted to raise this issue early, before it becomes a problem for users. If this issue tracker is not the right place, please feel free to move this discussion elsewhere. I would also be available for testing any potential solution, since I have access to a macOS Catalina beta.