Installation crashes macOS 12.1 (21C52) "Monterey"

[jonesnoaht]

Describe the bug

Installing Nix on a Mac Mini running macOS 12.1 (21C52) on Intel architecture returns the following to the standard output after which the entire screen goes black except for the cursor, the desktop reloads, the computer reconnects to Wi-Fi, and it is apparent that all running applications had been quit.

curl -L https://nixos.org/nix/install | sh            
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4046  100  4046    0     0   9270      0 --:--:-- --:--:-- --:--:--  9270
downloading Nix 2.5.1 binary tarball for x86_64-darwin from 'https://releases.nixos.org/nix/nix-2.5.1/nix-2.5.1-x86_64-darwin.tar.xz' to '/var/folders/j4/rxv02tp54x7fzmbwncn72vzm0000gn/T/nix-binary-tarball-unpack.XXXXXXXXXX.009h0QOM'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 33.5M  100 33.5M    0     0  6649k      0  0:00:05  0:00:05 --:--:-- 6838k
Switching to the Multi-user Installer
Welcome to the Multi-User Nix Installation

This installation tool will set up your computer with the Nix package
manager. This will happen in a few stages:

1. Make sure your computer doesn't already have Nix. If it does, I
   will show you instructions on how to clean up your old install.

2. Show you what I am going to install and where. Then I will ask
   if you are ready to continue.

3. Create the system users and groups that the Nix daemon uses to run
   builds.

4. Perform the basic installation of the Nix files daemon.

5. Configure your shell to import special Nix Profile files, so you
   can use Nix.

6. Start the Nix daemon.

Would you like to see a more detailed list of what I will do?
No TTY, assuming you would say yes :)

I will:

 - make sure your computer doesn't already have Nix files
   (if it does, I will tell you how to clean them up.)
 - create local users (see the list above for the users I'll make)
 - create a local group (nixbld)
 - install Nix in to /nix
 - create a configuration file in /etc/nix
 - set up the "default profile" by creating some Nix-related files in
   /var/root
 - back up /etc/bashrc to /etc/bashrc.backup-before-nix
 - update /etc/bashrc to include some Nix configuration
 - back up /etc/zshrc to /etc/zshrc.backup-before-nix
 - update /etc/zshrc to include some Nix configuration
 - create a Nix volume and a LaunchDaemon to mount it
 - create a LaunchDaemon (at /Library/LaunchDaemons/org.nixos.nix-daemon.plist) for nix-daemon

Ready to continue?
No TTY, assuming you would say yes :)

---- let's talk about sudo -----------------------------------------------------
This script is going to call sudo a lot. Normally, it would show you
exactly what commands it is running and why. However, the script is
run in a headless fashion, like this:

  $ curl -L https://nixos.org/nix/install | sh

or maybe in a CI pipeline. Because of that, I'm going to skip the
verbose output in the interest of brevity.

If you would like to
see the output, try like this:

  $ curl -L -o install-nix https://nixos.org/nix/install
  $ sh ./install-nix


~~> Fixing any leftover Nix volume state
Before I try to install, I'll check for any existing Nix volume config
and ask for your permission to remove it (so that the installer can
start fresh). I'll also ask for permission to fix any issues I spot.

---- Found existing Nix volume -------------------------------------------------
  special:	disk1s7
     uuid:	48B62EE1-740F-450A-AC53-A132C72DCF34
encrypted:	no

~~> Checking for artifacts of previous installs
Before I try to install, I'll check for signs Nix already is or has
been installed on this system.

---- Nix config report ---------------------------------------------------------
        Temp Dir:	/var/folders/j4/rxv02tp54x7fzmbwncn72vzm0000gn/T/tmp.HnjBXVAvNY
        Nix Root:	/nix
     Build Users:	32
  Build Group ID:	30000
Build Group Name:	nixbld

build users:
    Username:	UID
     _nixbld1:	301
     _nixbld2:	302
     _nixbld3:	303
     _nixbld4:	304
     _nixbld5:	305
     _nixbld6:	306
     _nixbld7:	307
     _nixbld8:	308
     _nixbld9:	309
     _nixbld10:	310
     _nixbld11:	311
     _nixbld12:	312
     _nixbld13:	313
     _nixbld14:	314
     _nixbld15:	315
     _nixbld16:	316
     _nixbld17:	317
     _nixbld18:	318
     _nixbld19:	319
     _nixbld20:	320
     _nixbld21:	321
     _nixbld22:	322
     _nixbld23:	323
     _nixbld24:	324
     _nixbld25:	325
     _nixbld26:	326
     _nixbld27:	327
     _nixbld28:	328
     _nixbld29:	329
     _nixbld30:	330
     _nixbld31:	331
     _nixbld32:	332

Ready to continue?
No TTY, assuming you would say yes :)

---- Preparing a Nix volume ----------------------------------------------------
    Nix traditionally stores its data in the root directory /nix, but
    macOS now (starting in 10.15 Catalina) has a read-only root directory.
    To support Nix, I will create a volume and configure macOS to mount it
    at /nix.

~~> Configuring /etc/synthetic.conf to make a mount-point at /nix

~~> Creating a Nix volume
Volume Nix Store on disk1s7 force-unmounted

~~> Configuring /etc/fstab to specify volume mount options

~~> Configuring LaunchDaemon to mount 'Nix Store'

Steps To Reproduce

Run curl -L https://nixos.org/nix/install | sh and enter super-user password when prompted.

Expected behavior

I would have expected the progression to the next step.

nix-env --version output

error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted

Additional context

I have installed Nix successfully on a previous Mac running macOS 11.

I doubt it is relevant, but I used zsh.

[abathur]

First I can recall hearing of these symptoms, so we'll have to explore a bit I guess.

If you check console.app, is there a crash report that matches the time of the install/reboot?

[jonesnoaht]

There was no crash report, and unfortunately the earliest entry in launchd.log was an hour too late.

When I attempt to run curl -L https://nixos.org/nix/install | sh now, I get the following error.

~~> Configuring LaunchDaemon to mount 'Nix Store'
/nix/var/nix/profiles/default/bin/nix-env
error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted

I was able to adjust the permissions of that directory using chmod, but after running curl -L https://nixos.org/nix/install | sh again, I received the same error.

[abathur]

What do ls -la /nix/var/nix/profiles/ and diskutil apfs list say?

You can also drop by the "Nix on macOS" room in the NixOS space on Matrix if doing this in real-time would be better (though I'll be out for a bit this evening).

[jonesnoaht]

I had to run take care of some things. I really appreciate your help.

ls -la /nix/var/nix/profiles/
# total 0
# drwxr-xr-x  5 root  nixbld  160 Dec 19 11:58 .
# drwxr-xr-x  8 root  nixbld  256 Dec  5 19:11 ..
# lrwxr-xr-x  1 root  nixbld   14 Dec 19 11:58 default -> default-1-link
# lrwxr-xr-x  1 root  nixbld   60 Dec 19 11:58 default-1-link -> /nix/store/dh68v00n54caa962clb636fsk55bcc1w-user-environment
# drwxr-xr-x  3 root  nixbld   96 Dec  5 19:11 per-user

diskutil apfs list
# APFS Containers (3 found)
# |
# +-- Container disk1 85CEAA93-1785-4FA5-B91F-F7C4A192CC78
# |   ====================================================
# |   APFS Container Reference:     disk1
# |   Size (Capacity Ceiling):      250656219136 B (250.7 GB)
# |   Capacity In Use By Volumes:   146917990400 B (146.9 GB) (58.6% used)
# |   Capacity Not Allocated:       103738228736 B (103.7 GB) (41.4% free)
# |   |
# |   +-< Physical Store disk0s2 131C814A-99AD-4DE3-9BB4-2047D395211A
# |   |   -----------------------------------------------------------
# |   |   APFS Physical Store Disk:   disk0s2
# |   |   Size:                       250656219136 B (250.7 GB)
# |   |
# |   +-> Volume disk1s1 2EA53E69-5B12-39A2-ACFA-277A938DC61F
# |   |   ---------------------------------------------------
# |   |   APFS Volume Disk (Role):   disk1s1 (Data)
# |   |   Name:                      Mac HD - Data (Case-insensitive)
# |   |   Mount Point:               /System/Volumes/Data
# |   |   Capacity Consumed:         127015645184 B (127.0 GB)
# |   |   Sealed:                    No
# |   |   FileVault:                 No
# |   |
# |   +-> Volume disk1s2 F0884BC8-EC65-45AB-95A0-751C707F8A1F
# |   |   ---------------------------------------------------
# |   |   APFS Volume Disk (Role):   disk1s2 (Preboot)
# |   |   Name:                      Preboot (Case-insensitive)
# |   |   Mount Point:               /System/Volumes/Preboot
# |   |   Capacity Consumed:         330444800 B (330.4 MB)
# |   |   Sealed:                    No
# |   |   FileVault:                 No
# |   |
# |   +-> Volume disk1s3 C4CF3348-EA7A-4D84-9999-3E1680619A8E
# |   |   ---------------------------------------------------
# |   |   APFS Volume Disk (Role):   disk1s3 (Recovery)
# |   |   Name:                      Recovery (Case-insensitive)
# |   |   Mount Point:               Not Mounted
# |   |   Capacity Consumed:         1096470528 B (1.1 GB)
# |   |   Sealed:                    No
# |   |   FileVault:                 No
# |   |
# |   +-> Volume disk1s4 88F4C168-A4BA-41C9-B868-DF7AEBD75CCE
# |   |   ---------------------------------------------------
# |   |   APFS Volume Disk (Role):   disk1s4 (VM)
# |   |   Name:                      VM (Case-insensitive)
# |   |   Mount Point:               /System/Volumes/VM
# |   |   Capacity Consumed:         2148552704 B (2.1 GB)
# |   |   Sealed:                    No
# |   |   FileVault:                 No
# |   |
# |   +-> Volume disk1s5 4994E416-17B5-40F3-9CAD-7C6E8DFA54DD
# |   |   ---------------------------------------------------
# |   |   APFS Volume Disk (Role):   disk1s5 (System)
# |   |   Name:                      Mac HD (Case-insensitive)
# |   |   Mount Point:               Not Mounted
# |   |   Capacity Consumed:         15752601600 B (15.8 GB)
# |   |   Sealed:                    Yes
# |   |   FileVault:                 No
# |   |   |
# |   |   Snapshot:                  09EB8AFE-1EC5-4FA6-8170-6F5ADFDEE02B
# |   |   Snapshot Disk:             disk1s5s1
# |   |   Snapshot Mount Point:      /
# |   |   Snapshot Sealed:           Yes
# |   |
# |   +-> Volume disk1s7 48B62EE1-740F-450A-AC53-A132C72DCF34
# |       ---------------------------------------------------
# |       APFS Volume Disk (Role):   disk1s7 (No specific role)
# |       Name:                      Nix Store (Case-insensitive)
# |       Mount Point:               /nix
# |       Capacity Consumed:         433131520 B (433.1 MB)
# |       Sealed:                    No
# |       FileVault:                 No
# |
# +-- Container disk4 CD2DDC1A-7323-41CC-A5F7-8B63F8ECB5E9
# |   ====================================================
# |   APFS Container Reference:     disk4
# |   Size (Capacity Ceiling):      2000155533312 B (2.0 TB)
# |   Capacity In Use By Volumes:   285920219136 B (285.9 GB) (14.3% used)
# |   Capacity Not Allocated:       1714235314176 B (1.7 TB) (85.7% free)
# |   |
# |   +-< Physical Store disk3s2 8EF99B2D-C542-4222-8B04-0383A2F55205
# |   |   -----------------------------------------------------------
# |   |   APFS Physical Store Disk:   disk3s2
# |   |   Size:                       2000155533312 B (2.0 TB)
# |   |
# |   +-> Volume disk4s1 78742C73-3156-4F6E-9BF1-CF48EE1B100D
# |       ---------------------------------------------------
# |       APFS Volume Disk (Role):   disk4s1 (No specific role)
# |       Name:                      Birb Small Storage (Case-sensitive)
# |       Mount Point:               /Volumes/Birb Small Storage
# |       Capacity Consumed:         285620342784 B (285.6 GB)
# |       Sealed:                    No
# |       FileVault:                 No
# |
# +-- Container disk6 45AB19CB-2670-4AB0-822D-08D23338816C
#     ====================================================
#     APFS Container Reference:     disk6
#     Size (Capacity Ceiling):      8001247207424 B (8.0 TB)
#     Capacity In Use By Volumes:   2022713548800 B (2.0 TB) (25.3% used)
#     Capacity Not Allocated:       5978533658624 B (6.0 TB) (74.7% free)
#     |
#     +-< Physical Store disk5s2 F24EE779-0EBB-465F-9AA9-79A124065617
#     |   -----------------------------------------------------------
#     |   APFS Physical Store Disk:   disk5s2
#     |   Size:                       8001247207424 B (8.0 TB)
#     |
#     +-> Volume disk6s1 9F0017B4-C0EC-413E-AD0B-6E2E12F9CFD5
#     |   ---------------------------------------------------
#     |   APFS Volume Disk (Role):   disk6s1 (No specific role)
#     |   Name:                      BirbStorage (Case-insensitive)
#     |   Mount Point:               /Volumes/BirbStorage
#     |   Capacity Consumed:         1732355698688 B (1.7 TB)
#     |   Sealed:                    No
#     |   FileVault:                 Yes (Unlocked)
#     |
#     +-> Volume disk6s3 7A5E9C63-5A12-4351-AF90-2BFE15A2C5AF
#     |   ---------------------------------------------------
#     |   APFS Volume Disk (Role):   disk6s3 (Backup)
#     |   Name:                      Brittany Pink Macbook Air Backup (Case-sensitive)
#     |   Mount Point:               /Volumes/Brittany Pink Macbook Air Backup
#     |   Capacity Consumed:         189655662592 B (189.7 GB)
#     |   Sealed:                    No
#     |   FileVault:                 No
#     |
#     +-> Volume disk6s4 4A20FD43-FAFF-43E5-9A5C-5959D86CBFFF
#         ---------------------------------------------------
#         APFS Volume Disk (Role):   disk6s4 (Backup)
#         Name:                      Noah Mac Mini Backup (Case-sensitive)
#         Mount Point:               /Volumes/Noah Mac Mini Backup 1
#         Capacity Consumed:         99846213632 B (99.8 GB)
#         Sealed:                    No
#         FileVault:                 Yes (Unlocked)

[abathur]

Those look fine.

I have a hunch, at least. Does the installer hang for a long time at the final line in your initial output?

I have been assuming you meant that the crash happened immediately, but I have seen a small number of reports of the launchctl kickstart step hanging (for reasons we still don't understand AFAIK). If this hangs for long enough, the system's watchdogd might be stepping in and triggering a reboot?

Mostly just confirming for now--not really sure what the next steps are yet. I guess you could look in Console.app while this is running and see if there's any clues?

[jonesnoaht]

@abathur,

I apologize for taking so long to circle back. I went through the install script, and I found that my system crashes when I run sudo nix-env -i /nix/store/wiqcjg66s7sb6cais8pifrk3l9cpkrmq-nss-cacert-3.66 and actually any time sudo nix-env -i is run at all. I cannot get a crash report from Console, but I can say that if I run without sudo I receive the following error message:

could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted

Running sudo chmod -R 755 /nix/var/nix/profiles/per-user makes no improvement.

Thank you so much for your time!! I have no idea how to approach this problem. In the future, I'm switching completely to Linux.

[abathur]

I went through the install script, and I found that my system crashes when I run sudo nix-env -i /nix/store/wiqcjg66s7sb6cais8pifrk3l9cpkrmq-nss-cacert-3.66 and actually any time sudo nix-env -i is run at all.

Two questions:

1. Can you elaborate a little on "my system crashes"?
2. Can you try running these nix-env commands with -vvvv? (It may vomit quite a bit. If it crashes in such a way that you can't read the output, redirecting it to a file may help.) I'm not sure, but perhaps it'll either say something helpful or give a better clue as to what it is doing when it falls over.

Knowing that it is in the cacert step makes me wonder if this is a manifestation of that EOF error (since we have documented cases of the EOF error, there.)

Since you say it happens on any nix-env -i invocation, I'm also wondering if something else is getting corrupted (perhaps by the same issue as the EOF?)

[jonesnoaht]

Can you elaborate a little on "my system crashes"?

Status bar at the top of the screen disappears, the desktop wallpaper disappears, all applications crash including the dock, finder, etc., and then the status bar reappears sans any additional services (e.g., Dropbox), the desktop wallpaper reappears, Finder starts again (so the icons on the desktop appear again), and the dock reappears. All GUI applications are no-longer running except for the Finder.

Emacs, which was started from the terminal and run in the background emacs &, which does have a windowed GUI, stayed open, though, so that was interesting.

Can you try running these nix-env commands with -vvvv? (It may vomit quite a bit. If it crashes in such a way that you can't read the output, redirecting it to a file may help.) I'm not sure, but perhaps it'll either say something helpful or give a better clue as to what it is doing when it falls over.

Awesome. I will do this now.

[jonesnoaht]

Unfortunately I still do not have a verbose output.

IMG_1586.mp4

IMG_1587.mp4

Knowing that it is in the cacert step makes me wonder if this is a manifestation of that EOF error (since we have documented cases of the EOF error, there.)

Interesting. I can try to recreate the issue that this user was having to see if it's related.

Since you say it happens on any nix-env -i invocation, I'm also wondering if something else is getting corrupted (perhaps by the same issue as the EOF?)

Any suggestions as to what I should try?

Again, many thanks!!

[jonesnoaht]

I disabled system-integrity protection to see if that made a difference. I again filmed the process because a couple of times prior I thought I had seen an output flash just before the system shut down. However, each time there would be no error logged on the console.

I was able to capture a few frames with the output in a video. I extracted that frame, ran OCR, and then cleaned up the output (may still not be perfect). The output was as follows:

-7.76.1', '/nix/store/nafyfv5lq0n0biqakzcpb15cq2dkwdz8-libxm12-2.9.12', '/nix/store/q7g8rbn3ab26zy2s7q0q28r2cz721hi4-env-manifest.nix', '/nix/store/q8pcnpxg8i61s44c3fxcn26pbxw5smxi-boost-1.69.0', '/nix/store/r0023xa974x14h83jwrcc8x6i54n9b78-libxm12-2.9.12', '/nix/store/s3dharda6mj935q2pfgw373w0n8459m1-apple-framework-IOkit', '/nix/store/wil7lqvyfb18hkdv@bdrf5h0f9y5v91m-ICU-66108', '/nix/store/wiqcjg66s7sb6caisBpifrk3l9cpkrmq-nss-cacert-3.66', '/nix/store/x0r180c6xm8zgfnczhi278311y56k888-nghttp2-1.43.0-lib', '/nix/store/yjfc15qdr545sh8g3xqqd3khffhxipij-icu4c-69.1', '/nix/store/231hmk06a669vhjaj6ahn5x217312bsy-libkrb5-1.18', '/nix/store/zqigm4h2qjjBpr6247aj4zf3rasrb6fy-nghttp2-1.43.0-lib'
building of '/nix/store/6wndkfwgrdlcigsnvwr22c07lbqvv2w-user-environment.drv!*' from .drv file: woken up
building of '/nix/store/6wndkfwgrdlc6igsnvwr22c071bqvv2w-user-environment.drv!*' from .drv file: trying to build
locking path '/nix/store/1asx1n738cyg@vvas22z5f31kzphza39-user-environment'
lock acquired on '/nix/store/1asx1n738cyg@vvas22z5f31kzphza39-user-environment.lock
removing invalid path '/nix/store/1asx1n738cyg@vvas22z5f31kzphza39-user-environment'
building of '/nix/store/6wndkfwgrdlcbigsnvwr22c071bevv2w-user-environment.drv!*' from .drv file: woken up
found build user 'noahjones'
trying user 'noahjones'
killing all processes running under uid '501'

It is truncated because my terminal window is only like 80 by 24. I will try again but with a larger window and a smaller font size.

[jonesnoaht]

full.output.mov

This was a little more challenging for OCR and to get adequate resolution, but here is the full output.

[jonesnoaht]

I think I have misconfigured something so that it kills itself? Under which UID should this be running?

[abathur]

Haven't fully looked at this, just seeing it on my phone and the videos aren't loading, but AFAIK the build users should just be nixbld{1,32}. Not certain how nix looks up the build users, but if it is just by group this might mean your user is in the nixbld group?

[abathur]

There may be a more-efficient way to check, but you can at least run something like dscl . -read /Groups/nixbld and see if your user is in the GroupMembership section. If so, removing it may get this ~fixed.

(I don't imagine it is, but you might want to also confirm that your user's primary group isn't set to nixbld with dscl . -read /Users/noahjones PrimaryGroupID. I'm not sure if this would require any additional steps to clean up...)

If your user is in the group, I think you can remove it with sudo dseditgroup -o edit -t user -d noahjones nixbld.

[stale]

I marked this as stale due to inactivity. → More info