Stuck trying to boot #22

justinmoon · 2022-03-21T21:14:20Z

I'm on an M1 MacBook and I followed all the instructions in the YouTube video & instructions in this repo. I ran make iso/nixos.iso to build an ISO. Tried with VMWare Fusion downloaded from here and brew install --cask vmware-fusion-tech-preview.

I get to this screen and select the top option

But I get stuck here

Any ideas how to fix this?

The text was updated successfully, but these errors were encountered:

futurepaul · 2022-03-22T00:00:56Z

Same here

mitchellh · 2022-04-01T16:14:08Z

I just noticed this as well. This looks like a kernel issue (as in, broadly related to the kernel, kernel config, etc.).

Booting works with Kernel 5.15.26, but fails for me .31.

I'm unsure what to do so I reported a Fusion TP issue: https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Failure-to-boot-Linux-Kernel-5-15-31-works-with-5-15-26/m-p/2901884#M1245

Meanwhile, I pushed a revert commit to ensure we stay on .26 for now. I'll look into pinning that later but hopefully we can get this resolved.

mitchellh · 2022-04-01T16:37:25Z

I've been told this might be a kernel bug, and the fix is already in tree (the kernel tree) but not released.

mitchellh · 2022-04-01T17:04:24Z

Waiting for this commit to get released: torvalds/linux@27599aa (we think its that commit)

gnattu · 2022-04-10T17:29:05Z

Unfortunately it is not that commit. I tried the 5.18-rc1 and no luck.

mitchellh · 2022-04-12T06:20:14Z

:(

https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Linux-5-17-EFI-stub-does-not-boot-hangs-at-loading-kernel/m-p/2900884#M1189

https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Failure-to-boot-Linux-Kernel-5-15-31-works-with-5-15-26/m-p/2901884#M1245

mitchellh · 2022-04-12T06:27:22Z

It looks like other hypervisors are having similar issues so it looks like we'll have to wait for VMware to update, is my guess, and stick to older kenels until then. https://gitlab.com/qemu-project/qemu/-/issues/899

gnattu · 2022-04-12T09:43:00Z

I posted a very tricky workaround here

This patchset removes ID_AA64ISAR2_EL1 from 5.17 kernel which prevents the VM to boot. Not a good approach at all, as we are removing a kernel feature to support a broken hypervisor.

I personally use custom built kernel for a long time because VMware related drivers are not enabled by default on Arch Linux ARM's kernel config, so to apply an extra patch-set is not a big deal for me, but this would be a big problem for a lot who relies on prebuilt kernels.

nateberkopec · 2022-04-14T02:00:38Z

The image built by iso/build.sh still exhibits this issue 🤔 How do we pin the kernel version there?

mitchellh · 2022-04-15T21:26:05Z

The image built by iso/build.sh still exhibits this issue 🤔 How do we pin the kernel version there?

I don't know, but this is a good question. I rarely rebootstrap from an ISO so I don't regularly hit this. There should be a way to pin a specific nixpkgs for the ISO build. I'm also looking into the YOLO solution that @gnattu posted, its very easy to apply kernel patches with Nix, but the downside is compiling a kernel is no small task. Early on, I had to compile a patched kernel for M1+Fusion support (like, right when M1 came out) and even on M1 devices it takes like 20 to 30 minutes.

This is bleeding-edge, and requires - The public preview of VMWare Fusion: https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/ct-p/3022 - Building an aarch64 ISO manually, see: https://github.com/TobiasBales/nixos-m1-arm-builder/ - The ISO has to be built with an older kernel, which can be done from the 21.05 or earlier channel Getting this to work required a number of tweaks: - Pinning a custom kernel because of a VMWare Fusion bug mitchellh/nixos-config#22 , https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Failure-to-boot-Linux-Kernel-5-15-31-works-with-5-15-26/m-p/2901884 - Dropping some unsupported packages like `zoom-us`, `python-language-server`, `dropbox`, etc. - Create a fork of vmware-guest that turns off some features, lifted from https://github.com/mitchellh/nixos-config/blob/main/modules/vmware-guest.nix I could not use this module exactly as something wasn't working with the `disabledModules` setting for me, so I aliased it as `vmware-patched`. - Hardcode screen scale since VMWare Fusion isn't passing the resolution along right now. - Upgrade home-manager to fix a few packages - Disabled openjfx dependencies for all JDKs, since JavaFX does not have a built for aarch64 yet. Among known breakages, Chrome doesn't appear to work yet, though it does install, and there are some strange boot messages, but otherwise things appear to be in a workable state so far.

This is bleeding-edge, and requires - The public preview of VMWare Fusion: https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/ct-p/3022 - Building an aarch64 ISO manually, see: https://github.com/TobiasBales/nixos-m1-arm-builder/ - The ISO has to be built with an older kernel, which can be done from the 21.05 or earlier channel Getting this to work required a number of tweaks: - Pinning a custom kernel because of a VMWare Fusion bug mitchellh/nixos-config#22 , https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Failure-to-boot-Linux-Kernel-5-15-31-works-with-5-15-26/m-p/2901884 - Dropping some unsupported packages like `zoom-us`, `python-language-server`, `dropbox`, etc. - Create a fork of vmware-guest that turns off some features, lifted from https://github.com/mitchellh/nixos-config/blob/main/modules/vmware-guest.nix - Hardcode screen scale since VMWare Fusion isn't passing the resolution along right now. - Upgrade home-manager to fix a few packages - Disabled openjfx dependencies for all JDKs, since JavaFX does not have a built for aarch64 yet. Among known breakages, Chrome doesn't appear to work yet, though it does install, and there are some strange boot messages, but otherwise things appear to be in a workable state so far.

marko911 · 2022-08-08T11:43:21Z

Any update on this ? Anyone able to get an iso with an older kernel version ?

justinmoon · 2022-08-08T12:21:43Z

I ended up getting it working with an older kernal, but don't remember the details.

marko911 · 2022-08-08T12:26:14Z

I ended up getting it working with an older kernal, but don't remember the details.

Would love to know how you actually specify kernel when building iso or finding iso with specific kernel ?

justinmoon · 2022-08-08T12:42:15Z

I ended up getting it working with an older kernal, but don't remember the details.

Would love to know how you actually specify kernel when building iso or finding iso with specific kernel ?

I used nixos 21.05 in VMWare Fusion Tech Preview. Not sure about more than that.

dmartzol · 2022-08-08T15:19:09Z

I was not able to install 21.05 either on M1 using VMWare Fusion Tech Preview.

Any update on this ? Anyone able to get an iso with an older kernel version ?

@marko911, you can find ISOs here: https://releases.nixos.org/?prefix=nixos/

marko911 · 2022-08-08T15:22:16Z

@dmartzol I was able to install 21.05 but now running the bootstrap script fails with

building the system configuration...
error: attribute 'shellDryRun' missing

       at /nix/store/d5j5vgsfvwkaqmc1qhwaay5mp51yaqkm-source/modules/programs/bash.nix:12:9:

           11|     checkPhase = ''
           12|       ${pkgs.stdenv.shellDryRun} "$target"
             |         ^
           13|     '';

dmartzol · 2022-08-08T16:26:59Z

@dmartzol I was able to install 21.05 but now running the bootstrap script fails with

building the system configuration...
error: attribute 'shellDryRun' missing

       at /nix/store/d5j5vgsfvwkaqmc1qhwaay5mp51yaqkm-source/modules/programs/bash.nix:12:9:

           11|     checkPhase = ''
           12|       ${pkgs.stdenv.shellDryRun} "$target"
             |         ^
           13|     '';

Yes, sorry when I said I was not able to install I was referring exactly to this same error

marko911 · 2022-08-09T09:12:39Z

I think the issue is that all the config files are using 22.05 as the version and so the missing attribute error throws because that attribute doesn't exist in the 21.05 version which is using old kernel ? My best guess. I'm new to this ecosystem

nbareil · 2022-08-09T12:24:25Z

For the record, the VMWare issue was fixed in 22H2 (released on July 28th this year), it is compatible with the kernel v5.18.6 (and 5.19.0 as well) available in NixOS 22.05.

🍾 🥳

marko911 · 2022-08-09T14:04:02Z

@nbareil Yes, I just re-did it and it does work. But I still get the same error as above

(root@172.16.218.128) Password: 
building the system configuration...
error: attribute 'shellDryRun' missing

       at /nix/store/d5j5vgsfvwkaqmc1qhwaay5mp51yaqkm-source/modules/programs/bash.nix:12:9:

           11|     checkPhase = ''
           12|       ${pkgs.stdenv.shellDryRun} "$target"
             |         ^
           13|     '';

nbareil · 2022-08-09T14:33:02Z

error: attribute 'shellDryRun' missing

Isn't it nix-community/home-manager#2722 ?

marko911 · 2022-08-09T19:00:10Z

error: attribute 'shellDryRun' missing

Isn't it nix-community/home-manager#2722 ?

No , because the versions for me aren't different. they're both 22.05

marko911 · 2022-08-10T09:19:19Z

For anyone wondering, currently if you pull this repo and use aarch64 with 22.05 nixos , you must change lines 50 and 51 in flake.nix from

inherit overlays home-manager;
nixpkgs = inputs.nixpkgs-old-kernel;

to

inherit overlays nixpkgs home-manager;

nateberkopec · 2022-08-16T02:18:40Z

Confirmed that master works on a new install/VM. I think this can be closed.

justinmoon · 2022-08-16T21:35:15Z

Confirmed that master works on a new install/VM. I think this can be closed.

What exactly did you do? I just this ISO with VMWare Fusion Tech Preview on an M1 and it never gets past this. I'm not even getting to apply any of the stuff in this repo ...

mitchellh · 2022-08-18T17:57:29Z

The original issue is indeed fixed with VMW TP 22H2.

nateberkopec · 2022-08-22T06:05:01Z

@justinmoon 🤷‍♂️ I'm making my own ISO as the README suggests.

mitchellh closed this as completed Aug 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stuck trying to boot #22

Stuck trying to boot #22

justinmoon commented Mar 21, 2022 •

edited

futurepaul commented Mar 22, 2022

mitchellh commented Apr 1, 2022 •

edited

mitchellh commented Apr 1, 2022 •

edited

mitchellh commented Apr 1, 2022

gnattu commented Apr 10, 2022

mitchellh commented Apr 12, 2022

mitchellh commented Apr 12, 2022

gnattu commented Apr 12, 2022

nateberkopec commented Apr 14, 2022

mitchellh commented Apr 15, 2022

marko911 commented Aug 8, 2022

justinmoon commented Aug 8, 2022

marko911 commented Aug 8, 2022

justinmoon commented Aug 8, 2022

dmartzol commented Aug 8, 2022

marko911 commented Aug 8, 2022

dmartzol commented Aug 8, 2022

marko911 commented Aug 9, 2022

nbareil commented Aug 9, 2022 •

edited

marko911 commented Aug 9, 2022

nbareil commented Aug 9, 2022

marko911 commented Aug 9, 2022

marko911 commented Aug 10, 2022

nateberkopec commented Aug 16, 2022

justinmoon commented Aug 16, 2022 •

edited

mitchellh commented Aug 18, 2022

nateberkopec commented Aug 22, 2022

Stuck trying to boot #22

Stuck trying to boot #22

Comments

justinmoon commented Mar 21, 2022 • edited

futurepaul commented Mar 22, 2022

mitchellh commented Apr 1, 2022 • edited

mitchellh commented Apr 1, 2022 • edited

mitchellh commented Apr 1, 2022

gnattu commented Apr 10, 2022

mitchellh commented Apr 12, 2022

mitchellh commented Apr 12, 2022

gnattu commented Apr 12, 2022

nateberkopec commented Apr 14, 2022

mitchellh commented Apr 15, 2022

marko911 commented Aug 8, 2022

justinmoon commented Aug 8, 2022

marko911 commented Aug 8, 2022

justinmoon commented Aug 8, 2022

dmartzol commented Aug 8, 2022

marko911 commented Aug 8, 2022

dmartzol commented Aug 8, 2022

marko911 commented Aug 9, 2022

nbareil commented Aug 9, 2022 • edited

marko911 commented Aug 9, 2022

nbareil commented Aug 9, 2022

marko911 commented Aug 9, 2022

marko911 commented Aug 10, 2022

nateberkopec commented Aug 16, 2022

justinmoon commented Aug 16, 2022 • edited

mitchellh commented Aug 18, 2022

nateberkopec commented Aug 22, 2022

justinmoon commented Mar 21, 2022 •

edited

mitchellh commented Apr 1, 2022 •

edited

mitchellh commented Apr 1, 2022 •

edited

nbareil commented Aug 9, 2022 •

edited

justinmoon commented Aug 16, 2022 •

edited