NixOS guest?

[LATBauerdick]

I would really love to have a NixOS guest -- at the moment I use lima with the nix package manager on an archlinux guest. I think in theory NixOS should work, looking at the stated requirements. I also have a qemu NixOS VM, that works just fine. It does not have the great lima integration features, so I really want it running from lima.

I guess the best way forward would be to just go ahead and do it myself, but I am new to lima and could not (easily) find a thread how to start. Are there some example instructions somewhere, or where should I look to e.g. how it was done for archlinux or Debian?

Any pointers are much appreciated.

[jandubois]

I guess the best way forward would be to just go ahead and do it myself

Yes, it would! 😄

The first step is to find a somewhat stable/official location to download cloud images from.

The second step is implementing support for the package manager in pkg/cidata/cidata.TEMPLATE.d/boot/30-install-packages.sh.

The third step is trying it out, checking if something is broken, and fixing it. 😈

[ghost]

I am also interested in a NixOS guest and may be able to provide some development.

I did try this with the minimal 64-bit iso image I found at NixOS' download.

It did not work correctly and I did not see anything in serial.log.

From what I researched NixOS itself has a way to build isos. We could leverage that to build a pared down image with the appropriate configuration to integrate with lima.
This comment gave me that idea: https://discourse.nixos.org/t/how-to-build-a-standalone-nixos-qemu-vm/5688/3

So we could try step 2 and 3 of what @jandubois notes!

[LATBauerdick]

great! I think I already roughly know how to do the first step, to create a working qemu image and install additional stuff, via a script, starting from the official/current NixOS .iso image (pulling it from the NixOS archive).

I will put this together over the holidays and report back.

I am however pretty hazy about the next steps, and will have to look at the repository in detail to see how it works with alpine or arch linux and go from there.

I am far from an expert though, help will be much appreciated!

[jandubois]

Just a quick note that there are 2 different scenarios: booting from an image vs. booting from an ISO. We latter is only done for Alpine right now, and we build a custom ISO to make sure it includes cloud-init (or our own lima-init replacement). Most ISOs are either install or rescue media and do not support this.

There is special code in Lima to mount a persistent data volume and move some directories to it, so they survive a reboot. This code probably will need some adjustments if you want to boot NixOS from an ISO instead of a regular image.

Let me know if you run into any issues, and I'll try to help out!

[afbjorklund]

Apparently NixOS thinks that cloud-init is too bloated to go into the default image, but it has been packaged:

https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/system/cloud-init.nix

• Issue Support user-data in minimal ISO image (Was: Add cloud-init service to minimal ISO) NixOS/nixpkgs#39076

I don't think it is the python3 itself that is the biggest bloat, but it (ci) does have a truckload of dependencies...

[jandubois]

Apparently NixOS thinks that cloud-init is too bloated to go into the default image, but it has been packaged:

If you build you own image, you could also use the lima-init I wrote for our Alpine ISO instead. I it just a shell script and some awk code, and supports everything needed by Lima from cloud-init. It may need some extra work to make it compatible with nix.

[afbjorklund]

You have to fork cloud-init as well, since the list of distros is hardcoded within... So it might be easier to start with lima-init

[vielmetti]

Also registering interest in this development!

[patryk4815]

Today i created my nixos image https://github.com/patryk4815/ctftools/tree/master/lima-vm
And it is working with limactl :)

[patryk4815]

@stuta you need build it localy, just run this command and you will get image :)
nix build .#packages.aarch64-linux.box

[stuta]

I'm missing some context here. So I installed default Ubuntu to OSX M1 Lima and then installed Nix to Ubuntu with sh <(curl -L https://nixos.org/nix/install) --daemon.

Then I did go to lima-vm -folder and ran: nix --extra-experimental-features nix-command --extra-experimental-features flakes build .#packages.aarch64-linux.box

After a long download I get an error error: a 'aarch64-linux' with features {kvm} is required to build '/nix/store/yxy5233kbb26kv3i18ylq7mczm6nf8cw-nixos-disk-image.drv', but I am a 'aarch64-linux' with features {benchmark, big-parallel, nixos-test}.

[patryk4815]

ah yes, you need "kvm" feature. But anyway you can "bypass" this check.
Put this into /etc/nix/nix.conf:

system-features = nixos-test benchmark big-parallel kvm

(qemu in this case will use TCG mode not kvm)

[stuta]

I had to do limactl stop & start for this setting in /etc/nix/nix.conf to work. Now it built the disk image.

[stuta]

Just to clarify for others: running nix build .#packages.aarch64-linux.box created a symlink called result (/nix/store/rhg5kv530fg93ylhncywmpsfc4x01hqw-nixos-disk-image) and inside that folder is a new disk image.

Move image from lima to OSX lima-vm/imgs -folder with name nixos-aarch64.img. Then fix aarch64.yml (line 2 has a space, remove it and fix memory and others). Then move OSX ~/.lima/default to somewhere else and run:

limactl delete -f default
limactl start --name=default aarch64.yml
lima

Inside lima run:

uname -a
Linux nixos 5.17.3 #1-NixOS SMP Wed Apr 13 17:27:43 UTC 2022 aarch64 GNU/Linux

Thanks!