Prebuilt lxc/lxd images of nixos?

[evelant]

I can't find much information about running nixos in an lxc container. Are there any images for nixos such as those found on https://us.images.linuxcontainers.org/ ? Would it be difficult to create and list an image there?

[samueldr]

Hi, #9735 has some discussion as to the past situation of LXC images of nixos.

Earlier this year I wrote this hacky walkthrough to get an LXC container of NixOS going in Proxmox.

As far as support? I don't know if the issues I had with nixos-rebuild were specific to how Proxmox works, but it works, if one were to disregard the warnings and errors on rebuild.

I don't know what differences Promox has with other LXC solutions, so I can't really say how much of it applies.

And as far as distributing images elsewhere, it's kind of a hassle, just as our iso images aren't distributed and mirrored at many locations. This is because every time the nixos release updates, a new iso image (and container images) have been built for that new update. It would need a constant feed of not-that-long-lived images.

[asbachb]

I guess the most convenient way for now would be that hydra builds that image. This could be easily imported via lxc import.

[stale]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[nathan-at-least]

This issue is not stale for me, a new-comer to nixos. I use lxd images for development and prototyping, so it would be my primary platform for a nixos installation.

[NULLx76]

I'm also still interested in using NixOS inside of LXC containers as the overhead compared to VMs is that much lower. A variety of issues I encountered are also mentioned in #9735

[asbachb]

Tbh I'm unsure if this will happen. Someone would need to invest quite some time to make NixOS a first class citizen of lxc/lxd. That guy needs to be educated in NixOS and LXC/LXD with is I guess a rare combination. I tried to get generating NixOS in distrobuilder somehow but failed due the lack of knowledge.

Maybe it's worth starting a dedicated discourse thread or updating the wiki to gather which problems currently occur and which tasks might be performed. But as you see it's quite hard to get some momentum to that topic.

[stale]

I marked this as stale due to inactivity. → More info

[mberry]

I tried to get generating NixOS in distrobuilder

Could you post your distrobuilder yaml config? I was hoping to give this a try

[asbachb]

@mberry I don't think I still have it. Even if I find it it's not in a state of being used.

Your best chance of getting it to work you'll find here: https://discourse.nixos.org/t/running-nixos-18-09-in-an-lxd-3-8-container/1804

But I'm not sure it's still working tbh.

[adamcstephens]

There are hydra jobs that can be used to import a new image:

• https://hydra.nixos.org/job/nixos/release-22.05/nixos.lxdImage.x86_64-linux
• https://hydra.nixos.org/job/nixos/release-22.05/nixos.lxdMeta.x86_64-linux

I believe security.nesting: true is required, and you'll have to manually configure the container for now.

The distrobuilder config requires more work to get going. matthewbauer has made an attempt, and so have I. Neither are complete.

[asbachb]

This should work without security.nesting: true:

lxc profile show nixos

config:
  raw.lxc: |-
    lxc.init.cmd = /sbin/init
    lxc.mount.entry = proc mnt/proc proc create=dir 0 0
    lxc.apparmor.profile = unconfined

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/lxd-distrobuilder-support-for-nixos/21375/1

[adamcstephens]

The container images have been renamed and VM images have been added, for both aarch64 and x86_64. I should note that aarch64 has not been well tested, so please open an issue if you encounter problems.

Container images should have security.nesting: true set

https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdContainerImage.x86_64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdContainerMeta.x86_64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdContainerImage.aarch64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdContainerMeta.aarch64-linux

Virtual machines should have security.secureboot: false set

https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImage.x86_64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImageMeta.x86_64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImage.aarch64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImageMeta.aarch64-linux

custom images

You can also build your own images by importing the relevant LXD module (lxc-container or lxd-virtual-machine) and then building the necessary outputs.

container

import modules/virtualisation/lxc-container.nix and build config.system.build.tarball for the root tarball and config.system.build.metadata for the metadata

virtual-machine

import modules/virtualisation/lxd-virtual-machine.nix and build config.system.build.qemuImage for the root qcow2 image and config.system.build.metadata for the metadata