lxc

[Lassulus]

should fix: #18
run with: lxc image import $(./nixos-generate -f lxc-metadata) $(./nixos-generate -f lxc)
shell can be entered with lxc exec $my-nixos-lxc -- /run/current-system/sw/bin/bash

stuff to do:

• get init from /init instead of /sbin/init so we don't need to rebuild the tarball again
• set PATH somehow so we can run stuff

[lblasc]

@Lassulus suggestion for the /init, in lxc metadata custom init path can be set:

  raw.lxc: |-
    lxc.init_cmd = /init

[lblasc]

I was wrong, /init can be only set in profile not metadata.

[Lassulus]

can we somehow set the profile from inside the rootfs or metadata tarball?

[lblasc]

Just read the metadata code, I don't see the way to do this, seems like the only way is to create symlink to init path in image (like you did). @stgraber any suggestions on topic? thx!

[davidak]

Testing this

Setup LXC & LXD

  virtualisation.lxc = {
    enable = true;
    lxcfs.enable = true;
    defaultConfig = "lxc.include = ${pkgs.lxcfs}/share/lxc/config/common.conf.d/00-lxcfs.conf";
  };

  virtualisation.lxd.enable = true;

Add your user to group "lxd".

  users.extraUsers.davidak = {
    isNormalUser = true;
    extraGroups = [ "wheel" "networkmanager" "audio" "video" "docker" "lxd" ];
    openssh.authorizedKeys.keys = [ pubkey.gb ];
  };

Logout and login to have a new session with that group membership.

configure lxd

[root@ethmoid:~]# lxd init

install from this branch

$ nix-env -f https://api.github.com/repos/nix-community/nixos-generators/tarball/lxc -i

import image

$ lxc image import $(nixos-generate -f lxc-metadata) $(nixos-generate -f lxc)
Image imported with fingerprint: dd6e07dd201cb153d6036d44b5075674d5f45a97b69c8abb694b91ba6440f467

show image

$ lxc image list
+-------+--------------+--------+-------------+--------+----------+-------------------------------+
| ALIAS | FINGERPRINT  | PUBLIC | DESCRIPTION |  ARCH  |   SIZE   |          UPLOAD DATE          |
+-------+--------------+--------+-------------+--------+----------+-------------------------------+
|       | dd6e07dd201c | no     | NixOS       | x86_64 | 139.10MB | Aug 29, 2019 at 10:28am (UTC) |
+-------+--------------+--------+-------------+--------+----------+-------------------------------+

launch container from image

$ lxc launch dd6e07dd201c nixos
Creating nixos
Starting nixos

And finally, you can enter the container!

$ lxc exec nixos -- /run/current-system/sw/bin/bash

[davidak]

set PATH somehow so we can run stuff

@Lassulus what is the problem there? i can run stuff :D

[root@nixos:~]# nix-env -iA nixos.hello
installing 'hello-2.10'
building '/nix/store/wr5fzsvfsr5k3b8vcj9ag07pzaddbfb9-user-environment.drv'...
created 2 symlinks in user environment

[root@nixos:~]# hello 
Hello, world!

Even when including in the image with environment.systemPackages = with pkgs; [ hello ];.

[stgraber]

We specifically do not allow images to pass container configuration or profiles through as we treat images as untrusted and so they shouldn't have access to mechanisms that would allow privilege escalation or access to the host filesystem.

For the init binary specifically, this is something that in theory could be added to the metadata (as it's not security sensitive), but there would be quite a bit of complexity related to storing that information and passing it on to containers that were created from it (just pre-filling raw.lxc wouldn't quite work for this).

Is there a reason why you can't create a sbin directory and put a symlink from /sbin/init to /init?
This is the approach we've seen taken in such cases (for example Android).

[Lassulus]

@davidak If I remember correctly, it was because, if you run it with the default shell it was not able to find any binaries. If that is the intended usecase this is not a problem at all, but I guess people without prior NixOS experience would be confused by the non default location of bash.

@stgraber thanks a lot for your answer, in the current approach I do this symlink, but I have to rebuild the upstream tarball again. If I could use /sbin/init directly I could use the upstream tarball which would speedup evaluation time and reduce code amount. If this is to complex I will stick with this approach.

[davidak]

people without prior NixOS experience would be confused by the non default location of bash.

maybe add another symlink :D

/bin/sh exists on NixOS
/bin/bash don't

but that's a general discussion about the philosophy of NixOS and probably had been taken with any newcomer...

In any case, i would like to have this merged since it's already working quiet well. Any issue can be documented and fixed later. NixOS on LXC is requested in some places!

[asbachb]

@davidak I wonder how you get nix-env -iA nixos.hello to work.

My guest

[root@nixos:/nix/var/nix]# nix-env -iA hello
error: attribute 'hello' in selection path 'hello' not found

[root@nixos:/nix/var/nix]# nix-channel --update
unpacking channels...
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
error: while setting up the build environment: mounting /proc: Operation not permitted
error: program '/nix/store/cngq5a1c1k0aqblz6wxyc90c2232hm9b-nix-2.2.2/bin/nix-env' failed with exit code 1

[davidak]

I wonder how you get nix-env -iA nixos.hello to work.

I have documented every step here: #19 (comment)

I havn't done any additional commands to get it working. It just worked for me. So it would be interesting what's the difference between our setups.

My system nixpkgs is at version nixos-19.03.173202.31d476b8797. You probably have a newer version since i havn't updates some weeks.

Please create a new issue since this PR is merged and closed.