ZFS filesystems must be created with exec=on

[hadmut]

Hi,
I've installed lxc and lxc-templates1.1.5-0ubuntu1ubuntu14.04.1ppa1 on a Ubuntu 14.04 machine with zfs from the ubuntu ppa.

lxc-create works correctly, as long as I don't mention zfs.

But with zfs, something goes wrong with the installation paths:

lxc-create -n abcdef -t ubuntu -B zfs --zfsroot raid1/lxc -- --release trusty

results in

lxc-create -n abcdef -t ubuntu -B zfs --zfsroot raid1/lxc -- --release trusty

Checking cache download in /var/cache/lxc/trusty/rootfs-amd64 ...
Copy /var/cache/lxc/trusty/rootfs-amd64 to /usr/lib/x86_64-linux-gnu/lxc ...
Copying rootfs to /usr/lib/x86_64-linux-gnu/lxc ...
chroot: failed to run command ‘useradd’: Permission denied
lxc-create: lxccontainer.c: create_run_template: 1201 container creation template for abcdef failed

for some reason it tries to chroot to /usr/lib/x86_64-linux-gnu/lxc , which is, of course, wrong and does not work.

[hallyn]

Could you try the download template? lxc-create -n abcdef -t download -B zfs --zfsroot raid1/lxc -- -d ubuntu -r trusty -a amd64 When a container is created or started, its rootfs does in fact get mounted under /usr/lib/x86_64-linux-gnu/lxc, so that path was probably correct. Something else appears to be going wrong.

[hadmut]

That command with the download template works, creates a zfs file system with the expected linux structure.

[hallyn]

Thanks. Could you try running the ubuntu template again, then check for any DENIED messages in journalctl output, /var/log/syslog, or /var/log/audit/audit.log?

To summarize your finding so far,

sudo lxc-create -t download -n x1 -B zfs --zfsroot raid1/lxc -- -d ubuntu -r trusty -a amd64

works,

sudo lxc-create -t ubuntu -n x1 -- -r trusty

works, but

sudo lxc-create -t ubuntu -n x1 -B zfs --zfsroot raid1/lxc -- -r trusty

fails?

[hadmut]

Exactly. No log entries. But I've found the problem:

My zfs root has the noexec option set. It therefore does

10439 execve("/sbin/zfs", ["zfs", "create", "-omountpoint=/var/lib/lxc/x1/rootfs", "raid1/lxc/x1"], [/* 33 vars /]) = 0
10440 execve("/bin/mount", ["/bin/mount", "-t", "zfs", "-o", "defaults,noatime,dev,noexec,rw,suid,nomand,zfsutil", "raid1/lxc/x1", "/var/lib/lxc/x1/rootfs"], [/ 33 vars /]) = 0
10441 execve("/sbin/mount.zfs", ["/sbin/mount.zfs", "raid1/lxc/x1", "/var/lib/lxc/x1/rootfs", "-o", "rw,noexec,noatime,zfsutil"], [/ 29 vars */]) = 0

...

10468 execve("/usr/sbin/useradd", ["useradd", "--create-home", "-s", "/bin/bash", "ubuntu"], [/* 34 vars */]) = -1 EACCES (Permission denied)

The simple reason why the download template works is that it simply does not execute anything. After setting the exec option to raid1/lxc it works. However, it would be better to set it on raid1/lxc/x1

So creating a zfs filesystem should be done with setting option exec=on

[tcharding]

Is this still open? I attempted unsuccessfully to recreate the issue, created a pool with zpool called bugpool. Set option noexec

$ sudo zfs get exec bugpool
NAME     PROPERTY  VALUE  SOURCE
bugpool  exec      off    local

Then created and started a container

$ sudo lxc-create -t ubuntu -n x1 -B zfs --zfsroot bugpool/lxc -l trace -o /tmp/lxc-bug.out -- -r trusty
...
$ sudo lxc-info x1
Name:           x1
State:          RUNNING
PID:            27670
...

[brauner]

The zfs driver would probably need to be changed to do this when creating new zfs datasets.

[danboid]

Does lxc-create still exist? I've installed the LXD 4 snap and there is no lxc-create in my path.

exec seems to be on by default when you create pools manually under 20.04.

[hallyn]

lxc-create is a part of 'lxc', not 'lxd':

serge@sl ~$ dpkg -S /usr/bin/lxc-create
lxc-utils: /usr/bin/lxc-create