[Question] [lxc] possible to use fscrypt inside a Linux Container (lxc)

[graysky2]

I'd like to use fscrypt within a Linux Container (LXC) to encrypt a directory therein. I'm finding that the user space util, fscrypt does not like the fact that it is containerized. For example, I start the container and ssh into it:

# fscrypt setup
Defaulting to policy_version 2 because kernel supports it.
Customizing passphrase hashing difficulty for this system...
Created global config file at "/etc/fscrypt.conf".
[ERROR] fscrypt setup: /: not a mountpoint

Is there something I can modify with my config or via passing a switch upon setup or is this something that needs to be handled within my LXC config?

[josephlr]

Filesystem encryption certainly should work inside of containers. What are the contents of /proc/self/mountinfo inside the container when you're trying to run fscrypt setup?

This might be a bug in fscrypt, but I'm not sure.

[graysky2]

@josephlr

Within the container:

% cat /proc/self/mountinfo
1073 1024 8:5 /var/lib/lxc/base/rootfs / rw,relatime master:1 - ext4 /dev/sda5 rw
1074 1073 0:73 / /dev rw,relatime - tmpfs none rw,size=492k,mode=755
1075 1073 0:72 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
1076 1077 0:72 /sys/net /proc/sys/net rw,nosuid,nodev,noexec,relatime - proc proc rw
1077 1075 0:72 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
1078 1075 0:72 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
1079 1073 0:74 / /sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
1080 1079 0:74 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs rw
1081 1080 0:74 / /sys/devices/virtual/net rw,relatime - sysfs sysfs rw
1082 1081 0:74 /devices/virtual/net /sys/devices/virtual/net rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
1083 1073 8:5 /var/cache/pacman/pkg /var/cache/pacman/pkg rw,relatime master:1 - ext4 /dev/sda5 rw
1084 1073 8:5 /srv/repo/x86_64 /srv/http/x86_64 rw,relatime master:1 - ext4 /dev/sda5 rw
1085 1074 0:26 /2 /dev/lxc/console rw,nosuid,noexec,relatime master:4 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
1086 1074 0:26 /2 /dev/console rw,nosuid,noexec,relatime master:4 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
1025 1077 0:73 /.lxc-boot-id /proc/sys/kernel/random/boot_id ro,nosuid,nodev,noexec,relatime - tmpfs none rw,size=492k,mode=755
1026 1074 0:75 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1027 1074 0:75 /ptmx /dev/ptmx rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1028 1074 0:75 /0 /dev/lxc/tty1 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1029 1074 0:75 /1 /dev/lxc/tty2 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1030 1074 0:75 /2 /dev/lxc/tty3 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1031 1074 0:75 /3 /dev/lxc/tty4 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666,max=1024
1032 1074 0:76 / /dev/shm rw,nosuid,nodev - tmpfs tmpfs rw
1033 1073 0:77 / /run rw,nosuid,nodev - tmpfs tmpfs rw,mode=755
1034 1080 0:78 / /sys/fs/cgroup ro,nosuid,nodev,noexec - tmpfs tmpfs ro,mode=755
1035 1034 0:28 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime - cgroup2 cgroup2 rw,nsdelegate
1036 1034 0:29 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,xattr,name=systemd
1037 1034 0:42 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,hugetlb
1038 1034 0:38 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,blkio
1039 1034 0:37 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,perf_event
1040 1034 0:40 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,memory
1041 1034 0:41 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,freezer
1042 1034 0:39 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpu,cpuacct
1043 1034 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,net_cls,net_prio
1044 1034 0:36 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpuset
1045 1034 0:34 / /sys/fs/cgroup/rdma rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,rdma
1046 1034 0:35 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,pids
1047 1034 0:32 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime - cgroup cgroup rw,devices
1048 1074 0:79 / /dev/hugepages rw,relatime - hugetlbfs hugetlbfs rw,pagesize=2M
1049 1074 0:68 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
1050 1073 0:80 / /tmp rw,nosuid,nodev - tmpfs tmpfs rw
1545 1033 0:85 / /run/user/1000 rw,nosuid,nodev,relatime - tmpfs tmpfs rw,size=390000k,mode=700,uid=1000,gid=100

I also tried on the host moving /var/lib/lxc/base/rootfs to just /rootfs and then bind mounting it so that it might appear to be a mount point in the container:

# mv /var/lib/lxc/base/rootfs /
# mkdir /var/lib/lxc/base/rootfs
# mount -o bind /rootfs /var/lib/lxc/base/rootfs
# start lxc@base
# lxc-attach -n base


% grep rootfs /proc/self/mountinfo
779 673 8:5 /rootfs / rw,relatime master:1 - ext4 /dev/sda5 rw
% sudo fscrypt setup       
Defaulting to policy_version 2 because kernel supports it.
Customizing passphrase hashing difficulty for this system...
Created global config file at "/etc/fscrypt.conf".
[ERROR] fscrypt setup: /: not a mountpoint

But I get the same error as you see.

[ebiggers]

Yes this is a bug. Since v0.2.6, in order to handle bind mounts better, fscrypt doesn't store metadata at every individual mountpoint of a filesystem but rather only at the root of the filesystem.

Since you have:

1073 1024 8:5 /var/lib/lxc/base/rootfs / rw,relatime master:1 - ext4 /dev/sda5 rw
1083 1073 8:5 /var/cache/pacman/pkg /var/cache/pacman/pkg rw,relatime master:1 - ext4 /dev/sda5 rw
1084 1073 8:5 /srv/repo/x86_64 /srv/http/x86_64 rw,relatime master:1 - ext4 /dev/sda5 rw

... the root of the filesystem isn't mounted, so fscrypt doesn't count any of these as a "real" mountpoint.

I think the expected behavior is that the metadata for this filesystem be stored at / (which is /var/lib/lxc/base/rootfs outside the container)?

Earlier we had considered selecting a mountpoint whose subtree contains all the other subtrees, but that wouldn't for you anyway since /var/lib/lxc/base/rootfs, /var/cache/pacman/pkg, and /srv/repo/x86_64 are all nonoverlapping. So I think we need to instead select a mountpoint that contains all other mountpoints: / contains /var/cache/pacman/pkg and /srv/http/x86_64.

[graysky2]

I think the expected behavior is that the metadata for this filesystem be stored at / (which is /var/lib/lxc/base/rootfs outside the container)?

Yes, this is outside of the container albeit on a different partition (I have /dev/nvme0n1p3 mounted to /var/lib/lxc on my root file system which is itself on /dev/nvme0n1p2).

I am thinking it would be better in the case of containers to store the metadata within the rootfs of the container rather than on the host system. So in the container's "/" rather than on the host's "/" as this this should be part of the container not the host. This is how ecryptfs-mount-private works.

Is this possible or could this be possible using fscrypt?

[ebiggers]

Yes, this is outside of the container albeit on a different partition (I have /dev/nvme0n1p3 mounted to /var/lib/lxc on my root file system which is itself on /dev/nvme0n1p2).

I'm not sure what you mean here, since your /proc/self/mountinfo says the device is /dev/sda5, not /dev/nvme0n1p3.

I am thinking it would be better in the case of containers to store the metadata within the rootfs of the container rather than on the host system. So in the container's "/" rather than on the host's "/" as this this should be part of the container not the host. This is how ecryptfs-mount-private works.

Is this possible or could this be possible using fscrypt?

That's basically what I suggested in my comment above. If we permitted non-root subtrees and if we selected the "main" mountpoint of a filesystem to be the one that contains the other mountpoints of that filesystem, then the metadata for /dev/sda5's mountpoints in your container would end up being stored in the container's /. I need to implement it though.

Note, however, that encrypted directories created from within the container wouldn't be easily unlockable from outside the container, as the container and host system would disagree about where the fscrypt metadata is stored. Maybe that is okay though.

[graysky2]

@ebiggers - Sorry, I have been experimenting with this on several machines. The output from my original entry above was indeed in /dev/sda5 on that particular host. I mistakenly mentioned the nvme paths of another machine.

That's basically what I suggested in my comment above. If we permitted non-root subtrees and if we selected the "main" mountpoint of a filesystem to be the one that contains the other mountpoints of that filesystem, then the metadata for /dev/sda5's mountpoints in your container would end up being stored in the container's /.

I need to implement it though. Note, however, that encrypted directories created from within the container wouldn't be easily unlockable from outside the container, as the container and host system would disagree about where the fscrypt metadata is stored. Maybe that is okay though.

To me, that is the best way to containerize the encryption such that it is independent from the host (with the exception of residing on a shared filesystems). I would be grateful if you can modify the code to do it 😄

Please let me know if you would like some help in testing this.

[ebiggers]

Pull request opened: #213. @graysky2, feel free to test it.

[graysky2]

I patched against 0.2.7 and installed it within the container. If I ssh into the container, I can:

• As root, fscrypt setup worked creating /.fscrypt in the container
• As my user, I can fscrypt {lock,unlock,encrypt} as expected in the container. For example, ~/.config/chromium therein.

On the host (ie outside of the container), I am able to see the contents of /var/lib/lxc/playtime/rootfs/home/squishy/.config/chromium/ ... is this expected?

EDIT: I guess it would be since /var/lib/lxc/ is mounted on the host as well.

[ebiggers]

Yes, that's expected. When encrypted files are unlocked (or locked), they are unlocked (or locked) for everyone. Encryption is orthogonal to access control. Like any other files, standard mechanisms like UNIX mode bits can be used to restrict access. Note that while access control mechanisms typically don't restrict access by root, they have full control of the system anyway.

[graysky2]

I figured that 😄 Your patch seem to be working just fine in the LXCs.

As an aside, I added a simple systemd user service unit to lock the target directory when the LXC is shutdown which also seems to be working. There might be a more elegant method to achieve this.

# 
# ~ /.config/systemd/user/re-encrypt.service
# 

[Unit]
Description=encrypt on shutdown
DefaultDependencies=no
Before=shutdown.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c "/usr/bin/fscrypt lock /home/facade/.config/chromium"
TimeoutStartSec=0

[Install]
WantedBy=shutdown.target

[jorong1]

Me and my friend were very excited seeing this issue closed but we are still having one issue to it working properly. We can't tell if this is LXC inside-the-container specific or issue with fscrypt.

user@container$ fscrypt lock .protected
[ERROR] fscrypt lock: inode cache can only be dropped as root

Either this command should be run as root to properly clear the inode cache, or it should be run with --drop-caches=false (this may leave encrypted files and directories in an accessible state).

And seems using --drop-caches=falce leaves the files within the host-side root FS in visible cleartext.

Here are our application versions. fscrypt is compiled from source, go is binary from go website:
root@myhost$ lxd --version
3.0.3

joro@container1$ fscrypt --version
fscrypt version v0.2.7-1-gfe86093
joro@container1$ go version
go version go1.14.2 linux/amd64
joro@container1$ uname -sr
Linux 4.15.0-96-generic

If I am doing drop-caches wrong, sorry to disrupt.
Regards

[ebiggers]

With v1 encryption policies, fscrypt lock requires global root.

We already fixed this by introducing v2 encryption policies, but you need to update your kernel to a version that supports them (v5.4 or later). Then re-run fscrypt setup and re-create your encrypted directories.

[jorong1]

Thank you for letting me know. I knew I was missing something simple! I will check my repos for available 5.4+ kernels, or compile one from source.