Masking files/directories

[flx42]

The Linux section of the OCI runtime spec has an array of strings called maskedPaths:

maskedPaths (array of strings, OPTIONAL) will mask over the provided paths inside the container so that they cannot be read. The values MUST be absolute paths in the container namespace.

Here's an example from the default OCI spec generated by runc:

		"maskedPaths": [
			"/proc/kcore",
			"/proc/latency_stats",
			"/proc/timer_list",
			"/proc/timer_stats",
			"/proc/sched_debug",
			"/sys/firmware",
			"/proc/scsi"
		],

This is how runc handles it:
https://github.com/opencontainers/runc/blob/63e6708c74c1cc46091ec92ea9df5aca4d82e803/libcontainer/rootfs_linux.go#L777-L790

// For files, maskPath bind mounts /dev/null over the top of the specified path.
// For directories, maskPath mounts read-only tmpfs over the top of the specified path.

Straighforward, but in LXC I don't see a true equivalent. If you know in advance if the target is a file or directory, you can do the right lxc.mount.entry. Here, files (/proc/kcore) and directories (/sys/firmware) are in the same list, and also some paths might not exist (/proc/latency_stats for me).
The solution I've used is to add both mounts, but both optional:
brauner@c816bd8

If the target is a file, the first mount should succeed and the second one will fail.
If the target is a directory, the first mount will fail and the second one should succeed.
If the target doesn't exist, both mounts will fail.

There is a gap: if the target exists, but both mounts fail for another reason. We have no way of guaranteeing that at least one mount will succeed if the target exists.

What do you think? Should we add another configuration options to handle this case natively? Any other idea?

[hallyn]

It seems to me this should be more of a system-wide policy thing. In which case, the default profile for all containers would do this automatically. Why on earth should it be up to the container runtime spec to specify this kind of host security detail?

…

-serge

[flx42]

If you control the OCI runtime spec, you can do everything you want security-wise.
There are cases where you do want to do privileged operations inside your container, so I guess that's the reason why it's per-container.

It's outside of the scope of the spec, but I understand your point, maybe the runtime should be free to add its own security restrictions.

[hallyn]

Ok so in any case the general feature is fine. I think a good way forward would be:

1. implement lxc.pathmask = which does what you describe
2. maybe add some default lxc.pathmask's to the common config file

@brauner, @tych0 what do you think?

[tych0]

Is there a reason we can't use deny $evil_path rwklx in the apparmor profile? I guess lxc doesn't have a way to append rules to the apparmor profile like lxd does, but perhaps that would be a more flexible alternative?

[tych0]

I guess that makes the behavior slightly different than the OCI spec, since you'd get EPERM instead of a file of zero length when you opened it, but since nobody is presumably looking at these files anyway, it's probably ok.

[flx42]

It doesn't look like the spec mandates how it should be implemented. I also found this comment that agrees with you :)
opencontainers/runtime-spec#582 (comment)
I actually prefer EPERM, an empty file looks like you failed your bind-mount ;)

So, you're right, we could use apparmor. However, I don't feel it means we have to discard the idea of adding a new option. Apparmor vs bind-mount would be an implementation detail, with bind-mount being the fallback when apparmor is not available.

[hallyn]

So that sounds like we want a generic lxc.pathmask mechanism, which would be implemented so as to use apparmor or selinux when active, or a bind mount otherwise?

How many users are there who actually aren't using apparmor?

(Keeping in mind that the bind mount approach has other implications such as inability to further create nested containers without jumping through hoops)

[flx42]

Hm, good point. Let's start with apparmor then, we will need a way to add those rules to the profile.
OK with that @brauner?

[brauner]

I'm fine with doing it via AppArmor. I'm against lxc.pathmask for now though.

[flx42]

Yeah, agree, we can revisit at a later time. I'll let you close this issue @brauner if you want.

[brauner]

So, here's my idea. Doing it via AppArmor by default is fine. But I'm against the lxc.pathmask solution for LXC. Plain LXC users that want to hide paths and files can do this right now by using lxc.mount.entry. For the OCI config parser we can simply establish and additional internal protocol. This can be as simple as, while maskedPaths or whatever it's called is parsed we set a boolean on the corresponding mount entry hidden to true. (Might need to change the or add a struct for the mount entries.) Then when we go on to mount them we check whether this boolean is set to true and if so and there's no AppArmor support enabled we will stat the source and determine whether it is a file or not. When it's a file we yank /dev/null over it otherwise we'll simply strap a tmpfs on top of it. Thoughts?
//cc @hallyn, flx42

[flx42]

This can be as simple as, while maskedPaths or whatever it's called is parsed we set a boolean on the corresponding mount entry hidden to true. (Might need to change the or add a struct for the mount entries.)

I was following you until this point. What's the corresponding mount entry?
If I want to hide /sys/firmware and my corresponding mount entry is /sys, how would I do that? Wouldn't I need a new mount entry anyway for applying this hidden flag to a subset of sysfs?

[brauner]

I need a new mount entry anyway for applying this hidden flag to a subset of sysfs?

Now I'm not following: why is your corresponding mount entry /sys if you want to hide /sys/firmware? Wouldn't maskedPath list /sys/firmware and hence, you could add a mount entry for /sys/firmware to the container's mount list and set the boolean hidden to true?