osutil: workaround overlayfs on ubuntu 18.10 #5974
Conversation
This patch adds a workaround for apparmor and overlayfs not playing
together on the ephemeral Ubuntu 18.10 server images. On such images
there's an overlayfs mounted over / with the upper directory in
/media/root-rw/overlay. Snapd detects this and generates a directive
with read access to said directory. At runtime we get a denial, however,
one that looks like this:
[ 1588.858154] audit: type=1400 audit(1539338016.165:576):
apparmor="DENIED" operation="open" profile="/usr/lib/snapd/snap-confine"
name="/overlay/" pid=8735 comm="snap-confine"
profile="/usr/lib/snapd/snap-confine" requested_mask="r"
denied_mask="r" fsuid=0 ouid=0
As we can see apparmor decided to resolve the path to "/overlay/" (which
notably does not exist in the filesystem at all). The reason for that is
not understood but as a special-case workaround we detect this and
return "/overlay" instead.
Bug-Link: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1797218
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
jdstrand
left a comment
jdstrand
left a comment
There was a problem hiding this comment.
Approving but please adjust the comments. Feel free to add more precise details if desired.
osutil/overlay_linux.go
Outdated
| } | ||
|
|
||
| if dir == "/media/root-rw/overlay" { | ||
| // Apparmor doesn't really support overlayfs correctly. |
There was a problem hiding this comment.
Please remove this editorial comment.
osutil/overlay_linux.go
Outdated
| // apparmor resolves the path the rules look for "/overlay" | ||
| // for some unknown reason. As a quick hack to unblock the | ||
| // Ubuntu 18.10 release, detect this condition and return | ||
| // the path that apparmor would detect. |
There was a problem hiding this comment.
Please update this comment. Something like the following:
// On the Ubuntu server ephemeral image, '/' is setup via
// overlayroot (on at least 18.10), which uses a combination
// of overlayfs and chroot. This differs from the livecd setup
// so special case the detection logic to look for the known
// upperdir for this configuration, and return the required
// path. See LP: #1797218 for details.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
|
For reference, here is /proc/mounts from an overlay root system: The relevant mounts are below. I suspect that '/overlay' is from '/media/root-rw/overlay'. The code that does that is overlayroot/scripts/init-bottom/overlayroot |
smoser
left a comment
smoser
left a comment
There was a problem hiding this comment.
I think the bug is actually in the general reading of /proc/mounts, specifically here. It assumes that the source (fs_spec per fstab(5)) is a directory path ('/cow'). That isn't really guaranteed and is really just a happenstance of the live-cd setup.
The real fix I think would be to read upper-dir and traverse backwards through /proc/mounts to find what other mounts (specifically fs_file per fstab(5)) that upper is on.
| } | ||
|
|
||
| if dir == "/media/root-rw/overlay" { | ||
| // On the Ubuntu server ephemeral image, '/' is setup via |
There was a problem hiding this comment.
It may help you to have a more specific comment here.
It's not "Ubuntu server" that does this.
Its 'overlayroot' (package) that sets things up this way.
overlayroot package is configured by kernel command line or /etc/overlayroot.conf or /etc/overlayroot.local.conf in all supported Ubuntu. The package is installed by default in Ubuntu Server but is available in Ubuntu since 12.04.
There was a problem hiding this comment.
Also just note that the 'overlay' portion of '/media/root-rw/overlay' is configurable via overlayroot.conf or kernel command line. overlayroot=tmpfs:dir=/mydir . dir defaults to '/overlay'.
| if dir == "/media/root-rw/overlay" { | ||
| // On the Ubuntu server ephemeral image, '/' is setup via | ||
| // overlayroot (on at least 18.10), which uses a combination | ||
| // of overlayfs and chroot. This differs from the livecd setup |
There was a problem hiding this comment.
no chroot us used. pivot is done into the rootfs from the initramfs, but that is the common path of initramfs.
| }, { | ||
| mountinfo: "31 1 0:26 / / rw,relatime shared:1 - overlay overlay rw,lowerdir=//filesystem.squashfs,upperdir=/cow/bad\"upper,workdir=/cow/work", | ||
| }, { | ||
| // The special cased version for 18.10 server release |
There was a problem hiding this comment.
reference 'overlayroot' here rather than 'Ubuntu 18.10 server release'.
|
Could you expand on why returning /overlay resolves the issue? I would have expected that it should return the complete upperdir path. |
|
@raharper because (probably due to pivot_root) this is what the apparmor denial specifies. For whatever reason apparmor chooses to use |
5 participants
This patch adds a workaround for apparmor and overlayfs not playing
together on the ephemeral Ubuntu 18.10 server images. On such images
there's an overlayfs mounted over / with the upper directory in
/media/root-rw/overlay. Snapd detects this and generates a directive
with read access to said directory. At runtime we get a denial, however,
one that looks like this:
As we can see apparmor decided to resolve the path to "/overlay/" (which
notably does not exist in the filesystem at all). The reason for that is
not understood but as a special-case workaround we detect this and
return "/overlay" instead.
Bug-Link: https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1797218
Signed-off-by: Zygmunt Krynicki zygmunt.krynicki@canonical.com