Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SElinux not correctly working with overlayfs #1727

Open
cyrus-mc opened this issue Nov 10, 2015 · 29 comments
Open

SElinux not correctly working with overlayfs #1727

cyrus-mc opened this issue Nov 10, 2015 · 29 comments

Comments

@cyrus-mc
Copy link

@cyrus-mc cyrus-mc commented Nov 10, 2015

Environment

OS: CentOS 7.1
Kernel: 3.10.0
rkt: 0.10.0
systemd: 208

When trying to start any container, with selinux set to Enforcing I received the following error:

/usr/lib/systemd/systemd: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Permission denied

Disabling selinux (disabled or Permissive) solved the issue.

@yifan-gu

This comment has been minimized.

Copy link
Contributor

@yifan-gu yifan-gu commented Nov 10, 2015

@yifan-gu

This comment has been minimized.

Copy link
Contributor

@yifan-gu yifan-gu commented Nov 10, 2015

From irc discussion, @cyrus-mc is using stage1-coreos.aci

@jonboulle

This comment has been minimized.

Copy link
Contributor

@jonboulle jonboulle commented Nov 11, 2015

hmm, is this the v0.10.0 release from GitHub, or did you build it yourself?
Could you please paste the entire output of what happens when you try to run it?

@ericchiang

This comment has been minimized.

Copy link
Contributor

@ericchiang ericchiang commented Nov 18, 2015

I've gotten the same issue when using rkt v0.10.0 and v0.11.0 (from releases page) on Fedora 23. I have to disable SELinux to run pods.

Run output:

$ sudo rkt run --insecure-skip-verify sha512-3f25067fc69f
rkt: using image from local store for image name coreos.com/rkt/stage1-coreos:0.11.0
/usr/lib/systemd/systemd: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Permission denied

Full audit logs here.

Interesting part:

type=AVC msg=audit(1447866807.475:2032): avc:  denied  { read } for  pid=11851 comm="systemd" name="libselinux.so.1" dev="dm-2" ino=6569480 scontext=system_u:system_r:svirt_lxc_net_t:s0:c330,c605 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
type=AVC msg=audit(1447866807.475:2033): avc:  denied  { read } for  pid=11851 comm="systemd" name="libselinux.so.1" dev="dm-2" ino=6569480 scontext=system_u:system_r:svirt_lxc_net_t:s0:c330,c605 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
type=USER_AVC msg=audit(1447866807.516:2034): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='Unknown permission stop for class system exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
type=USER_AVC msg=audit(1447866807.517:2035): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='Unknown permission stop for class system exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
@jonboulle

This comment has been minimized.

Copy link
Contributor

@jonboulle jonboulle commented Nov 18, 2015

@mjg59 any ideas here?

@jonboulle jonboulle added this to the v0.13.0 milestone Dec 2, 2015
@krnowak

This comment has been minimized.

Copy link
Collaborator

@krnowak krnowak commented Dec 10, 2015

From OOB communication with @mjg59 we can say that rkt is not supported by Fedora/RHEL/CentOS selinux policy.

So, either we add a disable-selinux option to configure or we wait until Fedora policy gets fixed.

Another issue might be that we expect overlays to support selinux and CoreOS has a patched kernel for that.

@alban

This comment has been minimized.

Copy link
Member

@alban alban commented Dec 10, 2015

The overlay fs patch for SELinux support is here with others in the series.

@alban alban modified the milestones: v0.15.0, v0.14.0 Dec 18, 2015
@alban alban modified the milestones: v0.16.0, v0.15.0 Jan 8, 2016
@alban

This comment has been minimized.

Copy link
Member

@alban alban commented Jan 19, 2016

@lsm5 @rhatdan @mjg59

The command ausearch -m avc -ts recent returns:

----
time->Tue Jan 19 14:54:36 2016
type=AVC msg=audit(1453211676.627:617): avc:  denied  { read } for  pid=7201 comm="systemd" name="libselinux.so.1" dev="dm-1" ino=541346 scontext=system_u:system_r:svirt_lxc_net_t:s0:c9,c955 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
----
time->Tue Jan 19 14:54:36 2016
type=AVC msg=audit(1453211676.627:618): avc:  denied  { read } for  pid=7201 comm="systemd" name="libselinux.so.1" dev="dm-1" ino=541346 scontext=system_u:system_r:svirt_lxc_net_t:s0:c9,c955 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0

So it is systemd inside the container trying to read libselinux.so.1 inside the container. ls -liZ returns:

541346 -rwxr-xr-x. 1 root root unconfined_u:object_r:var_lib_t:s0                145440 Sep  9 07:44 /var/lib/rkt/cas/tree/deps-sha512-f5143c3cd24d997aea9a1c0e4e641f4ed9a44edeb4fdfe231a3b8059475100ca/rootfs/usr/lib64/libselinux.so.1
541346 -rwxr-xr-x. 1 root root system_u:object_r:svirt_sandbox_file_t:s0:c9,c955 145440 Sep  9 07:44 /var/lib/rkt/pods/run/0570eb27-fba5-4a2d-8550-9becf0d9ec70/stage1/rootfs/usr/lib64/libselinux.so.1

They are using the same inode because the first one is the overlay fs lower directory, and the second one is in the overlay fs mount.

@alban

This comment has been minimized.

Copy link
Member

@alban alban commented Jan 19, 2016

Same error without using overlay fs:

type=AVC msg=audit(1453212466.826:869): avc:  denied  { read } for  pid=9315 comm="systemd" name="libselinux.so.1" dev="dm-1" ino=541825 scontext=system_u:system_r:svirt_lxc_net_t:s0:c581,c949 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0

541825 -rwxr-xr-x. 1 root root unconfined_u:object_r:var_lib_t:s0 145440 Sep  9 07:44 /var/lib/rkt/pods/run/2787d853-c031-483f-a400-450a7be183d0/stage1/rootfs/usr/lib64/libselinux.so.1
@alban alban added the help wanted label Jan 19, 2016
@rhatdan

This comment has been minimized.

Copy link
Contributor

@rhatdan rhatdan commented Jan 20, 2016

You can not use SELinux and Overlay at this time. Well I guess you could label everything under /var/lib/rkt/pods as system_u:object_r:svirt_sandbox_file_t:s0 and it might work better. Not sure how you disable SELinux for rkt process separation, but you might have to. Red Hat Kernel Engineers continue to look to improve OverlayFS with SELinux but nothing so far.

@alban

This comment has been minimized.

Copy link
Member

@alban alban commented Jan 20, 2016

@rhatdan I tested as well with rkt run --no-overlay so that rkt does not use overlay fs, but I get the same error, see comment above.

Then, I tried:

$ sudo chcon -R  -u system_u -t svirt_sandbox_file_t /var/lib/rkt/pods
$ sudo ./build-rkt-0.15.0+git/bin/rkt run --debug --insecure-options=image --interactive --no-overlay docker://busybox
(...)
Welcome to Linux!

Initializing machine ID from container UUID.
bind(/run/systemd/notify) failed: Permission denied
Failed to bind private socket: Permission denied
Failed to fully start up daemon: No such file or directory
Failed to add watch on /run/systemd/ask-password: No such file or directory

So it works a bit further but still fails. ausearch -m avc -ts recent has plenty of errors; the first ones are:

----
time->Wed Jan 20 16:03:38 2016
type=AVC msg=audit(1453302218.553:2106): avc:  denied  { write } for  pid=4329 comm="systemd" name="systemd" dev="tmpfs" ino=682171 scontext=system_u:system_r:svirt_lxc_net_
t:s0:c433,c492 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=dir permissive=0
----
time->Wed Jan 20 16:03:38 2016
type=AVC msg=audit(1453302218.553:2107): avc:  denied  { write } for  pid=4329 comm="systemd" name="systemd" dev="tmpfs" ino=682171 scontext=system_u:system_r:svirt_lxc_net_
t:s0:c433,c492 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=dir permissive=0
----
time->Wed Jan 20 16:03:38 2016
type=AVC msg=audit(1453302218.553:2108): avc:  denied  { read write } for  pid=4329 comm="systemd" name="tty" dev="tmpfs" ino=682156 scontext=system_u:system_r:svirt_lxc_net
_t:s0:c433,c492 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=chr_file permissive=0
----
@alban alban modified the milestones: v1.0.0, v0.16.0 Jan 21, 2016
@rhatdan

This comment has been minimized.

Copy link
Contributor

@rhatdan rhatdan commented Jan 21, 2016

This looks like this content "systemd dir" and "tty chr_file" are being created by the user process on /tmp and then somehow being used within the container.

I guess rkt guys will need to figure out what is happening, I have not used rkt.

@iaguis iaguis modified the milestones: v1+, v1.0.0 Jan 26, 2016
@jonboulle jonboulle modified the milestones: v1.1.0, v1+ Jan 29, 2016
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 5, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).

Conflicts:
	pkg/kubelet/rkt/rkt.go
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 5, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).

Conflicts:
	pkg/kubelet/rkt/rkt.go
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 6, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).

Conflicts:
	pkg/kubelet/rkt/rkt.go
@jonboulle jonboulle modified the milestones: v1+, v1.6.0 May 12, 2016
@jonboulle

This comment has been minimized.

Copy link
Contributor

@jonboulle jonboulle commented May 12, 2016

Removing from specific milestone as still dependent on external work.

yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 13, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 13, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 16, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 23, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 23, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 25, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu pushed a commit to yifan-gu/kubernetes that referenced this issue May 25, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu added a commit to yifan-gu/kubernetes that referenced this issue May 31, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu added a commit to yifan-gu/kubernetes that referenced this issue Jun 1, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu added a commit to yifan-gu/kubernetes that referenced this issue Jun 1, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
yifan-gu added a commit to yifan-gu/kubernetes that referenced this issue Jun 1, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
k8s-github-robot added a commit to kubernetes/kubernetes that referenced this issue Jun 2, 2016
Automatic merge from submit-queue

rkt: Add pod selinux support.

Currently only pod level selinux context is supported, besides when
running selinux, we will not be able to use the overlay fs, see:
rkt/rkt#1727 (comment).


cc @kubernetes/sig-node  @alban @mjg59 @pmorie
mtaufen added a commit to mtaufen/kubernetes that referenced this issue Jun 6, 2016
Currently only pod level selinux context is supported, besides when
running selinux, for now we will not be able to use the overlay fs
except for coreos, see:
rkt/rkt#1727 (comment).
@sym3tri

This comment has been minimized.

Copy link
Contributor

@sym3tri sym3tri commented Jun 10, 2016

am having this same issue in Fedora 23

@iaguis

This comment has been minimized.

Copy link
Member

@iaguis iaguis commented Jun 13, 2016

am having this same issue in Fedora 23

The fixes (fedora-selinux/selinux-policy#100 and fedora-selinux/selinux-policy#108) are in Fedora 24. There's one more fix (fedora-selinux/selinux-policy#114) which I'm not sure it's included in 24, but it is in Rawhide.

@lucab lucab changed the title /usr/lib/systemd/systemd: error while loading shared libraries: libselinux.so.1: cannot open shared object file: Permission denied SElinux not correctly working with overlayfs Jul 7, 2016
@alban

This comment has been minimized.

Copy link
Member

@alban alban commented Jul 15, 2016

The pending kernel patches for SELinux + Overlayfs:
[RFC PATCH 0/9][V3] Overlayfs SELinux Support

/cc @mjg59

@rhatdan

This comment has been minimized.

Copy link
Contributor

@rhatdan rhatdan commented Jul 15, 2016

Yes, it currently looks like this will land in 4.9 kernel.

@jonboulle

This comment has been minimized.

Copy link
Contributor

@jonboulle jonboulle commented Sep 12, 2016

is this resolved?

@rhatdan

This comment has been minimized.

Copy link
Contributor

@rhatdan rhatdan commented Sep 12, 2016

SELinux and Overlayfs are currently working in the Fedora Rawhide Kernel and Fedora 25.

@wenjianhn

This comment has been minimized.

Copy link

@wenjianhn wenjianhn commented Jun 26, 2017

Overlayfs supports selinux since Linux 4.9.

The related commits are the following:

a518b0a selinux: Implement dentry_create_files_as() hook
2602625 security, overlayfs: Provide hook to correctly label newly created
files
c957f6d selinux: Pass security pointer to determine_inode_label()
19472b6 selinux: Implementation for inode_copy_up_xattr() hook
121ab82 security,overlayfs: Provide security hook for copy up of xattrs for
overlay file
56909eb selinux: Implementation for inode_copy_up() hook
d8ad8b4 security, overlayfs: provide copy up security hook for unioned files 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.