Skip to content
This repository has been archived by the owner on Feb 24, 2020. It is now read-only.

stage1/prepare-app: don't mount /sys/fs/cgroup in stage2 #2681

Merged
merged 3 commits into from May 24, 2016
Merged

stage1/prepare-app: don't mount /sys/fs/cgroup in stage2 #2681

merged 3 commits into from May 24, 2016

Conversation

iaguis
Copy link
Member

@iaguis iaguis commented May 24, 2016

The previous two patches make rkt not do a recursive bind-mount on /sys
unless we use user namespaces (the kernel doesn't allow that). However,
SELinux doesn't allow mounting /sys/fs/cgroup manually in that case.

Since the spec doesn't say apps should have a view of /sys/fs/cgroup,
let's not mount it in stage2.

Fixes #2351

@iaguis iaguis added this to the v1.7.0 milestone May 24, 2016
@iaguis
Copy link
Member Author

iaguis commented May 24, 2016

On Fedora rawhide:

$ getenforce
Enforcing
$ sudo ./build-rkt-1.6.0+git/bin/rkt run kinvolk.io/aci/busybox:1.24 --interactive
image: using image from file /var/tmp/rkt/build-rkt-1.6.0+git/bin/stage1-host.aci
image: using image from local store for image name kinvolk.io/aci/busybox:1.24
networking: loading networks from /etc/rkt/net.d
networking: loading network default with type ptp
/ # mount | grep sys
tmpfs on /dev/null type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/zero type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/full type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/random type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/urandom type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/tty type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
tmpfs on /dev/net/tun type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,mode=755)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
tmpfs on /proc/sys/kernel/random/boot_id type tmpfs (ro,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,nodev,mode=755)
tmpfs on /proc/kmsg type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,nodev,mode=755)
tmpfs on /dev/shm type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,nodev)
devpts on /dev/pts type devpts (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
tmpfs on /run/systemd/journal type tmpfs (rw,context="system_u:object_r:svirt_sandbox_file_t:s0:c83,c671",nosuid,nodev,mode=755)
sysfs on /sys type sysfs (ro,seclabel,nosuid,nodev,noexec,relatime)
/ # exit

On my host:

$ sudo ./rkt-monitor too-many-apps.podmanifest
prepare-app(13384): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13391): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13845): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13853): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13372): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13385): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13585): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
prepare-app(13344): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13671): seconds alive: 4  avg CPU: 5.499009%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13824): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13790): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13347): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13726): seconds alive: 4  avg CPU: 4.379131%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13627): seconds alive: 4  avg CPU: 0.582759%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13743): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13353): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13591): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
worker-binary(13600): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13765): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13877): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13697): seconds alive: 4  avg CPU: 1.162000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13746): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13901): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13753): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13904): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13873): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13351): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13388): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13618): seconds alive: 4  avg CPU: 4.078320%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13708): seconds alive: 4  avg CPU: 0.870418%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13721): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13738): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13849): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13343): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13595): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13665): seconds alive: 4  avg CPU: 4.809204%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13678): seconds alive: 4  avg CPU: 7.376779%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13763): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13788): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13861): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13888): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13348): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13373): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13718): seconds alive: 4  avg CPU: 4.809348%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13831): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13869): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13896): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13880): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
systemd-journal(13341): seconds alive: 5  avg CPU: 0.135815%  avg Mem: 6 mB  peak Mem: 6 mB
prepare-app(13352): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13584): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
worker-binary(13593): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13643): seconds alive: 4  avg CPU: 5.245878%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13735): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13804): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13731): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13797): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13604): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13704): seconds alive: 4  avg CPU: 0.580262%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13741): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13838): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13886): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13710): seconds alive: 4  avg CPU: 3.203592%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13759): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13867): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13346): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13609): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
worker-binary(13623): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13680): seconds alive: 4  avg CPU: 5.504373%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13711): seconds alive: 4  avg CPU: 2.908307%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13771): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13890): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
worker-binary(13612): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13762): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13781): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13814): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13840): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13358): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
worker-binary(13598): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13642): seconds alive: 4  avg CPU: 3.497555%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13713): seconds alive: 4  avg CPU: 3.638243%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13829): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13844): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13356): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13768): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13846): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13847): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13893): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
rkt(13302): seconds alive: 7  avg CPU: 26.756880%  avg Mem: 5 mB  peak Mem: 20 mB
prepare-app(13360): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13383): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13652): seconds alive: 4  avg CPU: 5.245667%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13696): seconds alive: 4  avg CPU: 2.041350%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13856): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
systemd(13337): seconds alive: 5  avg CPU: 16.570744%  avg Mem: 11 mB  peak Mem: 12 mB
(r-binary)(13719): seconds alive: 4  avg CPU: 1.165949%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13807): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13842): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13826): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13378): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13390): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13716): seconds alive: 4  avg CPU: 2.910807%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13748): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13751): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13774): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13819): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13848): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
worker-binary(13589): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 836 kB  peak Mem: 836 kB
(r-binary)(13638): seconds alive: 4  avg CPU: 3.060403%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13800): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13891): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13365): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13370): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13393): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
prepare-app(13394): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13822): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13835): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13345): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13782): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13883): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13603): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13737): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
prepare-app(13374): seconds alive: 1  avg CPU: 0.000000%  avg Mem: 0 B  peak Mem: 0 B
(r-binary)(13620): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13681): seconds alive: 4  avg CPU: 6.084009%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13808): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13686): seconds alive: 4  avg CPU: 2.333047%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13785): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB
(r-binary)(13795): seconds alive: 4  avg CPU: 0.000000%  avg Mem: 10 mB  peak Mem: 10 mB

@iaguis
stage1/prepare-app: don't mount /sys/fs/cgroup in stage2
2954a58 
The previous two patches make rkt not do a recursive bind-mount on /sys
unless we use user namespaces (the kernel doesn't allow that). However,
SELinux doesn't allow mounting `/sys/fs/cgroup` manually in that case.

Since the spec doesn't say apps should have a view of /sys/fs/cgroup,
let's not mount it in stage2.
@alban alban mentioned this pull request May 24, 2016
Closed
@alban
Copy link
Member

alban commented May 24, 2016

LGTM if green and if it really works.

Fixes #2675 as well.

@iaguis
Copy link
Member Author

iaguis commented May 24, 2016

I started the too-many-apps.podmanifest file generated by rkt-monitor and I can see systemd taking almost 0 CPU after the pod starts (around 4s):

24658 root      20   0   46.2m  12.9m   0.0  0.1   0:01.30 S                      `- systemd                      
24661 root      20   0   30.4m   6.7m   0.0  0.0   0:00.09 S                          `- systemd-journal          
24869 root      20   0    2.7m   0.8m   0.0  0.0   0:00.04 S                          `- worker-binary            
24872 root      20   0    2.7m   0.8m   0.0  0.0   0:00.11 S                          `- worker-binary            
24874 root      20   0    2.7m   0.8m   0.0  0.0   0:00.23 S                          `- worker-binary            
24876 root      20   0    2.7m   0.8m   0.0  0.0   0:00.21 S                          `- worker-binary            
24878 root      20   0    2.7m   0.8m   0.0  0.0   0:00.37 S                          `- worker-binary            
24879 root      20   0    2.7m   0.8m   0.0  0.0   0:00.25 S                          `- worker-binary            
24881 root      20   0    2.7m   0.8m   0.0  0.0   0:00.27 S                          `- worker-binary            
24884 root      20   0    2.7m   0.8m   0.0  0.0   0:00.35 S                          `- worker-binary           
[...]

@iaguis iaguis merged commit 1c221bb into rkt:master May 24, 2016
@alban alban mentioned this pull request May 24, 2016
Closed
@alban alban mentioned this pull request Jul 12, 2016
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
area/performance priority/P0 required-by/rktnetes reviewed/lgtm technology/systemd
Projects
None yet
Milestone
v1.7.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@iaguis @alban