SELinux on Fedora: rkt started as systemd service: execv() failed: No such file or directory

[alban]

After #2308 was merged, I ran the tests on Fedora Rawhide with tests/aws.sh with the host flavor. A lot of tests fail with "execv() failed" but it is not always reproducible:

=== RUN   TestInteractive
--- FAIL: TestInteractive (7.31s)
    rkt_interactive_test.go:83: Running test #0: Check tty without interactive
    rkt_tests.go:113: Running command: /var/tmp/rkt/builds/build-rkt-host-/build-rkt-1.2.1+git/tmp/functional/rkt --dir=/tmp/datadir-240431676 --local-config=/tmp/localdir-165758059 --system-config=/tmp/systemdir-791001294 --debug --insecure-options=image run --mds-register=false /var/tmp/rkt/builds/build-rkt-host-/build-rkt-1.2.1+git/tmp/functional/test-tmp/rkt-inspect-interactive.aci
    rkt_interactive_test.go:102: Expected "stdin is not a terminal" but not found #0: image: using image from file /var/tmp/rkt/builds/build-rkt-host-/build-rkt-1.2.1+git/tmp/functional/stage1-host.aci
        image: using image from file /var/tmp/rkt/builds/build-rkt-host-/build-rkt-1.2.1+git/tmp/functional/test-tmp/rkt-inspect-interactive.aci
        stage0: Preparing stage1
        stage0: Writing image manifest
        stage0: Loading image sha512-2bbb0f890045ed40150a4cc8a0b41cfbff962bca101f63c083585360ba0d3370
        stage0: Writing image manifest
        stage0: Writing pod manifest
        stage0: Setting up stage1
        stage0: Wrote filesystem to /tmp/datadir-240431676/pods/run/9cf5e94c-2dd7-4f19-bbb7-a5f881235447
        stage0: Pivoting to filesystem /tmp/datadir-240431676/pods/run/9cf5e94c-2dd7-4f19-bbb7-a5f881235447
        stage0: Execing /init
        networking: loading networks from /tmp/localdir-165758059/net.d
        networking: loading network default with type ptp
        Spawning container rkt-9cf5e94c-2dd7-4f19-bbb7-a5f881235447 on /tmp/datadir-240431676/pods/run/9cf5e94c-2dd7-4f19-bbb7-a5f881235447/stage1/rootfs.
        Press ^] three times within 1s to kill container.
        execv() failed: No such file or directory
        Container rkt-9cf5e94c-2dd7-4f19-bbb7-a5f881235447 failed with error code 1.

[alban]

It only happens when rkt is executed in a systemd service and SELinux is enabled.

[alban]

To reproduce the issue, I used:

systemd-run /usr/bin/rkt run --insecure-options=image,ondisk --no-overlay docker://redis

The message "No such file or directory" was misleading: strace reveals that there was EACCESS errors before:

execve("/usr/lib/systemd/systemd", ["/usr/lib/systemd/systemd", "--default-standard-output=tty", "--log-target=null", "--show-status=0"], [/* 6 vars */]) = -1 EACCES (Permission denied)
execve("/lib/systemd/systemd", ["/lib/systemd/systemd", "--default-standard-output=tty", "--log-target=null", "--show-status=0"], [/* 6 vars */]) = -1 EACCES (Permission denied)
execve("/sbin/init", ["/sbin/init", "--default-standard-output=tty", "--log-target=null", "--show-status=0"], [/* 6 vars */]) = -1 ENOENT (No such file or directory)

The SELinux errors are:

AVC avc:  denied  { transition } for  pid=21591 comm="ld-linux-x86-64" path="/usr/lib64/systemd/systemd" dev="xvda1" ino=169603 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:svirt_lxc_net_t:s0:c478,c671 tclass=process permissive=0
AVC avc:  denied  { transition } for  pid=21591 comm="ld-linux-x86-64" path="/usr/lib64/systemd/systemd" dev="xvda1" ino=169603 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:svirt_lxc_net_t:s0:c478,c671 tclass=process permissive=0

[alban]

/cc @rhatdan Do you have an idea about this SELinux issue ^ when rkt is started via systemd-run?

[alban]

I tried with a .service file with the "SELinuxContext" option:

[Unit]
Description=etcd

[Service]
Slice=machine.slice
ExecStart=/usr/bin/rkt run coreos.com/etcd:v2.2.5
KillMode=mixed
Restart=no
SELinuxContext=system_u:system_r:unconfined_t:s0

And it made it work. Should we recommend this in the documentation?
https://github.com/coreos/rkt/blob/master/Documentation/using-rkt-with-systemd.md#simple-unit-file

[alban]

Filed systemd/systemd#2896 to request a SELinuxContext parameter in systemd-run.

[iaguis]

Great monologue :)

[rhatdan]

We actually need to work on the policy. The problem here is we need to add a transiton from initrc_t to svirt_lxc_net_t.

[rhatdan]

cat myrkt.te
policy_module(myrkt,1.0)
gen_require(`
    type init_t, initrc_t;
')


virt_transition_svirt_sandbox(init_t, system_r)
virt_transition_svirt_sandbox(initrc_t, system_r)

Now compile this te file and install it. You need to have selinux-policy-devel
installed.

make -f /usr/share/selinux/devel/Makefile myrkt.pp
semodule -i myrkt.pp

Now you should be able to run the rkt container without the unconfined_t line.

[rhatdan]

fedora-selinux/selinux-policy#114

[alban]

@rhatdan thanks, it works fine with your myrkt.pp.

[alban]

There are other SELinux issues but I will file them as separate issues as they are found.

[yifan-gu]

@alban Seems like we don't need to wait for #2361 to support selinux options in kubernetes/kubernetes#23773

We can just use the SELinuxContext option in the service file, right?

[alban]

@yifan-gu sure, Kubernetes can use SELinuxContext=.

Kubernetes writes the service file itself and does not use systemd-run, does it? I am asking because of systemd/systemd#2896

[yifan-gu]

@alban Yes, it is writing the service file to disk today

[yifan-gu]

@alban What is the SELinuxContext I need to set to make rkt run as unit on fedora? I keep getting permission denied even if I don't specify any selinux context.

But I can run manually through cmd line.

[alban]

Needs to figure out which version of Fedora has the fix: fedora-selinux/selinux-policy#114 (comment)

[yifan-gu]

@alban I tried with no-overlay and the context you mentioned in #2322 (comment)
It works.

[alban]

Moving milestone, since this is tracking an external target (whether the fix is in all versions of Fedora).