systemctl start not working when ReadWriteDirectories is a symlink

[xorbug]

This bug has been reported on debian bts first:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792187
I copy/paste relevant infos:

Package: systemd
Version: 221-1
Severity: normal

Dear Maintainer,

I installed tor (The onion router) the other day and when i started it
(either via /usr/sbin/service or systemctl) i went through this:

$ sudo systemctl start tor.service

Job for tor.service failed because the control process exited with
error code. See "systemctl status tor.service" and "journalctl -xe"
for details.

$ systemctl status tor.service

● tor.service - Anonymizing overlay network for TCP
   Loaded: loaded (/lib/systemd/system/tor.service; enabled; vendor
preset: enabled)
   Active: failed (Result: start-limit) since Sun 2015-07-12 01:47:54
CEST; 45s ago
  Process: 19035 ExecStartPre=/usr/bin/install -Z -m 02750 -o
debian-tor -g debian-tor -d /var/run/tor (code=exited,
status=226/NAMESPACE)

$ sudo journalctl -xe

[...]
Jul 12 01:47:54 blade systemd[1]: Starting Anonymizing overlay network
for TCP...
-- Subject: Unit tor.service has begun start-up
-- Defined-By: systemd
-- Support:
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit tor.service has begun starting up.
Jul 12 01:47:54 blade systemd[19030]: tor.service: Failed at step
NAMESPACE spawning /usr/bin/install: Too many levels of symbolic links
-- Subject: Process /usr/bin/install could not be executed
-- Defined-By: systemd
-- Support:
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- The process /usr/bin/install could not be executed and failed.
-- 
-- The error number returned by this process is 40.
Jul 12 01:47:54 blade kernel: Chromium OS LSM: Mount path with
symlinks prohibited - pid=19030 cmdline="(install)
"
Jul 12 01:47:54 blade systemd[1]: tor.service: Control process exited,
code=exited status=226
Jul 12 01:47:54 blade systemd[1]: Failed to start Anonymizing overlay
network for TCP.
-- Subject: Unit tor.service has failed
-- Defined-By: systemd
-- Support:
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit tor.service has failed.
-- 
-- The result is failed.
Jul 12 01:47:54 blade systemd[1]: tor.service: Unit entered failed
state.
Jul 12 01:47:54 blade systemd[1]: tor.service: Failed with result
'exit-code'.
Jul 12 01:47:54 blade systemd[1]: tor.service: Service hold-off time
over, scheduling restart.
[...]

At first i thought it was a kernel issue (beware also that my kernel
is a chrome os kernel, not the one shipped by Debian, if that
matters). Anyways running the commands in the tor unit file by hand,
one by one in a terminal, leads to a correct execution. The same
renaming/removing the tor unit file and starting the service using the
init file in /etc/init.d.

After a quick jump on the #tor IRC channel we concluded that this may
be an issue on the systemd side, and after reading something around
the web we tried to tweak the Hardening section of the unit file. And
indeed we found that ReadWriteDirectories is set to /var/run which on
my system is a link to /run. Changing ReadWriteDirectories to /run and
running 'systemctl daemon-reload' solved the issue and now the service
is starting fine.

Let me know if more infos are needed.
Thanks.

-- Package-specific info:

-- System Information:
Debian Release: stretch/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: armhf (armv7l)

Kernel: Linux 3.8.11 (SMP w/2 CPU cores)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages systemd depends on:
ii adduser 3.113+nmu3
ii libacl1 2.2.52-2
ii libapparmor1 2.9.2-3
ii libaudit1 1:2.4.2-1
ii libblkid1 2.26.2-6
ii libc6 2.19-18
ii libcap2 1:2.24-9
ii libcap2-bin 1:2.24-9
ii libcryptsetup4 2:1.6.6-5
ii libgcc1 1:5.1.1-12
ii libgcrypt20 1.6.3-2
ii libkmod2 20-1
ii liblzma5 5.1.1alpha+20120614-2.1
ii libmount1 2.26.2-6
ii libpam0g 1.1.8-3.1
ii libseccomp2 2.2.1-2
ii libselinux1 2.3-2+b1
ii libsystemd0 221-1
ii mount 2.26.2-6
ii sysv-rc 2.88dsf-59.2
ii udev 221-1
ii util-linux 2.26.2-6

Versions of packages systemd recommends:
ii dbus 1.8.18-1
ii libpam-systemd 221-1

Versions of packages systemd suggests:
pn systemd-ui

-- Configuration Files:
/etc/systemd/logind.conf changed [not included]

-- no debconf information

This is the unit file included in the tor package:

[Unit]
Description=Anonymizing overlay network for TCP
After=network.target nss-lookup.target

[Service]
Type=notify
NotifyAccess=all
PIDFile=/var/run/tor/tor.pid
PermissionsStartOnly=yes
ExecStartPre=/usr/bin/install -Z -m 02750 -o debian-tor -g debian-tor -d
/var/run/tor
ExecStartPre=/usr/bin/tor --defaults-torrc
/usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0
--verify-config
ExecStart=/usr/bin/tor --defaults-torrc
/usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0
ExecReload=/bin/kill -HUP ${MAINPID}
KillSignal=SIGINT
TimeoutSec=45
Restart=on-failure
LimitNOFILE=65536

# Hardening
PrivateTmp=yes
PrivateDevices=yes
ProtectHome=yes
ProtectSystem=full
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/tor
ReadWriteDirectories=-/var/log/tor
ReadWriteDirectories=-/var/run
CapabilityBoundingSet=CAP_SETUID CAP_SETGID CAP_NET_BIND_SERVICE
CAP_DAC_OVERRIDE CAP_CHOWN CAP_FOWNER

[Install]
WantedBy=multi-user.target

[alban]

• Do you also have a RootDirectory= statement in your service file? If so, this looks like similar issues in nspawn doesn't handle bind-mounts for symlinks correctly #2860 (nspawn) and in namespace: unify limit behavior on non-directory paths #3327 (comment) (systemd).
• Is /var/run a symlink to /run or to ../run? If it is the former, could you try to set it up like the latter? Relative paths should work better.

$ ls -l /var/run
lrwxrwxrwx. 1 root root 6 Dec 11 15:29 /var/run -> ../run

/cc @lucab @alepuccetti

[xorbug]

Hi @alban, i'm really sorry but i'm afraid i can not provide further infos since i no longer own that ARM system (it was almost 1 year ago). Thank you anyways, hope this report may still be useful at some point.

[fkleon]

Same issue here on Debian Testing with systemd 230-5.

@alban

• There is no RootDirectory statement in the service file.
• /var/run is a symlink to /run. I changed it to the relative path ../run, and that works!

Another workaround is changing the ReadWriteDirectory to just /run as described by the OP.

[poettering]

This is going to be fixed by #4018.

[evverx]

This was fixed by #4018