systemd --user services are not starting automatically

[shibumi]

Hello,
I have different service-files in /etc/systemd/user/. I have enabled them via systemctl --user enable <myService>.service, but they are not starting automatically.

Here is an example of one of my services:

[Unit]
Description=urxvt daemon
Documentation=man:urxvtd(1)

[Service]
ExecStart=/usr/bin/urxvtd -q 

[Install]
WantedBy=multi-user.target

You can see that the service should start urxvt-daemon if multi-user.target is reached (I have some other services with graphical.target this doesn't work also). This is already reached when I login via TTY. (Hint: I am not using a loginmanager. I login via TTY and start my X-Server manually via startx if I need it). The problem is is that this service (and some others) are not starting automatically. If I checkout journalctl --user I can't even see a try to start the service. If I start the service manually it's working fine.

I have by the way the same behaviour for timers. They are also not started automatically when I have enabled them as user-service.

For me this is a bug (maybe I am doing something wrong, please correct me).

My systemd version is the following:

systemd 229
+PAM -AUDIT -SELINUX -IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN

[daurnimator]

Have you enabled linger?
sudo loginctl enable-linger "$USER"

[keszybz]

There is no multi-user.target in --user mode. User services should by WantedBy=default.target.

[shibumi]

@daurnimator I don't see the reason why I should enable linger for this issue? I mean linger is for the case if I logout. So that my services are still running after logging out.

@keszybz even if I use default.target it's not working. The services are not starting automatically at login

[daurnimator]

@daurnimator I don't see the reason why I should enable linger for this issue? I mean linger is for the case if I logout. So that my services are still running after logging out.

Linger is required to bring up user instances on boot; but I guess you didn't explicitly say you wanted that.

[keszybz]

OK, so let's debug this:

1. is default.target running? What does systemctl --user -t targer say?
2. is your service wanted by default.target? What does systemctl --user list-dependencies default.target say?
3. Finally, what does systemctl --user status <your-service>.service say?

[shibumi]

@daurnimator I want to bring them up only when I login not on boot.

@keszybz

Output of: systemctl --user -t target

UNIT           LOAD   ACTIVE SUB    DESCRIPTION
basic.target   loaded active active Basic System
default.target loaded active active Default
paths.target   loaded active active Paths
sockets.target loaded active active Sockets
timers.target  loaded active active Timers

Output of systemctl --user list-dependencies default.target

default.target
● ├─dwupdate.timer
● ├─gpg-agent.service
● ├─pulseaudio.service
● ├─urxvtd.service
● └─basic.target
●   ├─paths.target
●   ├─sockets.target
●   │ ├─dbus.socket
●   │ └─pulseaudio.socket
●   └─timers.target

You can see my 4 default units. 1 timer (dwupdate.timer) and 3 services.
They all are red. The rest is green.

If i check a status of a service I get this:

systemctl --user status gpg-agent.service

● gpg-agent.service - Gnupg Agent
   Loaded: loaded (/etc/systemd/user/gpg-agent.service; enabled; vendor
preset: enabled
   Active: inactive (dead)
     Docs: man:gpg-agent(1)

You can see that the service is enabled.
Here is the output of journalctl --user

Feb 22 15:41:24 motoko systemd[392]: Starting D-Bus User Message Bus Socket.
Feb 22 15:41:24 motoko systemd[392]: Reached target Timers.
Feb 22 15:41:24 motoko systemd[392]: Listening on Sound System.
Feb 22 15:41:24 motoko systemd[392]: Reached target Paths.
Feb 22 15:41:24 motoko systemd[392]: Listening on D-Bus User Message Bus
Socket.
Feb 22 15:41:24 motoko systemd[392]: Reached target Sockets.
Feb 22 15:41:24 motoko systemd[392]: Reached target Basic System.
Feb 22 15:41:24 motoko systemd[392]: Reached target Default.
Feb 22 15:41:24 motoko systemd[392]: Startup finished in 36ms.
Feb 22 15:41:33 motoko systemd[392]: Started D-Bus User Message Bus.
Feb 22 15:41:42 motoko dbus-daemon[491]: Activating service
name='org.a11y.Bus'
Feb 22 15:41:42 motoko dbus-daemon[491]: Successfully activated service
'org.a11y.Bus'
Feb 22 15:41:45 motoko systemd[392]: Time has been changed
Feb 22 15:43:55 motoko systemd[392]: Reloading.
Feb 22 15:44:00 motoko systemd[392]: Reloading.
Feb 22 15:44:08 motoko systemd[392]: Reloading.
Feb 22 15:44:30 motoko systemd[392]: Reloading.
Feb 22 15:44:50 motoko systemd[392]: Reloading.

Here is for example my gpg-agent.service

[Unit]
Description=Gnupg Agent
Documentation=man:gpg-agent(1)

[Service]
Type=forking
ExecStart=/usr/bin/gpg-agent  --homedir %h/.gnupg --use-standard-socket --daemon

[Install]
WantedBy=default.target

When I start the service manually it's working:
systemctl --user start gpg-agent.service && systemctl --user status gpg-agent.service

● gpg-agent.service - Gnupg Agent
   Loaded: loaded (/etc/systemd/user/gpg-agent.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2016-02-22 16:02:12 CET; 4s ago
     Docs: man:gpg-agent(1)
  Process: 1364 ExecStart=/usr/bin/gpg-agent --homedir /home/chris/.gnupg --use-standard-socket --daemon (
 Main PID: 1365 (gpg-agent)
   CGroup: /user.slice/user-1000.slice/user@1000.service/gpg-agent.service
           └─1365 /usr/bin/gpg-agent --homedir /home/chris/.gnupg --use-standard-socket --daemon

Feb 22 16:02:12 motoko systemd[392]: Starting Gnupg Agent...
Feb 22 16:02:12 motoko systemd[392]: Started Gnupg Agent.

[keszybz]

If it's red, than it means that the service was started, and failed. You should look at the logs (journalctl --user --user-unit gpg-agent or so) and figure out why it fails...

[shibumi]

@keszybz
You are so? I think 'red' means that the service is not running. For example: when I stop the service it will also be red.

I've checked the logs. There is nothing in the logs. Systemd has even not tried to start the services.

Could some other user with systemd v229 could please test this scenario?:

• Add service to /etc/systemd/user
• enable the service
• logout and login as the user and check if it's started or not.

[benjarobin]

@keszybz

If it's red, than it means that the service was started, and failed

This is true for the command status, but this is wrong or there is a bug for the command list-dependencies

[keszybz]

You are right, list-dependencies indeed shows a red dot. This is inconsistent with status, and thus misleading, but that's another story.

[benjarobin]

@keszybz I did open a "bug" report for that : #2718

[keszybz]

OK, I see something is wrong. systemctl --user enable gpg-agent didn't do anything, but later when I tried again, it installed a symlink as expected. I don't have time to debug this now, but it seems something fishy is going on.

[shibumi]

@keszybz after using reenable on all my custom services and one reboot it's working now. I don't know why I need to enable it twice.. second try works.

There are also different people on the arch-general mailinglist who complain about --user services:

https://lists.archlinux.org/pipermail/arch-general/2016-February/040797.html

but seems like they have other problems.. Just want to mention that here. I wrote them that they should open an issue here.

[grazzolini]

I can reproduce this bug by having a timer with Persistent=true (I guess BootSec=0 would also trigger). If the service the timer triggers was supposed to run, and with Persistent=true is most likely the case, systemd will start the manager for the user and reach the default target, and will start the timers/services. But after some seconds pam will complain about timeout and the manager service for the user will die with SIGRTMIN+24. I need to manually login as root, stop the processes and logout/login again.

For now I removed the Persistent=true flag. But I believe that if I happen to login at the exact time the timer was supposed to be run, this issue will still happen. And issuing two (or more) reenable's does not help as it did with @shibumi.

login[734]: pam_systemd(login:session): Failed to create session: Connection timed out
systemd[1012]: Received SIGRTMIN+24 from PID 1148 (kill).
systemd[1017]: pam_unix(systemd-user:session): session closed for user user
systemd[1]: Failed to start User Manager for UID 1000.
systemd[1]: user@1000.service: Unit entered failed state.
systemd[1]: user@1000.service: Failed with result 'timeout'.

[poettering]

@shibumi: so the problem at hand appears to be that "systemctl enable --user" is borked somehow? Any chance you can find a way to reproduce the issue?

@grazzolini sounds like a different problem. The systemd --user instance that is running in user@.service informs the system systemd instance about being ready as soon as it's job queue ran empty for the first time. Now, if you have a timer unit that (due to Persistant=yes) is triggered immediately at boot it might happen that the long running service it starts is immediately added to this job queue and thus means the job queue stays busy for a long time and the ready notification never happens. There's a bug open somewhere to change the ready notification to be triggered already as soon as default.target is reached, or when the queue runs empty, whatever comes first. Anyway, that's a different topic, so let's not continue discussin about this here. Thanks.

[grazzolini]

@poettering
Sorry for the noise. Given that the @shibumi mentioned the archlinux mail list, and the error I run into is the same described there, I presumed his problem and mine are related.

Your description fits exactly into my problem, the services being triggered are indeed long running processes. Good to know that this already is a known issue.

[shibumi]

@poettering
it's working fine since my last post.: #2690 (comment)

Not sure If i could reproduce this error. I will setup a new machine the next days. Maybe I can reproduce the issue there

[poettering]

OK, will close this here now. If you manage to reproduce this, please reopen or comment here! Thanks!

[DocMAX]

Same issue,

systemctl --user enable syncthing.service
loginctl enable-linger docmax

but the service is not getting started on boot?!