systemd support

[anarcat]

So I've looked at how to run SMD in a more standard way. I originally setup smd-loop and ran that in a terminal, but it seemed annoying to have to start that in each session. I considered using the applet, but I don't really need to have that stuff cluttering my desktop and i don't have a standard GNOME/whatever desktop environment.

So I turned towards systemd: it already runs other stuff in my user session and it's designed to handle processes like these.

My first attempt was to start smd-loop, and that worked fairly well. It's pretty simple, just drop this in ~/.config/systemd/user/smd-loop.service:

[Unit]
Description=syncmaildir loop

[Service]
ExecStart=/usr/bin/smd-loop

[Install]
WantedBy=multi-user.target

... and then enable and start the thing:

systemctl --user enable smd-loop
systemctl --user start smd-loop

The output of smd-loop is visible through systemctl --user status smd-loop and also of course its own usual logfile.

But then I thought: what does smd-loop do really... I don't really need it! It can be reimplemented with systemd services and timers! So you need this service for pull, in ~/.config/systemd/user/smd-pull.service:

[Unit]
Description=pull emails with syncmaildir
Wants=network.target

[Service]
Type=oneshot
ExecStart=/usr/bin/smd-pull --show-tags

[Install]
WantedBy=multi-user.target

And you need a timer file as well, in ~/.config/systemd/user/smd-pull.timer:

[Unit]
Description=run smd-pull regularly

[Timer]
OnUnitActiveSec=2min

[Install]
WantedBy=timers.target

The above sets up a pull every 2 minutes. Configure to your wishes. Then you enable the whole thing:

systemctl --user enable smd-pull.service
systemctl --user enable smd-pull.timer
systemctl --user start smd-pull.timer

To start the "service", you can use:

systemctl --user start smd-pull.service

Obviously, you can still run smd-pull by hand, but the above keeps the log in systemd. status also does the right thing:

$ systemctl status smd-push.service 
● smd-push.service - push emails with syncmaildir
   Loaded: loaded (/home/anarcat/.config/systemd/user/smd-push.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2018-05-29 11:06:24 EDT; 8s ago
  Process: 10415 ExecStart=/usr/bin/smd-push --show-tags (code=exited, status=0/SUCCESS)
 Main PID: 10415 (code=exited, status=0/SUCCESS)

mai 29 11:06:21 curie systemd[2144]: Starting push emails with syncmaildir...
mai 29 11:06:24 curie smd-push[10415]: default: smd-client@smd-server-anarcat: TAGS: stats::new-mails(0), del-mails(0), bytes-received(0), xdelta-received(366)
mai 29 11:06:24 curie systemd[2144]: Started push emails with syncmaildir.

Rinse and repeat for smd-push.

So there you go. I'm not sure how to integrate this directly in SMD, but it sure is neat here. ;) How should I proceed? docs (README)? or the files in misc/? or systemd/?

[anarcat]

one thing that can happen in the multiple service scenario is this:

mai 29 12:20:07 curie systemd[2144]: Starting push emails with syncmaildir... 
mai 29 12:20:07 curie systemd[2144]: Starting pull emails with syncmaildir... 
mai 29 12:20:07 curie smd-pull[11026]: 11025 ?        00:00:00 smd-push 
mai 29 12:20:07 curie smd-pull[11026]: Already running. 
mai 29 12:20:07 curie smd-pull[11026]: If this is not the case, remove /home/anarcat/.smd/lock by hand. 
mai 29 12:20:07 curie smd-pull[11026]: any: smd-pushpull@localhost: TAGS: error::context(locking) probable-cause(another-instance-is-running) human-intervention(necessary) suggested-actions(run(kill 11025) run(rm /home/anarcat/.smd/lock)) 
mai 29 12:20:07 curie systemd[2144]: smd-pull.service: Main process exited, code=exited, status=1/FAILURE 
mai 29 12:20:07 curie systemd[2144]: Failed to start pull emails with syncmaildir. 
mai 29 12:20:07 curie systemd[2144]: smd-pull.service: Unit entered failed state. 
mai 29 12:20:07 curie systemd[2144]: smd-pull.service: Failed with result 'exit-code'. 

but then it will try again later... maybe there are some tweaks to work around that issue?

[anarcat]

testing the following tweak:

--- i/.config/systemd/user/smd-push.service
+++ w/.config/systemd/user/smd-push.service
@@ -1,6 +1,7 @@
 [Unit]
 Description=push emails with syncmaildir
 Wants=network.target
+After=smd-pull.service
 
 [Service]
 Type=oneshot

[gares]

Thanks!

systemd is pretty standard these days, so it should go to the README, IMO.
smd-loop is used by smd-applet (that reads its output, especially the tags), so, unless systemd can replace it in this use case too, its doc should stay.

BTW smd-loop (or maybe smd-applet, I don't remember now) does not complain in case a network error is detected once (twice in a row). It helps when the network is unreliable. What does systemd in these case? retry? stop?

[anarcat]

network is a bit i need to work on. normally, systemd should "know" when the network is "up" or "down", but i haven't looked at how it actually works in practice. my desktop environment is a bit of a chimera made of bits and parts of multiple things, so it's probably not standard at all.. normally, i guess there would eb a network target, which I depend on in Wants=network.target, but from what i can tell it doesn't actually work.

so i need to test that bit.. my workstation is always online, so i haven't had that scenario yet, but i'll be testing on the laptop soon and report back here.

in any case, i'll work on a quick howto for the readme when i figured out the whole universe. ;)

[anarcat]

so just for the record... i just stumbled upon a case where i ran smd-pull by hand and the systemd timer fired at the same time. the result is:

mai 30 07:05:41 angela systemd[2233]: Starting pull emails with syncmaildir... 
mai 30 07:05:41 angela smd-pull[12297]: 12129 ?        00:00:00 smd-pull 
mai 30 07:05:41 angela smd-pull[12297]: Already running. 
mai 30 07:05:41 angela smd-pull[12297]: If this is not the case, remove /home/anarcat/.smd/lock by hand. 
mai 30 07:05:41 angela smd-pull[12297]: any: smd-pushpull@localhost: TAGS: error::context(locking) probable-cause(another-instance-is-running) human-intervention(necessary) suggested-actions(run(kill 12129) run(rm /home/anarcat/.smd/lock)) 
mai 30 07:05:41 angela systemd[2233]: smd-pull.service: Main process exited, code=exited, status=1/FAILURE 
mai 30 07:05:41 angela systemd[2233]: Failed to start pull emails with syncmaildir. 
mai 30 07:05:41 angela systemd[2233]: smd-pull.service: Unit entered failed state. 
mai 30 07:05:41 angela systemd[2233]: smd-pull.service: Failed with result 'exit-code'. 

then i felt "oh noes, it won't restart again because systemd will think it's dead, dang it!" but by default, systemd retries, and it did just that cleanly a faithful 2 minutes later:

mai 30 07:07:50 angela systemd[2233]: Starting pull emails with syncmaildir... 
mai 30 07:07:50 angela systemd[1]: Started PC/SC Smart Card Daemon. 
mai 30 07:07:50 angela gpg-agent[3278]: scdaemon[3280] pcsc_list_readers failed: unknown PC/SC error code (0x8010002e) 
mai 30 07:07:50 angela gpg-agent[3278]: no authentication key for ssh on card: No such device 
mai 30 07:07:57 angela smd-pull[13317]: default: smd-client@localhost: TAGS: stats::new-mails(0), del-mails(0), bytes-received(0), xdelta-received(217) 
mai 30 07:07:57 angela systemd[2233]: Started pull emails with syncmaildir. 

... well, more or less two minutes. there are probably ways to tweak retry times and so on - i haven't looked. and i haven't looked at network issues yet. here ssh and ntpd and things get reloaded when wifi is cycled on suspend/resume, but i am not sure those are systemd hooks as much as NetworkManager calling systemd directly.

so still investigating, but so far so good: things generally work.

[anarcat]

after a chat with people in #systemd and #debian-systemd (thanks folks!) I was able to confirm that network.target is not only not the right target, but it's also not available at all in the --user session. the right target is network-online.target, and the right way to depend on it is with After=network-online.target... but this is only relevant if smd would be ran as a --system service.

now as for actually hooking in "network up" events, that is network-manager-dependent according to my research. that means basically writing a DBUS daemon, which is far from worth it, in my opinion. :)

[anarcat]

Update: here's my current configuration.

==> .config/systemd/user/smd-pull.service <==
[Unit]
Description=pull emails with syncmaildir
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
# --show-tags gives email counts
#
# --verbose shows the actual mails changed and was after a catastrophic
# failure where 150k emails were deleted
ExecStart=/usr/bin/smd-pull --verbose --show-tags

[Install]
WantedBy=multi-user.target

==> .config/systemd/user/smd-pull.timer <==
[Unit]
Description=run smd-pull regularly

[Timer]
OnCalendar=*:0/2

[Install]
WantedBy=timers.target

==> .config/systemd/user/smd-push.service <==
[Unit]
Description=push emails with syncmaildir
Wants=network-online.target
After=smd-pull.service

[Service]
Type=oneshot
# --show-tags gives email counts
#
# --verbose shows the actual mails changed and was after a catastrophic
# failure where 150k emails were deleted
ExecStart=/usr/bin/smd-push --verbose --show-tags

[Install]
WantedBy=multi-user.target

==> .config/systemd/user/smd-push.timer <==
[Unit]
Description=run smd-push regularly

[Timer]
OnCalendar=*:0/2

[Install]
WantedBy=timers.target

I am using OnCalendar because timers won't fire automatically with OnUnitActiveSec. I don't remember why exactly - it was strange.

I've also created the /var/log/journal directory and restarted systemd-journald to make sure logs are kept by systemd.

[gares]

Could you please make a PR? I'm happy to merge it.
I'd like to make a release "soon".

[anarcat]

what should the PR look like? patch to the README?

[gares]

I think you could put a .service file somewhere, maybe in misc/, and mention its existence in the README in the installation section.

[anarcat]

alright done, see #12. i added it in the "howto" section, as it seemed to fit more naturally there, but maybe I'm mistaken?