Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use systemd.timer instead of a cron job #10

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mvo5
Copy link
Owner

@mvo5 mvo5 commented Mar 17, 2016

The rational is that we need to spread the load on the mirrors
that apt update and unattended-upgrades cause. To do so, we
leverage the RandomizeDelay feature of systemd. The other advantage
is that the timer is not run at a fixed daily.daily time but
instead every 24h. This also fixes the problem that the randomized
deplay in the current apt.cron.daily causes other cron jobs to
be deplayed.

@mvo5
Copy link
Owner Author

mvo5 commented Mar 17, 2016

Yay, thanks for the excellent feedback Martin!

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch 2 times, most recently from 05a26ef to 12e42c0 Compare March 17, 2016 11:00

dh_bugfiles -p$@
dh_lintian -p$@
dh_installexamples -p$@ $(BLD)/docs/examples/*
dh_installman -p$@ $(wildcard $(patsubst %,doc/en/%.[158],$(apt_MANPAGES)) $(patsubst %,doc/*/%.*.[158],$(apt_MANPAGES)))
dh_installcron -p$@
dh_systemd_enable -p$@ apt-daily.timer apt-daily.service
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a build-depends on dh-systemd

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch 10 times, most recently from e0a2e90 to 17a8faa Compare March 17, 2016 20:36
# same time. The systemd time is better at this than the fixed
# cron.daily time
if [ -d /run/systemd/system ]; then
echo exit 0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spurious echo?

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch 2 times, most recently from 1b2393c to 3b0db0c Compare March 18, 2016 07:10
@mvo5
Copy link
Owner Author

mvo5 commented Mar 18, 2016

Fwiw, this is not the final version yet unfortunately. The issue is that the OnUnit{,In}ActiveSec stops when the systemd is suspended. Which breaks our "run the script at least every 24h". We could use an OnCalendar event, however it appears that systemd has no anacron like functionatliy. So if we do a OnCalendar daily at 6:00 with randomized delays of e.g. 2h then all systems starting at 8:01 would not run the timer job. What we really need is a OnUnitActiveSec that keeps increasing the time on suspend. Alternatively we could run it much more frequently (the apt.systemd.daily script will ensure that actual network activity only happens when at least 24h are over), something like OnInActiveSec=2h RandomDelay=2h or somesuch. Still not ideal to wake the script every 2h+-rand(2)h :/

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch from 3b0db0c to cd3b7a7 Compare March 18, 2016 10:44
@fsateler
Copy link

I think the Persistent flag is for such anacron-like functionality.
On 18 Mar 2016 04:16, "Michael Vogt" notifications@github.com wrote:

Fwiw, this is not the final version yet unfortunately. The issue is that
the OnUnit{,In}ActiveSec stops when the systemd is suspended. Which breaks
our "run the script at least every 24h". We could use an OnCalendar event,
however it appears that systemd has no anacron like functionatliy. So if we
do a OnCalendar daily at 6:00 with randomized delays of e.g. 2h then all
systems starting at 8:01 would not run the timer job. What we really need
is a OnUnitActiveSec that keeps increasing the time on suspend.
Alternatively we could run it much more frequently (the apt.systemd.daily
script will ensure that actual network activity only happens when at least
24h are over), something like OnInActiveSec=2h RandomDelay=2h or somesuch.
Still not ideal to wake the script every 2h+-rand(2)h :/


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#10 (comment)

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch 4 times, most recently from 1c40ac5 to 26bcc12 Compare March 18, 2016 11:02
@mvo5
Copy link
Owner Author

mvo5 commented Mar 18, 2016

Thanks @fsateler ! I think that does indeed do the trick :)

@paulgear
Copy link

@mvo5 Am I reading the patch correctly that there are two randomisations involved? The systemd configuration to run twice daily at a random hour between 04:00 and 16:00 and then another 30-minute random delay within the script itself?

It seems to me that by running twice daily we will be essentially doubling the load on the mirrors [1]. It should be nice & even thanks to the hour randomisation, but this is still a net increase in mirror traffic, which was certainly not the goal of https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1549143

I'd really like you to consider leaving this daily by default rather than twice daily.

[1] Possibly not quite double because there won't be updates every time this runs, but it will still hit and most times download the indexes (because the main archives update on an as-needed basis rather than a fixed schedule).

@julian-klode
Copy link
Contributor

You could even let it run every hour, as the job has it's own timestamp files which it checks and only acts if they are old enough.

@paulgear
Copy link

paulgear commented Apr 1, 2016

@julian-klode & @mvo5 - Thanks for the explanations about the timestamp files; I think this will be a big improvement.

@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch from 26bcc12 to 758191e Compare April 1, 2016 10:24
The rational is that we need to spread the load on the mirrors
that apt update and unattended-upgrades cause. To do so, we
leverage the RandomizeDelay feature of systemd. The other advantage
is that the timer is not run at a fixed daily.daily time but
instead every 24h. This also fixes the problem that the randomized
deplay in the current apt.cron.daily causes other cron jobs to
be deplayed.

A compatibility cron job is also provided for systems that do not
use systemd.

Note that the time is fired two times a day, but the logic inside
of apt.systemd.daily will ensure (via stamp files) that the
servers are hit at most every 24h. Firing two times a day helps
with the worst case update time and it also helps with systems
that are not always on.

LP: #246381, #727685
Closes: #600262, #709675, #663290
@mvo5 mvo5 force-pushed the feature/use-systemd-timers branch from 758191e to 61de1c3 Compare April 1, 2016 10:36
@gkope
Copy link

gkope commented Apr 19, 2016

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants