Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undocumented systemd magic breaks certain renewal workflows #5398

Closed
csdev opened this issue Jan 9, 2018 · 4 comments

Comments

Projects
None yet
4 participants
@csdev
Copy link

commented Jan 9, 2018

On Ubuntu 16.04+, certbot automatically runs a renewal command twice a day via a systemd timer. This magic feature is totally undocumented.

Furthermore, the documentation on cert renewal (https://certbot.eff.org/docs/using.html#renewing-certificates) suggests that the user should set up their own renewal command, and place it in a crontab. The user's renewal command may include custom options, such as renewal/deployment hooks. However, the default renewal command in the systemd service may preempt the user's renewal command, renew the certificate, and completely bypass the user's custom renewal/deployment hooks.

Rough example of how to reproduce:
Register and request the initial cert, using the procedure at https://certbot.eff.org/docs/using.html#manual

certbot register --non-interactive --agree-tos --email 'admin@email.com.example' --no-eff-email

certbot certonly --manual --non-interactive --manual-public-ip-logging-ok \
    --domains "domain.com.example" \
    --preferred-challenges dns \
    --manual-auth-hook /opt/my_certbot_hooks/auth_hook.sh \
    --manual-cleanup-hook /opt/my_certbot_hooks/cleanup_hook.sh

Deploy the cert (deployment commands do not matter). Then, make a crontab that runs the following command once a week:

certbot renew --non-interactive \
    --renew-hook "/opt/my_certbot_hooks/renew_hook.sh"

When the cert nears expiration, it is likely that the systemd service will run first and renew it. As a result, our renewal command will be a no-op and renew_hook.sh will never be called.

@ArchangeGabriel

This comment has been minimized.

Copy link

commented Jan 9, 2018

  1. Note that --renew-hook does not exist anymore, they are now pre, post and deploy hooks.
  2. This is an issue with Ubuntu package at the very most, not with Certbot itself AFAICT. So report this against Ubuntu (e.g. on Launchpad) or to the package maintainer.
  3. From the doc (but maybe your certbot version is too old for that to work, especially I don’t remember if that was available for renew-hook):

You can also specify hooks by placing files in subdirectories of Certbot’s configuration directory. Assuming your configuration directory is /etc/letsencrypt, any executable files found in /etc/letsencrypt/renewal-hooks/pre, /etc/letsencrypt/renewal-hooks/deploy, and /etc/letsencrypt/renewal-hooks/post will be run as pre, deploy, and post hooks respectively when any certificate is renewed with the renew subcommand.

  1. You can deactivate the provided systemd timer to work around the problem, but I suspect you’re going to have to do that again upon each upgrade of certbot. Way better: you can override the systemd service that is run, see below and here.

First, you need to find the timer, for instance /etc/systemd/system/timers.target.wants/certbot.timer on my system.

This timer trigger the execution of the renewal service, that has the same name, and is likely available at something like /usr/lib/systemd/system/certbot.service (just check to be sure). There you can see what exactly the service does. And you can add overrides in a folder using the service name like this /etc/systemd/system/certbot.service.d/. In your case, that would be:
00-renew-hook.conf

[Service]
ExecStart=
ExecStart=/usr/bin/certbot renew --non-interactive --renew-hook "/opt/my_certbot_hooks/renew_hook.sh"

The first empty ExecStart= line is for removing the one in the system unit.

@SwartzCr

This comment has been minimized.

Copy link
Contributor

commented Jan 17, 2018

@csdev I think this is a reasonable complaint, and echoed in #5034 (comment)
On Debian (and thus Ubuntu) we install both a cron job and systemd timer, which is not reflected in our documentation, and probably should be.
@hlieberman would you be willing to work with me to document this?

@SwartzCr

This comment has been minimized.

Copy link
Contributor

commented Jan 24, 2018

There is now a PR for this #5460 and a corresponding issue for the website here, hopefully I'll have a PR done by later today, and have the website updated this week, certbot/website#305

@joohoi

This comment has been minimized.

Copy link
Member

commented Mar 29, 2019

This issue hasn't been updated for quite a long time, and things have changed quite a bit since. I'm closing this issue as deprecated, but feel free to comment if this is not the case, and we can reopen the issue.

@joohoi joohoi closed this Mar 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.