New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confconsole - Letsencrypt support #766

Closed
OnGle opened this Issue Feb 1, 2017 · 63 comments

Comments

Projects
None yet
4 participants
@OnGle
Member

OnGle commented Feb 1, 2017

root@lamp ~# apt-cache policy confconsole
confconsole:
  Installed: 0.9.4+94+gc2196a0
  Candidate: 0.9.4+94+gc2196a0
  Version table:
 *** 0.9.4+94+gc2196a0 0
        900 http://tkl-dev-test.s3-website-us-east-1.amazonaws.com/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
     0.9.4+54+g99603c7 0
        850 http://archive.turnkeylinux.org/debian/ jessie/main amd64 Packages

The letsencrypt plugin for confconsole requires /usr/local/bin/letsencrypt.sh which currently causes an error due to it not being present.

@JedMeister JedMeister added this to the 14.2 milestone Feb 1, 2017

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 1, 2017

Member

related to #546 & #369

Member

JedMeister commented Feb 1, 2017

related to #546 & #369

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 1, 2017

Member

@qrntz - where does the /usr/local/bin/letsencrypt.sh come from? Is it included in another repo somewhere or something? I vaguely recall you saying that you were using a thrid party acme/LE client from GH, but I don't recall what it was called.

Member

JedMeister commented Feb 1, 2017

@qrntz - where does the /usr/local/bin/letsencrypt.sh come from? Is it included in another repo somewhere or something? I vaguely recall you saying that you were using a thrid party acme/LE client from GH, but I don't recall what it was called.

@JedMeister JedMeister changed the title from Confconsole letsencrypt no file found to Confconsole letsencrypt not file found Feb 2, 2017

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 2, 2017

Member

@qrntz - After having a dig around, I found one explicitly called letsencrypt.sh. However there is another (that is marginally older - by 2 days; 5th Dec 2015 vs 7th Dec 2015) called dehydrated. It was originally known as "letsencrypt.sh".

The latter appear to be a more vibrant and active project and also appears to be more full featured. If it's of no substantial consequence, I'm inclined to use the latter. But if you have already implemented and tested the former, then we may as well run with that... Thoughts?

Member

JedMeister commented Feb 2, 2017

@qrntz - After having a dig around, I found one explicitly called letsencrypt.sh. However there is another (that is marginally older - by 2 days; 5th Dec 2015 vs 7th Dec 2015) called dehydrated. It was originally known as "letsencrypt.sh".

The latter appear to be a more vibrant and active project and also appears to be more full featured. If it's of no substantial consequence, I'm inclined to use the latter. But if you have already implemented and tested the former, then we may as well run with that... Thoughts?

@qrntz

This comment has been minimized.

Show comment
Hide comment
@qrntz

qrntz Feb 2, 2017

Member

@JedMeister, "dehydrated" is the proper one. It used to be called "letsencrypt.sh" when I wrote the plugin. The other one was not mentioned in any comparisons I have researched at the time so I guess it was already abandoned (or virtually unknown) back then.

Member

qrntz commented Feb 2, 2017

@JedMeister, "dehydrated" is the proper one. It used to be called "letsencrypt.sh" when I wrote the plugin. The other one was not mentioned in any comparisons I have researched at the time so I guess it was already abandoned (or virtually unknown) back then.

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 2, 2017

Member

@qrntz - Awesome thanks mate. 👍

FWIW its in backports. Nice! 😄

Member

JedMeister commented Feb 2, 2017

@qrntz - Awesome thanks mate. 👍

FWIW its in backports. Nice! 😄

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 3, 2017

Member

In discussions with @OnGle - we're thinking that we should make 'dehydrated' a "Recommends" in the Confconsole package.

To assist users who upgrade Confconsole on earlier versions of TurnKey, Confconsole should give a meaningful error (pointing to a doc page on the TurnKey site, e.g. www.turnkeylinux.org/docs/lets-encrypt) if the binary is not available. Once we rebase on Stretch, then we can make "dehydrated" a hard "Dependency" (assuming that it makes it into Stretch main).

Member

JedMeister commented Feb 3, 2017

In discussions with @OnGle - we're thinking that we should make 'dehydrated' a "Recommends" in the Confconsole package.

To assist users who upgrade Confconsole on earlier versions of TurnKey, Confconsole should give a meaningful error (pointing to a doc page on the TurnKey site, e.g. www.turnkeylinux.org/docs/lets-encrypt) if the binary is not available. Once we rebase on Stretch, then we can make "dehydrated" a hard "Dependency" (assuming that it makes it into Stretch main).

@OnGle

This comment has been minimized.

Show comment
Hide comment
@OnGle

OnGle Feb 3, 2017

Member

These 2 commits should replace the error with a cleaner message including a link to the not-yet-created docs page JedMeister mentions above.

OnGle/confconsole@0396812
OnGle/confconsole@f610e0f

Member

OnGle commented Feb 3, 2017

These 2 commits should replace the error with a cleaner message including a link to the not-yet-created docs page JedMeister mentions above.

OnGle/confconsole@0396812
OnGle/confconsole@f610e0f

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 3, 2017

Member

This issue should be (sort of) resolved in the latest dev-test package (confconsole_0.9.4+101+gf610e0f_all.deb). If it can't find dehydrated it should give a meaningful error that points to (currently non-existent) TKL doc page (explaining that dehydrated needs to be installed).

We'll ship v14.2 with dehydrated pre-installed (from Jessie backports) but don't want to (re)package it ourselves. For the time being I was hoping to uploaded the jessie-backports package of dehydrated to the dev-test repo to make installing it super easy, but for some reason it keeps failing.

So if you want to actually test Let's Encrypt, you will need to (configure Jessie backports](https://backports.debian.org/Instructions/) and install from there.

Member

JedMeister commented Feb 3, 2017

This issue should be (sort of) resolved in the latest dev-test package (confconsole_0.9.4+101+gf610e0f_all.deb). If it can't find dehydrated it should give a meaningful error that points to (currently non-existent) TKL doc page (explaining that dehydrated needs to be installed).

We'll ship v14.2 with dehydrated pre-installed (from Jessie backports) but don't want to (re)package it ourselves. For the time being I was hoping to uploaded the jessie-backports package of dehydrated to the dev-test repo to make installing it super easy, but for some reason it keeps failing.

So if you want to actually test Let's Encrypt, you will need to (configure Jessie backports](https://backports.debian.org/Instructions/) and install from there.

@JedMeister JedMeister referenced this issue Feb 3, 2017

Merged

Bugfix spree #8

@JedMeister JedMeister added the feature label Feb 24, 2017

@JedMeister JedMeister self-assigned this Feb 24, 2017

@JedMeister JedMeister changed the title from Confconsole letsencrypt not file found to Confconsole - Letsencrypt WIP Feb 24, 2017

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 24, 2017

Member

Ok, after another week... We have something that I think (read really hope) is pretty much good to go... Testing is welcome (actually it's encouraged!).

The latest package confconsole_0.9.4+122+ga395036_all.deb is available from my dev-test repo.

Dependencies: authbind, python-bottle, dehydrated (dehydrated needs to be installed from jessie-backports)

Alternatively, to avoid configuring repos etc, you could just download them with wget and install with dpkg:

apt-get update
apt-get install authbind python-bottle
wget http://ftp.us.debian.org/debian/pool/main/d/dehydrated/dehydrated_0.3.1-3~bpo8+1_all.deb
dpkg -i dehydrated_0.3.1-3~bpo8+1_all.deb
wget https://s3.amazonaws.com/tkl-dev-test/pool/c/co/confconsole_0.9.4%2B122%2Bga395036_all.deb
dpkg -i confconsole_0.9.4+122+ga395036_all.deb

Please note, that by default this will use the production Let's Encrypt servers to gain a certificate. They have a cert renewal limit of 20 per week. So for testing purposes I highly recommend that you configure your server to use the Let's Encrypt staging server. The easiest way to do that, is to copy across the staging dehydrated config file from /usr/share/confconsole/letsencrypt:

cp /usr/share/confconsole/letsencrypt/dehydrated-staging-confconsole.config /etc/dehydrated/confconsole.config

By default the dehydrated-wrapper will use /etc/dehydrated/confconsole.config. If that doesn't exist it will copy across /usr/share/confconsole/letsencrypt/dehydrated-confconsole.config. So to go back to using the production Let's Encrypt server, simply delete the config file and on next run, the dehydrated-wrapper will use the confconsole default config (which uses the production server).

I haven't yet done significant testing, but have done enough to confirm that all the components work fairly reliably individually. I haven't fully tested everything all bundled together. I also haven't double checked the functioning of the included cron job.

FWIW the current state of the confconsole code can be found here: https://github.com/JedMeister/confconsole/tree/letsencrypt-addwater

Member

JedMeister commented Feb 24, 2017

Ok, after another week... We have something that I think (read really hope) is pretty much good to go... Testing is welcome (actually it's encouraged!).

The latest package confconsole_0.9.4+122+ga395036_all.deb is available from my dev-test repo.

Dependencies: authbind, python-bottle, dehydrated (dehydrated needs to be installed from jessie-backports)

Alternatively, to avoid configuring repos etc, you could just download them with wget and install with dpkg:

apt-get update
apt-get install authbind python-bottle
wget http://ftp.us.debian.org/debian/pool/main/d/dehydrated/dehydrated_0.3.1-3~bpo8+1_all.deb
dpkg -i dehydrated_0.3.1-3~bpo8+1_all.deb
wget https://s3.amazonaws.com/tkl-dev-test/pool/c/co/confconsole_0.9.4%2B122%2Bga395036_all.deb
dpkg -i confconsole_0.9.4+122+ga395036_all.deb

Please note, that by default this will use the production Let's Encrypt servers to gain a certificate. They have a cert renewal limit of 20 per week. So for testing purposes I highly recommend that you configure your server to use the Let's Encrypt staging server. The easiest way to do that, is to copy across the staging dehydrated config file from /usr/share/confconsole/letsencrypt:

cp /usr/share/confconsole/letsencrypt/dehydrated-staging-confconsole.config /etc/dehydrated/confconsole.config

By default the dehydrated-wrapper will use /etc/dehydrated/confconsole.config. If that doesn't exist it will copy across /usr/share/confconsole/letsencrypt/dehydrated-confconsole.config. So to go back to using the production Let's Encrypt server, simply delete the config file and on next run, the dehydrated-wrapper will use the confconsole default config (which uses the production server).

I haven't yet done significant testing, but have done enough to confirm that all the components work fairly reliably individually. I haven't fully tested everything all bundled together. I also haven't double checked the functioning of the included cron job.

FWIW the current state of the confconsole code can be found here: https://github.com/JedMeister/confconsole/tree/letsencrypt-addwater

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 24, 2017

Member

^^^ @DocCyblade ^^^

If you get a chance to see how it works for you (and see if you can break it) within the next few days that would be absolutely incredible! No pressure, but it'd be super awesome if you have time! 😄

Member

JedMeister commented Feb 24, 2017

^^^ @DocCyblade ^^^

If you get a chance to see how it works for you (and see if you can break it) within the next few days that would be absolutely incredible! No pressure, but it'd be super awesome if you have time! 😄

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Feb 24, 2017

Member

I'll have some time this weekend! If it can break I'll break it!

I'll be testing it mostly on a hub server, but if I can I'll test it on a server inside my network.

Member

DocCyblade commented Feb 24, 2017

I'll have some time this weekend! If it can break I'll break it!

I'll be testing it mostly on a hub server, but if I can I'll test it on a server inside my network.

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 27, 2017

Member

Oops, I just realised that there is a missing dependency. I'm still having issues getting it working myself but that was a show stopper! I've updated my previous post to include the new dependency (python-bottle).

Member

JedMeister commented Feb 27, 2017

Oops, I just realised that there is a missing dependency. I'm still having issues getting it working myself but that was a show stopper! I've updated my previous post to include the new dependency (python-bottle).

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Feb 27, 2017

Member

Hmm, still not working for me on a clean install... :( I wonder what is going wrong?! It was wokring fine during my previous testing. There must be something I tweaked on my old server which I forgot to note.

I discovered my previous issue, but actually, there are still some other bugs. And also I want to tidy up the logging a little. Currently, it writes to 4 different log files. Whilst that's ok for debugging, it's not really very nice for users. I'm also wondering if it might actually be worth reducing the volume of logging (at least by default). Anyway, that's a bit of an aside for now...

Apologies on the (another) false start Ken. I hope you didn't spend too much time on it over the weekend. I'll fix it and put up a new version ASAP. I'll update my post above and add a new post here when I'm done.

Member

JedMeister commented Feb 27, 2017

Hmm, still not working for me on a clean install... :( I wonder what is going wrong?! It was wokring fine during my previous testing. There must be something I tweaked on my old server which I forgot to note.

I discovered my previous issue, but actually, there are still some other bugs. And also I want to tidy up the logging a little. Currently, it writes to 4 different log files. Whilst that's ok for debugging, it's not really very nice for users. I'm also wondering if it might actually be worth reducing the volume of logging (at least by default). Anyway, that's a bit of an aside for now...

Apologies on the (another) false start Ken. I hope you didn't spend too much time on it over the weekend. I'll fix it and put up a new version ASAP. I'll update my post above and add a new post here when I'm done.

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Feb 27, 2017

Member

Well good news was I did get to it, very busy weekend. I have a server ready however to test it, just let me know

Member

DocCyblade commented Feb 27, 2017

Well good news was I did get to it, very busy weekend. I have a server ready however to test it, just let me know

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 1, 2017

Member

Ok, finally have an updated version of confconsole which should now include a working instance of the Let's Encrypt plugin. Here's the proof!:

screenshot from 2017-03-01 16 25 03
screenshot from 2017-03-01 16 25 06

As noted previously, you will need to install dehydrated from jessie-backports. The other dependencies are python-bottle and authbind. Both of these can be installed from Jessie's main repo. To update my "down and dirty" copy/pastable install instructions:

apt-get update && apt-get install python-bottle authbind 
wget http://ftp.us.debian.org/debian/pool/main/d/dehydrated/dehydrated_0.3.1-3~bpo8+1_all.deb
dpkg -i dehydrated_0.3.1-3~bpo8+1_all.debwget https://s3.amazonaws.com/tkl-dev-test/pool/c/co/confconsole_0.9.4%2B124%2Bga6ebfe6_all.deb
dpkg -i confconsole_0.9.4+124+ga6ebfe6_all.deb

If you wish to test with the Let's Encrypt staging server (recommended as an initial first step for now):

cp /usr/share/confconsole/letsencrypt/dehydrated-staging-confconsole.config /etc/dehydrated/confconsole.config

After you have tested and are happy, and wish to get a proper Let's Encrypt cert, you'll need to force it (by default Dehydrated won't update certificates that are valid for the next 30 days). The easiest way to do that is to adjust the dehydrated-wrapper to add the --force switch when it calls dehydrated. Here's a sed command to do just that:

sed -i "/dehydrated/ s|--cron --config|--cron --force --config|" /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper

When you have your cert, I encourage you to revert back to the default (i.e. no --force). It probably won't matter as the default cron job (you can enable/disable it within confconsole too) checks the expiry date of the cert before it even runs dehydrated, but it's a good idea IMO:

sed -i "/dehydrated/ s|--cron --force --config|--cron --config|" /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper
Member

JedMeister commented Mar 1, 2017

Ok, finally have an updated version of confconsole which should now include a working instance of the Let's Encrypt plugin. Here's the proof!:

screenshot from 2017-03-01 16 25 03
screenshot from 2017-03-01 16 25 06

As noted previously, you will need to install dehydrated from jessie-backports. The other dependencies are python-bottle and authbind. Both of these can be installed from Jessie's main repo. To update my "down and dirty" copy/pastable install instructions:

apt-get update && apt-get install python-bottle authbind 
wget http://ftp.us.debian.org/debian/pool/main/d/dehydrated/dehydrated_0.3.1-3~bpo8+1_all.deb
dpkg -i dehydrated_0.3.1-3~bpo8+1_all.debwget https://s3.amazonaws.com/tkl-dev-test/pool/c/co/confconsole_0.9.4%2B124%2Bga6ebfe6_all.deb
dpkg -i confconsole_0.9.4+124+ga6ebfe6_all.deb

If you wish to test with the Let's Encrypt staging server (recommended as an initial first step for now):

cp /usr/share/confconsole/letsencrypt/dehydrated-staging-confconsole.config /etc/dehydrated/confconsole.config

After you have tested and are happy, and wish to get a proper Let's Encrypt cert, you'll need to force it (by default Dehydrated won't update certificates that are valid for the next 30 days). The easiest way to do that is to adjust the dehydrated-wrapper to add the --force switch when it calls dehydrated. Here's a sed command to do just that:

sed -i "/dehydrated/ s|--cron --config|--cron --force --config|" /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper

When you have your cert, I encourage you to revert back to the default (i.e. no --force). It probably won't matter as the default cron job (you can enable/disable it within confconsole too) checks the expiry date of the cert before it even runs dehydrated, but it's a good idea IMO:

sed -i "/dehydrated/ s|--cron --force --config|--cron --config|" /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper
@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 1, 2017

Member

Sounds good, I guess to proper test it we will need to wait to make sure it gets updated in 30 Days? I'll do a quick build on a hub server and then one that I a hosting in house and we shall see. :-)

If it will break I'll break it!

Member

DocCyblade commented Mar 1, 2017

Sounds good, I guess to proper test it we will need to wait to make sure it gets updated in 30 Days? I'll do a quick build on a hub server and then one that I a hosting in house and we shall see. :-)

If it will break I'll break it!

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 1, 2017

Member

Cool mate, please let me know how you go. 👍

Unfortunately the time frame isn't quite as simple as that. The certificate is valid for 90 days from issue and dehydrated won't update it (other than via --force) unless it expires within the next 30 days. So to wait for it to "naturally" occur will take ~60 days... I guess we could temporarily leave the --force switch enabled and tweak the cron job?

Anyway, the way I've implemented it, every time dehydrated checks, it brings down the webserver, even if it doesn't need to do anything. So rather than check more often than we need to; the cron job I've implemented does it a little differently. It's a weekly cron job which tests the certificate expiry date (using openssl). Only if the certificate expires within the next 30 days, will it launch our dehydrated-wrapper script. So in theory it should only launch dehydrated if the certificate can be renewed (without --force). The cron job runs weekly though so it will attempt to renew at the earliest possible time (within the 30 days you have to renew).

Anyway, your suggestion has made me think more about testing the cron job. What we could do is create a self-signed cert which expires really soon, and reconfigure the cron job to run daily. Then see if it gets a new (non self-signed) cert. I'll aim to set that up tomorrow.

Also, I've only tested it with LAMP based apps. In theory it should work with all apps, Nginx, Lightty and even Tomcat! Also Core too! 😄 But I still need to test...!

There was something else I was going to mention, but I've forgotten now...

Member

JedMeister commented Mar 1, 2017

Cool mate, please let me know how you go. 👍

Unfortunately the time frame isn't quite as simple as that. The certificate is valid for 90 days from issue and dehydrated won't update it (other than via --force) unless it expires within the next 30 days. So to wait for it to "naturally" occur will take ~60 days... I guess we could temporarily leave the --force switch enabled and tweak the cron job?

Anyway, the way I've implemented it, every time dehydrated checks, it brings down the webserver, even if it doesn't need to do anything. So rather than check more often than we need to; the cron job I've implemented does it a little differently. It's a weekly cron job which tests the certificate expiry date (using openssl). Only if the certificate expires within the next 30 days, will it launch our dehydrated-wrapper script. So in theory it should only launch dehydrated if the certificate can be renewed (without --force). The cron job runs weekly though so it will attempt to renew at the earliest possible time (within the 30 days you have to renew).

Anyway, your suggestion has made me think more about testing the cron job. What we could do is create a self-signed cert which expires really soon, and reconfigure the cron job to run daily. Then see if it gets a new (non self-signed) cert. I'll aim to set that up tomorrow.

Also, I've only tested it with LAMP based apps. In theory it should work with all apps, Nginx, Lightty and even Tomcat! Also Core too! 😄 But I still need to test...!

There was something else I was going to mention, but I've forgotten now...

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 3, 2017

Member

FWIW, during our brief chat yesterday, I was prompted to double check the file permissions of all the files that dehydrated (and the wrapper and hook script we provide) use. And they all look good (i.e. only accessible by root, except for the ACME challenges, which need to be accessible by www-data):

root@lamp ~# ls -lA /etc/ssl/private/
total 20
-r-------- 1 root root     3243 Mar  1 04:31 cert.key
-r-------- 1 root root     7094 Mar  1 04:31 cert.pem
-r-------- 1 root root      245 Feb 27 01:07 dhparams.pem
-rw-r----- 1 root ssl-cert 1704 Apr  8  2016 ssl-cert-snakeoil.key
root@lamp ~# ls -lA /var/lib/dehydrated/
total 12
drwx------ 4 root root 4096 Feb 27 03:51 accounts
drwxr-xr-x 2 root root 4096 Mar  1 04:31 acme-challenges
drwx------ 3 root root 4096 Feb 27 03:37 certs
Member

JedMeister commented Mar 3, 2017

FWIW, during our brief chat yesterday, I was prompted to double check the file permissions of all the files that dehydrated (and the wrapper and hook script we provide) use. And they all look good (i.e. only accessible by root, except for the ACME challenges, which need to be accessible by www-data):

root@lamp ~# ls -lA /etc/ssl/private/
total 20
-r-------- 1 root root     3243 Mar  1 04:31 cert.key
-r-------- 1 root root     7094 Mar  1 04:31 cert.pem
-r-------- 1 root root      245 Feb 27 01:07 dhparams.pem
-rw-r----- 1 root ssl-cert 1704 Apr  8  2016 ssl-cert-snakeoil.key
root@lamp ~# ls -lA /var/lib/dehydrated/
total 12
drwx------ 4 root root 4096 Feb 27 03:51 accounts
drwxr-xr-x 2 root root 4096 Mar  1 04:31 acme-challenges
drwx------ 3 root root 4096 Feb 27 03:37 certs
@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 8, 2017

Member

Ok, I think this is pretty much done now. Looking forward to hearing how your cron job went...

FWIW I have another build of confconsole: confconsole_0.9.4+134+g90c1a74_all.deb

This one includes a minor update to the cron job (added --force as per discussions).

It also has another plugin now too; Mail relay which provides a way to easily configure a third party SMTP relay (for sending emails). I'd love your feedback on that too, although I guess we should discuss that in a separate thread. Probably this one: #482

Member

JedMeister commented Mar 8, 2017

Ok, I think this is pretty much done now. Looking forward to hearing how your cron job went...

FWIW I have another build of confconsole: confconsole_0.9.4+134+g90c1a74_all.deb

This one includes a minor update to the cron job (added --force as per discussions).

It also has another plugin now too; Mail relay which provides a way to easily configure a third party SMTP relay (for sending emails). I'd love your feedback on that too, although I guess we should discuss that in a separate thread. Probably this one: #482

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 8, 2017

Member

I'll update my test servers and let them update over the next 24 hrs

Member

DocCyblade commented Mar 8, 2017

I'll update my test servers and let them update over the next 24 hrs

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 8, 2017

Member

No worries.

Please note though, that if you update confconsole on a server which has an existing dev install it won't update the existing cron job (in /etc/cron.daily). The easiest way to do that (other than purging the package, manually removing all the extra files and re-configuring everything from scratch) would be to just copy the cron job across. I.e.

cp /usr/share/confconsole/letsencrypt/dehydrated-confconsole.cron /etc/cron.daily/confconsole-dehydrated

Don't forget to make it executable! 😄

chmod +x /etc/cron.daily/confconsole-dehydrated

Assuming all your other config is still in place, it should "just work"! 😄

Member

JedMeister commented Mar 8, 2017

No worries.

Please note though, that if you update confconsole on a server which has an existing dev install it won't update the existing cron job (in /etc/cron.daily). The easiest way to do that (other than purging the package, manually removing all the extra files and re-configuring everything from scratch) would be to just copy the cron job across. I.e.

cp /usr/share/confconsole/letsencrypt/dehydrated-confconsole.cron /etc/cron.daily/confconsole-dehydrated

Don't forget to make it executable! 😄

chmod +x /etc/cron.daily/confconsole-dehydrated

Assuming all your other config is still in place, it should "just work"! 😄

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 8, 2017

Member

Thanks for the heads up, I ran your commands to copy it over. We shall see if it works

Member

DocCyblade commented Mar 8, 2017

Thanks for the heads up, I ran your commands to copy it over. We shall see if it works

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 8, 2017

Member

No worries. I look forward to hearing how it goes.

Member

JedMeister commented Mar 8, 2017

No worries. I look forward to hearing how it goes.

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 8, 2017

Member

Another new version of confconsole: confconsole_0.9.4+140+g0b038ce_all.deb

This one includes the email stuff as mentioned previously, but it also has some significant changes to the SLL bits, so worth posting here... The changes made include:

  • fixed support for Tomcat appliance (didn't work previously) (JedMeister/confconsole@cba67ae)
  • fixed a minor bug I had missed previously in the switch handling routine (JedMeister/confconsole@dafc14f)
  • added support for DEBUG mode (global variable; i.e. if export DEBUG=y then inside the script set -x)(JedMeister/confconsole@916b3d5)
  • added some new global variables that the hook script can get (rather than hard code paths in the hook)
  • trap interrupts and clean up prior to exit; should reduce the chances of a server being left in a broken state, even if something bad happens. Basically makes the whole operation much more robust. E.g. try starting the wrapper from the commandline, then do a Ctrl-C while it's in the middle of doing something! 😄

The last 2 (plus some other tidying up) are a single commit (JedMeister/confconsole@05e5919). Ideally they should have probably been split into separate commits, but I had already done the work when I realised I hadn't done a commit between; and I wasn't about to undo it just to make the git history nicer...

Special note: This version changes both the dehydrated-wrapper and the hook script. The wrapper will auto overwrite on package update, but not the hook script. The quick and dirty way to ensure that you have the latest (on a server which you've already been testing confconsole on and have just installed the latest version on) is to manually copy it across:

cp /usr/share/confconsole/letsencrypt/dehydrated-confconsole.hook.sh /etc/dehydrated/confconsole.hook.sh

I have tested the scripts themselves a bit, but probably not enough considering the refactoring I've done this afternoon (all my testing was on a Tomcat appliance). I haven't yet actually tested the package itself. If you get a chance, I'd love to hear of anything that isn't working or is broken... But I'm really hoping it's good to go!

Member

JedMeister commented Mar 8, 2017

Another new version of confconsole: confconsole_0.9.4+140+g0b038ce_all.deb

This one includes the email stuff as mentioned previously, but it also has some significant changes to the SLL bits, so worth posting here... The changes made include:

  • fixed support for Tomcat appliance (didn't work previously) (JedMeister/confconsole@cba67ae)
  • fixed a minor bug I had missed previously in the switch handling routine (JedMeister/confconsole@dafc14f)
  • added support for DEBUG mode (global variable; i.e. if export DEBUG=y then inside the script set -x)(JedMeister/confconsole@916b3d5)
  • added some new global variables that the hook script can get (rather than hard code paths in the hook)
  • trap interrupts and clean up prior to exit; should reduce the chances of a server being left in a broken state, even if something bad happens. Basically makes the whole operation much more robust. E.g. try starting the wrapper from the commandline, then do a Ctrl-C while it's in the middle of doing something! 😄

The last 2 (plus some other tidying up) are a single commit (JedMeister/confconsole@05e5919). Ideally they should have probably been split into separate commits, but I had already done the work when I realised I hadn't done a commit between; and I wasn't about to undo it just to make the git history nicer...

Special note: This version changes both the dehydrated-wrapper and the hook script. The wrapper will auto overwrite on package update, but not the hook script. The quick and dirty way to ensure that you have the latest (on a server which you've already been testing confconsole on and have just installed the latest version on) is to manually copy it across:

cp /usr/share/confconsole/letsencrypt/dehydrated-confconsole.hook.sh /etc/dehydrated/confconsole.hook.sh

I have tested the scripts themselves a bit, but probably not enough considering the refactoring I've done this afternoon (all my testing was on a Tomcat appliance). I haven't yet actually tested the package itself. If you get a chance, I'd love to hear of anything that isn't working or is broken... But I'm really hoping it's good to go!

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 13, 2017

Member

So it seems that the cron job did not work. I checked the logs all I get is the below:
screen shot 2017-03-12 at 9 38 29 pm

I also noted the server very very sluggish so I ran htop and see this...
screen shot 2017-03-12 at 9 35 05 pm

So something is running a muck! If I have time I'll wipe and reload one of the servers. I mess around a bit see if I can't manually test the cron job to see if I can find out what it's doing.

Member

DocCyblade commented Mar 13, 2017

So it seems that the cron job did not work. I checked the logs all I get is the below:
screen shot 2017-03-12 at 9 38 29 pm

I also noted the server very very sluggish so I ran htop and see this...
screen shot 2017-03-12 at 9 35 05 pm

So something is running a muck! If I have time I'll wipe and reload one of the servers. I mess around a bit see if I can't manually test the cron job to see if I can find out what it's doing.

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 13, 2017

Member

So I had a go at manually calling /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper and it ran just fine. I then called it with --force and that when things went left.

My screen filled up with this:
screen shot 2017-03-12 at 9 54 06 pm

I quickly canceled out, and scrolled up to see this
screen shot 2017-03-12 at 9 53 42 pm

It seems that the wrapper is looping. I am looking at the code now to see what is what. Maybe I can fix it!

Member

DocCyblade commented Mar 13, 2017

So I had a go at manually calling /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper and it ran just fine. I then called it with --force and that when things went left.

My screen filled up with this:
screen shot 2017-03-12 at 9 54 06 pm

I quickly canceled out, and scrolled up to see this
screen shot 2017-03-12 at 9 53 42 pm

It seems that the wrapper is looping. I am looking at the code now to see what is what. Maybe I can fix it!

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 13, 2017

Member

After mucking around I found the issue. I found the source for confconsole and the wrapper looks different than the one I am using. I am going to wipe and reload see whats what.

Member

DocCyblade commented Mar 13, 2017

After mucking around I found the issue. I found the source for confconsole and the wrapper looks different than the one I am using. I am going to wipe and reload see whats what.

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 13, 2017

Member

So it seems my test servers were mucked up and was using old files. In the process of spinning them all up (nginx/lighty/lamp) and will test again

Member

DocCyblade commented Mar 13, 2017

So it seems my test servers were mucked up and was using old files. In the process of spinning them all up (nginx/lighty/lamp) and will test again

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 13, 2017

Member

I spun up three new servers and did all the step here.

I changed the name, worked great
Tried getting a cert and all three did not seem to work.

confconsole:
  Installed: 0.9.4+140+g0b038ce
  Candidate: 0.9.4+140+g0b038ce
  Version table:
 *** 0.9.4+140+g0b038ce 0
        900 http://tkl-dev-test.s3-website-us-east-1.amazonaws.com/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
     0.9.4+54+g99603c7 0
        850 http://archive.turnkeylinux.org/debian/ jessie/main amd64 Packages
INFO: Deploying challenge for cctest-lighty.tklapp.com (2017-03-13 03:18:01)
INFO: Serving /var/lib/dehydrated/acme-challenges/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE on http://cctest-lighty.tklapp.com/.well-known/acme-challenge/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE (2017-03-13 03:18:01)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-lighty.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:18:02] "GET /.well-known/acme-challenge/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:18:03)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-lighty.tklapp.com to /etc/ssl/private (2017-03-13 03:18:04)
INFO: fullchain:  (2017-03-13 03:18:04)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-lighty.tklapp.com/privkey.pem (2017-03-13 03:18:04)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting lighttpd
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:04)
INFO: Deploying challenge for cctest-lamp.tklapp.com (2017-03-13 03:17:57)
INFO: Serving /var/lib/dehydrated/acme-challenges/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 on http://cctest-lamp.tklapp.com/.well-known/acme-challenge/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 (2017-03-13 03:17:57)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-lamp.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:17:58] "GET /.well-known/acme-challenge/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:17:59)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-lamp.tklapp.com to /etc/ssl/private (2017-03-13 03:18:00)
INFO: fullchain:  (2017-03-13 03:18:00)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-lamp.tklapp.com/privkey.pem (2017-03-13 03:18:00)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting apache2
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:01)
INFO: Deploying challenge for cctest-nginx.tklapp.com (2017-03-13 03:17:59)
INFO: Serving /var/lib/dehydrated/acme-challenges/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 on http://cctest-nginx.tklapp.com/.well-known/acme-challenge/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 (2017-03-13 03:17:59)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-nginx.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:18:00] "GET /.well-known/acme-challenge/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:18:01)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-nginx.tklapp.com to /etc/ssl/private (2017-03-13 03:18:02)
INFO: fullchain:  (2017-03-13 03:18:02)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-nginx.tklapp.com/privkey.pem (2017-03-13 03:18:02)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting nginx
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:02)
Member

DocCyblade commented Mar 13, 2017

I spun up three new servers and did all the step here.

I changed the name, worked great
Tried getting a cert and all three did not seem to work.

confconsole:
  Installed: 0.9.4+140+g0b038ce
  Candidate: 0.9.4+140+g0b038ce
  Version table:
 *** 0.9.4+140+g0b038ce 0
        900 http://tkl-dev-test.s3-website-us-east-1.amazonaws.com/ jessie/main amd64 Packages
        100 /var/lib/dpkg/status
     0.9.4+54+g99603c7 0
        850 http://archive.turnkeylinux.org/debian/ jessie/main amd64 Packages
INFO: Deploying challenge for cctest-lighty.tklapp.com (2017-03-13 03:18:01)
INFO: Serving /var/lib/dehydrated/acme-challenges/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE on http://cctest-lighty.tklapp.com/.well-known/acme-challenge/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE (2017-03-13 03:18:01)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-lighty.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:18:02] "GET /.well-known/acme-challenge/UEdPjgccuDF2uE4krQryGlv-NlxmvAQetRpT6GjVXWE HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:18:03)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-lighty.tklapp.com to /etc/ssl/private (2017-03-13 03:18:04)
INFO: fullchain:  (2017-03-13 03:18:04)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-lighty.tklapp.com/privkey.pem (2017-03-13 03:18:04)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting lighttpd
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:04)
INFO: Deploying challenge for cctest-lamp.tklapp.com (2017-03-13 03:17:57)
INFO: Serving /var/lib/dehydrated/acme-challenges/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 on http://cctest-lamp.tklapp.com/.well-known/acme-challenge/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 (2017-03-13 03:17:57)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-lamp.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:17:58] "GET /.well-known/acme-challenge/cNG-DETv8EaW_2Sfo6S0JxGljtUzuLP6NMRw48Yp5i4 HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:17:59)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-lamp.tklapp.com to /etc/ssl/private (2017-03-13 03:18:00)
INFO: fullchain:  (2017-03-13 03:18:00)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-lamp.tklapp.com/privkey.pem (2017-03-13 03:18:00)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting apache2
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:01)
INFO: Deploying challenge for cctest-nginx.tklapp.com (2017-03-13 03:17:59)
INFO: Serving /var/lib/dehydrated/acme-challenges/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 on http://cctest-nginx.tklapp.com/.well-known/acme-challenge/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 (2017-03-13 03:17:59)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for cctest-nginx.tklapp.com...
66.133.109.36 - - [13/Mar/2017 03:18:00] "GET /.well-known/acme-challenge/iU6aZs4TpLiyDGyWjkyHdS8kls5pJun442soQwWzru8 HTTP/1.1" 200 87
INFO: Stopping add-water daemon (2017-03-13 03:18:01)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for cctest-nginx.tklapp.com to /etc/ssl/private (2017-03-13 03:18:02)
INFO: fullchain:  (2017-03-13 03:18:02)
INFO: keyfile: /var/lib/dehydrated/certs/cctest-nginx.tklapp.com/privkey.pem (2017-03-13 03:18:02)
 + Done!
INFO: dehydrated complete
WARNING: Certificate update was not completed, restoring original cert & key.
INFO: starting nginx
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-13 03:18:02)
@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 13, 2017

Member

Thanks for your testing Ken.

I t looks like your initial issues were because of the buggy wrapper. As you discovered I had inadvertently caused a loop. I fixed that in the most recent version. FWIW this was the fix).

As to your more recent issues, it looks like the refactoring I did has introduced a new bug. What I was trying to do was make it more robust (so if it failed you wouldn't be left with a broken server). But it seems that it is a little overzealous! According to your logs, it successfully updated the certificate, but it thought something went wrong so it rolled it back to your original certificate.

I'll look into it ASAP. I was hoping to build tonight, but it seems we're still not quite ready 😢 I need to do a little admin catchup this morning (which I have neglected for months but I really need to get done).

Member

JedMeister commented Mar 13, 2017

Thanks for your testing Ken.

I t looks like your initial issues were because of the buggy wrapper. As you discovered I had inadvertently caused a loop. I fixed that in the most recent version. FWIW this was the fix).

As to your more recent issues, it looks like the refactoring I did has introduced a new bug. What I was trying to do was make it more robust (so if it failed you wouldn't be left with a broken server). But it seems that it is a little overzealous! According to your logs, it successfully updated the certificate, but it thought something went wrong so it rolled it back to your original certificate.

I'll look into it ASAP. I was hoping to build tonight, but it seems we're still not quite ready 😢 I need to do a little admin catchup this morning (which I have neglected for months but I really need to get done).

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 14, 2017

Member

Didn't get a chance to look at it today 😢

Hopefully I'll get to it tomorrow. Also Alon suggested to remove the variable check and just rely on the exit code of Dehydrated, so I'll test that that works as it should and we'll do that...

Member

JedMeister commented Mar 14, 2017

Didn't get a chance to look at it today 😢

Hopefully I'll get to it tomorrow. Also Alon suggested to remove the variable check and just rely on the exit code of Dehydrated, so I'll test that that works as it should and we'll do that...

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 15, 2017

Member

Ok, I have built and uploaded a new version of confconsole: confconsole_0.9.4+141+g1280550_all.deb

I have removed the variable from the hook script and the wrapper. It now just relies on the dehydrated exit code. I checked that it still trapped the errors and even when I did Ctrl-C during the dehydrated bit (not the wrapper) it still finished cleanly (restored the backup certs and restarted services), so that seems all good.

Hopefully, this is it...! 🤞

FWIW I have installed this latest version on my test LAMP server. I copy/pasted from your instructions although I just ran apt-get install confconsole because I didn't want to bother upgrading everything...

Currently it has the self signed cert (so still gets the warning). Have a look for yourself here: https://www.jeremydavis.org/

I have enabled the cron job, so hopefully it will renew really soon...

Member

JedMeister commented Mar 15, 2017

Ok, I have built and uploaded a new version of confconsole: confconsole_0.9.4+141+g1280550_all.deb

I have removed the variable from the hook script and the wrapper. It now just relies on the dehydrated exit code. I checked that it still trapped the errors and even when I did Ctrl-C during the dehydrated bit (not the wrapper) it still finished cleanly (restored the backup certs and restarted services), so that seems all good.

Hopefully, this is it...! 🤞

FWIW I have installed this latest version on my test LAMP server. I copy/pasted from your instructions although I just ran apt-get install confconsole because I didn't want to bother upgrading everything...

Currently it has the self signed cert (so still gets the warning). Have a look for yourself here: https://www.jeremydavis.org/

I have enabled the cron job, so hopefully it will renew really soon...

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 16, 2017

Member

I just checked in on my server and the cron job still hadn't run yet. So to expedite testing I rolled the clock forward like this:

date +%T -s "06:24:55"

By default cron.daily jobs are configured to run at 06:25:00. And it ran and successfully updated the cert! Yay!

screenshot from 2017-03-16 11 31 28

FYI here's the log entry:

[2017-03-16 06:25:03] cron: Checking if SSL certificate needs update
[2017-03-16 06:25:03] cron: /etc/ssl/private/cert.pem has expired or will do so within 30 days. Attempting renewal.
INFO: dehydrated-wrapper started (2017-03-16 06:25:03)
INFO: found apache2 listening on port 80
INFO: stopping apache2
INFO: running dehydrated
# INFO: Using main config file /etc/dehydrated/confconsole.config
Processing jeremydavis.org with alternative names: www.jeremydavis.org lamp.jeremydavis.org
 + Checking domain name(s) of existing cert... unchanged.
 + Checking expire date of existing cert...
 + Valid till Jun  1 06:13:00 2017 GMT (Longer than 30 days). Ignoring because renew was forced!
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting challenge for jeremydavis.org...
 + Requesting challenge for www.jeremydavis.org...
 + Requesting challenge for lamp.jeremydavis.org...
INFO: Deploying challenge for jeremydavis.org (2017-03-16 06:25:11)
INFO: Serving /var/lib/dehydrated/acme-challenges/IiogCsICxMt4u39SasF8Lqj9oqNVN911zQ_fgnpoDog on http://jeremydavis.org/.well-known/acme-challenge/IiogCsICxMt4u39SasF8Lqj9oqNVN911zQ_fgnpoDog (2017-03-16 06:25:11)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:13)
 + Challenge is valid!
INFO: Deploying challenge for www.jeremydavis.org (2017-03-16 06:25:13)
INFO: Serving /var/lib/dehydrated/acme-challenges/qytCatkWj1imZJDM_vsRwkE6N-EGvWN4DIjmHnCwJoI on http://www.jeremydavis.org/.well-known/acme-challenge/qytCatkWj1imZJDM_vsRwkE6N-EGvWN4DIjmHnCwJoI (2017-03-16 06:25:13)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for www.jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:14)
 + Challenge is valid!
INFO: Deploying challenge for lamp.jeremydavis.org (2017-03-16 06:25:14)
INFO: Serving /var/lib/dehydrated/acme-challenges/J711DoCXlj58y8t2IvknwLYRLO_4xRX1bB1afAQyCbM on http://lamp.jeremydavis.org/.well-known/acme-challenge/J711DoCXlj58y8t2IvknwLYRLO_4xRX1bB1afAQyCbM (2017-03-16 06:25:14)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for lamp.jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:16)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for jeremydavis.org to /etc/ssl/private (2017-03-16 06:25:18)
INFO: fullchain:  (2017-03-16 06:25:18)
INFO: keyfile: /var/lib/dehydrated/certs/jeremydavis.org/privkey.pem (2017-03-16 06:25:18)
 + Done!
INFO: dehydrated complete
INFO: Cleaning backup cert & key
INFO: starting apache2
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-16 06:25:19)
Member

JedMeister commented Mar 16, 2017

I just checked in on my server and the cron job still hadn't run yet. So to expedite testing I rolled the clock forward like this:

date +%T -s "06:24:55"

By default cron.daily jobs are configured to run at 06:25:00. And it ran and successfully updated the cert! Yay!

screenshot from 2017-03-16 11 31 28

FYI here's the log entry:

[2017-03-16 06:25:03] cron: Checking if SSL certificate needs update
[2017-03-16 06:25:03] cron: /etc/ssl/private/cert.pem has expired or will do so within 30 days. Attempting renewal.
INFO: dehydrated-wrapper started (2017-03-16 06:25:03)
INFO: found apache2 listening on port 80
INFO: stopping apache2
INFO: running dehydrated
# INFO: Using main config file /etc/dehydrated/confconsole.config
Processing jeremydavis.org with alternative names: www.jeremydavis.org lamp.jeremydavis.org
 + Checking domain name(s) of existing cert... unchanged.
 + Checking expire date of existing cert...
 + Valid till Jun  1 06:13:00 2017 GMT (Longer than 30 days). Ignoring because renew was forced!
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting challenge for jeremydavis.org...
 + Requesting challenge for www.jeremydavis.org...
 + Requesting challenge for lamp.jeremydavis.org...
INFO: Deploying challenge for jeremydavis.org (2017-03-16 06:25:11)
INFO: Serving /var/lib/dehydrated/acme-challenges/IiogCsICxMt4u39SasF8Lqj9oqNVN911zQ_fgnpoDog on http://jeremydavis.org/.well-known/acme-challenge/IiogCsICxMt4u39SasF8Lqj9oqNVN911zQ_fgnpoDog (2017-03-16 06:25:11)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:13)
 + Challenge is valid!
INFO: Deploying challenge for www.jeremydavis.org (2017-03-16 06:25:13)
INFO: Serving /var/lib/dehydrated/acme-challenges/qytCatkWj1imZJDM_vsRwkE6N-EGvWN4DIjmHnCwJoI on http://www.jeremydavis.org/.well-known/acme-challenge/qytCatkWj1imZJDM_vsRwkE6N-EGvWN4DIjmHnCwJoI (2017-03-16 06:25:13)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for www.jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:14)
 + Challenge is valid!
INFO: Deploying challenge for lamp.jeremydavis.org (2017-03-16 06:25:14)
INFO: Serving /var/lib/dehydrated/acme-challenges/J711DoCXlj58y8t2IvknwLYRLO_4xRX1bB1afAQyCbM on http://lamp.jeremydavis.org/.well-known/acme-challenge/J711DoCXlj58y8t2IvknwLYRLO_4xRX1bB1afAQyCbM (2017-03-16 06:25:14)
Daemonizing add-water server
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:80/
Hit Ctrl-C to quit.

 + Responding to challenge for lamp.jeremydavis.org...
INFO: Stopping add-water daemon (2017-03-16 06:25:16)
 + Challenge is valid!
 + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
INFO: writing cert.pem & cert.key for jeremydavis.org to /etc/ssl/private (2017-03-16 06:25:18)
INFO: fullchain:  (2017-03-16 06:25:18)
INFO: keyfile: /var/lib/dehydrated/certs/jeremydavis.org/privkey.pem (2017-03-16 06:25:18)
 + Done!
INFO: dehydrated complete
INFO: Cleaning backup cert & key
INFO: starting apache2
INFO: starting stunnel4
INFO: dehydrated-wrapper completed successfully. (2017-03-16 06:25:19)
@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 16, 2017

Member

My only concern is that that is a pretty big log entry, for just updating 3 certs (well actually only 1 cert with 2 alternate subdomains).

Previously I had the cron job writing to a separate log file (/var/log/confconsole/dehydrated-wrapper-cron.log) with the dehydrated part still writing to the default log (/var/log/confconsole/letsencrypt.log).

I do have logrotate set up (see here). That makes it rotate monthly and keep 18mths worth of logs before it starts deleting them. Also it doesn't immediately archive the log when it's rotated (via delaycompress), just in case something is writing to the logfile at the time of rotation. I haven't been able to do a "real world" test, but I have done some basic testing:

root@lamp ~# logrotate --debug  /etc/logrotate.d/confconsole 
reading config file /etc/logrotate.d/confconsole

Handling 1 logs

rotating pattern: /var/log/confconsole/*.log  monthly (18 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/confconsole/dehydrated-wrapper-cron.log
  log does not need rotating
considering log /var/log/confconsole/letsencrypt.log
  log does not need rotating

Note, that even though I have changed the logging behaviour, I still have the old dehydrated-wrapper-cron.log file in place as you can see above.

I also tested it like this:

root@lamp ~# logrotate --force --verbose  /etc/logrotate.d/confconsole 
reading config file /etc/logrotate.d/confconsole

Handling 1 logs

rotating pattern: /var/log/confconsole/*.log  forced from command line (18 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/confconsole/dehydrated-wrapper-cron.log
  log needs rotating
considering log /var/log/confconsole/letsencrypt.log
  log needs rotating
rotating log /var/log/confconsole/dehydrated-wrapper-cron.log, log->rotateCount is 18
dateext suffix '-20170316'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/confconsole/dehydrated-wrapper-cron.log.1 does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.19.gz (rotatecount 18, logstart 1, i 18), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz (rotatecount 18, logstart 1, i 17), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz (rotatecount 18, logstart 1, i 16), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz (rotatecount 18, logstart 1, i 15), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz (rotatecount 18, logstart 1, i 14), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz (rotatecount 18, logstart 1, i 13), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz (rotatecount 18, logstart 1, i 12), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz (rotatecount 18, logstart 1, i 11), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz (rotatecount 18, logstart 1, i 10), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz (rotatecount 18, logstart 1, i 9), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz (rotatecount 18, logstart 1, i 8), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz (rotatecount 18, logstart 1, i 7), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz (rotatecount 18, logstart 1, i 6), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz (rotatecount 18, logstart 1, i 5), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz (rotatecount 18, logstart 1, i 4), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz (rotatecount 18, logstart 1, i 3), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz (rotatecount 18, logstart 1, i 2), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz (rotatecount 18, logstart 1, i 1), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.0.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz (rotatecount 18, logstart 1, i 0), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.0.gz does not exist
log /var/log/confconsole/dehydrated-wrapper-cron.log.19.gz doesn't exist -- won't try to dispose of it
renaming /var/log/confconsole/dehydrated-wrapper-cron.log to /var/log/confconsole/dehydrated-wrapper-cron.log.1
creating new /var/log/confconsole/dehydrated-wrapper-cron.log mode = 0640 uid = 0 gid = 0
rotating log /var/log/confconsole/letsencrypt.log, log->rotateCount is 18
dateext suffix '-20170316'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/confconsole/letsencrypt.log.1 does not exist
renaming /var/log/confconsole/letsencrypt.log.18.gz to /var/log/confconsole/letsencrypt.log.19.gz (rotatecount 18, logstart 1, i 18), 
old log /var/log/confconsole/letsencrypt.log.18.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.17.gz to /var/log/confconsole/letsencrypt.log.18.gz (rotatecount 18, logstart 1, i 17), 
old log /var/log/confconsole/letsencrypt.log.17.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.16.gz to /var/log/confconsole/letsencrypt.log.17.gz (rotatecount 18, logstart 1, i 16), 
old log /var/log/confconsole/letsencrypt.log.16.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.15.gz to /var/log/confconsole/letsencrypt.log.16.gz (rotatecount 18, logstart 1, i 15), 
old log /var/log/confconsole/letsencrypt.log.15.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.14.gz to /var/log/confconsole/letsencrypt.log.15.gz (rotatecount 18, logstart 1, i 14), 
old log /var/log/confconsole/letsencrypt.log.14.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.13.gz to /var/log/confconsole/letsencrypt.log.14.gz (rotatecount 18, logstart 1, i 13), 
old log /var/log/confconsole/letsencrypt.log.13.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.12.gz to /var/log/confconsole/letsencrypt.log.13.gz (rotatecount 18, logstart 1, i 12), 
old log /var/log/confconsole/letsencrypt.log.12.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.11.gz to /var/log/confconsole/letsencrypt.log.12.gz (rotatecount 18, logstart 1, i 11), 
old log /var/log/confconsole/letsencrypt.log.11.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.10.gz to /var/log/confconsole/letsencrypt.log.11.gz (rotatecount 18, logstart 1, i 10), 
old log /var/log/confconsole/letsencrypt.log.10.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.9.gz to /var/log/confconsole/letsencrypt.log.10.gz (rotatecount 18, logstart 1, i 9), 
old log /var/log/confconsole/letsencrypt.log.9.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.8.gz to /var/log/confconsole/letsencrypt.log.9.gz (rotatecount 18, logstart 1, i 8), 
old log /var/log/confconsole/letsencrypt.log.8.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.7.gz to /var/log/confconsole/letsencrypt.log.8.gz (rotatecount 18, logstart 1, i 7), 
old log /var/log/confconsole/letsencrypt.log.7.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.6.gz to /var/log/confconsole/letsencrypt.log.7.gz (rotatecount 18, logstart 1, i 6), 
old log /var/log/confconsole/letsencrypt.log.6.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.5.gz to /var/log/confconsole/letsencrypt.log.6.gz (rotatecount 18, logstart 1, i 5), 
old log /var/log/confconsole/letsencrypt.log.5.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.4.gz to /var/log/confconsole/letsencrypt.log.5.gz (rotatecount 18, logstart 1, i 4), 
old log /var/log/confconsole/letsencrypt.log.4.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.3.gz to /var/log/confconsole/letsencrypt.log.4.gz (rotatecount 18, logstart 1, i 3), 
old log /var/log/confconsole/letsencrypt.log.3.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.2.gz to /var/log/confconsole/letsencrypt.log.3.gz (rotatecount 18, logstart 1, i 2), 
old log /var/log/confconsole/letsencrypt.log.2.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.1.gz to /var/log/confconsole/letsencrypt.log.2.gz (rotatecount 18, logstart 1, i 1), 
old log /var/log/confconsole/letsencrypt.log.1.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.0.gz to /var/log/confconsole/letsencrypt.log.1.gz (rotatecount 18, logstart 1, i 0), 
old log /var/log/confconsole/letsencrypt.log.0.gz does not exist
log /var/log/confconsole/letsencrypt.log.19.gz doesn't exist -- won't try to dispose of it
renaming /var/log/confconsole/letsencrypt.log to /var/log/confconsole/letsencrypt.log.1
creating new /var/log/confconsole/letsencrypt.log mode = 0640 uid = 0 gid = 0

Initially I thought I should adjust the permissions of the files:

root@lamp ~# ls -lh /var/log/confconsole/
total 28K
-rw-r----- 1 root     root   0 Mar 16 06:55 dehydrated-wrapper-cron.log
-rw-r--r-- 1 www-data root 945 Mar 12 06:47 dehydrated-wrapper-cron.log.1
-rw-r----- 1 root     root   0 Mar 16 06:55 letsencrypt.log
-rw-r--r-- 1 www-data root 17K Mar 16 06:25 letsencrypt.log.1

However, after manually running /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper --force (as the cron job would) all is well:

root@lamp ~# ls -lh /var/log/confconsole/
total 32K
-rw-r----- 1 www-data root    0 Mar 16 06:55 dehydrated-wrapper-cron.log
-rw-r--r-- 1 www-data root  945 Mar 12 06:47 dehydrated-wrapper-cron.log.1
-rw-r----- 1 www-data root 3.6K Mar 16 06:59 letsencrypt.log
-rw-r--r-- 1 www-data root  17K Mar 16 06:25 letsencrypt.log.1

I'd be interested in your thoughts. Perhaps we could leave it as is for now, and improve it later for next release?

Member

JedMeister commented Mar 16, 2017

My only concern is that that is a pretty big log entry, for just updating 3 certs (well actually only 1 cert with 2 alternate subdomains).

Previously I had the cron job writing to a separate log file (/var/log/confconsole/dehydrated-wrapper-cron.log) with the dehydrated part still writing to the default log (/var/log/confconsole/letsencrypt.log).

I do have logrotate set up (see here). That makes it rotate monthly and keep 18mths worth of logs before it starts deleting them. Also it doesn't immediately archive the log when it's rotated (via delaycompress), just in case something is writing to the logfile at the time of rotation. I haven't been able to do a "real world" test, but I have done some basic testing:

root@lamp ~# logrotate --debug  /etc/logrotate.d/confconsole 
reading config file /etc/logrotate.d/confconsole

Handling 1 logs

rotating pattern: /var/log/confconsole/*.log  monthly (18 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/confconsole/dehydrated-wrapper-cron.log
  log does not need rotating
considering log /var/log/confconsole/letsencrypt.log
  log does not need rotating

Note, that even though I have changed the logging behaviour, I still have the old dehydrated-wrapper-cron.log file in place as you can see above.

I also tested it like this:

root@lamp ~# logrotate --force --verbose  /etc/logrotate.d/confconsole 
reading config file /etc/logrotate.d/confconsole

Handling 1 logs

rotating pattern: /var/log/confconsole/*.log  forced from command line (18 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/confconsole/dehydrated-wrapper-cron.log
  log needs rotating
considering log /var/log/confconsole/letsencrypt.log
  log needs rotating
rotating log /var/log/confconsole/dehydrated-wrapper-cron.log, log->rotateCount is 18
dateext suffix '-20170316'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/confconsole/dehydrated-wrapper-cron.log.1 does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.19.gz (rotatecount 18, logstart 1, i 18), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.18.gz (rotatecount 18, logstart 1, i 17), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.17.gz (rotatecount 18, logstart 1, i 16), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.16.gz (rotatecount 18, logstart 1, i 15), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.15.gz (rotatecount 18, logstart 1, i 14), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.14.gz (rotatecount 18, logstart 1, i 13), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.13.gz (rotatecount 18, logstart 1, i 12), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.12.gz (rotatecount 18, logstart 1, i 11), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.11.gz (rotatecount 18, logstart 1, i 10), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.10.gz (rotatecount 18, logstart 1, i 9), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.9.gz (rotatecount 18, logstart 1, i 8), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.8.gz (rotatecount 18, logstart 1, i 7), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.7.gz (rotatecount 18, logstart 1, i 6), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.6.gz (rotatecount 18, logstart 1, i 5), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.5.gz (rotatecount 18, logstart 1, i 4), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.4.gz (rotatecount 18, logstart 1, i 3), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.3.gz (rotatecount 18, logstart 1, i 2), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.2.gz (rotatecount 18, logstart 1, i 1), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz does not exist
renaming /var/log/confconsole/dehydrated-wrapper-cron.log.0.gz to /var/log/confconsole/dehydrated-wrapper-cron.log.1.gz (rotatecount 18, logstart 1, i 0), 
old log /var/log/confconsole/dehydrated-wrapper-cron.log.0.gz does not exist
log /var/log/confconsole/dehydrated-wrapper-cron.log.19.gz doesn't exist -- won't try to dispose of it
renaming /var/log/confconsole/dehydrated-wrapper-cron.log to /var/log/confconsole/dehydrated-wrapper-cron.log.1
creating new /var/log/confconsole/dehydrated-wrapper-cron.log mode = 0640 uid = 0 gid = 0
rotating log /var/log/confconsole/letsencrypt.log, log->rotateCount is 18
dateext suffix '-20170316'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/confconsole/letsencrypt.log.1 does not exist
renaming /var/log/confconsole/letsencrypt.log.18.gz to /var/log/confconsole/letsencrypt.log.19.gz (rotatecount 18, logstart 1, i 18), 
old log /var/log/confconsole/letsencrypt.log.18.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.17.gz to /var/log/confconsole/letsencrypt.log.18.gz (rotatecount 18, logstart 1, i 17), 
old log /var/log/confconsole/letsencrypt.log.17.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.16.gz to /var/log/confconsole/letsencrypt.log.17.gz (rotatecount 18, logstart 1, i 16), 
old log /var/log/confconsole/letsencrypt.log.16.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.15.gz to /var/log/confconsole/letsencrypt.log.16.gz (rotatecount 18, logstart 1, i 15), 
old log /var/log/confconsole/letsencrypt.log.15.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.14.gz to /var/log/confconsole/letsencrypt.log.15.gz (rotatecount 18, logstart 1, i 14), 
old log /var/log/confconsole/letsencrypt.log.14.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.13.gz to /var/log/confconsole/letsencrypt.log.14.gz (rotatecount 18, logstart 1, i 13), 
old log /var/log/confconsole/letsencrypt.log.13.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.12.gz to /var/log/confconsole/letsencrypt.log.13.gz (rotatecount 18, logstart 1, i 12), 
old log /var/log/confconsole/letsencrypt.log.12.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.11.gz to /var/log/confconsole/letsencrypt.log.12.gz (rotatecount 18, logstart 1, i 11), 
old log /var/log/confconsole/letsencrypt.log.11.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.10.gz to /var/log/confconsole/letsencrypt.log.11.gz (rotatecount 18, logstart 1, i 10), 
old log /var/log/confconsole/letsencrypt.log.10.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.9.gz to /var/log/confconsole/letsencrypt.log.10.gz (rotatecount 18, logstart 1, i 9), 
old log /var/log/confconsole/letsencrypt.log.9.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.8.gz to /var/log/confconsole/letsencrypt.log.9.gz (rotatecount 18, logstart 1, i 8), 
old log /var/log/confconsole/letsencrypt.log.8.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.7.gz to /var/log/confconsole/letsencrypt.log.8.gz (rotatecount 18, logstart 1, i 7), 
old log /var/log/confconsole/letsencrypt.log.7.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.6.gz to /var/log/confconsole/letsencrypt.log.7.gz (rotatecount 18, logstart 1, i 6), 
old log /var/log/confconsole/letsencrypt.log.6.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.5.gz to /var/log/confconsole/letsencrypt.log.6.gz (rotatecount 18, logstart 1, i 5), 
old log /var/log/confconsole/letsencrypt.log.5.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.4.gz to /var/log/confconsole/letsencrypt.log.5.gz (rotatecount 18, logstart 1, i 4), 
old log /var/log/confconsole/letsencrypt.log.4.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.3.gz to /var/log/confconsole/letsencrypt.log.4.gz (rotatecount 18, logstart 1, i 3), 
old log /var/log/confconsole/letsencrypt.log.3.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.2.gz to /var/log/confconsole/letsencrypt.log.3.gz (rotatecount 18, logstart 1, i 2), 
old log /var/log/confconsole/letsencrypt.log.2.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.1.gz to /var/log/confconsole/letsencrypt.log.2.gz (rotatecount 18, logstart 1, i 1), 
old log /var/log/confconsole/letsencrypt.log.1.gz does not exist
renaming /var/log/confconsole/letsencrypt.log.0.gz to /var/log/confconsole/letsencrypt.log.1.gz (rotatecount 18, logstart 1, i 0), 
old log /var/log/confconsole/letsencrypt.log.0.gz does not exist
log /var/log/confconsole/letsencrypt.log.19.gz doesn't exist -- won't try to dispose of it
renaming /var/log/confconsole/letsencrypt.log to /var/log/confconsole/letsencrypt.log.1
creating new /var/log/confconsole/letsencrypt.log mode = 0640 uid = 0 gid = 0

Initially I thought I should adjust the permissions of the files:

root@lamp ~# ls -lh /var/log/confconsole/
total 28K
-rw-r----- 1 root     root   0 Mar 16 06:55 dehydrated-wrapper-cron.log
-rw-r--r-- 1 www-data root 945 Mar 12 06:47 dehydrated-wrapper-cron.log.1
-rw-r----- 1 root     root   0 Mar 16 06:55 letsencrypt.log
-rw-r--r-- 1 www-data root 17K Mar 16 06:25 letsencrypt.log.1

However, after manually running /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper --force (as the cron job would) all is well:

root@lamp ~# ls -lh /var/log/confconsole/
total 32K
-rw-r----- 1 www-data root    0 Mar 16 06:55 dehydrated-wrapper-cron.log
-rw-r--r-- 1 www-data root  945 Mar 12 06:47 dehydrated-wrapper-cron.log.1
-rw-r----- 1 www-data root 3.6K Mar 16 06:59 letsencrypt.log
-rw-r--r-- 1 www-data root  17K Mar 16 06:25 letsencrypt.log.1

I'd be interested in your thoughts. Perhaps we could leave it as is for now, and improve it later for next release?

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 17, 2017

Member

Spun up three more test servers, followed the same steps as before. Manuel certs worked with out a problem. I am waiting to see if the cert will renew in the morning

Member

DocCyblade commented Mar 17, 2017

Spun up three more test servers, followed the same steps as before. Manuel certs worked with out a problem. I am waiting to see if the cert will renew in the morning

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 17, 2017

Member

Ok last testing release (I hope): confconsole_0.9.4+146+gbf0dfd6_all.deb

This version should contain everything that will be in the final v1.0.0 release (which v14.2 will ship with).

Things changed in this version:

Hopefully that's all good. Nearly ready to build...! 😄

Member

JedMeister commented Mar 17, 2017

Ok last testing release (I hope): confconsole_0.9.4+146+gbf0dfd6_all.deb

This version should contain everything that will be in the final v1.0.0 release (which v14.2 will ship with).

Things changed in this version:

Hopefully that's all good. Nearly ready to build...! 😄

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 17, 2017

Member

I'll give this one a test this weekend

Member

DocCyblade commented Mar 17, 2017

I'll give this one a test this weekend

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 20, 2017

Member

Ok another build: confconsole_0.9.4+150+g1aff37b_all.deb. It's in the test repo. Only changes since previous version are:

  • reduced the logging of the cron job to one line: either nothing to do, or running update (note will log additional line if it tries to update and fails).
  • adjusted the logrotate config to only keep 6 mths of logs (was 18).

I will be putting this forward to Alon for code review as proposed version 1.0.0 release of confconsole.

FWIW I have merged the email SMTP config. And the docs. Within the next day or 2 I will merge this and tag it 1.0.0. And then we're right to release Core v14.2! 😄

Member

JedMeister commented Mar 20, 2017

Ok another build: confconsole_0.9.4+150+g1aff37b_all.deb. It's in the test repo. Only changes since previous version are:

  • reduced the logging of the cron job to one line: either nothing to do, or running update (note will log additional line if it tries to update and fails).
  • adjusted the logrotate config to only keep 6 mths of logs (was 18).

I will be putting this forward to Alon for code review as proposed version 1.0.0 release of confconsole.

FWIW I have merged the email SMTP config. And the docs. Within the next day or 2 I will merge this and tag it 1.0.0. And then we're right to release Core v14.2! 😄

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 20, 2017

Member

I'll give this a good testing this evening.

Member

DocCyblade commented Mar 20, 2017

I'll give this a good testing this evening.

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 20, 2017

Member

Thanks mate. Always appreciate your vigilant testing! 👍

Member

JedMeister commented Mar 20, 2017

Thanks mate. Always appreciate your vigilant testing! 👍

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 21, 2017

Member

Ok, another version released. Now hopefully THIS one will be the last! 🤞 😄

confconsole_0.9.4+153+g9d48e77_all.deb uploaded and in the test repo

in this version:

  • sets hostname in /etc/hosts now too (see #795)
  • dehydrated hook script, now double check vars (thanks Ken for your assistance with that)
  • dehydrated hook script, improved logging
Member

JedMeister commented Mar 21, 2017

Ok, another version released. Now hopefully THIS one will be the last! 🤞 😄

confconsole_0.9.4+153+g9d48e77_all.deb uploaded and in the test repo

in this version:

  • sets hostname in /etc/hosts now too (see #795)
  • dehydrated hook script, now double check vars (thanks Ken for your assistance with that)
  • dehydrated hook script, improved logging
@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 21, 2017

Member

Unless you have anything further to add, or can find any new bugs within the next day or 2, it'll be v1.0.0!

Member

JedMeister commented Mar 21, 2017

Unless you have anything further to add, or can find any new bugs within the next day or 2, it'll be v1.0.0!

@DocCyblade

This comment has been minimized.

Show comment
Hide comment
@DocCyblade

DocCyblade Mar 21, 2017

Member

I had a quick test of the new version, hostname was fixed and logging seemed to be working as well. My three test servers (light/nginx/apache) all updated correctly this morning and got a new cert.

my only concern would be the below, I know you had to make it a warning to make it log, but maybe add something to it like [2017-03-21 10:20:30] confconsole.hook.sh: WARNING: Cert request successful. Writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

As if you read the log, it almost looked like something did not work, when in fact it did. Other than that. It works

from /var/log/confconsole/letsencrypt.log

[2017-03-21 10:18:31] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:acme:error:connection",
    "detail": "Could not connect to test-server01.tklapp.com",
    "status": 400
  },
  "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/VNwltnGOjU2onKL6WUe_3_TJ_14NaDBfIViGxhyrxU0/845072545",
  "token": "6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w",
  "keyAuthorization": "6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w.cGF71z5AWJdQ59DVnLwiKqq3KKGMGMBcELpIK53PQB4",
  "validationRecord": [
    {
      "url": "http://test-server01.tklapp.com/.well-known/acme-challenge/6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w",
      "hostname": "test-server01.tklapp.com",
      "port": "80",
      "addressesResolved": [
        "34.205.16.57"
      ],
      "addressUsed": "34.205.16.57"
    }
  ]
})
[2017-03-21 10:19:07] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2017-03-21 10:19:07] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert & key.
[2017-03-21 10:19:08] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
[2017-03-21 10:20:30] confconsole.hook.sh: WARNING: writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

Member

DocCyblade commented Mar 21, 2017

I had a quick test of the new version, hostname was fixed and logging seemed to be working as well. My three test servers (light/nginx/apache) all updated correctly this morning and got a new cert.

my only concern would be the below, I know you had to make it a warning to make it log, but maybe add something to it like [2017-03-21 10:20:30] confconsole.hook.sh: WARNING: Cert request successful. Writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

As if you read the log, it almost looked like something did not work, when in fact it did. Other than that. It works

from /var/log/confconsole/letsencrypt.log

[2017-03-21 10:18:31] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:acme:error:connection",
    "detail": "Could not connect to test-server01.tklapp.com",
    "status": 400
  },
  "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/VNwltnGOjU2onKL6WUe_3_TJ_14NaDBfIViGxhyrxU0/845072545",
  "token": "6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w",
  "keyAuthorization": "6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w.cGF71z5AWJdQ59DVnLwiKqq3KKGMGMBcELpIK53PQB4",
  "validationRecord": [
    {
      "url": "http://test-server01.tklapp.com/.well-known/acme-challenge/6eh8ZBea_w9_tBU8FAxSAWisfNzxCbKIL64M8w14A8w",
      "hostname": "test-server01.tklapp.com",
      "port": "80",
      "addressesResolved": [
        "34.205.16.57"
      ],
      "addressUsed": "34.205.16.57"
    }
  ]
})
[2017-03-21 10:19:07] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2017-03-21 10:19:07] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert & key.
[2017-03-21 10:19:08] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.
[2017-03-21 10:20:30] confconsole.hook.sh: WARNING: writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 21, 2017

Member

Great feedback, thanks Ken (as per always really, but I never get tired of it!)

Ok, so here's what I've done. I've changed "warning" to "success" in the hook_log function. It didn't actually need to be a "warning" to be picked up by the wrapper script (and written to the log) it just had to output to stderr. I also added your suggested message. So a new (successful) log entry should look like this:

[2017-03-21 10:20:30] confconsole.hook.sh: SUCCESS: Cert request successful. Writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

The changes I made are here: JedMeister/confconsole@dcea898

I'm going to merge all this now and tag it v1.0.0! 😄

Although obviously post any issues if you come across any!

Member

JedMeister commented Mar 21, 2017

Great feedback, thanks Ken (as per always really, but I never get tired of it!)

Ok, so here's what I've done. I've changed "warning" to "success" in the hook_log function. It didn't actually need to be a "warning" to be picked up by the wrapper script (and written to the log) it just had to output to stderr. I also added your suggested message. So a new (successful) log entry should look like this:

[2017-03-21 10:20:30] confconsole.hook.sh: SUCCESS: Cert request successful. Writing cert.pem & cert.key for test-server02.tklapp.com to /etc/ssl/private

The changes I made are here: JedMeister/confconsole@dcea898

I'm going to merge all this now and tag it v1.0.0! 😄

Although obviously post any issues if you come across any!

@JedMeister

This comment has been minimized.

Show comment
Hide comment
@JedMeister

JedMeister Mar 29, 2017

Member

Closing now. Should be all good as of the latest confconsole commit: https://github.com/turnkeylinux/confconsole/tree/18aa7e6d082667522ebecdf6777990b1bbed81cf

Member

JedMeister commented Mar 29, 2017

Closing now. Should be all good as of the latest confconsole commit: https://github.com/turnkeylinux/confconsole/tree/18aa7e6d082667522ebecdf6777990b1bbed81cf

@JedMeister JedMeister closed this Mar 29, 2017

@JedMeister JedMeister changed the title from Confconsole - Letsencrypt WIP to Confconsole - Letsencrypt support Mar 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment