TP Delivery Service Generate SSL update, new letsencrypt generate and renew API endpoints #3534

mattjackson220 · 2019-04-29T20:42:25Z

What does this PR (Pull Request) do?

This PR integrates the use of Let's Encrypt (LE) as a free, automated certificate authority in order to generate signed certificates for delivery services. It updates Traffic Portal (TP) to include LE as an option when generating certificates and adds Traffic Ops (TO) endpoints in order to orchestrate the certificate generation. TO will call LE to get a token, then place that token in the TO database. Traffic Router (TR) has been updated to watch the new table in the TO database, and when an update is seen, place the token at the appropriate location (_acme-challenge.domain.com). LE will verify the token and provide the signed certificate to TO, which then saves the certificate and key into Traffic Vault.

It also sets up a cron job to automatically renew certificates as they near expiration, and send a summary email if desired.

This PR fixes Let's Encrypt support #2159

This PR is related to PR #1906 and is a potential solution.

Which Traffic Control components are affected by this PR?

Documentation
Traffic Ops
Traffic Portal
Traffic Router

What is the best way to verify this PR?

Testing this requires a publicly accessible domain, and access to a working SMTP server! To test, set up cdn.conf to contain the following options:

"lets_encrypt" : {
	"send_expiration_email": true,
	"convert_self_signed": true,
	"renew_days_before_expiration": 100,
	"environment": "staging"
},
"smtp" : {
	"enabled" : true,
	"to_email" : "email to receive updates",
	"from_email" : "email to be shown as sender",
	"user" : "SMTP server username",
	"password" : "SMTP server password",
	"address" : "SMTP server location (localhost:25 for sendmail)"
}

Note: settting renew_days_before_expiration to 100 will allow the autorenew testing to renew LE certificates during testing since LE certs expire every 90 days. This is normally set to 30 as the default. Also, setting environment to production will generate valid, signed certificates if desired.

Then, update TO, TP, and TR in an environment that is accessible to Let's Encrypt in order for it to complete the domain validation. Set up or find a delivery service for HTTPS, DNS with HTTPS, and steering with HTTPS.
For each variation:

Go to More -> Manage SSL Keys -> More -> Generate SSL Keys and click on Use Let's Encrypt and verify the Common Name is what you want. Click Generate Keys With Lets Encrypt and then Yes on the pop up.
The call to LE has been made but will take a few minutes. Verify that a record has been added to /api/1.4/letsencrypt/dnsrecords and wait for updates in the Change Log.
Once a success message has been received, the certificates should show up under the Manage SSL Keys page.
Queue updates for the associated CDN in order to get the new certificates out to the caches.
Once the servers have been updated, verify that the new certificates are used at the HTTPS endpoint for the delivery service.
Repeat for other Delivery Service types

To verify the auto-renewal script is working, update /etc/cron.d/autorenew_certs to run more frequently (currently set at every sunday night) and replace { "base_url": "https://127.0.0.1" } with { "base_url": "https://127.0.0.1", "user": "your TO user", "pass": "your TO password" }

Wait until the cron job is run again and verify that the LE certs you generated in the first tests were renewed. Verify that an email summary was sent to the email configured in the cdn.conf file, including all certificates that were renewed or any that will expire within the specified time.

The following criteria are ALL met by this PR

This PR includes tests OR I have explained why tests are unnecessary
This PR includes documentation OR I have explained why documentation is unnecessary
This PR includes an update to CHANGELOG.md OR such an update is not necessary
This PR includes any and all required license headers
This PR ensures that database migration sequence is correct OR this PR does not include a database migration
This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY

asfgit · 2019-04-29T20:46:09Z

Can one of the admins verify this patch?

JBevillC · 2019-05-21T15:27:41Z

I'm also working (still) on the x509 validation TO code/architecture of imported certificates. We have decided to shift to an architecture that requires the full cert chain in traffic vault so that certificates can be validated fully without having to rely on an external CA trust store and whether or not they are are the same on multiple TO hosts.

How are you storing the certificate in Traffic Vault once received? Can you possibly store and/or add the option to store the full cert chain in traffic vault? The format should be in reverse order: server-cert, intermediate-CA-N, intermediate-CA 2, intermediate-CA-1, CA-Root

Thoughts?.

asfgit · 2019-06-21T18:40:35Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3891/
Test FAILed.

asfgit · 2019-07-09T21:49:10Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3947/
Test PASSed.

asfgit · 2019-07-25T06:53:03Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4032/
Test PASSed.

asfgit · 2019-07-29T21:35:25Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4050/
Test PASSed.

asf-ci · 2019-08-28T20:38:27Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4188/
Test PASSed.

asf-ci · 2019-09-05T15:30:16Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4203/
Test PASSed.

asf-ci · 2019-09-05T16:03:27Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4204/
Test PASSed.

asf-ci · 2019-10-01T17:06:08Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4371/

asf-ci · 2019-10-01T20:46:19Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4373/

asf-ci · 2019-10-04T11:28:12Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4395/

asf-ci · 2019-10-07T22:10:35Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4409/

ocket8888 · 2019-10-08T19:23:01Z

Can you rebase this on the new SMTP configuration stuff that's in master?

asf-ci · 2019-10-10T23:33:03Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4447/

asf-ci · 2019-10-14T15:27:23Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4491/

asf-ci · 2019-10-14T15:32:49Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4492/

ocket8888 · 2019-10-14T15:46:20Z

retest this please

asf-ci · 2019-10-14T15:52:21Z

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4493/

… changelog

…ving "Not Assigned" as auth type

traffic_ops/traffic_ops_golang/routing/routes.go

ocket8888

Got some unit test failures

traffic_ops/traffic_ops_golang/deliveryservice/autorenewcerts.go

ocket8888

Looks good, LE support works, supporting scripts work, unit and integration tests show that existing functionality isn't broken (ran all Go unit tests, TO Go client/API integration tests and TR unit tests), and docs look complete, thorough and compile without warnings or errors.

zrhoffman · 2020-04-07T19:01:30Z

docs/source/api/letsencrypt_dnsrecords.rst

+.. code-block:: http
+	:caption: Request Example
+
+	GET /api/1.4/letsencrypt/dnsrecord?fqdn=_acme-challenge.demo1.example.com. HTTP/1.1


@mattjackson220 This should be letsencrypt/dnsrecords

mattjackson220 changed the title ~~TP Delivery Service Generate SSL update, new letsencrypt generate and…~~ WIP - TP Delivery Service Generate SSL update, new letsencrypt generate and… May 2, 2019

mattjackson220 force-pushed the lets_encrypt branch from b20928d to ebb8649 Compare September 5, 2019 15:34

ocket8888 added documentation related to documentation new feature A new feature, capability or behavior SSL support for/problems with SSL features Traffic Ops related to Traffic Ops labels Sep 5, 2019

mattjackson220 force-pushed the lets_encrypt branch from ff65af4 to c4d2290 Compare October 1, 2019 19:26

mattjackson220 changed the title ~~WIP - TP Delivery Service Generate SSL update, new letsencrypt generate and…~~ TP Delivery Service Generate SSL update, new letsencrypt generate and… Oct 3, 2019

mattjackson220 marked this pull request as ready for review October 3, 2019 21:29

mattjackson220 force-pushed the lets_encrypt branch from 8446758 to d9ff72f Compare October 10, 2019 19:39

mattjackson220 force-pushed the lets_encrypt branch from 5f7ac6d to c25a638 Compare October 14, 2019 15:29

mattjackson220 added 18 commits February 14, 2020 15:04

autorenew, formatted email, documentation, async workflow and logs to…

86d0b94

… changelog

minor updates and SMTP code relocation

1858216

minor update so certs created before this update show in the UI as ha…

7a81730

…ving "Not Assigned" as auth type

merge with other SMTP config

59158c3

updates per comments

3ccf4c4

updates per comments

8a89f59

Updates per comments

3927087

Vendored in lego dependency

5617856

update for minor fix

9028aa8

License update

75f2cb6

updated for 1.5 and per comments

8e7adfa

updated TR to use api/1.5 for LE records

55eeaf7

minor updates for 1.5

464a708

fix for autorenew

58da5fe

updates per comments

0a12bc3

updated autorenewal to be async

8a405a9

updates per comments

1275bcf

updates per comments

822c0f4

mattjackson220 force-pushed the lets_encrypt branch from 799f6b9 to 822c0f4 Compare February 14, 2020 22:10

updated LE for 2.0

cff25f6

ocket8888 reviewed Feb 16, 2020

View reviewed changes

traffic_ops/traffic_ops_golang/routing/routes.go Show resolved Hide resolved

updates per comments

41752f9

ocket8888 reviewed Feb 17, 2020

View reviewed changes

fixed unit tests

3e7f5c8

ocket8888 approved these changes Feb 17, 2020

View reviewed changes

ocket8888 merged commit 18fe13a into apache:master Feb 17, 2020

zrhoffman reviewed Apr 7, 2020

View reviewed changes

zrhoffman mentioned this pull request Sep 24, 2020

SteeringTest fails when LetsEncryptDnsChallengeWatcher hangs #5069

Closed

zrhoffman added this to Done in Self Service Change Integrity Oct 8, 2020

zrhoffman mentioned this pull request Oct 8, 2020

SSL Automate Standard Cert Generation and Application #1906

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TP Delivery Service Generate SSL update, new letsencrypt generate and renew API endpoints #3534

TP Delivery Service Generate SSL update, new letsencrypt generate and renew API endpoints #3534

mattjackson220 commented Apr 29, 2019 •

edited by ocket8888

asfgit commented Apr 29, 2019

JBevillC commented May 21, 2019 •

edited

asfgit commented Jun 21, 2019

asfgit commented Jul 9, 2019

asfgit commented Jul 25, 2019

asfgit commented Jul 29, 2019

asf-ci commented Aug 28, 2019

asf-ci commented Sep 5, 2019

asf-ci commented Sep 5, 2019

asf-ci commented Oct 1, 2019

asf-ci commented Oct 1, 2019

asf-ci commented Oct 4, 2019

asf-ci commented Oct 7, 2019

ocket8888 commented Oct 8, 2019

asf-ci commented Oct 10, 2019

asf-ci commented Oct 14, 2019

asf-ci commented Oct 14, 2019

ocket8888 commented Oct 14, 2019

asf-ci commented Oct 14, 2019

ocket8888 left a comment

ocket8888 left a comment

zrhoffman Apr 7, 2020

TP Delivery Service Generate SSL update, new letsencrypt generate and renew API endpoints #3534

TP Delivery Service Generate SSL update, new letsencrypt generate and renew API endpoints #3534

Conversation

mattjackson220 commented Apr 29, 2019 • edited by ocket8888

What does this PR (Pull Request) do?

Which Traffic Control components are affected by this PR?

What is the best way to verify this PR?

The following criteria are ALL met by this PR

asfgit commented Apr 29, 2019

JBevillC commented May 21, 2019 • edited

asfgit commented Jun 21, 2019

asfgit commented Jul 9, 2019

asfgit commented Jul 25, 2019

asfgit commented Jul 29, 2019

asf-ci commented Aug 28, 2019

asf-ci commented Sep 5, 2019

asf-ci commented Sep 5, 2019

asf-ci commented Oct 1, 2019

asf-ci commented Oct 1, 2019

asf-ci commented Oct 4, 2019

asf-ci commented Oct 7, 2019

ocket8888 commented Oct 8, 2019

asf-ci commented Oct 10, 2019

asf-ci commented Oct 14, 2019

asf-ci commented Oct 14, 2019

ocket8888 commented Oct 14, 2019

asf-ci commented Oct 14, 2019

ocket8888 left a comment

Choose a reason for hiding this comment

ocket8888 left a comment

Choose a reason for hiding this comment

zrhoffman Apr 7, 2020

Choose a reason for hiding this comment

mattjackson220 commented Apr 29, 2019 •

edited by ocket8888

JBevillC commented May 21, 2019 •

edited