Acme auto renew

CHANGELOG.md

@@ -20,6 +20,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Traffic Ops: Added validation to ensure that the `weight` parameter of `parent.config` is a float
 - Added license files to the RPMs
 - Added ACME certificate renewals and ACME account registration using external account binding
+- Added functionality to automatically renew ACME certificates.
 
 ### Fixed
 - [#5445](https://github.com/apache/trafficcontrol/issues/5445) - When updating a registered user, ignore updates on registration_sent field.

docs/source/admin/traffic_ops.rst

@@ -316,6 +316,13 @@ This file deals with the configuration parameters of running Traffic Ops itself.
 	:kid:           The key ID provided by the :abbr:`ACME (Automatic Certificate Management Environment)` provider for ref:`external_account_binding`.
 	:hmac_encoded:  The :abbr:`HMAC (Hashed Message Authentication Code)` key provided by the :abbr:`ACME (Automatic Certificate Management Environment)` provider for ref:`external_account_binding`. This should be in Base64 URL encoded.
 
+:acme_renewal: This object contains the information for the automatic renewal script for certificates.
+
+	.. versionadded:: 5.1
+
+	:renew_days_before_expiration: Set the number of days before expiration date to renew certificates.
+	:summary_email: The email address to use for summarizing certificate expiration and renewal status. If it is blank, no email will be sent.
+
 :geniso: This object contains configuration options for system ISO generation.
 
 	:iso_root_path: Sets the filesystem path to the root of the ISO generation directory. For default installations, this should usually be set to :file:`/opt/traffic_ops/app/public`.
@@ -350,8 +357,16 @@ This file deals with the configuration parameters of running Traffic Ops itself.
 
 	:user_email: A required email address to create an account with Let's Encrypt or to receive expiration updates. If this is not included then `rate limits <https://letsencrypt.org/docs/rate-limits>`_ may apply for the number of certificates.
 	:send_expiration_email: A boolean option to send email summarizing certificate expiration status
+
+		.. deprecated:: 5.1
+			Future versions of Traffic Ops will not support this legacy configuration option, see acme_renewal: { summary_email: <string> } instead.
+
 	:convert_self_signed: A boolean option to convert self signed to Let's Encrypt certificates as they expire. This only works for certificates labeled as Self Signed in the Certificate Source field.
 	:renew_days_before_expiration: Set the number of days before expiration date to renew certificates.
+
+		.. deprecated:: 5.1
+			Future versions of Traffic Ops will not support this legacy configuration option, see acme_renewal: { renew_days_before_expiration: <int> } instead.
+
 	:environment: This specifies which Let's Encrypt environment to use: 'staging' or 'production'. It defaults to 'production'.
 
 :portal: This section provides information regarding a connected UI with which users interact, so that emails can include links to it.

docs/source/admin/traffic_router.rst

@@ -833,9 +833,27 @@ Let's Encrypt can be set up through :ref:`cdn.conf` by updating the following fi
 	+------------------------------+---------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 	| renew_days_before_expiration | int     | No       | Number of days before expiration date to renew certificates                                                                                                            |
 	+------------------------------+---------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-	| environment                  | string  | No       | Let's Encrypt environment to use.  Options are 'staging' or 'production'. Defaults to 'production'                                                                     |
+	| environment                  | string  | No       | Let's Encrypt environment to use.  Options are 'staging' or 'production'. Defaults to 'production'.                                                                    |
 	+------------------------------+---------+----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
 
+Automatic Certificate Renewal
+-----------------------------
+If desired, an automated certificate renewal script is located at :file:`/traffic_ops/etc/cron.d/autorenew_certs`. This job is setup to be run, but the file must be updated with the username and password for Traffic Ops in order to be run.  In :ref:`cdn.conf` the following fields can be defined in order to alter the number of days in advance to renew and send a summary email after renewal.
+
+.. note:: In order for this to work, the AuthType field for the certificate must match the ACME provider in the :ref:`cdn.conf`.
+
+.. important:: After the automatic renewal script has run, a queue and snapshot must be run manually in order for the certificates to be used.
+
+.. table:: Fields to update to run the automatic renewal script under `acme_renewal`:
+
+	+------------------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------------+
+	| Name                         | Type    | Required | Description                                                                                                                |
+	+==============================+=========+==========+============================================================================================================================+
+	| summary_email                | boolean | No       | The email address to use for summarizing certificate expiration and renewal status. If it is blank, no email will be sent. |
+	+------------------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------------+
+	| renew_days_before_expiration | int     | No       | Number of days before expiration date to renew certificates. Default is 30 days.                                           |
+	+------------------------------+---------+----------+----------------------------------------------------------------------------------------------------------------------------+
+
 .. table:: Fields to update for sending emails under `smtp`
 
 	+------------+------------------+----------------------------------------------------------------------+

docs/source/api/v1/deliveryservices_sslkeys_generate_letsencrypt.rst

@@ -62,7 +62,7 @@ Response Structure
 
 	{ "alerts": [{
 		"level": "success",
-		"text": "Beginning async call to Let's Encrypt for ds-01.  This may take a few minutes."
+		"text": "Beginning async call to Let's Encrypt for ds-01. This may take a few minutes."
 	}]}
 
 .. [#needOne] Either the ``key`` or the ``deliveryservice`` field must be provided. If both are provided, then they must match.

docs/source/api/v1/letsencrypt_autorenew.rst

@@ -36,54 +36,20 @@ No parameters available
 
 Response Structure
 ------------------
-:LetsEncryptExpirations: A list of objects with information regarding certificate expiration for all Let's Encrypt certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:SelfSignedExpirations:  A list of objects with information regarding certificate expiration for all self signed certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:OtherExpirations:       A list of objects with information regarding certificate expiration for all other certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
 
 .. code-block:: http
 	:caption: Response Example
 
 	HTTP/1.1 200 OK
 	Content-Type: application/json
 
-	{ "response": {
-		"LetsEncryptExpirations": [
-			{
-				"XmlId":"demo2",
-				"Version":1,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Lets Encrypt",
-				"Error":null
-			}
-		],
-		"SelfSignedExpirations": [
-			{
-				"XmlId":"demo1",
-				"Version":3,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Self Signed",
-				"Error":null
-			}
-		],
-		"OtherExpirations":null
-	}}
+	{ "alerts": [
+		{
+			"text": "This endpoint is deprecated, please use letsencrypt/autorenew instead",
+			"level": "warning"
+		},
+		{
+			"text": "Beginning async call to renew certificates. This may take a few minutes.",
+			"level": "success"
+		}
+	]}

docs/source/api/v2/deliveryservices_sslkeys_generate_letsencrypt.rst

@@ -60,7 +60,7 @@ Response Structure
 
 	{ "alerts": [{
 		"level": "success",
-		"text": "Beginning async call to Let's Encrypt for ds-01.  This may take a few minutes."
+		"text": "Beginning async call to Let's Encrypt for ds-01. This may take a few minutes."
 	}]}
 
 .. [#needOne] Either the ``key`` or the ``deliveryservice`` field must be provided. If both are provided, then they must match.

docs/source/api/v2/letsencrypt_autorenew.rst

@@ -34,54 +34,20 @@ No parameters available
 
 Response Structure
 ------------------
-:LetsEncryptExpirations: A list of objects with information regarding certificate expiration for all Let's Encrypt certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:SelfSignedExpirations:  A list of objects with information regarding certificate expiration for all self signed certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:OtherExpirations:       A list of objects with information regarding certificate expiration for all other certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
 
 .. code-block:: http
 	:caption: Response Example
 
 	HTTP/1.1 200 OK
 	Content-Type: application/json
 
-	{ "response": {
-		"LetsEncryptExpirations": [
-			{
-				"XmlId":"demo2",
-				"Version":1,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Lets Encrypt",
-				"Error":null
-			}
-		],
-		"SelfSignedExpirations": [
-			{
-				"XmlId":"demo1",
-				"Version":3,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Self Signed",
-				"Error":null
-			}
-		],
-		"OtherExpirations":null
-	}}
+	{ "alerts": [
+		{
+			"text": "This endpoint is deprecated, please use letsencrypt/autorenew instead",
+			"level": "warning"
+		},
+		{
+			"text": "Beginning async call to renew certificates. This may take a few minutes.",
+			"level": "success"
+		}
+	]}

docs/source/api/v3/deliveryservices_sslkeys_generate_letsencrypt.rst

@@ -60,7 +60,7 @@ Response Structure
 
 	{ "alerts": [{
 		"level": "success",
-		"text": "Beginning async call to Let's Encrypt for ds-01.  This may take a few minutes."
+		"text": "Beginning async call to Let's Encrypt for ds-01. This may take a few minutes."
 	}]}
 
 .. [#needOne] Either the ``key`` or the ``deliveryservice`` field must be provided. If both are provided, then they must match.

docs/source/api/v3/letsencrypt_autorenew.rst

@@ -34,54 +34,20 @@ No parameters available
 
 Response Structure
 ------------------
-:LetsEncryptExpirations: A list of objects with information regarding certificate expiration for all Let's Encrypt certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:SelfSignedExpirations:  A list of objects with information regarding certificate expiration for all self signed certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
-
-:OtherExpirations:       A list of objects with information regarding certificate expiration for all other certificates
-
-	:XmlId:       The :term:`Delivery Service`'s uniquely identifying :ref:`ds-xmlid`
-	:Version:     An integer that defines the "version" of the key - which may be thought of as the sequential generation; that is, the higher the number the more recent the key
-	:Expiration:  The expiration date of the certificate for the :term:`Delivery Service` in :rfc:`3339` format
-	:AuthType:    The authority type of the certificate for the :term:`Delivery Service`
-	:Error:       Any errors received in the renewal process
 
 .. code-block:: http
 	:caption: Response Example
 
 	HTTP/1.1 200 OK
 	Content-Type: application/json
 
-	{ "response": {
-		"LetsEncryptExpirations": [
-			{
-				"XmlId":"demo2",
-				"Version":1,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Lets Encrypt",
-				"Error":null
-			}
-		],
-		"SelfSignedExpirations": [
-			{
-				"XmlId":"demo1",
-				"Version":3,
-				"Expiration":"2020-08-18T13:53:06Z",
-				"AuthType":"Self Signed",
-				"Error":null
-			}
-		],
-		"OtherExpirations":null
-	}}
+	{ "alerts": [
+		{
+			"text": "This endpoint is deprecated, please use letsencrypt/autorenew instead",
+			"level": "warning"
+		},
+		{
+			"text": "Beginning async call to renew certificates. This may take a few minutes.",
+			"level": "success"
+		}
+	]}

docs/source/api/v4/acme_autorenew.rst

@@ -0,0 +1,49 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+.. _to-api-acnme-autorenew:
+
+******************
+``acme_autorenew``
+******************
+
+``POST``
+========
+Generates SSL certificates and private keys for all :term:`Delivery Services` that have certificates expiring within the configured time. This uses:abbr:`ACME (Automatic Certificate Management Environment)` or Let's Encrypt depending on the certificate.
+
+:Auth. Required: Yes
+:Roles Required: "admin" or "operations"
+:Response Type:  ``undefined``
+
+Request Structure
+-----------------
+No parameters available
+
+
+Response Structure
+------------------
+
+.. code-block:: http
+	:caption: Response Example
+
+	HTTP/1.1 202 Accepted
+	Content-Type: application/json
+
+	{ "alerts": [
+		{
+			"text": "Beginning async call to renew certificates. This may take a few minutes.",
+			"level": "success"
+		}
+	]}

docs/source/api/v4/deliveryservices_sslkeys_generate_letsencrypt.rst

@@ -60,7 +60,7 @@ Response Structure
 
 	{ "alerts": [{
 		"level": "success",
-		"text": "Beginning async call to Let's Encrypt for ds-01.  This may take a few minutes."
+		"text": "Beginning async call to Let's Encrypt for ds-01. This may take a few minutes."
 	}]}
 
 .. [#needOne] Either the ``key`` or the ``deliveryservice`` field must be provided. If both are provided, then they must match.