Added CDNi Capacity and Telemetry

CHANGELOG.md

@@ -4,6 +4,9 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 ## [unreleased]
+### Added
+- Added a new Traffic Ops endpoint to `GET` capacity and telemetry data for CDNi integration.
+
 ### Fixed
 - Update traffic_portal dependencies to mitigate `npm audit` issues.
 

docs/source/admin/cdni.rst

@@ -0,0 +1,36 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+.. _cdni_admin:
+
+****************************
+CDNi Administration
+****************************
+
+:abbr:`CDNi (Content Delivery Network Interconnect)` specifications define the standards for interoperability within the :abbr:`CDN (Content Delivery Network)` and open caching ecosystems set forth by the :abbr:`IETF (Internet Engineering Task Force)`. This integration utilizes the :abbr:`APIs (Application Programming Interfaces)` defined by the :abbr:`SVA (Streaming Video Alliance)`.
+
+.. seealso:: For complete details on CDNi and the related API specifications see :rfc:`8006`, :rfc:`8007`, :rfc:`8008`, and the :abbr:`SVA (Streaming Video Alliance)` documents titled `Footprint and Capabilities Interface: Open Caching API`, `Open Caching API Implementation Guidelines`, `Configuration Interface: Part 1 Specification - Overview & Architecture`, `Configuration Interface: Part 2 Specification – CDNi Metadata Model Extensions`, and `Configuration Interface: Part 3 Specification – Publishing Layer APIs`.
+
+In short, these documents describe the :abbr:`CDNi (Content Delivery Network Interconnect)` metadata interface that enables interconnected :abbr:`CDNs (Content Delivery Networks)` to exchange content distribution metadata to enable content acquisition and delivery. These define the interfaces through which a :abbr:`uCDN (Upstream Content Delivery Network)` and a :abbr:`dCDN (Downstream Content Delivery Network)` can communicate configuration and capacity information.
+
+For our use case, it is assumed that :abbr:`ATC (Apache Traffic Control)` is the :abbr:`dCDN (Downstream Content Delivery Network)`.
+
+	..  Note:: This is currently under construction and will be for a while. This document will be updated as new features are supported.
+
+/OC/FCI/advertisement
+=====================
+.. seealso:: :ref:`to-api-oc-fci-advertisement`
+
+The advertisement response is unique for the :abbr:`uCDN (Upstream Content Delivery Network)` and contains the complete footprint and capabilities information structure the :abbr:`dCDN (Downstream Content Delivery Network)` wants to expose. This endpoint will return an array of generic :abbr:`FCI (Footprint and Capabilities Advertisement Interface)` base objects, including type, value, and footprint for each. Currently supported base object types are `FCI.Capacitiy` and `FCI.Telemetry` but these will be expanded in the future.

docs/source/admin/index.rst

@@ -52,3 +52,4 @@ Once everything is installed, you will need to configure the servers to talk to
 	t3c/index.rst
 	traffic_vault.rst
 	quick_howto/index.rst
+	cdni.rst

docs/source/admin/traffic_ops.rst

@@ -494,6 +494,13 @@ This file deals with the configuration parameters of running Traffic Ops itself.
 
 	.. versionadded:: 6.1
 
+:cdni: This is an optional section of configurations for :abbr:`CDNi (Content Delivery Network Interconnect)` operations.
+
+	.. versionadded:: 6.2
+
+	:dcdn_id: A string representing this :abbr:`CDN (Content Delivery Network)` to be used in the :abbr:`JWT (JSON Web Token)` and subsequently in :abbr:`CDNi (Content Delivery Network Interconnect)` operations.
+	:jwt_decoding_secret: A string used to decode the :abbr:`JWT (JSON Web Token)` to get information for :abbr:`CDNi (Content Delivery Network Interconnect)` operations.
+
 
 Example cdn.conf
 ''''''''''''''''

docs/source/api/v4/oc_fci_advertisement.rst

@@ -0,0 +1,132 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+..     http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+.. _to-api-oc-fci-advertisement:
+
+************************
+``OC/FCI/advertisement``
+************************
+
+``GET``
+=======
+Returns the complete footprint and capabilities information structure the :abbr:`dCDN (Downstream Content Delivery Network)` wants to expose to a given :abbr:`uCDN (Upstream Content Delivery Network)`.
+
+:Auth. Required: No
+:Roles Required: "admin" or "operations"
+:Permissions Required: CDNI:READ
+:Response Type:  Array
+:Headers Required: "Authorization"
+
+Request Structure
+-----------------
+.. table:: Request Required Headers
+
+	+-----------------+------------------------------------------------------------------------------------------------------------------------------+
+	|    Name         | Description                                                                                                                  |
+	+=================+==============================================================================================================================+
+	|  Authorization  | A :abbr:`JWT (JSON Web Token)` provided by the :abbr:`dCDN (Downstream Content Delivery Network)` to identify the            |
+	|                 | :abbr:`uCDN (Upstream Content Delivery Network)`. This token must include the following claims:                              |
+	|                 |                                                                                                                              |
+	|                 | - ``iss`` Issuer claim as a string key for the :abbr:`uCDN (Upstream Content Delivery Network)`                              |
+	|                 | - ``aud`` Audience claim as a string key for the :abbr:`dCDN (Downstream Content Delivery Network)`                          |
+	|                 | - ``exp`` Expiration claim as the expiration date as a Unix epoch timestamp (in seconds)                                     |
+	+-----------------+------------------------------------------------------------------------------------------------------------------------------+
+
+Response Structure
+------------------
+:capabilities:     An array of generic :abbr:`FCI (Footprint and Capabilities Advertisement Interface)` base objects.
+:capability-type:  A string of the type of base object.
+:capability-value: An array of the value for the base object.
+:footprints:       An array of footprints impacted by this generic base object.
+
+	.. note:: These are meant to be generic and therefore there is not much information in these documents. For further information please see :rfc:`8006`, :rfc:`8007`, :rfc:`8008`, and the :abbr:`SVA (Streaming Video Alliance)` documents titled `Footprint and Capabilities Interface: Open Caching API`, `Open Caching API Implementation Guidelines`, `Configuration Interface: Part 1 Specification - Overview & Architecture`, `Configuration Interface: Part 2 Specification – CDNi Metadata Model Extensions`, and `Configuration Interface: Part 3 Specification – Publishing Layer APIs`.
+
+.. code-block:: json
+	:caption: Example /OC/FCI/advertisement Response
+
+	{
+		"capabilities": [
+			{
+				"capability-type": "FCI.CapacityLimits",
+				"capability-value": [
+					{
+						"total-limits": [
+							{
+								"limit-type": "egress",
+								"maximum-hard": 5000,
+								"maximum-soft": 2500,
+								"telemetry-source": {
+									"id": "capacity_metrics",
+									"metric": "capacity"
+								}
+							}
+						],
+						"host-limits": [
+							{
+								"host": "example.com",
+								"limits": [
+									{
+										"limit-type": "requests",
+										"maximum-hard": 100,
+										"maximum-soft": 50,
+										"telemetry-source": {
+											"id": "request_metrics",
+											"metric": "requests"
+										}
+									}
+								]
+							}
+						]
+					}
+				],
+				"footprints": [
+					{
+						"footprint-type": "countrycode",
+						"footprint-value": [
+							"us"
+						]
+					}
+				]
+			},
+			{
+				"capability-type": "FCI.Telemetry",
+				"capability-value": {
+					"sources": [
+						{
+							"id": "capacity_metrics",
+							"type": "generic",
+							"metrics": [
+								{
+									"name": "capacity",
+									"time-granularity": 0,
+									"data-percentile": 50,
+									"latency": 0
+								}
+							]
+						}
+					]
+				},
+				"footprints": [
+					{
+						"footprint-type": "countrycode",
+						"footprint-value": [
+							"us"
+						]
+					}
+				]
+			}
+		]
+	}
+

traffic_ops/app/conf/cdn.conf

@@ -98,5 +98,9 @@
         "organization" : "",
         "country" : "",
         "state" : ""
+    },
+    "cdni" : {
+        "dcdn_id" : "",
+        "jwt_decoding_secret" : ""
     }
 }

traffic_ops/app/db/migrations/2022011112591400_added_cdni_capacity_table.down.sql

@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+DROP TABLE IF EXISTS cdni_total_limits;
+DROP TABLE IF EXISTS cdni_host_limits;
+DROP TABLE IF EXISTS cdni_telemetry_metrics;
+DROP TABLE IF EXISTS cdni_telemetry;
+DROP TABLE IF EXISTS cdni_footprints;
+DROP TABLE IF EXISTS cdni_capabilities;

traffic_ops/app/db/migrations/2022011112591400_added_cdni_capacity_table.up.sql

@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+CREATE TABLE IF NOT EXISTS cdni_capabilities (
+                                                 id bigserial NOT NULL,
+                                                 type text NOT NULL,
+                                                 ucdn text NOT NULL,
+                                                 last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_capabilities PRIMARY KEY (id)
+);
+
+CREATE TABLE IF NOT EXISTS cdni_footprints (
+                                               id bigserial NOT NULL,
+                                               footprint_type text NOT NULL,
+                                               footprint_value text[] NOT NULL,
+                                               ucdn text NOT NULL,
+                                               capability_id bigint NOT NULL,
+                                               last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_footprints PRIMARY KEY (id),
+    CONSTRAINT fk_cdni_footprint_capabilities FOREIGN KEY (capability_id) REFERENCES cdni_capabilities(id) ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE IF NOT EXISTS cdni_telemetry (
+                                        id text NOT NULL,
+                                        type text NOT NULL,
+                                        capability_id bigint NOT NULL,
+                                        last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_telemetry PRIMARY KEY (id),
+    CONSTRAINT fk_cdni_telemetry_capabilities FOREIGN KEY (capability_id) REFERENCES cdni_capabilities(id) ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE IF NOT EXISTS cdni_telemetry_metrics (
+                                        name text NOT NULL,
+                                        time_granularity bigint NOT NULL,
+                                        data_percentile bigint NOT NULL,
+                                        latency int NOT NULL,
+                                        telemetry_id text NOT NULL,
+                                        last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_telemetry_metrics PRIMARY KEY (name),
+    CONSTRAINT fk_cdni_telemetry_metrics_telemetry FOREIGN KEY (telemetry_id) REFERENCES cdni_telemetry(id) ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE IF NOT EXISTS cdni_total_limits (
+                                        limit_type text NOT NULL,
+                                        maximum_hard bigint NOT NULL,
+                                        maximum_soft bigint NOT NULL,
+                                        telemetry_id text NOT NULL,
+                                        telemetry_metric text NOT NULL,
+                                        capability_id bigint NOT NULL,
+                                        last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_total_limits PRIMARY KEY (capability_id, telemetry_id),
+    CONSTRAINT fk_cdni_total_limits_telemetry FOREIGN KEY (telemetry_id) REFERENCES cdni_telemetry(id) ON UPDATE CASCADE ON DELETE CASCADE,
+    CONSTRAINT fk_cdni_total_limits_capabilities FOREIGN KEY (capability_id) REFERENCES cdni_capabilities(id) ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE IF NOT EXISTS cdni_host_limits (
+                                        limit_type text NOT NULL,
+                                        maximum_hard bigint NOT NULL,
+                                        maximum_soft bigint NOT NULL,
+                                        telemetry_id text NOT NULL,
+                                        telemetry_metric text NOT NULL,
+                                        capability_id bigint NOT NULL,
+                                        host text NOT NULL,
+                                        last_updated timestamp with time zone DEFAULT now() NOT NULL,
+    CONSTRAINT pk_cdni_host_limits PRIMARY KEY (capability_id, telemetry_id, host),
+    CONSTRAINT fk_cdni_host_limits_telemetry FOREIGN KEY (telemetry_id) REFERENCES cdni_telemetry(id) ON UPDATE CASCADE ON DELETE CASCADE,
+    CONSTRAINT fk_cdni_total_limits_capabilities FOREIGN KEY (capability_id) REFERENCES cdni_capabilities(id) ON UPDATE CASCADE ON DELETE CASCADE
+);