Remove support for PKI and PKIz tokens

This is the first step of several to remove PKI token support in
keystone. A large issue in removing PKI support is support for the
revocation list must be maintained.

This patch removes support for the token format, it's surrounding tests
and examples that are generated. Additionally, some wording has been
changed around the CLI and config options to make the distinction
between keys and certs used for PKI tokens and those used for getting
the revocation list (a list of tokens that are revoked, which is signed).

Future patches will:

- Remove the keystone-manage commands for generating certs

- Modify the revocation list (at /auth/tokens/OS-PKI/revoked) to return
a 403 if pki is not configured (instead of raising a 500). We cannot
remove the API as that would break an API contract.

- Options to configure PKI will be marked as deprecated

- If PKI is configured a normal signed list will be returned (same
behavior as today)

- Follow up patch to keystonemiddleware will make sure auth_token does
not rely on the revocation api at all.

Related-Bug: 1626778
Related-Bug: 1626779

Co-Authored-By: Boris Bobrov <bbobrov@mirantis.com>
bp removed-as-of-ocata
Change-Id: Icf1ebced44a675c88fb66a6c0431208ff5181574

doc/source/configuration.rst

@@ -437,8 +437,8 @@ configuring the following property.
   :class:`keystone.token.providers.uuid.Provider`
 
 
-UUID, PKI, PKIZ, or Fernet?
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+UUID or Fernet?
+^^^^^^^^^^^^^^^
 
 Each token format uses different technologies to achieve various performance,
 scaling and architectural requirements.
@@ -449,29 +449,6 @@ transport and are thus URL-friendly. They must be persisted by the identity
 service in order to be later validated. Revoking them is simply a matter of
 deleting them from the token persistence backend.
 
-Both PKI and PKIZ tokens contain JSON payloads that represent the entire token
-validation response that would normally be retrieved from keystone. The payload
-is then signed using `Cryptographic Message Syntax (CMS)
-<http://en.wikipedia.org/wiki/Cryptographic_Message_Syntax>`_. The combination
-of CMS and the exhaustive payload allows PKI and PKIZ tokens to be verified
-offline using keystone's public signing key. The only reason for them to be
-persisted by the identity service is to later build token revocation *lists*
-(explicit lists of tokens that have been revoked), otherwise they are
-theoretically ephemeral when supported by token revocation *events* (which
-describe invalidated tokens rather than enumerate them). PKIZ tokens add zlib
-compression after signing to achieve a smaller overall token size. To make them
-URL-friendly, PKI tokens are base64 encoded and then arbitrarily manipulated to
-replace unsafe characters with safe ones whereas PKIZ tokens use conventional
-base64url encoding. Due to the size of the payload and the overhead incurred by
-the CMS format, both PKI and PKIZ tokens may be too long to fit in either
-headers or URLs if they contain extensive service catalogs or other additional
-attributes. Some third-party applications such as web servers and clients may
-need to be recompiled from source to customize the limitations that PKI and
-PKIZ tokens would otherwise exceed). Both PKI and PKIZ tokens require signing
-certificates which may be created using ``keystone-manage pki_setup`` for
-demonstration purposes (this is not recommended for production deployments: use
-certificates issued by an trusted CA instead).
-
 Fernet tokens contain a limited amount of identity and authorization data in a
 `MessagePacked <http://msgpack.org/>`_ payload. The payload is then wrapped as
 a `Fernet <https://github.com/fernet/spec>`_ message for transport, where
@@ -481,7 +458,7 @@ established using ``keystone-manage fernet_setup`` and periodically rotated
 using ``keystone-manage fernet_rotate``.
 
 .. WARNING::
-    UUID, PKI, PKIZ, and Fernet tokens are all bearer tokens, meaning that they
+    UUID and Fernet tokens are both bearer tokens, meaning that they
     must be protected from unnecessary disclosure to prevent unauthorized
     access.
 
@@ -1338,7 +1315,7 @@ through the normal REST API. At the moment, the following calls are supported:
 * ``mapping_engine``: Test your federation mapping rules.
 * ``mapping_populate``: Prepare domain-specific LDAP backend
 * ``mapping_purge``: Purge the identity mapping table.
-* ``pki_setup``: Initialize the certificates used to sign tokens.
+* ``pki_setup``: Initialize the certificates used to sign revocation lists.
 * ``saml_idp_metadata``: Generate identity provider metadata.
 * ``token_flush``: Purge expired tokens
 

doc/source/man/keystone-manage.rst

@@ -53,7 +53,7 @@ Available commands:
 * ``mapping_populate``: Prepare domain-specific LDAP backend.
 * ``mapping_purge``: Purge the identity mapping table.
 * ``mapping_engine``: Test your federation mapping rules.
-* ``pki_setup``: Initialize the certificates used to sign tokens. **deprecated**
+* ``pki_setup``: Initialize the certificates used to sign revocation lists. **deprecated**
 * ``saml_idp_metadata``: Generate identity provider metadata.
 * ``token_flush``: Purge expired tokens.
 

doc/source/token-support-matrix.ini

@@ -56,8 +56,6 @@
 # drivers should maintain their own equivalent document, and merge it with this
 # when their code merges into core.
 driver-impl-uuid=UUID tokens
-driver-impl-pki=PKI tokens
-driver-impl-pkiz=PKIZ tokens
 driver-impl-fernet=Fernet tokens
 
 [operation.create_unscoped_token]
@@ -68,8 +66,6 @@ notes=All token providers must be capable of issuing tokens without an explicit
 cli=openstack --os-username=<username> --os-user-domain-name=<domain>
   --os-password=<password> token issue
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [operation.create_project_scoped_token]
@@ -80,8 +76,6 @@ cli=openstack --os-username=<username> --os-user-domain-name=<domain>
   --os-password=<password> --os-project-name=<project>
   --os-project-domain-name=<domain> token issue
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [operation.create_domain_scoped_token]
@@ -92,8 +86,6 @@ notes=Domain-scoped tokens are not required for all use cases, and for some use
 cli=openstack --os-username=<username> --os-user-domain-name=<domain>
   --os-password=<password> --os-domain-name=<domain> token issue
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [operation.create_trust_scoped_token]
@@ -104,8 +96,6 @@ notes=Tokens scoped to a trust convey only the user impersonation and
 cli=openstack --os-username=<username> --os-user-domain-name=<domain>
   --os-password=<password> --os-trust-id=<trust> token issue
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [operation.create_token_using_oauth]
@@ -114,8 +104,6 @@ status=optional
 notes=OAuth access tokens can be exchanged for keystone tokens.
 cli=
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [operation.create_token_with_bind]
@@ -125,8 +113,6 @@ notes=Tokens can express a binding to an additional authentication method, such
   as kerberos or x509.
 cli=
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=missing
 
 [operation.revoke_token]
@@ -138,8 +124,6 @@ notes=Tokens may be individually revoked, such as when a user logs out of
   revoked token was previously used to create additional tokens).
 cli=openstack token revoke
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [feature.online_validation]
@@ -149,8 +133,6 @@ notes=Keystone must be able to validate the tokens that it issues when
   presented with a token that it previously issued.
 cli=
 driver-impl-uuid=complete
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=complete
 
 [feature.offline_validation]
@@ -161,8 +143,6 @@ notes=Services using Keystone for authentication may want to validate tokens
   performance and scalability.
 cli=
 driver-impl-uuid=missing
-driver-impl-pki=complete
-driver-impl-pkiz=complete
 driver-impl-fernet=missing
 
 [feature.non_persistent]
@@ -174,6 +154,4 @@ notes=If a token format does not require persistence (such as to a SQL
   operations such as `keystone-manage token_flush`.
 cli=
 driver-impl-uuid=missing
-driver-impl-pki=partial
-driver-impl-pkiz=partial
 driver-impl-fernet=complete