WL#14837: consistent TLS connection options

The existing TLS interface both for connection strings and connection
configuration objects is too strict and, at the same time, is not clear
about the role of additional TLS options such as verifying server
certificates are issued or revoked by a CA in a given chain.

Also, there is currently no option to verify the server identity
comparing the hostname and the common name specified by its own
certificate.

This patch introduces some changes to relax the validation requirements
of TLS-related options and to enable implicit or explicit server
identity checks.

Two specific working TLS modes - "VERIFY_CA" and "VERIFY_IDENTITY" are
now available to enable these certificate validation features with
connection strings, and a new secure context option
"checkServerIdentity" is now available and allow applications to verify
the server hostname using the details of the server certificate. This
can happen implicitly using a custom function or using the builtin
Node.js validation function.

https://nodejs.org/docs/v12.0.0/api/tls.html#tls_tls_checkserveridentity_hostname_cert

Author: ruiquelhas

docs/tutorial/Secure_Sessions.md

@@ -166,19 +166,22 @@ Applications are free to use older TLSv1 and TLSv1.1 compatible ciphersuites (li
 
 Non-TLS ciphersuites, including the `MD5`, `SSLv3` and other older sets are not supported and will be ignored if the application wants to use them. If none of the ciphers provided by the application is actually supported by the client, an error will be thrown.
 
-#### Certificate authority validation
+#### Certificate Authority Validation
 
 When creating a connection to the database using TLS, the connector can validate if the server certificate was signed by a certificate authority (CA) and, at the same time, ensure that the certificate was not revoked by that same authority, or any other in a given chain of trust.
 
 Just like with TLS-related core Node.js APIs, an application can provide one or more PEM formatted CA certificates and certificate revocation lists (CRL). The [secure context](https://nodejs.org/docs/v12.0.0/api/tls.html#tls_tls_createsecurecontext_options) uses a list of well-known CAs curated by Mozilla, which are completely replaced by the list of CAs provided by an application. This means that if the certificate was not signed by the root CA, the entire chain of trust down to the signing CA, should be included in the list. The same is true for the certificate revocation lists, because each CA has its own.
 
-Verifying if the server certificate was signed by the root CA can be done by providing the path to the CA file:
+Verifying if the server certificate was signed by the root CA can be done by providing the path to the CA file. When using a connection string, this feature needs to be explicitly enabled by setting value of `ssl-mode` to `VERIFY_CA` or `VERIFY_IDENTITY` (which will also validate the server identity against its own certificate).
+
+> **IMPORTANT**<br />
+> When using a connection string, if the `ssl-mode` option is not set whilst providing a path to a certificate authority file, the verification does not happen and the client will yield a warning message.
 
 ```javascript
 const mysqlx = require('@mysql/xdevapi');
 const path = require('path');
 
-mysqlx.getSession('mysqlx://localhost?ssl-ca=(/path/to/root/ca.pem)')
+mysqlx.getSession('mysqlx://localhost?ssl-mode=VERIFY_CA&ssl-ca=(/path/to/root/ca.pem)')
     .then(session => {
         // the connection succeeds if the server certificate was signed by the root CA
         console.log(session.inspect()); // { host: 'localhost', tls: true }
@@ -228,13 +231,13 @@ const mysqlx = require('@mysql/xdevapi');
 const path = require('path');
 
 // /path/to/chain/ca.pem should contain the entire chain of trust
-mysqlx.getSession('mysqlx://localhost?ssl-ca=(/path/to/chain/ca.pem)')
+mysqlx.getSession('mysqlx://localhost?ssl-mode=VERIFY_CA&ssl-ca=(/path/to/chain/ca.pem)')
     .then(session => {
         // the connection succeeds if the server certificate was signed by the root CA
         console.log(session.inspect()); // { host: 'localhost', tls: true }
     });
 
-mysqlx.getSession(`mysqlx://localhost?ssl-ca=${encodeURIComponent('/path/to/chain/ca.pem')}`)
+mysqlx.getSession(`mysqlx://localhost?ssl-mode=VERIFY_CA&ssl-ca=${encodeURIComponent('/path/to/chain/ca.pem')}`)
     .then(session => {
         // the connection succeeds if the server certificate was signed by the root CA
         console.log(session.inspect()); // { host: 'localhost', tls: true }
@@ -315,13 +318,13 @@ const fs = require('fs')
 const mysqlx = require('@mysql/xdevapi');
 const path = require('path');
 
-mysqlx.getSession('mysqlx://localhost?ssl-ca=(/path/to/root/ca.pem)&ssl-crl=(/path/to/root/crl.pem)')
+mysqlx.getSession('mysqlx://localhost?ssl-mode=VERIFY_CA&ssl-ca=(/path/to/root/ca.pem)&ssl-crl=(/path/to/root/crl.pem)')
     .then(session => {
         // the connection succeeds if the server certificate was not revoked by the root CA
         console.log(session.inspect()); // { host: 'localhost', tls: true }
     });
 
-mysqlx.getSession(`mysqlx://localhost?ssl-ca=${encodeURIComponent('/path/to/root/ca.pem')}&ssl-crl=${encodeURIComponent('/path/to/root/crl.pem')}`)
+mysqlx.getSession(`mysqlx://localhost?ssl-mode=VERIFY_CA&ssl-ca=${encodeURIComponent('/path/to/root/ca.pem')}&ssl-crl=${encodeURIComponent('/path/to/root/crl.pem')}`)
     .then(session => {
         // the connection succeeds if the server certificate was not revoked by the root CA
         console.log(session.inspect()); // { host: 'localhost', tls: true }
@@ -355,6 +358,101 @@ mysqlx.getSession(options)
 > **IMPORTANT**<br />
 > Due to the constraints imposed by connection strings, the fact that all core Node.js TLS-related APIs expect PEM file content as input, and the fact that Node.js and OpenSSL cannot verify CRLs concatenated in a single file, it is strongly recommended that applications always use a connection configuration object to specify one or more PEM files for CAs and CRLs using "fs.readFileSync()".
 
+### Checking the Server Identity
+
+Besides verifying if the server certificate was signed or revoked by a certificate in a given authority chain, it is also possible to additionally verify if the server is the effective owner of the certificate, by comparing the server hostname and the common name specifified by the certificate, according to a specific set of prerequisites. This check is only performed if the certificate first passes all the other checks, such as being issued by a given CA (required).
+
+When using a connection configuration object, this can be done by providing an additional `checkServerIdentity` property as part of the secure context. If its value is `true`, the server identity check will happen using the builtin [`tls.checkServerIdentity()`](https://nodejs.org/docs/v12.0.0/api/tls.html#tls_tls_checkserveridentity_hostname_cert) function, which expects both the server hostname and certificate common name to be exactly the same.
+
+```js
+const fs = require('fs')
+const mysqlx = require('@mysql/xdevapi');
+const path = require('path');
+
+let options = {
+    host: 'localhost',
+    tls: {
+        ca: path.join('/', 'path', 'to', 'ca.pem'),
+        checkServerIdentity: true
+    }
+};
+
+// CN = 'localhost'
+mysqlx.getSession(options)
+    .then(session => {
+        console.log(session.inspect()); // { host: 'localhost', tls: true }
+    });
+
+// CN = 'example.com'
+mysqlx.getSession(options)
+    .catch(err => {
+        console.log(err.message); // Hostname/IP does not match certificate's altnames: Host: localhost. is not cert's CN: example.com
+    });
+```
+
+An application can specify its own set of requirements to determine if the server identity is valid. This can be done by providing a custom `checkServerIdentity()` function. As an example, this can be useful, for instance to allow subdomains.
+
+```js
+const fs = require('fs')
+const mysqlx = require('@mysql/xdevapi');
+const path = require('path');
+
+let options = {
+    host: 'dev.mysql.com',
+    tls: {
+        ca: path.join('/', 'path', 'to', 'ca.pem'),
+        checkServerIdentity (hostname, cert) {
+            // Checks if the domain is the same.
+            if (cert.subject.CN === hostname.substring(hostname.length - cert.subject.CN.length)) {
+                return true;
+            }
+
+            // Instead of being thrown, the error needs to be returned back to the client.
+            return new Error(`"${hostname}" is not a valid subdomain of "${cert.subject.cn}"`);
+        }
+    }
+};
+
+// CN = 'mysql.com'
+mysqlx.getSession(options)
+    .then(session => {
+        console.log(session.inspect()); // { host: 'localhost', tls: true }
+    });
+
+// CN = 'example.com'
+mysqlx.getSession(options)
+    .catch(err => {
+        console.log(err.message); // "dev.mysql.com" is not a valid subdomain of "example.com"
+    });
+```
+
+By default, if the `checkServerIdentity` property is not specified, the check will not be performed.
+
+> **IMPORTANT**<br />
+> Custom `checkServerIdentity()` functions should "return" all errors instead "throwing" them. Further implementation details are availabe in the official Node.js [documentation](https://nodejs.org/docs/v12.0.0/api/tls.html#tls_tls_checkserveridentity_hostname_cert).
+
+When using a connection string, the API is a little bit more restricted. The only possibility is to set the value of the `ssl-mode` option to `VERIFY_IDENTITY` in order to explicitly enable the server identity check using the builtin `tls.checkServerIdentity()` function.
+
+```js
+const fs = require('fs')
+const mysqlx = require('@mysql/xdevapi');
+const path = require('path');
+
+const ca = path.join('path', 'to', 'ca.pem');
+
+// CN = 'localhost'
+mysqlx.getSession(`mysqlx://localhost?ssl-mode=VERIFY_IDENTITY&ssl-ca=${encodeURIComponent(ca)}`)
+    .then(session => {
+        console.log(session.inspect()); // { host: 'localhost', tls: true }
+    });
+
+// CN = 'example.com'
+mysqlx.getSession(`mysqlx://localhost?ssl-mode=VERIFY_IDENTITY&ssl-ca=${encodeURIComponent(ca)}`)
+    .catch(err => {
+        console.log(err.message); // Hostname/IP does not match certificate's altnames: Host: localhost. is not cert's CN: example.com
+    });
+```
+
 ### Authentication Mechanisms
 
 Currently, the MySQL X plugin supports the following authentication methods:

lib/DevAPI/Util/URIParser/parseSecurityOptions.js

@@ -30,7 +30,13 @@
 
 'use strict';
 
+const errors = require('../../../constants/errors');
+const logger = require('../../../logger');
 const parseQueryParameters = require('./parseQueryParameters');
+const util = require('util');
+const warnings = require('../../../constants/warnings');
+
+const log = logger('parser:uri:tls');
 
 module.exports = parse;
 
@@ -55,10 +61,9 @@ function parseItemList (input) {
  * @param {string} input - URI querystring
  * @returns {Object} Security section dictionary for URI properties object.
  */
-function parse (input) {
-    // TODO(Rui): use default agument values on node >= 6.0.0
-    const match = (input || '').trim().match(/^\?([^#]+)/) || [];
-    const params = parseQueryParameters(match[1], { allowDuplicates: false, ignoreCase: ['ssl-mode'] });
+function parse (input = '') {
+    const match = input.trim().match(/^\?([^#]+)/) || [];
+    const params = parseQueryParameters(match[1], { allowDuplicates: true, ignoreCase: ['ssl-mode'] });
 
     const isSecure = params['ssl-mode'] !== 'disabled';
     const options = Object.assign({}, params, { 'ssl-enabled': isSecure });
@@ -76,9 +81,35 @@ function parse (input) {
 
     delete options['ssl-mode'];
 
-    return Object.keys(options).reduce((result, key) => {
+    const tlsOptions = Object.keys(options).reduce((result, key) => {
         const match = key.trim().match(/^ssl-(.+)$/) || key.trim().match(/^tls-(.+)$/) || [];
 
         return !match[1] ? result : Object.assign(result, { [match[1]]: options[key] });
     }, {});
+
+    // If "ssl-mode" is "VERIFY_CA" or "VERIFY_IDENTITY", the path to the
+    // certificate authority PEM file MUST be provided.
+    if ((params['ssl-mode'] === 'verify_ca' || params['ssl-mode'] === 'verify_identity') && typeof tlsOptions.ca === 'undefined') {
+        throw new Error(util.format(errors.MESSAGES.ER_DEVAPI_CERTIFICATE_AUTHORITY_REQUIRED, params['ssl-mode'].toUpperCase()));
+    }
+
+    // If "ssl-mode" is neither "VERIFY_CA" nor "VERIFY_IDENTITY", any path
+    // provided for the certificate authority PEM file or the certificate
+    // revocation list PEM file should be ignored.
+    // Previously, this behaviour was a bit more strict, so we should ensure
+    // the user sees a warning.
+    if (params['ssl-mode'] !== 'verify_ca' && params['ssl-mode'] !== 'verify_identity') {
+        if (typeof params['ssl-ca'] !== 'undefined') {
+            log.warning('ssl-ca', warnings.MESSAGES.WARN_STRICT_CERTIFICATE_VALIDATION);
+        }
+
+        delete tlsOptions.ca;
+        delete tlsOptions.crl;
+    }
+
+    if (params['ssl-mode'] === 'verify_identity') {
+        tlsOptions.checkServerIdentity = true;
+    }
+
+    return tlsOptions;
 }

lib/constants/errors.js

@@ -64,9 +64,9 @@ exports.MESSAGES = {
     ER_DEVAPI_BAD_TLS_CA_PATH: 'The certificate authority (CA) file path is not valid.',
     ER_DEVAPI_BAD_TLS_CRL_PATH: 'The certificate revocation list (CRL) file path is not valid.',
     ER_DEVAPI_BAD_TLS_CIPHERSUITE_LIST: '%s is not a valid TLS ciphersuite list format.',
-    ER_DEVAPI_BAD_TLS_OPTIONS: 'Additional TLS options cannot be specified when TLS is disabled.',
     ER_DEVAPI_BAD_TLS_VERSION: '"%s" is not a valid TLS protocol version. Should be one of %s.',
     ER_DEVAPI_BAD_TLS_VERSION_LIST: '"%s" is not a valid TLS protocol list format.',
+    ER_DEVAPI_CERTIFICATE_AUTHORITY_REQUIRED: '%s requires a certificate authority.',
     ER_DEVAPI_COLLECTION_OPTIONS_NOT_SUPPORTED: 'Your MySQL server does not support the requested operation. Please update to MySQL 8.0.19 or a later version.',
     ER_DEVAPI_CONNECTION_CLOSED: 'This session was closed. Use "mysqlx.getSession()" or "mysqlx.getClient()" to create a new one.',
     ER_DEVAPI_CONNECTION_TIMEOUT: 'Connection attempt to the server was aborted. Timeout of %d ms was exceeded.',

lib/constants/warnings.js

@@ -44,7 +44,8 @@ exports.MESSAGES = {
     WARN_DEPRECATED_TABLE_UPDATE_EXPR_ARGUMENT: 'Passing an expression in Table.update() is a deprecated behavior and will not be supported in future versions. Use TableUpdate.where() instead.',
     WARN_DEPRECATED_TABLE_INSERT_OBJECT_ARGUMENT: 'Passing objects to Table.insert() is a deprecated behavior and will not be supported in future versions.',
     WARN_DEPRECATED_RESULT_GET_AFFECTED_ROWS_COUNT: 'Result.getAffectedRowsCount() is deprecated and will be removed in future versions. Use Result.getAffectedItemsCount() instead.',
-    WARN_DEPRECATED_SET_OFFSET: 'setOffset() is deprecated and will be removed in future versions. Use offset() instead.'
+    WARN_DEPRECATED_SET_OFFSET: 'setOffset() is deprecated and will be removed in future versions. Use offset() instead.',
+    WARN_STRICT_CERTIFICATE_VALIDATION: 'To verify if the server certificate was signed by a given certificate authority "ssl-mode" must be either "VERIFY_CA" or "VERIFY_IDENTITY".'
 };
 
 exports.CODES = {

lib/tls/secure-context.js

@@ -51,7 +51,21 @@ exports.create = function (options = {}) {
     // we use our own. We cannot have a default list of ciphersuites because
     // those are dependent on the TLS version that ends up being used.
     options = Object.assign({}, { versions: tlsVersions.allowed(), ciphersuites: [] }, options);
-    // We need to create the proper securiy context containing any
+    // Certificate identity verification can be performed using a builtin
+    // function or a custom application-provided function.
+    // https://nodejs.org/docs/v12.0.0/api/tls.html#tls_tls_checkserveridentity_hostname_cert
+    if (typeof options.checkServerIdentity === 'undefined') {
+        // If a custom function is not provided, we should use a no-op one that
+        // skips the identity check.
+        options.checkServerIdentity = () => { /* no-op */ };
+    } else if (options.checkServerIdentity === true) {
+        // If certificate identity verification is explicitly enabled, we can
+        // rely on the builtin "checkServerIdentity" function, which simply
+        // ensures the names match.
+        // Deleting the property will ensure the builting function is used.
+        delete options.checkServerIdentity;
+    }
+    // We need to create the proper security context containing any
     // certificates, TLS versions or ciphersuites that have been provided.
     // Applications can also provide any additional option supported by the
     // Node.js security context.
@@ -124,7 +138,6 @@ exports.create = function (options = {}) {
  * @private
  * @param {Object} params
  * @param {TLS} [params.tls] - wether TLS should be enabled or disabled
- * @param {boolean} [params.ssl] - deprecated property to enable or disable TLS
  * @param {DeprecatedSSLOptions} [params.sslOptions] - deprecated property to provide additional TLS options
  * @returns {boolean} Returns true all properties and values are valid.
  * @throws when TLS should be disabled and additional properties are provided,
@@ -133,13 +146,8 @@ exports.create = function (options = {}) {
  * one, or contains one that is not supported, or when the list of ciphersuites
  * does not include any valid one
  */
-exports.validate = function ({ tls, ssl, sslOptions }) {
-    const tlsIsDisabled = (tls && tls.enabled === false) || ssl === false;
+exports.validate = function ({ tls, sslOptions }) {
     const tlsOptions = Object.assign({}, tls, sslOptions);
-    // If TLS is disabled, we should not allow any additional options.
-    if (tlsIsDisabled === true && Object.keys(tlsOptions).some(k => k !== 'enabled')) {
-        throw new Error(errors.MESSAGES.ER_DEVAPI_BAD_TLS_OPTIONS);
-    }
 
     // Validate any CA file path or CA file content.
     if (!isValidPEM({ value: tlsOptions.ca }) && !isValidArray({ value: tlsOptions.ca, validator: isValidPEM }) && !isValidString({ value: tlsOptions.ca })) {

test/functional/extended/connection/ssl-support.js renamed to test/fixtures/scripts/connection/implicit-ssl-mode.js

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2020, 2021, Oracle and/or its affiliates.
+ * Copyright (c) 2021, Oracle and/or its affiliates.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License, version 2.0, as
@@ -30,26 +30,22 @@
 
 'use strict';
 
-/* eslint-env node, mocha */
+const mysqlx = require('../../../../');
 
-const config = require('../../../config');
-const expect = require('chai').expect;
-const mysqlx = require('../../../..');
+const config = JSON.parse(process.env.MYSQLX_CLIENT_CONFIG);
+const baseConfig = Object.assign({}, config, { schema: undefined, socket: undefined });
 
-describe('SSL/TLS support', () => {
-    context('when TLS is it not enabled in the server', () => {
-        // container as defined in docker-compose.yml
-        const baseConfig = { host: 'mysql-with-ssl-disabled', schema: undefined, socket: undefined };
+// additional configuration properties are provided via a JSON command argument
+const additionalConfig = JSON.parse(process.argv[2] || null);
+const scriptConfig = Object.assign({}, baseConfig, additionalConfig);
+const uri = `mysqlx://${scriptConfig.user}:${scriptConfig.password}@${scriptConfig.host}:${scriptConfig.port}?ssl-ca=${encodeURIComponent(scriptConfig.tls.ca)}`;
 
-        it('fails to connect if the client requires it', () => {
-            const secureConfig = Object.assign({}, config, baseConfig, { tls: { enabled: true } });
-            const error = 'The X Plugin version installed in the server does not support TLS. Check https://dev.mysql.com/doc/refman/8.0/en/x-plugin-ssl-connections.html for more details on how to enable secure connections.';
-
-            return mysqlx.getSession(secureConfig)
-                .then(() => expect.fail())
-                .catch(err => {
-                    expect(err.message).to.equal(error);
-                });
-        });
+mysqlx.getSession(uri)
+    .then(session => {
+        return session.close();
+    })
+    .catch(err => {
+        // errors in should be passed as JSON to the parent process via stderr
+        console.error(JSON.stringify({ message: err.message, stack: err.stack }));
+        return process.exit(1);
     });
-});

test/fixtures/tls/server/root/bad-cert.pem

@@ -0,0 +1,17 @@
+-----BEGIN CERTIFICATE-----
+MIICqTCCAZGgAwIBAgIBAjANBgkqhkiG9w0BAQsFADAPMQ0wCwYDVQQDDARyb290
+MB4XDTIxMTIxNjE3NTI0MloXDTIyMTIyNjE3NTI0MlowDzENMAsGA1UEAwwEbmFk
+YTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMgZRWcR1BgW9xUVLGiV
+hcLKN4SvI+Ps1dRGGHD1SiuZ73imBJfQqd/70ZE8kuRe1bQ47Ub/3Ns6pog1SCVR
+jWKnrEmbtO4YQZIWW56kX2RA2S1MHqxzoJAZsOvpLLW4yN3Rm8K7FNnEQ15+sw0f
+Ad+J2z0CvkUIt7TixWEACTUPQP+gklq1/8wMxUVZXbmVbQ+Ojqe3dYMsqxZf3lKQ
+FYEqL3sIGso0GRzA8PMg2qJyAQc6Qn5WvvSsySWtq9LxbHFpJ/Jd9QGlCbe595F4
+RVZICZPSHg4Y7QbcIceVSx6XXL60JyK6zl1tPzPidHFDhW50wYMQcg45uYwitn1c
+SjMCAwEAAaMQMA4wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAtjzu
+XTWvvVHFnJMEdoitsrpHR+SX0tE61ObLhGlrIiFFyZDogWos2LvB//+M7Xen1bNz
+z+zkCP2tkLg2ZOZs2BC+7gR/EM6XvloQ/xvjbMna04ixWNjew+kDhvY6i2FITiAu
+/htOL7lAFCXIzH/ZexkTxzkdXkekbkTl7rMAdcDvCI2axmlQWh3ZVYoQ/JnXK9l/
+v2VbdmjXkGD7I39Pi7a+0yVh+vX0HwqV/DYCjVB9Mi3uGZAv7/gPG6kywKGQ51Qn
+PIRKnZ3x64dgU5nZG5y0mbt7Ha354H9i84yRNbYEsGkJiDkq9LmoVUymvhN3KjJd
+l7GNzz9shTi948arug==
+-----END CERTIFICATE-----