tls-alpn-01 does not work with TLS <= 1.2 if client certificates are required

[mkauf]

There is a limitation when using the "tls-alpn-01" challenge type: mod_md cannot get a certificate from Let's Encrypt if client certificates are required and if TLS <= 1.2 is used (SSLVerifyClient require and SSLProtocol TLSv1.2). The TLS handshake initiated by the Let's Encrypt server fails, because the Let's Encrypt server does not have a client certificate.

This message is logged:

(22)Invalid argument: AH10056: processing www.example.com: remote error: tls: handshake failure

I think that the Let's Encrypt server actually does get the certificate containing the challenge response, but it does not accept it because the handshake fails. Maybe the Let's Encrypt server software could be improved.

Another approach would be to not require a client certificate when the "acme-tls/1" protocol has been negotiated. This Apache patch works for me:

diff --git a/modules/ssl/ssl_engine_kernel.c b/modules/ssl/ssl_engine_kernel.c
index d3f00c2df4..39e3c21806 100644
--- a/modules/ssl/ssl_engine_kernel.c
+++ b/modules/ssl/ssl_engine_kernel.c
@@ -2662,6 +2662,10 @@ int ssl_callback_alpn_select(SSL *ssl,
                     return SSL_TLSEXT_ERR_ALERT_FATAL;
                 }
             }
+
+            if (0 == strcmp("acme-tls/1", proposed)) {
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
+            }
         }
     }
 

Note that there is this "paranoid" check in ssl_io_filter_handshake() that checks again (after the TLS handshake) whether there is a client certificate:

/*
 * Make really sure that when a peer certificate
 * is required we really got one... (be paranoid)
 */
if ((sc->server->auth.verify_mode == SSL_CVERIFY_REQUIRE) &&
    !sslconn->client_cert)
{
    ap_log_cerror(APLOG_MARK, APLOG_INFO, 0, c, APLOGNO(02011)
                  "No acceptable peer certificate available");

    ssl_filter_io_shutdown(filter_ctx, c, 1);
    return APR_ECONNABORTED;
}

[icing]

Urgs, nasty things. I do agree that a SSL Verify settings should not apply here. However I am not sure how to best achieve that. Checking for a fixed string can only be a proof of concept here...

There have been issues with differences in TLS versions and when verify callbacks actually happen, so this code in mod_ssl is easy to break.

Since you have the setup: is the code lines 2355+ in ssl_engine_kernel.c too late to disable client certs? Because there it would fit, I guess.

[mkauf]

Since you have the setup: is the code lines 2355+ in ssl_engine_kernel.c too late to disable client certs? Because there it would fit, I guess.

It's not too late, this works:

diff --git a/modules/ssl/ssl_engine_kernel.c b/modules/ssl/ssl_engine_kernel.c
index d3f00c2df4..12f36e2779 100644
--- a/modules/ssl/ssl_engine_kernel.c
+++ b/modules/ssl/ssl_engine_kernel.c
@@ -2354,6 +2354,7 @@ static apr_status_t init_vhost(conn_rec *c, SSL *ssl)
                 if (set_challenge_creds(c, servername, ssl, cert, key) != APR_SUCCESS) {
                     return APR_EGENERAL;
                 }
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
             }
             else {
                 ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, APLOGNO(02044)
@@ -2661,6 +2662,7 @@ int ssl_callback_alpn_select(SSL *ssl,
                 if (set_challenge_creds(c, servername, ssl, cert, key) != APR_SUCCESS) {
                     return SSL_TLSEXT_ERR_ALERT_FATAL;
                 }
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
             }
         }
     }

Additionally we could call SSL_set_verify_result(ssl, X509_V_OK); to clear any existing verification result, but I think that none should exist at this point.

[tlhackque]

I also use client verification for some vhosts.
The code in mod_ssl has changed. The following looks right for 2.4.41, but other issues are keeping me from getting that far...

In any case, a solution needs to be checked in to mod_ssl. I looked at trunk; the code is unchanged (2357 maps to ~line 2401 at this writing). Has anyone filed a an issue about this with mod_ssl? (Or taken ownership?)

I haven't followed the code in detail, but one thing that occurs to me is that it's important that these "challenge" connections aren't cached (whether they succed or fail) or looked-up in the caches. Both be sure that the right certificates are sent, and to make sure that on the non-challenge connections, verification works as configured...

--- modules/ssl/ssl_engine_kernel.c~    2020-02-08 13:25:39.000000000 -0500
+++ modules/ssl/ssl_engine_kernel.c     2020-02-09 21:33:44.000000000 -0500
@@ -2357,10 +2357,11 @@
                  * that need to be answered with a special certificate and will
                  * otherwise not answer any requests. */
                 if (set_challenge_creds(c, servername, ssl, cert, key) != APR_SUCCESS) {
                     return APR_EGENERAL;
                 }
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
             }
             else {
                 ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, APLOGNO(02044)
                               "No matching SSL virtual host for servername "
                               "%s found (using default/first virtual host)",

[icing]

I applied the change as tested by @mkauf in r1873888 on Apache subversion trunk and will propose for backport into the 2.4.x branch.

[tlhackque]

Great. Many thanks. FWIW, I found the second case in 2.4, my backport now looks like this:

--- modules/ssl/ssl_engine_kernel.c.cv-base     2020-02-08 13:25:39.000000000 -0500
+++ modules/ssl/ssl_engine_kernel.c     2020-02-11 12:20:58.000000000 -0500
@@ -2357,10 +2357,11 @@
                  * that need to be answered with a special certificate and will
                  * otherwise not answer any requests. */
                 if (set_challenge_creds(c, servername, ssl, cert, key) != APR_SUCCESS) {
                     return APR_EGENERAL;
                 }
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
             }
             else {
                 ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, APLOGNO(02044)
                               "No matching SSL virtual host for servername "
                               "%s found (using default/first virtual host)",
@@ -2664,10 +2665,11 @@

             if (ssl_is_challenge(c, servername, &cert, &key)) {
                 if (set_challenge_creds(c, servername, ssl, cert, key) != APR_SUCCESS) {
                     return SSL_TLSEXT_ERR_ALERT_FATAL;
                 }
+                SSL_set_verify(ssl, SSL_VERIFY_NONE, ssl_callback_SSLVerify);
             }
         }
     }

     return SSL_TLSEXT_ERR_OK;

[icing]

Will be part of next Apache release.