Client-side mutual TLS, wss client connection does not seem to be sending client certificate

[michaelzg]

Versions

• akka-http 10.0.10
• akka version 2.5.4
• typesafe ssl-config 0.2.1
• scala 2.11.8
• TLS v1.2

Issue and Question

Mutual TLS to a WebSocket server using a akka-http client shows the akka client seemingly not sending the client certificate after verifying the server's identity. Therefore, I am wondering if Akka's client-side websocket connections support mutual tls currently?

If the Akka User List is more appropriate for this type of question, please let me know and I'll gladly move it there.

Thank you

Observations

When debugging the ssl handshake with akka.io.tcp.trace-logging = on I saw the following logs when establishing the wss connection that lead me question about mutual auth support:

DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Attempting connection to [/<host>:443]
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Connection established to [<host>:443]
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - [Actor[akka://default/user/StreamSupervisor-64/$$a#1774000492]] registered as connection handler
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Wrote [216] bytes to channel
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Read [1457] bytes.
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Wrote [7] bytes to channel
DEBUG TcpOutgoingConnection akka://default/system/IO-TCP/selectors/$a/63 - Got Close command, closing connection.

Most notably,

Wrote [216] bytes to channel
Read [1457] bytes.
Wrote [7] bytes to channel
Got Close command, closing connection.

I tried to break down the communication further:

The first 216 bytes look like the ClientHello (akka client).
The resulting 1457 bytes from the server that is the ServerHello that includes but not limited to:

• Handshake type ServerHello 0x02 (based on some hex's recorded from tcpdump)
• TLS v1.2, as seen from 0x0303 from tcpdump
• the certificate
• the CertificateRequest 0x0d

However, the client's response is only 7 bytes, which leads me to think the akka client is either not responding with its client certificate or I have configured something incorrectly.

More Contextual Info

The akka client is able to verify the the server's cert, as with logging enabled I am able to see logs that reflect that:

DEBUG CompositeX509TrustManager com.typesafe.sslconfig.ssl.CompositeX509TrustManager - checkServerTrusted: trustManager sun.security.ssl.X509TrustManagerImpl ...

Connection context setup

I'm creating the HttpsConnectionContext using the typesafe ssl-config classes similar to the following (added addition debugging settings)

val trustStoreConfig = immutable.Seq(
  TrustStoreConfig(None, Some("ca.cert")).withStoreType("PEM")
)

val trustManagerConfig = 
  TrustManagerConfig()
    .withTrustStoreConfigs(trustStoreConfig)

val keyStoreConfigs = immutable.Seq(
  KeyStoreConfig(None,Some("test.jks"))
    .withStoreType("jks")
    .withPassword(Some("password"))
)
val keyManagerConfig = 
  KeyManagerConfig()
    .withKeyStoreConfigs(keyStoreConfigs)

val sslHandshakeDebug = 
  SSLDebugHandshakeOptions()
    .withData(true)
    .withVerbose(true)
    
val sslDebugConfig = 
  SSLDebugConfig()
    .withAll(true)
    // i dont see logs related to the below unfortunately
    .withKeymanager(true)
    .withHandshake(Some(sslHandshakeDebug))

val sslSettings = 
  SSLConfigSettings()
    .withDefault(false)
    //to enable client auth in Http().createClientHttpsContext(sslConfig)
    .withSslParametersConfig(SSLParametersConfig().withClientAuth(Need))
    .withDebug(sslDebugConfig)
    .withKeyManagerConfig(keyManagerConfig)
    .withTrustManagerConfig(trustManagerConfig)

val sslConfig = AkkaSSLConfig().withSettings(sslSettings)
val httpsContext = Http().createClientHttpsContext(sslConfig)

and the websocket connection flow made like so:

val req = WebSocketRequest(s"wss://$host:$port")
Http(system).webSocketClientFlow(req, connectionContext = httpsContext)

JKS/certs

In testing I have verified that the keypair is valid and can be used to authenticate and fully connect (via mutual tls) to the server. The snippet of the keypair converted to jks format for use above:

$ keytool -list -v -keystore test.jks -storepass password

Alias name: example
Creation date: ...
Entry type: PrivateKeyEntry
Certificate chain length: 1

[jrudolph]

Hi @michaelzg, thanks for the report. Could you try if this is a ssl-config issue by setting up the SSLContext manually and see if it works in that case?

[michaelzg]

@jrudolph Yes, creating an SSLContext manually works! Thank you for that tip!

That means I am not using Http().createClientHttpsContext(sslConfig) and the subsequent instantiation of com.typesafe.sslconfig.ssl.ConfigSSLContextBuilder to create the SSLContext. Rather, doing the following with the java.security classes does allow mutual tls to connect as expected.

// load key stores and init factories ...
val sslContext = 
  SSLContext
    .getInstance("TLS")
    .init(kmFactory.getKeyManagers, tmFactory.getTrustManagers, secureRandom)
    
ConnectionContext.https(sslContext)

Now as for the why, I did spend a little time stepping through and compare both SSLContexts trying to pinpoint why the one generated from ssl-config and that generated manually could be different. Unfortunately, I did not find anything that jumped out at me...

Given that this does seem like a ssl-config issue, I'll close this one within akka-http. Let me know if you'd like me to open a new issue within ssl-config, however. I can try to help debugging over there.

[raboof]

@michaelzg sounds like this would be worth tracing down in https://github.com/typesafehub/ssl-config/issues