Kafka Internode communication breaks inspite of providing keyStore & Truststore files.

[ingared8]

I'm trying to use Kafka with a SSL enabled and generated the truststore files and keystore files using my kubernetes domain names (kafka-{0,1,2,3,4}.kafka-headless.namespcae.svc.cluster.local) as subjective alternative names. However the inter broker communication fails on the following exception.

org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
	at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1529)
	at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
	at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
	at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
	at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
	at org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:439)
	at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:304)
	at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:258)
	at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:125)
	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:487)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:425)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:510)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
	at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:330)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1614)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
	at sun.security.ssl.Handshaker$1.run(Handshaker.java:992)
	at sun.security.ssl.Handshaker$1.run(Handshaker.java:989)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1467)
	at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:393)
	at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:473)
	at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:331)
	... 8 more
Caused by: java.security.cert.CertificateException: No name matching kafka-kafka-2.kafka-kafka-headless.kafka.svc.cluster.local found
	at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:231)
	at sun.security.util.HostnameChecker.match(HostnameChecker.java:96)
	at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:455)
	at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1601)
	... 17 more
[2018-09-04 16:46:34,085] ERROR [Producer clientId=console-producer] Connection to node -1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
>[2018-09-04 16:46:34,192] ERROR [Producer clientId=console-producer] Connection to node -1 failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)```

[ingared8]

Should I need to adjust
ssl.endpoint.identification.algorithm=HTTPS
to
ssl.endpoint.identification.algorithm=None

[carrodher]

I am not a Kafka expert, but checking the documentation (https://kafka.apache.org/documentation/#security_confighostname), you should specify one value or another depending on the Kafka version you are using (apart from your specific configuration/environment)

[stale]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[stale]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.