SSLException on connection close

[SteveDemy]

An exception is thrown when a connector is closed every time a max lifetime expires. The issue is new with OpenJDK 11.0.1 (vs Oracle Java 10) and is easily solved if not using SSL.

With these connector properties defined (new with mySQL 8.0.13):

hikariDataSource.addDataSourceProperty("sslMode", "PREFERRED");
hikariDataSource.addDataSourceProperty("allowPublicKeyRetrieval", "true");

a lifetime expiry results in:

Sun Oct 28 23:48:26 EDT 2018 WARN: Caught while disconnecting...
EXCEPTION STACK TRACE:

** BEGIN NESTED EXCEPTION ** 

javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify

STACKTRACE:

javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
	at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1312)
	at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
	at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
	at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
	at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:135)
	at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:441)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

** END NESTED EXCEPTION **

From the Connector/J documentation: "PREFERRED" - (default) Establish encrypted connections if the server enabled them, otherwise fall back to unencrypted connections;

It is an easy fix:

hikariDataSource.addDataSourceProperty("sslMode", "DISABLED");
hikariDataSource.addDataSourceProperty("allowPublicKeyRetrieval", "false");

Possibly related discussion on StackOverflow pointing to an OpenJDK problem: https://stackoverflow.com/questions/52016415/jdk-11-ssl-error-on-valid-certificate-working-in-previous-versions

FYI, in case the connection close mechanism needs tweaking.

Environment

HikariCP version: 3.2.0
JDK version     : 11.0.1
Database        : MySQL
Driver version  : 8.0.13

[matelang]

What about the situations where SSL disabling is not an option and you have to verify server certificate? Is there any sort of workaround?

[matelang]

I also tried disabling TLS v1.3 according to JSSE documentation with providing the following JAVA_OPTION: -Djdk.tls.client.protocols=TLSv1.2. It did not solve the issue.

[SteveDemy]

I use SSL in the link between the client's browser and Tomcat-based application. That is unrelated, so I haven't tried altering the SSL settings in Java. The question here pertains to the Connector/J links between the application and the database, which are controlled by the HikariCP connection pool. Those connectors in mySQL 8.0.13 are subject to a new parameter, "sslMode". Without SSL along the link between application and database results in:

sslMode = PREFERRED -> errors thrown each time a connector is disconnected by HikariCP
sslMode = DISABLED -> no errors thrown

I haven't tested using SSL as my database and application are in the same machine.

[brettwooldridge]

This looks like a bug in the driver, not HikariCP. HikariCP is simply calling Connection.close(). If that results in an SSL exception from the driver, then the problem is definitely "below" HikariCP in the stack.

[davidgoate]

It looks like this is known to Oracle, I'm also having this issue and came here from this stack overflow: https://bugs.mysql.com/bug.php?id=93590

[phyntom]

just add this to your database url. I did it when I was connection to mysql database and it solved the issue spring.datasource.url=jdbc:mysql://localhost/demodb?useSSL=false

[davidgoate]

@phyntom that's all very well (useSSL=false) if one doesn't want to actually use SSL to protect data in transit. In some cases, it's even mandatory via the server database settings:

e.g. assume a user called app exists and a database called db: GRANT ALL ON db.* TO 'app'@'%' REQUIRE SSL;

[er1c]

Did this actually get fixed, or did the issue just get closed? I would love to get rid of these from our error logs.

[phyntom]

yes buy using the is spring.datasource.url=jdbc:mysql://localhost/demodb?useSSL=false and upgrading the mysql connector to 5.1.46 I got rid of them.Of course the mysql connector may differ depending on version of mysql you are using. But for me it worked when I upgraded from 5.1.37 to 5.1.46

[crazyproger]

@phyntom does this mean that SSL will be completely disabled for communication with MySQL? If so - I think it's very dangerous 'solution'.

[phyntom]

That is why I suggested you to find another driver and test it.I have remove also useSSL=false after the upgrade and all errors disappeared. So try another version

[er1c]

For reference: https://bugs.mysql.com/bug.php?id=93590 & https://bugs.openjdk.java.net/browse/JDK-8215102

There is an interesting answer here: https://bugs.openjdk.java.net/browse/JDK-8215102?focusedCommentId=14240050&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14240050

With a JDK code test here: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8215102-jtreg-test/01/webrev/openjdk-head-2.patch that can reproduce it regardless of the JDBC connector in the JDK11.

[18 Feb 11:34] Filipe Silva
Yes, the exception stack trace is an issue, but it's just clutter in the log. Java 11 just implemented SSL support differently and now this exception is being caught on a different place, and thus being logged while it wasn't before.

The real problem is how to control where sockets stay in TIME_WAIT state. This is being investigated at the moment.

[er1c]

Netty has a commit to "swallow"/suppress this Java11 exception/bug, is it worth adding to HikariCP? https://github.com/netty/netty/blob/41b02368153af86b1ddb19020ebf5e4f7c69aecd/handler/src/main/java/io/netty/handler/ssl/SslHandler.java#L1779

[er1c]

Never mind, it looks like Hikari already attempts to track the exception, it's getting logged lower down in the dependencies

[er1c]

I added: mysql/mysql-connector-j#32