BouncyCastleJsseProvider: Client raised fatal(2) internal_error(80) alert: Failed to read record

[jankostefancic]

I have run into an issue developing a HTTP client with the use of BouncyCastle libraries. Target versions is Java 1.6.0_45-b06 (but the error is also reproducible in Java 1.8.0_91 with the same version of BouncyCastle.)

BouncyCastle jdk15to18 167 (bcprov-jdk15to18-167.jar, bcpkix-jdk15to18-167.jar, bctls-jdk15to18-167.jar)

Accessing data with TLS 1.2 without client_auth works fine (there are some issues with specific sites that return handshake_failure(40), but that's another issue and luckily not for our integration). But when client_auth is required the code fails with the following (a little cryptic) error:

nov. 27, 2020 5:23:10 PM org.bouncycastle.jsse.provider.ProvTlsClient notifyAlertRaised
WARNING: Client raised fatal(2) internal_error(80) alert: Failed to read record
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
    at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
    at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
    at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
    at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at java.net.HttpURLConnection.getResponseCode(Unknown Source)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
    at HttpsClient.main(HttpsClient.java:109)

nov. 27, 2020 5:23:10 PM org.bouncycastle.jsse.provider.ProvTlsClient notifyAlertRaised
WARNING: Client raised fatal(2) internal_error(80) alert: Failed to read record
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
    at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
    at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
    at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
    at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at java.net.HttpURLConnection.getResponseCode(Unknown Source)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
    at HttpsClient.main(HttpsClient.java:109)

java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
    at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
    at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
    at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
    at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at java.net.HttpURLConnection.getResponseCode(Unknown Source)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
    at HttpsClient.main(HttpsClient.java:109)

Code without BC as SecurityProvider runs just fine in Java 1.8. But in our case BC is a requirement for TLS 1.2 support in Java 1.6.

I tried to analyze the SSL handshake with Wireshark, but I don't have enough knowledge to understand what's happening (failing).

What am I missing here? Or it's just a known limitation of BC?
Thanks for any suggestions.

[jankostefancic]

After extensive research and testing (we set up our own dev environment that replicated the endpoint, we were trying to connect to with client cert) we have found a solution (posted below) that works for us in our dev environment, but couldn't be configured on the endpoint. That's why we went to plan B - TLS termination proxy and it worked like a charm in our case, when we didn't have control of the server environment.

Solution for IIS
In our case the web service requiring client auth was running on Microsoft IIS. As stated "Unfortunately our TLS libraries do not support renegotiation (and we do not plan to add it, although there are corresponding TLS 1.3 features that we will)." in this open issue (#593). After reading the IIS documentation and some testing we successfully configured the environment in a way that TLS sessions could be established with BC TLS lib:

Step 1:

• Create an IIS site with default settings
• Run IISCrypto with the "Best practices" template:
  
  
  

Step 2:

• Enable SSLAlwaysNegoClientCert

Save the following text to a file called "Enable_SSL_Renegotiate.js"
var vdirObj=GetObject("IIS://localhost/W3svc/1"); // replace 1 on this line with the number of the web site you wish to configure WScript.Echo("Value of SSLAlwaysNegoClientCert Before: " + vdirObj.SSLAlwaysNegoClientCert); vdirObj.Put("SSLAlwaysNegoClientCert", true); vdirObj.SetInfo(); WScript.Echo("Value of SSLAlwaysNegoClientCert After: " + vdirObj.SSLAlwaysNegoClientCert);

Run the following command from an elevated / administrator command prompt:
cscript.exe enable_ssl_renegotiate.js