Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8268965: TCP Connection Reset when connecting simple socket to SSL server #4520

Closed
wants to merge 3 commits into from

Conversation

alexeybakhtin
Copy link
Contributor

@alexeybakhtin alexeybakhtin commented Jun 17, 2021

Please review the fix for JDK-8268965.

The new jtreg test is added for the described issue.
sun/security/ssl and javax/net/ssl tests are passed


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8268965: TCP Connection Reset when connecting simple socket to SSL server

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/4520/head:pull/4520
$ git checkout pull/4520

Update a local copy of the PR:
$ git checkout pull/4520
$ git pull https://git.openjdk.java.net/jdk pull/4520/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 4520

View PR using the GUI difftool:
$ git pr show -t 4520

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/4520.diff

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 17, 2021

👋 Welcome back abakhtin! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Jun 17, 2021
@openjdk
Copy link

openjdk bot commented Jun 17, 2021

@alexeybakhtin The following label will be automatically applied to this pull request:

  • security

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the security security-dev@openjdk.org label Jun 17, 2021
@mlbridge
Copy link

mlbridge bot commented Jun 17, 2021

Webrevs

@XueleiFan
Copy link
Member

I would like to have a look. I was wondering if there is a racing between close and read and if the closure could be hang. But I have no time to think more about them currently. Hopefully, I could have some cycles in the next couple of weeks.

@XueleiFan
Copy link
Member

Could I have more details about the problem? For example, the thread stacks for the socket reset or the thread stacks of the call to close() method. The test case does not call close() method of the socket or the I/O stream. Will the reset happen if the socket and I/O call the close() methods properly?

@alexeybakhtin
Copy link
Contributor Author

Hi Xuelei,
Thank you for your interest in this issue.
In this particular test case, a simple socket connects to the SSL server with a dummy message and waiting for the server's response.
SSL server fails to verify the message header and aborts the connection. The stack trace to socket close on the server side is:
java.lang.Throwable
at java.base/sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1785)
at java.base/sun.security.ssl.SSLSocketImpl.shutdown(SSLSocketImpl.java:1756)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:431)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:314)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:309)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:145)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:915)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1006)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:966)
at java.base/java.io.DataInputStream.readUnsignedShort(DataInputStream.java:343)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:595)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:570)
at SSLSocketReset$ServerThread.run(SSLSocketReset.java:97)
On the client side application waiting for response from the server but fails due to socket close by the server.
Current JDK code causes the client fails with:
java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:183)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:137)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
at java.base/java.io.DataInputStream.readFully(DataInputStream.java:201)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:614)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:570)
at SSLSocketReset.main(SSLSocketReset.java:60)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:833)

In case of proposed patch applied, the client fails with
java.io.EOFException
at java.base/java.io.DataInputStream.readFully(DataInputStream.java:203)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:614)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:570)
at SSLSocketReset.main(SSLSocketReset.java:60)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:833)

SSL server closes the socket during the handshake, so no changes if we try to close the socket from the application

@XueleiFan
Copy link
Member

Hi Alexey,

Thank you for the details thread stacks.

The basic idea of the fix looks good to me. But I did have a concern about the racing. If there is a read thread and a close/shutdown thread, there might be weird behaviors because each thread may only be able to read some of the information and make it impossible to decode the record. What do you think If placing the cleanup synchronized with readLock? For example, using the AppInputStream.deplete() method or a similar one that synch the reading.

Best,
Xuelei

@alexeybakhtin
Copy link
Contributor Author

Hi Xuelei,
Thank you for review. You are right. It is better to add readLock to exclude concurrent reading. Unfortunately I can not use appInput.deplete() here because of input stream already marked as closed (called from TransportContext.fatal(TransportContext.java:376)). So, I've added appInput.readLock around deplete()

Comment on lines 1780 to 1789
// Try to clear the kernel buffer to avoid TCP connection resets.
if (conContext.inputRecord instanceof SSLSocketInputRecord &&
isConnected) {
appInput.readLock.lock();
try {
((SSLSocketInputRecord) (conContext.inputRecord)).deplete(false);
} finally {
appInput.readLock.unlock();
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The blocking on close() may be not good in practice. I would use tryLock() rather than lock() so as to avoid closure blocking. tryLock() is not perfect, but it may be better than blocking the close().

BTW, you could use the intanceof pattern matching so as to avoid the cast (See https://openjdk.java.net/jeps/394).

            if (conContext.inputRecord instanceof
                                SSLSocketInputRecord inputRecord && isConnected) {
                 if (appInput.readLock.tryLock()) {
                    try {
                        inputRecord.deplete(false);
                    } finally {
                       appInput.readLock.unlock();
                    }
                 }
              }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Xuelei,
Thank you a lot again. Replaced lock with tryLock as you suggested

Copy link
Member

@XueleiFan XueleiFan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thank you!

@openjdk
Copy link

openjdk bot commented Jul 8, 2021

@alexeybakhtin This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8268965: TCP Connection Reset when connecting simple socket to SSL server

Reviewed-by: xuelei

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 298 new commits pushed to the master branch:

  • bca570c: 8269962: SA has unused Hashtable, Dictionary classes
  • 4a45d95: 8270021: Incorrect log decorators in gc/g1/plab/TestPLABEvacuationFailure.java
  • 30bba54: 8270064: Problem list tools/jdeprscan/tests/jdk/jdeprscan/TestRelease.java due to JDK-8270060
  • 2c883ee: 8267982: Set the node after peephole optimization to be removed
  • 4fbcce1: 8269886: Inaccurate error message for compressed hprof test
  • a96012f: 8269803: G1: remove unnecessary NoRefDiscovery
  • 4e18ec2: 8269993: [Test]: java/net/httpclient/DigestEchoClientSSL.java contains redundant @run tags
  • e54585b: 8268363: AArch64: Implement string_indexof_char intrinsic in SVE
  • 270fbcb: Merge
  • c812bbb: 8269929: (test) Add diagnostic info to ProceessBuilder/Basic.java for unexpected output
  • ... and 288 more: https://git.openjdk.java.net/jdk/compare/2d088fa91d18252a801db3b84ff87e261d63ebd4...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@XueleiFan) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jul 8, 2021
@alexeybakhtin
Copy link
Contributor Author

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Jul 8, 2021
@openjdk
Copy link

openjdk bot commented Jul 8, 2021

@alexeybakhtin
Your change (at version dc51d9e) is now ready to be sponsored by a Committer.

@VladimirKempik
Copy link

/sponsor

@openjdk
Copy link

openjdk bot commented Jul 8, 2021

Going to push as commit 6f171b9.
Since your change was applied there have been 299 commits pushed to the master branch:

  • 4f322a9: 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode
  • bca570c: 8269962: SA has unused Hashtable, Dictionary classes
  • 4a45d95: 8270021: Incorrect log decorators in gc/g1/plab/TestPLABEvacuationFailure.java
  • 30bba54: 8270064: Problem list tools/jdeprscan/tests/jdk/jdeprscan/TestRelease.java due to JDK-8270060
  • 2c883ee: 8267982: Set the node after peephole optimization to be removed
  • 4fbcce1: 8269886: Inaccurate error message for compressed hprof test
  • a96012f: 8269803: G1: remove unnecessary NoRefDiscovery
  • 4e18ec2: 8269993: [Test]: java/net/httpclient/DigestEchoClientSSL.java contains redundant @run tags
  • e54585b: 8268363: AArch64: Implement string_indexof_char intrinsic in SVE
  • 270fbcb: Merge
  • ... and 289 more: https://git.openjdk.java.net/jdk/compare/2d088fa91d18252a801db3b84ff87e261d63ebd4...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot closed this Jul 8, 2021
@openjdk openjdk bot added integrated Pull request has been integrated and removed ready Pull request is ready to be integrated rfr Pull request is ready for review sponsor Pull request is ready to be sponsored labels Jul 8, 2021
@openjdk
Copy link

openjdk bot commented Jul 8, 2021

@VladimirKempik @alexeybakhtin Pushed as commit 6f171b9.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integrated Pull request has been integrated security security-dev@openjdk.org
3 participants