Skip to content

Commit

Permalink
OpenSslEngine may use networkBIO after calling shutdown
Browse files Browse the repository at this point in the history
Motivation:
To ensure that all bytes queued in OpenSSL/tcnative internal buffers we invoke SSL_shutdown again to stimulate OpenSSL to write any pending bytes. If this call fails we may call SSL_free and the associated shutdown method to free resources. At this time we may attempt to use the networkBIO which has already been freed and get a NPE.

Modifications:
- Don't call bioLengthByteBuffer(networkBIO) if we have called shutdown() in ReferenceCountedOpenSslEngine

Result:
Fixes netty#6466
  • Loading branch information
Scottmitch committed Feb 28, 2017
1 parent 90a6104 commit 6695444
Showing 1 changed file with 8 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@

import java.nio.ByteBuffer;
import java.nio.ReadOnlyBufferException;
import java.security.AlgorithmConstraints;
import java.security.Principal;
import java.security.cert.Certificate;
import java.util.ArrayList;
Expand Down Expand Up @@ -479,9 +478,11 @@ public final SSLEngineResult wrap(
// It is possible when the outbound was closed there was not enough room in the non-application
// buffers to hold the close_notify. We should keep trying to close until we consume all the data
// OpenSSL can give us.
boolean didShutdown = doSSLShutdown();
if (!doSSLShutdown()) {
return newResultMayFinishHandshake(NOT_HANDSHAKING, 0, bytesProduced);
}
bytesProduced = bioLengthBefore - SSL.bioLengthByteBuffer(networkBIO);
return newResultMayFinishHandshake(didShutdown ? NEED_WRAP : NOT_HANDSHAKING, 0, bytesProduced);
return newResultMayFinishHandshake(NEED_WRAP, 0, bytesProduced);
}

// Flush any data that may be implicitly generated by OpenSSL (handshake, close, etc..).
Expand Down Expand Up @@ -1044,6 +1045,10 @@ public final synchronized void closeOutbound() {
}
}

/**
* Attempt to call {@link SSL#shutdownSSL(long)}.
* @return {@code false} if the call to {@link SSL#shutdownSSL(long)} was not attempted or returned an error.
*/
private boolean doSSLShutdown() {
if (SSL.isInInit(ssl) != 0) {
// Only try to call SSL_shutdown if we are not in the init state anymore.
Expand Down

0 comments on commit 6695444

Please sign in to comment.