Skip to content

Commit

Permalink
Fix a bug where SslHandler does not handle SSLv2Hello correctly
Browse files Browse the repository at this point in the history
Motivation:

When a SSLv2Hello message is received, SSLEngine expects the application buffer size to be more than 30KB which is larger than what SslBufferPool can provide.  SSLEngine will always return with BUFFER_OVERFLOW status, blocking the SSL session from continuing the handshake.

Modifications:

When SSLEngine.getSession().getApplicationBufferSize() returns a value larger than what SslBufferPool provides, allocate a temporary heap buffer.

Result:

SSLv2Hello is handled correctly.
  • Loading branch information
trustin committed Jun 10, 2014
1 parent 129c17a commit 2fa9400
Showing 1 changed file with 15 additions and 5 deletions.
20 changes: 15 additions & 5 deletions src/main/java/org/jboss/netty/handler/ssl/SslHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -1268,8 +1268,18 @@ private ChannelBuffer unwrap(
// always contain at least one record in decode(). Therefore, if SSLEngine.unwrap() returns
// BUFFER_OVERFLOW, it is always resolved by retrying after emptying the application buffer.
for (;;) {
final int outAppBufSize = engine.getSession().getApplicationBufferSize();
final ByteBuffer outAppBuf;
if (nioOutAppBuf.capacity() < outAppBufSize) {

This comment has been minimized.

Copy link
@normanmaurer

normanmaurer Jun 10, 2014

Member

shouldn't this check against remaining() ?

This comment has been minimized.

Copy link
@trustin

trustin Jun 10, 2014

Author Member

Not really because we always clear the buffer at the finally block below.

// SSLEngine wants a buffer larger than what the pool can provide.
// Allocate a temporary heap buffer.
outAppBuf = ByteBuffer.allocate(outAppBufSize);
} else {
outAppBuf = nioOutAppBuf;
}

try {
result = engine.unwrap(nioInNetBuf, nioOutAppBuf);
result = engine.unwrap(nioInNetBuf, outAppBuf);
switch (result.getStatus()) {
case CLOSED:
// notify about the CLOSED state of the SSLEngine. See #137
Expand All @@ -1283,21 +1293,21 @@ private ChannelBuffer unwrap(

break;
} finally {
nioOutAppBuf.flip();
outAppBuf.flip();

// Sync the offset of the inbound buffer.
nettyInNetBuf.readerIndex(
nettyInNetBufStartOffset + nioInNetBuf.position() - nioInNetBufStartOffset);

// Copy the unwrapped data into a smaller buffer.
if (nioOutAppBuf.hasRemaining()) {
if (outAppBuf.hasRemaining()) {
if (nettyOutAppBuf == null) {
ChannelBufferFactory factory = ctx.getChannel().getConfig().getBufferFactory();
nettyOutAppBuf = factory.getBuffer(initialNettyOutAppBufCapacity);
}
nettyOutAppBuf.writeBytes(nioOutAppBuf);
nettyOutAppBuf.writeBytes(outAppBuf);
}
nioOutAppBuf.clear();
outAppBuf.clear();
}
}

Expand Down

0 comments on commit 2fa9400

Please sign in to comment.