Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Netty-3.5.3-Final : ChannelBuffer doesn't handle bytearray like 3.4.5-Final does #487

Closed
marcottedan opened this Issue · 4 comments

2 participants

@marcottedan

Hey!

I have a major issue with Netty-3.5.3-Final that made me revert to netty-3.4.5-final. It looks like that Bytearray sent in the ChannelBuffer are ok but Bytearray read from a ChannelBuffer are no longer what they used to be. I tested my exact same code with the two different netty versions (3.4.5 vs 3.5.3). My test case use two Netty HTTP server (one contacting the other) and share the same netty version. I can reproduce my bug either on localhost or on a local-network server.

Have you modified the first bytes of the header?

Here's my Excel binary output (sorry, there's no way to attach a file in your issue system) As you can see, I output my bytearray, deflate it, output it again, receive it on my server, inflate it and output it. With netty 3-4-5 final I have the same result (which is correct). With netty 3-5-3 I have a massive amount of bytes and zLib is unable to understand the header. If I swap out my Netty lib, the whole thing starts to work again.

Login : netty-3.4.5.Final.jar :: UNCOMPRESSED

0 0 9 3 1 10 3 -127 15 99 111 109 46 102 114 105 109 97 115 116 117 100 105 111 46 115 110 111 119 115 116 111 114 109 46 97 117 116 104 101 110 116 105 99 97 116 105 111 110 46 99 111 114 101 46 108 111 103 105 110 46 97 109 102 46 71 101 116 84 111 107 101 110 82 101 113 117 101 115 116

#VALUE! #VALUE! " " � � "
" � #VALUE! � c o m . f r i m a s t u d i o . s n o w s t o r m . a u t h e n t i c a t i o n . c o r e . l o g i n . a m f . G e t T o k e n R e q u e s t

COMPRESSED IN DEFLATE FUNCTION

120 1 5 -63 49 14 -128 48 8 0 -64 -102 78 -70 -8 19 -66 -31 110 -4 0 -87 84 -119 2 -79 -48 56 -5 115 -17 82 26 -13 48 -27 111 46 38 80 27 11 122 -12 -99 13 92 -19 -11 -80 38 -128 61 78 -46 -32 -126 -63 -90 80 -84 17 -36 118 -80 2 74 -123 -123 98 -77 -117 116 -91 -89 -109 -57 15 41 73 28 -81

RECEIVED IN INFLATE FUNCTION

120 1 5 -63 49 14 -128 48 8 0 -64 -102 78 -70 -8 19 -66 -31 110 -4 0 -87 84 -119 2 -79 -48 56 -5 115 -17 82 26 -13 48 -27 111 46 38 80 27 11 122 -12 -99 13 92 -19 -11 -80 38 -128 61 78 -46 -32 -126 -63 -90 80 -84 17 -36 118 -80 2 74 -123 -123 98 -77 -117 116 -91 -89 -109 -57 15 41 73 28 -81

UNCOMPRESSED IN INFLATE FUNCTION

0 0 9 3 1 10 3 -127 15 99 111 109 46 102 114 105 109 97 115 116 117 100 105 111 46 115 110 111 119 115 116 111 114 109 46 97 117 116 104 101 110 116 105 99 97 116 105 111 110 46 99 111 114 101 46 108 111 103 105 110 46 97 109 102 46 71 101 116 84 111 107 101 110 82 101 113 117 101 115 116

#VALUE! #VALUE! " " � � "
" � #VALUE! � c o m . f r i m a s t u d i o . s n o w s t o r m . a u t h e n t i c a t i o n . c o r e . l o g i n . a m f . G e t T o k e n R e q u e s t

Login : netty-3.5.3.Final.jar :: UNCOMPRESSED

0 0 9 3 1 10 3 -127 15 99 111 109 46 102 114 105 109 97 115 116 117 100 105 111 46 115 110 111 119 115 116 111 114 109 46 97 117 116 104 101 110 116 105 99 97 116 105 111 110 46 99 111 114 101 46 108 111 103 105 110 46 97 109 102 46 71 101 116 84 111 107 101 110 82 101 113 117 101 115 116

#VALUE! #VALUE! " " � � "
" � #VALUE! � c o m . f r i m a s t u d i o . s n o w s t o r m . a u t h e n t i c a t i o n . c o r e . l o g i n . a m f . G e t T o k e n R e q u e s t

COMPRESSED IN DEFLATE FUNCTION

120 1 5 -63 49 14 -128 48 8 0 -64 -102 78 -70 -8 19 -66 -31 110 -4 0 -87 84 -119 2 -79 -48 56 -5 115 -17 82 26 -13 48 -27 111 46 38 80 27 11 122 -12 -99 13 92 -19 -11 -80 38 -128 61 78 -46 -32 -126 -63 -90 80 -84 17 -36 118 -80 2 74 -123 -123 98 -77 -117 116 -91 -89 -109 -57 15 41 73 28 -81

RECEIVED IN INFLATE FUNCTION

72 84 84 80 47 49 46 49 32 50 48 48 32 79 75 13 10 67 111 110 110 101 99 116 105 111 110 58 32 107 101 101 112 45 97 108 105 118 101 13 10 67 111 110 116 101 110 116 45 84 121 112 101 58 32 97 112 112 108 105 99 97 116 105 111 110 47 111 99 116 101 116 45 115 116 114 101 97 109 13 10 67 111 110 116 101 110 116 45 76 101 110 103 116 104 58 32 49 51 48 13 10 69 120 112 105 114 101 115 58 32 84 104 117 32 65 117 103 32 48 50 32 48 57 58 48 55 58 53 52 32 69 68 84 32 50 48 49 50 13 10 68 97 116 101 58 32 84 104 117 32 65 117 103 32 48 50 32 48 57 58 48 55 58 53 52 32 69 68 84 32 50 48 49 50 13 10 67 97 99 104 101 45 67 111 110 116 114 111 108 58 32 109 97 120 45 97 103 101 61 49 13 10 13 10 120 1 -29 100 102 -28 50 110 20 76 -50 -49 -43 75 43 -54 -52 77 44 46 41 77 -55 -52 -41 43 -50 -53 47 47 46 -55 47 -54 -43 75 44 45 -55 72 -51 43 -55 76 78 44 -55 -52 -49 -45 75 -50 47 74 -43 -53 -55 79 -49 -52 -45 75 -52 77 -45 115 79 45 9 -55 -49 78 -51 11 74 45 46 -56 -49 43 78 -27 46 1 -15 56 -117 83 83 83 -124 83 43 -110 83 11 64 -102 -40 100 45 44 -52 -51 -51 12 77 45 45 13 -115 44 -52 13 88 15 -34 116 -40 -68 -102 -127 -127 -127 17 0 28 31 43 78
UNCOMPRESSED IN INFLATE FUNCTION

ERROR

@normanmaurer
Collaborator

Would be possible to post a test case ? Also the OS and the Java version would be helpful

@marcottedan

PC OS : Windows 7 (my machine)
Server OS : Windows 2008 Server R2 Standard SP1
JDK 1.6u23

I'll try to give you a test case in a few moments (might be hard considering my application)

@marcottedan

Is this enough for you?

public class HttpEncoder extends HttpResponseEncoder
{
@Override
protected Object encode(ChannelHandlerContext ctx, Channel channel, Object msg) throws Exception
{
HttpResponse httpResponse;
try
{
//response is Object[], encode it with amf protocol
if(Message[].class.isAssignableFrom(msg.getClass()))
{
ByteArrayOutputStream out = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(out);
try
{
//apply amf message protocol
byte[] amfResponsesBytes = Amf3.serialize(msg);
//compress the amf byte array
dos.write(ZLib.deflate(amfResponsesBytes));
dos.flush();

                // :: BYTES HERE ARE CORRECT WITH NETTY 3.5.3 :: CORRECT WITH NETTY 3.4.5
                amfResponsesBytes = out.toByteArray();

                // build http response
                httpResponse = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
                httpResponse.setContent(ChannelBuffers.copiedBuffer(amfResponsesBytes));
                httpResponse.setHeader(HttpHeaders.Names.CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
                httpResponse.setHeader(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
            }
            finally
            {
                dos.close();
                out.close();
                dos = null;
                out = null;
            }
        }           
    }
    catch(Exception e)
    {
        Log.error(e);
        httpResponse = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.INTERNAL_SERVER_ERROR);
        httpResponse.setHeader(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
        httpResponse.setContent(ChannelBuffers.copiedBuffer("500", CharsetUtil.UTF_8));
    }

    setDefaultHeaders(httpResponse);
    return super.encode(ctx, channel, httpResponse);
}

}

private Object handleHttpRequest(ChannelHandlerContext ctx, Channel channel, HttpRequest request)
{
String contentType = request.getHeader(HttpHeaders.Names.CONTENT_TYPE); //retrieve the content-type of the request
HttpMethod httpMethod = request.getMethod();//retrieve the http method
String ip = IPUtil.getIp(channel.getRemoteAddress().toString());//retrieve the ip address of the client

try
{
    StatelessRequest decodedRequest;

    // :: BYTES HERE MAKE NO SENSE WITH NETTY 3.5.3 :: CORRECT WITH NETTY 3.4.5     
    byte[] bufferBytes = request.getContent().duplicate().array();

    // Uncompress
    bufferBytes = ZLib.inflate(bufferBytes);

    // Decode DataStream  :: FIXME this can be optimized by using channel buffer instead
    DataInputStream dis = new DataInputStream(new ByteArrayInputStream(bufferBytes));

    // Read Token
    String token = dis.readUTF();

    // Get AmfRequests
    int lenght = bufferBytes.length - token.getBytes().length;
    byte[] amfRequestBytes = null;
    if (lenght > 0)
    {
        amfRequestBytes = new byte[lenght];
        dis.read(amfRequestBytes);
    }

    Object[] amfRequests = (Object[]) Amf3.deserialize(amfRequestBytes);

    decodedRequest = new StatelessRequest(token, ip, amfRequests);

    //return the decoded response
    return decodedRequest;
}
catch(Exception e)
{
    Log.error(e);
    Log.error(request.getContent().duplicate().array().toString());
    return null;
}

}

@normanmaurer
Collaborator

@marcottedan the behavior you are see is because you use:

byte[] bufferBytes = request.getContent().duplicate().array();

This does not pay attention of the readerIndex and writerIndex. It worked before as we just create a new array and copy the bytes over in <= 3.5.1.Final, but you should never depend on that. It would even not work if you would use direct buffers, so thats the other downside.

In newer versions of netty we share the backed array and so we safe memory copies and so are faster. So the correct way of get all readable bytes is:

byte[] bufferBytes = new byte[request.getContent().readableBytes()];
request.getContent().getBytes(request.getContent().readerIndex(), bufferBytes);

Hope this helps to explain it.

@normanmaurer normanmaurer was assigned
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.