LineBasedFrameDecoder#findEndOfLine method exception

[stateIs0]

Expected behavior

decode

Actual behavior

throw IndexOutOfBoundsException

Caused by: java.lang.IndexOutOfBoundsException: index: 2, length: -1 (expected: range(0, 512))
	at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1360)
	at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1355)
	at io.netty.buffer.AbstractByteBuf.forEachByte(AbstractByteBuf.java:1261)
	at io.netty.handler.codec.LineBasedFrameDecoder.findEndOfLine(LineBasedFrameDecoder.java:165)
	at io.netty.handler.codec.LineBasedFrameDecoder.decode(LineBasedFrameDecoder.java:94)
	at io.netty.handler.codec.LineBasedFrameDecoder.decode(LineBasedFrameDecoder.java:78)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:512)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:450)
	... 18 more

Steps to reproduce

0. pipeline.addLast LineBasedFrameDecoder, and maxLength == 2.
1. Start the EchoServer
2. Start the EchoClient
3. throw IndexOutOfBoundsException

Minimal yet complete reproducer code (or URL to code)

EchoServer:

 p.addLast(new LineBasedFrameDecoder(2, false, false));
 p.addLast(new StringDecoder());
 p.addLast(new EchoServerHandler());

EchoClient:

  @Override
  public void channelActive(ChannelHandlerContext ctx) {
    ctx.writeAndFlush(Unpooled.wrappedBuffer(new byte[]{0, 1, 2}));
    ctx.writeAndFlush(Unpooled.wrappedBuffer(new byte[]{3, 4}));
    ctx.writeAndFlush(Unpooled.wrappedBuffer(new byte[]{'\n'}));
    ctx.writeAndFlush(Unpooled.wrappedBuffer(new byte[]{5}));
  }

Netty version

master

JVM version (e.g. java -version)

1.8.0_121-b13

OS version (e.g. uname -a)

Darwin PROMOTE 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

[stateIs0]

This was discovered by my friend @YunaiV.

[stateIs0]

@normanmaurer may need a counter to fix this bug.

[normanmaurer]

@stateIs0 thanks for reporting :) Fixed in #8257.