netty 4.1.20 makes the RPC calls to fail

[pasdam]

I'm using the latest gRPC version (1.9.0), which comes with netty 4.1.17, but I have another dependency which requires netty 4.1.20. If I use the latter the RPC calls on the client side fail with the following exception:

Exception in thread "main" io.grpc.StatusRuntimeException: UNKNOWN
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:221)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:202)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:131)
	...
Caused by: java.lang.IndexOutOfBoundsException: readerIndex(0) + length(10) exceeds writerIndex(0): PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 30)
	at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1403)
	at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1390)
	at io.netty.buffer.AbstractByteBuf.readSlice(AbstractByteBuf.java:856)
	at io.netty.handler.codec.http2.DefaultHttp2FrameWriter$DataFrameHeader.slice(DefaultHttp2FrameWriter.java:644)
	at io.netty.handler.codec.http2.DefaultHttp2FrameWriter.writeData(DefaultHttp2FrameWriter.java:160)
	at io.netty.handler.codec.http2.Http2OutboundFrameLogger.writeData(Http2OutboundFrameLogger.java:44)
	at io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder$FlowControlledData.write(DefaultHttp2ConnectionEncoder.java:411)
	at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.writeAllocatedBytes(DefaultHttp2RemoteFlowController.java:372)
	at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor$1.write(DefaultHttp2RemoteFlowController.java:562)
	at io.netty.handler.codec.http2.WeightedFairQueueByteDistributor$State.write(WeightedFairQueueByteDistributor.java:612)
	at io.netty.handler.codec.http2.WeightedFairQueueByteDistributor.distribute(WeightedFairQueueByteDistributor.java:293)
	at io.netty.handler.codec.http2.WeightedFairQueueByteDistributor.distributeToChildren(WeightedFairQueueByteDistributor.java:326)
	at io.netty.handler.codec.http2.WeightedFairQueueByteDistributor.distribute(WeightedFairQueueByteDistributor.java:272)
	at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.writePendingBytes(DefaultHttp2RemoteFlowController.java:643)
	at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.writePendingBytes(DefaultHttp2RemoteFlowController.java:267)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:200)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
	at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:991)
	at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:248)
	at io.grpc.netty.WriteQueue.flush(WriteQueue.java:136)
	at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:32)
	at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:44)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

[carl-mastrangelo]

Yes, a number of tests don't pass either. I am investigating, but not actively.

[Scottmitch]

FYI DefaultHttp2FrameWriter$DataFrameHeader was removed in Netty 4.1.18+ ... keep us posted if you think this is attributed to a Netty issue.

[darren-fu]

Get same exception while use Grpc 1.8.0 and project has other dependency need netty 4.1.19.final.

[ejona86]

We've updated to 4.1.21 on master and haven't seen anything anything resembling this.

[pasdam]

Attached you can find a simple project to reproduce it, I hope it can help.

Tested on Debian:

$ uname -a
Linux ldeb 4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2 (2018-01-04) x86_64 GNU/Linux

Note that the Maven dependency that uses a newer version of Netty is Redisson, if you remove that the client will work properly.

Attachments:
grpc-java_issue4025.tar.gz

[carl-mastrangelo]

@pasdam I can reproduce this locally.

[carl-mastrangelo]

@pasdam Updating to netty 4.1.22 removes the error. It may just be that 4.1.20 had a bug.

Since we have already updated to netty 4.1.22 in #4112, I think this should be fixed. gRPC 1.11.x should pick up the update.