LEAK: ByteBuf.release() was not called, 1.63.0

[y-kawawa]

What version of gRPC-Java are you using?

1.63.0

This issue does not occur in version 1.62.2.

What is your environment?

• Linux
• Java11

What did you expect to see?

ByteBuf is released

What did you see instead?

LEAK: ByteBuf.release() was not called before it's garbage-collected.

Steps to reproduce the bug

I am implementing a server-streaming RPC using grpc-java with the following behavior:

• The server delivers content in response to client requests and continues until disconnected.
• The server may also initiate disconnections.
• This behavior is implemented for multiple clients simultaneously.

After the server has been running for a while, the following error occurs:

LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:402)
	io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
	io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:124)
	io.grpc.netty.shaded.io.grpc.netty.NettyWritableBufferAllocator.allocate(NettyWritableBufferAllocator.java:51)
	io.grpc.internal.MessageFramer.writeKnownLengthUncompressed(MessageFramer.java:226)
	io.grpc.internal.MessageFramer.writeUncompressed(MessageFramer.java:172)
	io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:143)
	io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:66)
	io.grpc.internal.ServerCallImpl.sendMessageInternal(ServerCallImpl.java:168)
	io.grpc.internal.ServerCallImpl.sendMessage(ServerCallImpl.java:152)
	io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onNext(ServerCalls.java:380)
	com.example.Streaming.lambda$Process$0(Streaming.java:100)
	java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	java.base/java.lang.Thread.run(Thread.java:829)

Speculation

The changes in version 1.63.0 include #10258, which involves modifications in how stream resets are handled. This warning seems to occur in my environment particularly when the client closes the connection first, which may suggest a relationship to these changes.

I suspect the issue may be linked to the handling of closed connections and would appreciate further investigation into whether the changes in PR #10258 could be causing unintended side-effects in byte buffer management.

Others

There is a similar issue reported here: #11084, but it involves a different version and the stack trace is different, so I believe it might be a separate issue.

[ejona86]

Fixed by #11116

[sergiitk]

Note: the issue is fixed in v1.64.0, and the fix was backported to v1.63.1.