core: Close InputStream in NoopClientStream.writeMessage() to prevent resource leaks#12729
Conversation
… resource leaks NoopClientStream.writeMessage() silently discards the InputStream without closing it. When a Marshaller.stream() returns an InputStream backed by a ref-counted ByteBuf (e.g. Netty's PooledByteBufAllocator), this causes a direct memory leak. This affects any code path where writeMessage() is called on a NoopClientStream or its subclass FailingClientStream: - Context cancelled before stream start (ClientCallImpl line 197) - Compressor not found (ClientCallImpl line 219) - Deadline already exceeded (ClientCallImpl line 262, via FailingClientStream) - DelayedStream draining buffered messages after cancellation sets realStream to NoopClientStream.INSTANCE The fix calls GrpcUtil.closeQuietly(message) to ensure the InputStream is always closed, matching the contract in AbstractStream.writeMessage().
ejona86
left a comment
There was a problem hiding this comment.
Yeah, we wouldn't notice this with grpc-java's marshallers, but it should be closed.
I see a direct buffer was allocated as part of streamRequest(). gRPC will just copy out from that buffer again, so I suspect direct buffers don't actually help there. In ProtoInputStream we delay serializing so we can coordinate with the transport's buffering approach (e.g., pooled ByteBuf). However, it seems we've not optimized this in MessageFramer to avoid copying twice. This is related to 5a7f350 . It feels like we should be able to do something better here, although it'd take some thought and may not be clear what is "best." |
|
Yes, I've noticed that it was all copied into a It would be great if there could be some sort of shortcut when the |
|
Thank you! |
Summary
NoopClientStream.writeMessage()silently discards theInputStreamwithout closing it. When aMarshaller.stream()returns anInputStreambacked by a ref-counted NettyByteBuf, this causes a direct memory leak.This affects any code path where
writeMessage()is called onNoopClientStreamor its subclassFailingClientStream:start()time (ClientCallImplline 197)ClientCallImplline 219)ClientCallImplline 262, viaFailingClientStream)DelayedStreamdraining buffered messages after cancellation setsrealStreamtoNoopClientStream.INSTANCEThe fix calls
GrpcUtil.closeQuietly(message)to ensure theInputStreamis always closed, matching the contract inAbstractStream.writeMessage().Reproducer
We observed this with Oxia-java using LightProto-generated gRPC marshallers. When a standalone server is restarted, Netty's
ResourceLeakDetectorreports leakedByteBufinstances:Test plan
NoopClientStreamTest.writeMessageShouldCloseInputStream()that verifiesclose()is called on theInputStreampassed towriteMessage()