Repository navigation
gRPC streams stay readable when closed before they are read
Since 0.9.0, closing the InputStream returned by a LightProto gRPC Marshaller.stream() emptied it even if nobody had read it yet. This broke gRPC proxies that forward the stream they receive unchanged, such as Apache BookKeeper's stream-storage proxy. 0.9.3 keeps the content readable until the stream has been read to the end.
Keep gRPC marshaller streams readable when closed before they are read (#54)
gRPC's ClientCallImpl closes each message as soon as onMessage() returns. A proxy with an identity marshaller passes that same stream to the next hop, and over the in-process transport the next hop reads it later on its own executor. Since 0.9.0, close() emptied the stream and recycled its array (#27) or released its ByteBufs (#49), so the next hop read 0 bytes. In BookKeeper, TestStorageContainerStoreImpl got empty responses and ProxyPingPongServiceTest hung.
A stream now gives up its array or buffers only once it has been read to the end:
- Small messages: an early
close()does nothing, and the array is left to the garbage collector. - Large messages (gathered from the caller's buffers): an early
close()copies what is left into an array the stream owns and releases the ByteBufs, so nothing leaks if the stream is never read. The stream's methods are nowsynchronized, because a proxy may close it on one thread while the next hop reads it on another.
When Netty reads a stream to the end and then closes it, the array is still recycled with no extra allocation or copy. One behaviour changes: a send that fails before gRPC reads the stream (it exceeds the size limit, or a queued call is cancelled) no longer recycles its small-message array.