Skip to content

v0.9.3

Latest

Choose a tag to compare

@merlimat merlimat released this 13 Oct 23:18
5bdb9f5

gRPC streams stay readable when closed before they are read

Since 0.9.0, closing the InputStream returned by a LightProto gRPC Marshaller.stream() emptied it even if nobody had read it yet. This broke gRPC proxies that forward the stream they receive unchanged, such as Apache BookKeeper's stream-storage proxy. 0.9.3 keeps the content readable until the stream has been read to the end.

Keep gRPC marshaller streams readable when closed before they are read (#54)

gRPC's ClientCallImpl closes each message as soon as onMessage() returns. A proxy with an identity marshaller passes that same stream to the next hop, and over the in-process transport the next hop reads it later on its own executor. Since 0.9.0, close() emptied the stream and recycled its array (#27) or released its ByteBufs (#49), so the next hop read 0 bytes. In BookKeeper, TestStorageContainerStoreImpl got empty responses and ProxyPingPongServiceTest hung.

A stream now gives up its array or buffers only once it has been read to the end:

  • Small messages: an early close() does nothing, and the array is left to the garbage collector.
  • Large messages (gathered from the caller's buffers): an early close() copies what is left into an array the stream owns and releases the ByteBufs, so nothing leaks if the stream is never read. The stream's methods are now synchronized, because a proxy may close it on one thread while the next hop reads it on another.

When Netty reads a stream to the end and then closes it, the array is still recycled with no extra allocation or copy. One behaviour changes: a send that fails before gRPC reads the stream (it exceeds the size limit, or a queued call is cancelled) no longer recycles its small-message array.