Creating an issue here as suggested in reactor/reactor-core#2300 (comment)
Motivation
When working with binary data (bytes), it is sometimes necessary to send the data in chunks of exact size, with the possible exception for the last one, of course. Note that I'm not talking about TCP packets, but data sent by the client to the server without a POJO abstraction. This turns out to be surprisingly difficult to achieve in a non-blocking manner using the WebClient. The closest I've come is the following:
HttpClient.create()
.httpResponseDecoder { it.maxChunkSize(chunkSize) }
However, this doesn't guarantee that all chunks are exactly of size chunkSize, it only guarantees that the chunk size is less than or equal to chunkSize.
Desired solution
- Provide a
BodyExtractor that creates a Flux<DataBuffer>, where each DataBuffer contains exactly the number of bytes specified. It'll be the responsibility of the BodyExtractor to resize the packets it receives from Netty.
- Provide an encoder on the
TcpClient that can send chunks of exact size.
Considered alternatives
Short of writing my own Netty handlers (which isn't difficult, except for the pesky reference counting), the only available client is I could find that could do this is Ktor Http Client. Ktor is developed by JetBrains mostly for the server side, and to say their documentation is poor is an understatement. They are also not as battle-tested as WebClient, so I'd like to be able to use WebClient if possible.
Additional context
One of the use cases for this feature is streaming audio files. Uncompressed format like WAV comes with a header (44 bytes for WAV), that the recipient may not want. Using exact-size chunks would let us drop the header and only transmit the data. Netty provides the LengthFieldBasedFrameDecoder for this exact purpose.
Creating an issue here as suggested in reactor/reactor-core#2300 (comment)
Motivation
When working with binary data (bytes), it is sometimes necessary to send the data in chunks of exact size, with the possible exception for the last one, of course. Note that I'm not talking about TCP packets, but data sent by the client to the server without a POJO abstraction. This turns out to be surprisingly difficult to achieve in a non-blocking manner using the
WebClient. The closest I've come is the following:However, this doesn't guarantee that all chunks are exactly of size
chunkSize, it only guarantees that the chunk size is less than or equal tochunkSize.Desired solution
BodyExtractorthat creates aFlux<DataBuffer>, where eachDataBuffercontains exactly the number of bytes specified. It'll be the responsibility of theBodyExtractorto resize the packets it receives from Netty.TcpClientthat can send chunks of exact size.Considered alternatives
Short of writing my own Netty handlers (which isn't difficult, except for the pesky reference counting), the only available client is I could find that could do this is Ktor Http Client. Ktor is developed by JetBrains mostly for the server side, and to say their documentation is poor is an understatement. They are also not as battle-tested as
WebClient, so I'd like to be able to useWebClientif possible.Additional context
One of the use cases for this feature is streaming audio files. Uncompressed format like WAV comes with a header (44 bytes for WAV), that the recipient may not want. Using exact-size chunks would let us drop the header and only transmit the data. Netty provides the LengthFieldBasedFrameDecoder for this exact purpose.