Don't try to fill up the entire buffer on a read. #5
Conversation
An LZ4 frame is designed so that partial blocks can be "flushed" by the encoder at any time. A frame can be a long term stream sent over a network connection. The decoder needs to be sure that it only calls the underlying read once at a time, when necessary, and be able to return partially filled buffers. This allows the caller to respond to flushed data without the decoder blocking on a network read.
|
When I made my last pull request, I was under the assumption that to use lz4 over a network, each individual message would be an lz4 frame. Upon further investigation, I figured out that the lz4 frame standard is also intended to be used for streaming. An entire long-running network stream should be one giant lz4 frame. Part of the frame standard is the idea that the sender should be able to "flush" it's data at any point in time, and the other side should be able to read it. My change here should allow the other side to actually read and respond to a network message in this way. The down side is that this change could break people's programs. If people are using lz4-rs and assuming that the read function will never return until the entire buffer is full, this change will break them. Technically though, the contract for std::io::Read includes the possibility of returning early. |
Don't try to fill up the entire buffer on a read.
|
Big thanks :) |
|
Bumped 1.9.131 |
Update dependencies
An LZ4 frame is designed so that partial blocks can be "flushed" by
the encoder at any time. A frame can be a long term stream sent over
a network connection. The decoder needs to be sure that it only calls
the underlying read once at a time, when necessary, and be able to
return partially filled buffers. This allows the caller to respond
to flushed data without the decoder blocking on a network read.