-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Leftovers and concatenated compressed streams don't work #29
Comments
See #30 as well |
The behaviour of In #27 I had (I'm not convinced that A function with the behaviour that you're testing for would have to be quite a bit more complicated. You'd need to try decompressing leftovers in the hope that they were another archive; and then back up on yourself when that turned out to be wrong. I suppose you could build this too out of the existing (I'm even less convinced that the function that you're testing for would be useful enough to expose - it's kinda weird, and less useful even than the existing function as a building block). Suggest either:
|
@dimbleby a Yes, I agree that #27 didn't change the behavior of |
@dimbleby something that is not yet clear to me is whether this "concatenated stream" is a GZip only feature, or a Zlib feature. Do you know more about this? |
This concatenation support is explicitly mentioned in the gzip format http://www.gzip.org/zlib/rfc-gzip.html#file-format |
In https://www.ietf.org/rfc/rfc1950.txt it says:
So, it looks like concatenation is not supposed to work with Zlib. |
It's news to me, but it looks as though you're right that concatenated archives is specifically a gzip thing. What a minefield! So maybe what we want is:
(Above is written on the basis of the version of === If you want to expose building blocks, I'd still claim that the version of However, if you know of use cases where the version of |
For anyone who's looking for a quick solution, here's a full working code snippet: decompressAll ::
MonadIO m => P.Producer ByteString m r -> P.Producer ByteString m r
decompressAll p = do
er <- PGZip.decompress' p
case er of
Left leftover -> decompressAll leftover
Right r -> return r |
Both
decompress
anddecompress'
are expected to automatically process concatenated compressed input. However, this is currently not true fordecompress'
, and moreover, the leftover behavior in these cases is incorrect, as demonstrated by the (currently failing) test case added in c866d0fcc @dimbleby
The text was updated successfully, but these errors were encountered: