Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/zlib: #8650

Closed
gopherbot opened this issue Sep 4, 2014 · 3 comments
Closed

compress/zlib: #8650

gopherbot opened this issue Sep 4, 2014 · 3 comments

Comments

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 4, 2014

by TheRomanAlexander:

Tested on:
go version go1.2.2 linux/amd64
go version devel +a4312ec25ccd Sat Aug 30 00:56:52 2014 -0400 linux/amd64

Replication code:
https://gist.github.com/romanalexander/fd43345298c70c406440/
(Test too large for play.golang.org, also attached to report)

Issue:
zlib Reader.Read does not properly fill the requested buffer size. For example, when
requesting reader.Read(make([]byte, 99)) it will return 92 bytes without being at the
EOF. This is maybe a product of faulty buffering?
Using ioutil.ReadAll on the same reader instead of Reader.Read does not exhibit this
behavior and functions as intended.

Test Results:
Decompressing the entire compressed buffer at one time passes.
PASS
Streaming from the zlib buffer and comparing to the decompressed data fails.
FAIL Length mismatch! len1: 92 len2: 99 failed at pos 32676

The first test will attempt to decompress the entire compressed byte slice using
ioutil.ReadAll, which passes correctly.
The second test will attempt to decompress the compressed byte slice using random
Reader.Read sizes and compare it to the first test's decompressed byte slice. This test
fails.

Attachments:

  1. test.go (2187140 bytes)
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 4, 2014

Comment 1:

This doesn't sound like a bug.  In general the Read method, as documented at
http://golang.org/pkg/io/#Reader , does not promise that it will fill the entire buffer.
 If you need to fill the entire buffer, you should use ioutil.ReadAll, and you say that
works.

Labels changed: added repo-main, release-none.

@gopherbot
Copy link
Author

@gopherbot gopherbot commented Sep 4, 2014

Comment 2 by me@iscoelho.com:

Even so, if the zlib Reader has more data available, and it is not EOF, it should be
able to read it in one call. The issue here is that it happens predictably at certain
locations of the stream, which give signs of it being a bug and unintended behavior.

@bradfitz bradfitz removed the new label Dec 18, 2014
@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@dsnet
Copy link
Member

@dsnet dsnet commented May 9, 2016

compress/flate supports a flushing mechanism where Read returns all data immediately even if we could technically put in more bytes in the read buffer. This behavior is actually relied upon by some networking protocols, so I don't think we can ever support this change.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants