Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/gzip: Reader unable to parse headers with long comments #14639

Open
dsnet opened this issue Mar 4, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@dsnet
Copy link
Member

commented Mar 4, 2016

Using go1.6

RFC 1952 for gzip does not specify a length for the comment and name fields.

If FCOMMENT is set, a zero-terminated file comment is
present. This comment is not interpreted; it is only
intended for human consumption. The comment must consist of
ISO 8859-1 (LATIN-1) characters. Line breaks should be
denoted by a single line feed character (10 decimal).

The current implementation is inconsistent:

  • gzip.Writer permits the writing of any length comment/name string in gzip.Header.
  • gzip.Reader fails to read any comment/name string in gzip.Header longer than 511 bytes.

Playground example: https://play.golang.org/p/Zvjf8Q7jXe
Change 512 to 511 in the example, and it works again.

This causes issues reading gzip files produced by GZinga, which produces syntactically valid gzip files, but abuses the comment field to store meta data.

Update: I have no intention of fixing this unless there is someone who needs this functionality. Whatever fix we do will need to be careful that we don't introduce a potential vector for DOS attacks since gzip is a common Transfer-Encoding in HTTP. We don't want an infinitely long comment to cause a server to allocate an infinite amount of memory.

@dsnet

This comment has been minimized.

Copy link
Member Author

commented Apr 22, 2017

Bumping up milestone since #20083 provides good evidence that this is a real issue.

@dsnet dsnet modified the milestones: Go1.10, Unplanned Apr 22, 2017

@gopherbot

This comment has been minimized.

Copy link

commented Aug 8, 2017

Change https://golang.org/cl/53637 mentions this issue: compress/gzip: permit parsing of GZIP files with long header fields

@dsnet

This comment has been minimized.

Copy link
Member Author

commented Nov 8, 2017

Pushing to Go1.11 milestone... but I think a more general API for resource limitation might be the better approach. See #20169 for some discussion.

@dsnet dsnet modified the milestones: Go1.10, Go1.11 Nov 8, 2017

@dsnet dsnet modified the milestones: Go1.11, Unplanned Feb 16, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.