Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image/png: limit memory Decode can use while decoding #12512

Closed
opennota opened this issue Sep 5, 2015 · 6 comments
Closed

image/png: limit memory Decode can use while decoding #12512

opennota opened this issue Sep 5, 2015 · 6 comments

Comments

@opennota
Copy link

@opennota opennota commented Sep 5, 2015

With the DEFLATE compression used by the PNG image format one can compress a 50 gigapixel image to a 6 Mb PNG file. When decoding such a file, png.Decode throws a fatal error: runtime: out of memory.

image/png should provide a safety switch that allows the user to limit the amount of memory that png.Decode will use.

Test case: https://github.com/opennota/spark

Reference: https://www.bamsoftware.com/hacks/deflate.html

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 5, 2015

It does have such a safety switch already: http://golang.org/pkg/image/png/#DecodeConfig

I'm not sure there's anything to be done here that wouldn't be redundant with that. You could imagine a new Decoder type with some optional limit options, and have Decode return an error if limits are violated, I guess. That might be a little more friendly.

/cc @nigeltao @mpl

@bradfitz bradfitz changed the title Limit memory png.Decode can use while decoding image/png: limit memory Decode can use while decoding Sep 5, 2015
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 5, 2015

(this applies to all image decoders, really... not just png)

@opennota
Copy link
Author

@opennota opennota commented Sep 5, 2015

It does have such a safety switch already: http://golang.org/pkg/image/png/#DecodeConfig

Ah, yes. It seems there's no way to continue reading the entire image from the same reader without seeking to the start of the stream, though.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 5, 2015

There is a way, once you learn the rest of the standard library. :-)

Imagine you have a streamReader which is not seekable. (e.g. it's coming from the network, via an HTTP request or response or whatever). You can read the header, verify its properties, and then decode the rest of the image with the already-read header re-stitched on to the front like this:

var header bytes.Buffer
conf, err := png.DecodeConfig(io.TeeReader(streamReader, &header))
if err != nil || !confValid(conf) { ... }
im, err := png.Decode(io.MultiReader(&header, streamReader)

See the docs for io.TeeReader and io.MultiReader to see how that works.

@opennota
Copy link
Author

@opennota opennota commented Sep 6, 2015

@bradfitz
Wow, cool. I think the issue can be closed, then.

@minux minux closed this Sep 6, 2015
@nigeltao
Copy link
Contributor

@nigeltao nigeltao commented Sep 7, 2015

Yeah, what @bradfitz said.

As for a nicer API, this is issue #5050.

@golang golang locked and limited conversation to collaborators Sep 6, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.