-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: image/png: Encode should allow users to specify the filter type #51982
Comments
CC @nigeltao |
If this is a proposal, what's your proposed API for skipping filter selection? Bear in mind, though, that the standard library is especially constrained by the Go 1.x backwards compatibility promise, both in terms of (1) we can't change any existing API (e.g. adding new function arguments) or behavior if it will break previous users and (2) any new API we invent will be frozen and have to be supported for a long time. The best solution may not necessarily be "change the stdlib PNG encoder" but "create a new PNG encoder package, under a different import path" instead - one that isn't subject to these constraints. The stdlib PNG package is open source and easily forked. Along those lines, if you're looking for a PNG encoder more focused on compression speed than compression size, you might find inspiration in https://github.com/richgel999/fpng and https://github.com/veluca93/fpnge |
Hi Nigel I initially thought of adding a flag in the While adding I agree with you that in the near term, it is perhaps better for the community to explore different ideas outside the standard library, and then incorporate them back later. |
I'm not sure if that 10x number is right. Nonetheless, it's interesting that Sticking with @ianlancetaylor: absent additional language or stdlib support, is there a recommended way to get a |
I think that on most processors we could do slightly better if I think that the a large underlying array of a Note that I haven't tried to understand this issue. |
Sorry, the 10x was with respect to my original naive attempt of encoding with (image/png + *image.RGBA). The below file contains 3 profiles:
The durations are:
The difference between the standard library and a custom function is 5x, with 2x coming from byte alignment, and another 2.5x coming from hardcoding the
Regarding the mystery of stdlib.prof
custom.prof
|
@ianlancetaylor to tweak my earlier comment, the So... allocate too many bytes, |
SGTM |
In case anyone hit over the same bottlenecks, here is a solution limited to image.NRGBA |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Encode a 23500x45500
image.NRGBA
image.This image is around 4GB uncompressed.
What did you expect to see?
I expected the encoding to finish within 5 seconds.
What did you see instead?
It took longer than 20 seconds.
According to pprof, most time is spent on selecting the png filter type.
If the png library allowed us to simply select the
ftNone
, this would speed up dramatically.The below
pprof
shows this by simulatingftNone
with no compression.This file png_encode.zip is the pprof dump showing
png.filter
taking a lot time.One side note, even in the case of no compression, much time is still spent on
runtime.memmove
, any ideas how to eliminate this? Is this related to garbage collection?The text was updated successfully, but these errors were encountered: