Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image/gif: decoder calls defer in a loop for every frame affecting memory usage #22237

Closed
artyom opened this issue Oct 12, 2017 · 1 comment
Closed

Comments

@artyom
Copy link
Member

@artyom artyom commented Oct 12, 2017

Please answer these questions before submitting your issue. Thanks!

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Save https://play.golang.org/p/l7FgKFL6l0 to the gif_test.go file along with this attachment (save it under 1x1-40k.gif name) which is a 1x1 px 40k frames GIF.

Run test:

go test

What did you expect to see?

Test passes.

What did you see instead?

--- FAIL: TestDecodMemoryConsumption (1.35s)
	gif_test.go:24: Decode increased heap by 1014MB
FAIL

This happens because gif.decoder.decode() calls defers in a loop, causing multiple *lzw.decoders to stay referenced until decode() returns, thus increasing heap size proportionally to the number of frames:

go/src/image/gif/reader.go

Lines 226 to 227 in 764a6ac

lzwr := lzw.NewReader(br, lzw.LSB, int(litWidth))
defer lzwr.Close()

I plan to submit a CL.

System details

go version go1.9.1 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/tmp/go:/Users/artyom/go"
GORACE=""
GOROOT="/Users/artyom/Library/go"
GOTOOLDIR="/Users/artyom/Library/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lb/3rk8rqs53czgb4v35w_342xc0000gn/T/go-build010192856=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.9.1
uname -v: Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13
BuildVersion:	17A405
lldb --version: lldb-900.0.45
  Swift-4.0
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 12, 2017

Change https://golang.org/cl/70370 mentions this issue: image/gif: avoid setting defers in the decode loop

@gopherbot gopherbot closed this in 54fa10a Oct 23, 2017
@golang golang locked and limited conversation to collaborators Oct 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.