Skip to content

image/gif: decoder calls defer in a loop for every frame affecting memory usage #22237

@artyom

Description

@artyom

Please answer these questions before submitting your issue. Thanks!

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Save https://play.golang.org/p/l7FgKFL6l0 to the gif_test.go file along with this attachment (save it under 1x1-40k.gif name) which is a 1x1 px 40k frames GIF.

Run test:

go test

What did you expect to see?

Test passes.

What did you see instead?

--- FAIL: TestDecodMemoryConsumption (1.35s)
	gif_test.go:24: Decode increased heap by 1014MB
FAIL

This happens because gif.decoder.decode() calls defers in a loop, causing multiple *lzw.decoders to stay referenced until decode() returns, thus increasing heap size proportionally to the number of frames:

go/src/image/gif/reader.go

Lines 226 to 227 in 764a6ac

lzwr := lzw.NewReader(br, lzw.LSB, int(litWidth))
defer lzwr.Close()

I plan to submit a CL.

System details

go version go1.9.1 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/tmp/go:/Users/artyom/go"
GORACE=""
GOROOT="/Users/artyom/Library/go"
GOTOOLDIR="/Users/artyom/Library/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lb/3rk8rqs53czgb4v35w_342xc0000gn/T/go-build010192856=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.9.1
uname -v: Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13
BuildVersion:	17A405
lldb --version: lldb-900.0.45
  Swift-4.0

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions