image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT`

[secsys-go]

What version of Go are you using (go version)?

$ go version
go version go1.17.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN="/home/zjx/workspace/gowork/bin"
GOCACHE="/home/zjx/.cache/go-build"
GOENV="/home/zjx/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/zjx/workspace/gowork/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/zjx/workspace/gowork"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/home/zjx/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/zjx/.local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.8"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/zjx/workspace/gowork/src/go-fdg-exmaples/std/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1878510231=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I attempt to decode an png data, and the details can be seen at:

go.dev/play

What did you expect to see?

It decodes the data successfully or just returns an error

What did you see instead?

It triggered a fatal error like:

go env Output

fatal error: runtime: out of memory
runtime stack:

runtime.throw({0x4af45e, 0x31ce0000000})

/home/zjx/.local/go/src/runtime/panic.go:1198 +0x71

runtime.sysMap(0xc000400000, 0x426780, 0x7fff08857758)

/home/zjx/.local/go/src/runtime/mem_linux.go:169 +0x96

runtime.(*mheap).grow(0x5652e0, 0x18e70000)

/home/zjx/.local/go/src/runtime/mheap.go:1393 +0x225

runtime.(*mheap).allocSpan(0x5652e0, 0x18e70000, 0x0, 0x1)

/home/zjx/.local/go/src/runtime/mheap.go:1179 +0x165

runtime.(*mheap).alloc.func1()

/home/zjx/.local/go/src/runtime/mheap.go:913 +0x69

runtime.systemstack()

/home/zjx/.local/go/src/runtime/asm_amd64.s:383 +0x49
goroutine 1 [running]:

runtime.systemstack_switch()

/home/zjx/.local/go/src/runtime/asm_amd64.s:350 fp=0xc00010da00 sp=0xc00010d9f8 pc=0x458e20

runtime.(*mheap).alloc(0x0, 0xc00010daa8, 0x6f, 0x0)

/home/zjx/.local/go/src/runtime/mheap.go:907 +0x73 fp=0xc00010da50 sp=0xc00010da00 pc=0x422ab3

runtime.(*mcache).allocLarge(0x40b3fe, 0x31ce0000000, 0x87, 0x1)

/home/zjx/.local/go/src/runtime/mcache.go:227 +0x89 fp=0xc00010dab0 sp=0xc00010da50 pc=0x413949

runtime.mallocgc(0x31ce0000000, 0x499220, 0x1)

/home/zjx/.local/go/src/runtime/malloc.go:1088 +0x5c5 fp=0xc00010db30 sp=0xc00010dab0 pc=0x40bb45

runtime.makeslice(0xc0001b6000, 0x0, 0xc00010db98)

/home/zjx/.local/go/src/runtime/slice.go:98 +0x52 fp=0xc00010db58 sp=0xc00010db30 pc=0x444892

image.NewNRGBA64({{0x0, 0x0}, {0xc00010dc18, 0x1000000465c25}})

/home/zjx/.local/go/src/image/image.go:601 +0x66 fp=0xc00010dbd0 sp=0xc00010db58 pc=0x489fa6

image/png.(*decoder).readImagePass(0xc0000d7000, {0x7f125c277030, 0xc000180050}, 0x0, 0x0)

/home/zjx/.local/go/src/image/png/reader.go:493 +0x7b6 fp=0xc00010dda0 sp=0xc00010dbd0 pc=0x48c7f6

image/png.(*decoder).decode(0xc0000d7000)

/home/zjx/.local/go/src/image/png/reader.go:372 +0x1af fp=0xc00010de70 sp=0xc00010dda0 pc=0x48bb0f

image/png.(*decoder).parseIDAT(0xc0000d7000, 0xd7078)

/home/zjx/.local/go/src/image/png/reader.go:849 +0x25 fp=0xc00010de88 sp=0xc00010de70 pc=0x48f6e5

image/png.(*decoder).parseChunk(0xc0000d7000)

/home/zjx/.local/go/src/image/png/reader.go:908 +0x128 fp=0xc00010def8 sp=0xc00010de88 pc=0x48f888

image/png.Decode({0x4cb540, 0xc0001a6000})

/home/zjx/.local/go/src/image/png/reader.go:967 +0x11b fp=0xc00010df60 sp=0xc00010def8 pc=0x48fe5b

main.main()

/home/zjx/workspace/gowork/src/go-fdg-exmaples/std/image/png/pocTestIncolplete/poc.go:9 +0x56 fp=0xc00010df80 sp=0xc00010df60 pc=0x4903b6

runtime.main()

/home/zjx/.local/go/src/runtime/proc.go:255 +0x227 fp=0xc00010dfe0 sp=0xc00010df80 pc=0x4324a7

runtime.goexit()

/home/zjx/.local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc00010dfe8 sp=0xc00010dfe0 pc=0x45af01

exit status 2

Found by go-fuzz,

[robpike]

Your program says it exits without error, but it doesn't. My version, which checks the error returned by Decode, reports

% go run x.go
<nil> flate: corrupt input before offset 5
% 

https://go.dev/play/p/bAa9vbuJfgT

It may be running out of memory or CPU time on the playground, but the input is invalid and the error is reported on a proper machine.

Did you find this example by fuzzing?

[bcmills]

I am able to reproduce the reported behavior on Go at HEAD.

$ go version
go version devel go1.19-59ab6f351a Mon Jul 11 08:23:57 2022 +0000 linux/amd64

$ go mod init example
go: creating new go.mod: module example

$ cat > main.go
package main

import (
        "fmt"
        "image/png"
        "runtime"
        "strings"
)

func main() {
        fmt.Println(runtime.Version())
        _, err := png.Decode(strings.NewReader("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\xffd\x00\x00\x00\x10\x06\x00\x00\x00\x8bӧ\xed0000IDATx\x9c\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\xffd\x00\x00\x00\x10\x06\x00\x00"))
        fmt.Println("decoded:", err)
}

$ go run .
devel go1.19-59ab6f351a Mon Jul 11 08:23:57 2022 +0000
fatal error: runtime: out of memory

runtime stack:
runtime.throw({0x4b3483?, 0x80000?})
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/panic.go:1047 +0x5d fp=0x7fff582f0740 sp=0x7fff582f0710 pc=0x431afd
runtime.sysMapOS(0xc000400000, 0x31ce0000000?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mem_linux.go:187 +0x11b fp=0x7fff582f0788 sp=0x7fff582f0740 pc=0x4152fb
runtime.sysMap(0x5695a0?, 0x7f721d373000?, 0x428520?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mem.go:142 +0x35 fp=0x7fff582f07b8 sp=0x7fff582f0788 pc=0x414cd5
runtime.(*mheap).grow(0x5695a0, 0x18e70000?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:1459 +0x23d fp=0x7fff582f0828 sp=0x7fff582f07b8 pc=0x4255dd
runtime.(*mheap).allocSpan(0x5695a0, 0x18e70000, 0x0, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:1191 +0x1be fp=0x7fff582f08c0 sp=0x7fff582f0828 pc=0x424d3e
runtime.(*mheap).alloc.func1()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:910 +0x65 fp=0x7fff582f0908 sp=0x7fff582f08c0 pc=0x4247c5
runtime.systemstack()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:492 +0x49 fp=0x7fff582f0910 sp=0x7fff582f0908 pc=0x45b0a9

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:459 fp=0xc00005f9d0 sp=0xc00005f9c8 pc=0x45b040
runtime.(*mheap).alloc(0x413b2b?, 0x57fd20?, 0x7?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:904 +0x65 fp=0xc00005fa18 sp=0xc00005f9d0 pc=0x424705
runtime.(*mcache).allocLarge(0x7?, 0x31ce0000000, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mcache.go:233 +0x85 fp=0xc00005fa68 sp=0xc00005fa18 pc=0x413c65
runtime.mallocgc(0x31ce0000000, 0x49c4c0, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/malloc.go:1029 +0x57e fp=0xc00005fae0 sp=0xc00005fa68 pc=0x40bd5e
runtime.makeslice(0xc00005fb38?, 0xc00005fb38?, 0x446b72?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/slice.go:103 +0x52 fp=0xc00005fb08 sp=0xc00005fae0 pc=0x446b72
image.NewNRGBA64({{0xc000104bb8?, 0x484d6e?}, {0x100c000121088?, 0x4d1f08?}})
        /usr/local/google/home/bcmills/sdk/gotip/src/image/image.go:603 +0x66 fp=0xc00005fb80 sp=0xc00005fb08 pc=0x48da26
image/png.(*decoder).readImagePass(0x4d1f68?, {0x7f71f5ec6020, 0xc000078050}, 0x0?, 0x0)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:495 +0x845 fp=0xc00005fd58 sp=0xc00005fb80 pc=0x4902c5
image/png.(*decoder).decode(0xc000052c00)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:374 +0x1ae fp=0xc00005fe28 sp=0xc00005fd58 pc=0x48f56e
image/png.(*decoder).parseIDAT(0xc000052c00, 0x52c78?)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:859 +0x25 fp=0xc00005fe40 sp=0xc00005fe28 pc=0x493085
image/png.(*decoder).parseChunk(0xc000052c00)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:918 +0x129 fp=0xc00005feb0 sp=0xc00005fe40 pc=0x493229
image/png.Decode({0x4d1fe8?, 0xc00006e020})
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:977 +0x11b fp=0xc00005ff18 sp=0xc00005feb0 pc=0x4937fb
main.main()
        /tmp/tmp.oRxzJ95Mq5/main.go:12 +0xa5 fp=0xc00005ff80 sp=0xc00005ff18 pc=0x493d65
runtime.main()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:250 +0x212 fp=0xc00005ffe0 sp=0xc00005ff80 pc=0x434352
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00005ffe8 sp=0xc00005ffe0 pc=0x45d121

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004efb0 sp=0xc00004ef90 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.forcegchelper()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:302 +0xad fp=0xc00004efe0 sp=0xc00004efb0 pc=0x4345ad
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004efe8 sp=0xc00004efe0 pc=0x45d121
created by runtime.init.6
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:290 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004f790 sp=0xc00004f770 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.bgsweep(0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcsweep.go:278 +0x8e fp=0xc00004f7c8 sp=0xc00004f790 pc=0x4217ae
runtime.gcenable.func1()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:178 +0x26 fp=0xc00004f7e0 sp=0xc00004f7c8 pc=0x416686
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004f7e8 sp=0xc00004f7e0 pc=0x45d121
created by runtime.gcenable
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:178 +0x6b

goroutine 4 [GC scavenge wait]:
runtime.gopark(0xc00006c000?, 0x4d18c8?, 0x1?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004ff70 sp=0xc00004ff50 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.(*scavengerState).park(0x550e60)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc00004ffa0 sp=0xc00004ff70 pc=0x41f853
runtime.bgscavenge(0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcscavenge.go:617 +0x45 fp=0xc00004ffc8 sp=0xc00004ffa0 pc=0x41fe25
runtime.gcenable.func2()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:179 +0x26 fp=0xc00004ffe0 sp=0xc00004ffc8 pc=0x416626
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004ffe8 sp=0xc00004ffe0 pc=0x45d121
created by runtime.gcenable
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:179 +0xaa

goroutine 5 [finalizer wait]:
runtime.gopark(0x434a97?, 0x49?, 0xe0?, 0xe7?, 0xc00004e770?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004e628 sp=0xc00004e608 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.runfinq()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mfinal.go:180 +0x10f fp=0xc00004e7e0 sp=0xc00004e628 pc=0x41578f
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004e7e8 sp=0xc00004e7e0 pc=0x45d121
created by runtime.createfing
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mfinal.go:157 +0x45
exit status 2

[bcmills]

(CC @nigeltao)

[robpike]

Interesting. I am on an M1 with 128GB of memory and see an error, not a crash.

[rsc]

I ran Rob's version and Ctrl-'ed it after a few seconds and it looks like it is allocating 3.4 TB. Probably different systems handle that differently.

[rsc]

png.DecodeConfig runs fine and says 255x1677721600, so this may be another instance where untrusted inputs should be checked with DecodeConfig before being passed to Decode.

[robpike]

I tried it on a smaller M1 and still saw the error, not the crash.

My summary: this is a buggy program that does not protect against bad inputs as instructed, but the variant behavior on different CPU types and/or operating systems is worth investigating.

[secsys-go]

Your program says it exits without error, but it doesn't. My version, which checks the error returned by Decode, reports

% go run x.go
<nil> flate: corrupt input before offset 5
% 

https://go.dev/play/p/bAa9vbuJfgT

It may be running out of memory or CPU time on the playground, but the input is invalid and the error is reported on a proper machine.

Did you find this example by fuzzing?

Yeah, I found it by go-fuzz, and the program exceeded the memory limit of the system when applying for a slice according to mutated size of the image.

In fact, what I want to say at the last println in playground is "If it exits without crashes, you should see this sentence", and the link of playground has been updated as go.dev/play

[nigeltao]

this may be another instance where untrusted inputs should be checked with DecodeConfig before being passed to Decode.

I'd agree with that.

I'd also say that the bug is a dupe of #5050 (whose title says image/gif but really applies to all the image formats).

[ianlancetaylor]

One approach we could use for this kind of problem is a incremental image build, as in https://go.dev/cl/417477. @nigeltao Any opinions about that approach? Thanks.

[gopherbot]

Change https://go.dev/cl/417477 mentions this issue: image/png: build large images incrementally

[nigeltao]

One approach we could use for this kind of problem is a incremental image build, as in https://go.dev/cl/417477. @nigeltao Any opinions about that approach?

I dropped a code review comment on that page.