Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: potential memory leak in Go1.11.2 #29185

Closed
andig opened this issue Dec 12, 2018 · 4 comments

Comments

Projects
None yet
5 participants
@andig
Copy link

commented Dec 12, 2018

What version of Go are you using (go version)?

$ go version
go version go1.11.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/andig/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/andig/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.2/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/andig/htdocs/ingress/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/73/89ycv7qn51j4kbm04jsz9b840000gn/T/go-build037396939=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I realize this is very unlikely but I wouldn't expect this simple loop to show increasing memory usage:

for {
	time.Sleep(time.Second)
	runtime.GC() // this is optional
	var memstats runtime.MemStats
	runtime.ReadMemStats(&memstats)
	fmt.Printf("%db\n", memstats.Alloc)
}

What did you expect to see?

Consant memory usage

What did you see instead?

Rising memory:

❯ go run cmd/test/main.go
68360b
...
68376b
...
68832b
...
69288b
...
69744b
...
71536b
...
73328b

Omitting the GC() call makes the memory rise quicker in smaller steps.

@ALTree ALTree changed the title 1.11.2 Potential memory leak runtime: potential memory leak in Go1.11.2 Dec 12, 2018

@ALTree ALTree added this to the Go1.12 milestone Dec 12, 2018

@networkimprov

This comment has been minimized.

Copy link

commented Dec 13, 2018

How many iterations have you tried?

@mark-rushakoff

This comment has been minimized.

Copy link
Contributor

commented Dec 13, 2018

With this repro on go version devel +9a11b73635 Wed Dec 12 15:00:01 2018 +0000 darwin/amd64:

package main

import (
	"fmt"
	"runtime"
	"time"
)

func main() {
	for {
		time.Sleep(time.Second)
		// runtime.GC() // this is optional
		var memstats runtime.MemStats
		runtime.ReadMemStats(&memstats)
		// println(memstats.Alloc)
		fmt.Printf("%db\n", memstats.Alloc)
	}
}

And running go build then running the binary with GODEBUG=allocfreetrace=1, I see repeated output indicating that the allocations are happening in fmt.Printf (converting memstats.Alloc to an interface, and a defer while writing to stdout):

73688b
tracealloc(0xc00007e2d0, 0x10, runtime.uint64InterfacePtr)
goroutine 1 [running]:
runtime.mallocgc(0x10, 0x10a46a0, 0x104d400, 0xc00008e8e8)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/malloc.go:1011 +0x4ae fp=0xc00008e868 sp=0xc00008e7c8 pc=0x100a6de
runtime.convT64(0x11fe8, 0xc000086000)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/iface.go:345 +0x5b fp=0xc00008e898 sp=0xc00008e868 pc=0x10085bb
main.main()
	/tmp/m.go:16 +0x80 fp=0xc00008ff98 sp=0xc00008e898 pc=0x1093870
runtime.main()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/proc.go:200 +0x204 fp=0xc00008ffe0 sp=0xc00008ff98 pc=0x1029414
runtime.goexit()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x1050b91

tracealloc(0xc0000805c0, 0x40, runtime._defer)
goroutine 1 [running]:
runtime.systemstack_switch()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:311 fp=0xc00008e6a8 sp=0xc00008e6a0 pc=0x104ec60
runtime.newdefer(0x80000000008, 0x7ffff80000000000)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/panic.go:240 +0x17a fp=0xc00008e718 sp=0xc00008e6a8 pc=0x1026d4a
runtime.deferproc(0xc000000008, 0x10cda70)
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/panic.go:107 +0x3e fp=0xc00008e748 sp=0xc00008e718 pc=0x102681e
internal/poll.(*FD).Write(0xc000088000, 0xc00007e018, 0x7, 0x8, 0x0, 0x0, 0x0)
	/Users/mr/gotip/src/github.com/golang/go/src/internal/poll/fd_unix.go:258 +0x87 fp=0xc00008e7b0 sp=0xc00008e748 pc=0x1084c87
os.(*File).write(...)
	/Users/mr/gotip/src/github.com/golang/go/src/os/file_unix.go:266
os.(*File).Write(0xc000086000, 0xc00007e018, 0x7, 0x8, 0x1, 0x1, 0x10085bb)
	/Users/mr/gotip/src/github.com/golang/go/src/os/file.go:145 +0x74 fp=0xc00008e830 sp=0xc00008e7b0 pc=0x1085f24
fmt.Fprintf(0x10ddf20, 0xc000086000, 0x10c6b4c, 0x4, 0xc00008ff78, 0x1, 0x1, 0x7, 0x0, 0x0)
	/Users/mr/gotip/src/github.com/golang/go/src/fmt/print.go:200 +0xa5 fp=0xc00008e898 sp=0xc00008e830 pc=0x108bcb5
fmt.Printf(...)
	/Users/mr/gotip/src/github.com/golang/go/src/fmt/print.go:208
main.main()
	/tmp/m.go:16 +0xf7 fp=0xc00008ff98 sp=0xc00008e898 pc=0x10938e7
runtime.main()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/proc.go:200 +0x204 fp=0xc00008ffe0 sp=0xc00008ff98 pc=0x1029414
runtime.goexit()
	/Users/mr/gotip/src/github.com/golang/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x1050b91

But if I comment out the fmt.Printf and uncomment the println, the allocations immediately steady out and the same value of memstats.Alloc are printed, at least for the first minute or two.

I really doubt fmt.Printf is actually leaking memory. Is it just the runtime allowing space for defer and interface conversion to grow to a certain size before garbage collecting that area?

@Opposition

This comment has been minimized.

Copy link

commented Dec 13, 2018

Is it because var memstats runtime.MemStats is heap allocated when you use fmt.Printf?

Using fmt.Printf

$ go build -gcflags '-m' main.go
# command-line-arguments
./main.go:16:31: memstats.Alloc escapes to heap
./main.go:14:24: main &memstats does not escape
./main.go:16:13: main ... argument does not escape

Using println

$ go build -gcflags '-m' main.go
# command-line-arguments
./main.go:13:24: main &memstats does not escape

I think it is common for the escape analysis to allocate on the heap, if the function takes in an interface type.

@andig

This comment has been minimized.

Copy link
Author

commented Dec 14, 2018

Thank you all for the detailed feedback! From what I've understood this is the expected behaviour (memory increase until GC kicks in). Closing for now, please let me know if I did misunderstand the feedback.

@andig andig closed this Dec 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.