cmd/compile: compiler can unexpectedly preserve memory

[zardlee1937]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go1.9.1

Does this issue reproduce with the latest release?

yes. every release.

What operating system and processor architecture are you using (go env)?

GOARCH=amd64
GOBIN=C:\Go\bin
GOEXE=.exe
GOHOSTARCH=amd64
GOHOSTOS=windows
GOOS=windows
GOPATH=D:\golang
GORACE=
GOROOT=C:\Go
GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
GCCGO=gccgo
CC=gcc
GOGCCFLAGS=-m64 -fmessage-length=0 -fdebug-prefix-map=C:\Users\zhalei\AppData\Local\Temp\go-build309785515=/tmp/go-build -gno-record-gcc-switches
CXX=g++
CGO_ENABLED=1
PKG_CONFIG=pkg-config
CGO_CFLAGS=-g -O2
CGO_CPPFLAGS=
CGO_CXXFLAGS=-g -O2
CGO_FFLAGS=-g -O2
CGO_LDFLAGS=-g -O2

What did you do?

Write codes for test GC of golang.
Here is what i use to test.

package main

import "fmt"
import "time"

type Node struct {
	next     *Node
	payload  [64]byte
}

func main() {
	root := new(Node)
	curr := root
	i := 0
	
	lastTime := time.Now()
	for {
		currTime := time.Now()
		elapsed := currTime.Sub(lastTime)
		lastTime = currTime
		// 10ms = 10000000ns
		if elapsed > 10000000 {
			fmt.Println("StopTime:", elapsed)
		}
		
		curr.next = new(Node)
		curr = curr.next
		
		i++
		//3000 nodes max
		if i >= 3000 {
			i = 0
			root = curr
		}
	}
}

What did you expect to see?

the program run well.

What did you see instead?

memory never be released until everything stop work. and then, i take my pc power off. -_-

[davecheney]

Here is a smaller reproduction which grows linearly on my machine

package main

type Node struct {
        next    *Node
        payload [64]byte
}

func main() {
        curr := new(Node)
        for {
                curr.next = new(Node)
                curr = curr.next
        }
}

[mattn]

I think this is not a bug since curr.next refer the pointer even if the variable is updated. This is not weak refernece.

[davecheney]

@mattn i'm sorry I don't understand what you mean. I've stared at this code a bunch, and it looks like to me that on each iteration curr is replaced with curr.next, so the previous value of curr is no longer referenced.

  +------+---------+
  + next | payload |
  +------+---------+
       |
       |    +------+---------+
curr ->`--> | next | payload |
            +------+---------+

And so on.

I'm sure i'm missing something, so I'd appreciate someone helping me understand what I'm missing.

[ianlancetaylor]

At least @davecheney 's version is an interesting result of compiler optimization. The very first call to new(Node) never escapes, so the compiler allocates it on the stack. It's next pointer is set to point to the next allocated Node. The Node chain continues to build, but nothing ever clears the the stack allocated pointer, so all the Nodes appear live to the garbage collector.

Basically 1) the fact that you use the same Go variable doesn't mean that the compiler does the same; 2) the fact that you call new doesn't mean that you always get heap memory.

Not sure whether or how to fix this. Does it come up in a real program?

CC @randall77

[mattn]

As long as curr.next holds the reference of *Node (i.e. strong reference), it will not be freed, I belieave. So if make it gc, Node should have uintptr instead of *Node.

package main

import (
	"runtime"
	"unsafe"
)

type Node struct {
	next    uintptr
	payload [64]byte
}

func main() {
	curr := new(Node)
	for {
		curr.next = uintptr(unsafe.Pointer(new(Node)))
		curr = (*Node)(unsafe.Pointer(curr.next))
		runtime.GC()
	}
}

Of course this is a dangerous code. Because the value indicated by next may be a pointer already freed.

[davecheney]

@mattn

As long as curr.next holds the reference of *Node (i.e. strong reference), it will not be freed,

But curr is overwritten every loop, so on the next loop, curr.next is actually curr.next.next and so on. The previous value held in curr is now unreferenced so even though it's next field points to the current curr nothing points to it, the previous value of curr; ergo, it's garbage.

[davecheney]

@ianlancetaylor thanks for the explanation. I think, irrespective of if this comes up in real code or not, it's still quite a serious bug.

[mattn]

@davecheney
curr is overwritten but not overwritten the memory block pointed by curr.
pointer is a reference just not a value. if something refer the memory block, GC can sweep for marking.

[mattn]

@davecheney It GC collect the unreferenced-value immediately as you mentioned, following code can not make linked-list.

package main

type Node struct {
	next     *Node
	payload  [64]byte
}

func NewNode(curr *Node) *Node {
	newnode := new(Node)
	newnode.next = curr
	return newnode
}

func main() {
	curr := NewNode(nil)
	curr = NewNode(curr)
	curr = NewNode(curr)
}