Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.7.5 darwin/amd64
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jeromefroelich/golang"
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mc/_m9vnrfs0qjg9g8rl1h4s9n80000gn/T/go-build857571992=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
What did you do?
When using the reflect and unsafe packages to access the underlying data for a string or slice it seems the compiler does not consider assignments to the Data field of these structs when performing escape analysis. Consider the following program, a runnable example on the Go Playground is here
package main
import (
"fmt"
"reflect"
"unsafe"
)
type immutableBytes []byte
func toString(b immutableBytes) string {
var s string
if len(b) == 0 {
return s
}
strHeader := (*reflect.StringHeader)(unsafe.Pointer(&s))
strHeader.Data = (*reflect.SliceHeader)(unsafe.Pointer(&b)).Data
l := len(b)
strHeader.Len = l
return s
}
func getString() string {
b := immutableBytes("foobarbaz")
s := toString(b)
return s
}
//go:noinline
func getStringNoInline() string {
b := immutableBytes("foobarbaz")
s := toString(b)
return s
}
func main() {
s := getString()
fmt.Println(s)
s = getStringNoInline()
fmt.Println(s)
}
What did you expect to see?
The unsafe package states that the following pattern is valid:
Conversion of a reflect.SliceHeader or reflect.StringHeader Data field to or from Pointer.
Consequently, I would expect to see the same output for both Println calls. That is:
What did you see instead?
Instead, the second Println call prints random data. For example, in one run, the output was:
It seems that the problem is that the compiler does not "see" that the pointer to the underlying bytes is being returned in the string. Consequently, since the compiler thinks the bytes slice does not escape, it is allocating it on the stack. This isn't a problem in the call to getString because the function is inlined and so a pointer to the data is valid for subsequent instructions on the stack. However, if we force the compiler to not inline the function, as in getStringNoInline, then the data for the byte slice is once again allocated on the stack, but the pointer to is no longer valid when the function returns and its call stack is popped off the stack. Escape analysis output from the compiler seems to support this theory. Relevant lines are highlighted below:
$ go build -gcflags -m
# github.com/jeromefroe/test_unsafe_conversion
./main.go:11: can inline toString
./main.go:25: can inline getString
./main.go:27: inlining call to toString
./main.go:34: inlining call to toString
./main.go:39: inlining call to getString <-----
./main.go:39: inlining call to toString
./main.go:11: toString b does not escape
./main.go:17: toString &s does not escape
./main.go:18: toString &b does not escape
./main.go:26: getString immutableBytes("foobarbaz") does not escape <-----
./main.go:27: getString &s does not escape
./main.go:27: getString &b does not escape
./main.go:33: getStringNoInline immutableBytes("foobarbaz") does not escape <-----
./main.go:34: getStringNoInline &s does not escape
./main.go:34: getStringNoInline &b does not escape
./main.go:40: s escapes to heap
./main.go:43: s escapes to heap
./main.go:39: main immutableBytes("foobarbaz") does not escape
./main.go:39: main &s does not escape
./main.go:39: main &b does not escape
./main.go:40: main ... argument does not escape
./main.go:43: main ... argument does not escape
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?go version go1.7.5 darwin/amd64What operating system and processor architecture are you using (
go env)?What did you do?
When using the
reflectandunsafepackages to access the underlying data for a string or slice it seems the compiler does not consider assignments to theDatafield of these structs when performing escape analysis. Consider the following program, a runnable example on the Go Playground is hereWhat did you expect to see?
The unsafe package states that the following pattern is valid:
Consequently, I would expect to see the same output for both
Printlncalls. That is:What did you see instead?
Instead, the second
Printlncall prints random data. For example, in one run, the output was:It seems that the problem is that the compiler does not "see" that the pointer to the underlying bytes is being returned in the string. Consequently, since the compiler thinks the bytes slice does not escape, it is allocating it on the stack. This isn't a problem in the call to
getStringbecause the function is inlined and so a pointer to the data is valid for subsequent instructions on the stack. However, if we force the compiler to not inline the function, as ingetStringNoInline, then the data for the byte slice is once again allocated on the stack, but the pointer to is no longer valid when the function returns and its call stack is popped off the stack. Escape analysis output from the compiler seems to support this theory. Relevant lines are highlighted below: