Repository navigation
Description
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.7.5 darwin/amd64
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jeromefroelich/golang"
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mc/_m9vnrfs0qjg9g8rl1h4s9n80000gn/T/go-build857571992=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
What did you do?
When using the reflect and unsafe packages to access the underlying data for a string or slice it seems the compiler does not consider assignments to the Data field of these structs when performing escape analysis. Consider the following program, a runnable example on the Go Playground is here
package main
import (
"fmt"
"reflect"
"unsafe"
)
type immutableBytes []byte
func toString(b immutableBytes) string {
var s string
if len(b) == 0 {
return s
}
strHeader := (*reflect.StringHeader)(unsafe.Pointer(&s))
strHeader.Data = (*reflect.SliceHeader)(unsafe.Pointer(&b)).Data
l := len(b)
strHeader.Len = l
return s
}
func getString() string {
b := immutableBytes("foobarbaz")
s := toString(b)
return s
}
//go:noinline
func getStringNoInline() string {
b := immutableBytes("foobarbaz")
s := toString(b)
return s
}
func main() {
s := getString()
fmt.Println(s)
s = getStringNoInline()
fmt.Println(s)
}
What did you expect to see?
The unsafe package states that the following pattern is valid:
Conversion of a reflect.SliceHeader or reflect.StringHeader Data field to or from Pointer.
Consequently, I would expect to see the same output for both Println calls. That is:
foobarbaz
foobarbaz
What did you see instead?
Instead, the second Println call prints random data. For example, in one run, the output was:
foobarbaz
|�B���J
It seems that the problem is that the compiler does not "see" that the pointer to the underlying bytes is being returned in the string. Consequently, since the compiler thinks the bytes slice does not escape, it is allocating it on the stack. This isn't a problem in the call to getString because the function is inlined and so a pointer to the data is valid for subsequent instructions on the stack. However, if we force the compiler to not inline the function, as in getStringNoInline, then the data for the byte slice is once again allocated on the stack, but the pointer to is no longer valid when the function returns and its call stack is popped off the stack. Escape analysis output from the compiler seems to support this theory. Relevant lines are highlighted below:
$ go build -gcflags -m
# github.com/jeromefroe/test_unsafe_conversion
./main.go:11: can inline toString
./main.go:25: can inline getString
./main.go:27: inlining call to toString
./main.go:34: inlining call to toString
./main.go:39: inlining call to getString <-----
./main.go:39: inlining call to toString
./main.go:11: toString b does not escape
./main.go:17: toString &s does not escape
./main.go:18: toString &b does not escape
./main.go:26: getString immutableBytes("foobarbaz") does not escape <-----
./main.go:27: getString &s does not escape
./main.go:27: getString &b does not escape
./main.go:33: getStringNoInline immutableBytes("foobarbaz") does not escape <-----
./main.go:34: getStringNoInline &s does not escape
./main.go:34: getStringNoInline &b does not escape
./main.go:40: s escapes to heap
./main.go:43: s escapes to heap
./main.go:39: main immutableBytes("foobarbaz") does not escape
./main.go:39: main &s does not escape
./main.go:39: main &b does not escape
./main.go:40: main ... argument does not escape
./main.go:43: main ... argument does not escape
Activity
bradfitz commentedon Mar 28, 2017
Can you try Go 1.8?
jeromefroe commentedon Mar 28, 2017
Sure thing:
Seems the results are the same.
josharian commentedon Mar 28, 2017
@dr2chase @mdempsky
dr2chase commentedon Mar 28, 2017
Local discussion of this bug is that it's a special case flaw in the reflect package; we exposed those fields as uintptr, and so without a special case for that particular uintptr, the escape analysis code is "not wrong" because it's not actually pointers. Nonetheless, the special-case fix needs to be applied to the "not wrong" code in escape analysis.
There's a recent CL to some other part of the compiler dealing with the same issue elsewhere, I went looking for it but did not find it immediately.
josharian commentedon Mar 28, 2017
@dr2chase #19168?
dr2chase commentedon Mar 28, 2017
That's the one. Thanks.
gopherbot commentedon Mar 28, 2017
CL https://golang.org/cl/38738 mentions this issue.
gopherbot commentedon Apr 5, 2017
CL https://golang.org/cl/39616 mentions this issue.
[release-branch.go1.8] cmd/compile: added special case for reflect he…