runtime: MSAN failed to run on arm64 with non-PIE builds

[zhangfannie]

What version of Go are you using (go version)?

$ go version
go version devel +0dd120df7e Sun Aug 18 01:16:33 2019 +0000 linux/arm64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/fanzha02/.cache/go-build"
GOENV="/home/fanzha02/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/fanzha02/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/fanzha02/work/go_project/golang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/fanzha02/work/go_project/golang/pkg/tool/linux_arm64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/fanzha02/work/go_project/golang/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build281474518=/tmp/go-build -gno-record-gcc-switches"

What did you do?

cd golang/src
CC=clang ../bin/go tool dist test testsanitizers/msan
clang --version
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

What did you expect to see?

the msan tests report errors:
--- FAIL: TestMSAN/msan (4.06s)
msan_test.go:53: /tmp/TestMSAN490720188/msan exited with exit status 77
FATAL: Code 0x00000041d7a0 is out of application range. Non-PIE build?
FATAL: MemorySanitizer can not mmap the shadow memory.
FATAL: Make sure to compile with -fPIE and to link with -pie.
FATAL: Disabling ASLR is known to cause this error.
FATAL: If running under GDB, try 'set disable-randomization off'.
==20194==Process memory map follows:
0x000000400000-0x000000636000 /tmp/TestMSAN490720188/msan
0x000000646000-0x00000064b000 /tmp/TestMSAN490720188/msan
0x00000064b000-0x000000662000 /tmp/TestMSAN490720188/msan
0x000000662000-0x000002ba7000
0xffff905dc000-0xffff9094e000
0xffff9094e000-0xffff90a8e000 /lib/aarch64-linux-gnu/libc-2.27.so
0xffff90a8e000-0xffff90a9d000 /lib/aarch64-linux-gnu/libc-2.27.so
0xffff90a9d000-0xffff90aa1000 /lib/aarch64-linux-gnu/libc-2.27.so
0xffff90aa1000-0xffff90aa3000 /lib/aarch64-linux-gnu/libc-2.27.so
0xffff90aa3000-0xffff90aa7000
0xffff90aa7000-0xffff90aba000 /lib/aarch64-linux-gnu/libgcc_s.so.1
0xffff90aba000-0xffff90ac9000 /lib/aarch64-linux-gnu/libgcc_s.so.1
0xffff90ac9000-0xffff90aca000 /lib/aarch64-linux-gnu/libgcc_s.so.1
0xffff90aca000-0xffff90acb000 /lib/aarch64-linux-gnu/libgcc_s.so.1
0xffff90acb000-0xffff90ace000 /lib/aarch64-linux-gnu/libdl-2.27.so
0xffff90ace000-0xffff90ade000 /lib/aarch64-linux-gnu/libdl-2.27.so
0xffff90ade000-0xffff90adf000 /lib/aarch64-linux-gnu/libdl-2.27.so
0xffff90adf000-0xffff90ae0000 /lib/aarch64-linux-gnu/libdl-2.27.so
0xffff90ae0000-0xffff90b89000 /lib/aarch64-linux-gnu/libm-2.27.so
0xffff90b89000-0xffff90b98000 /lib/aarch64-linux-gnu/libm-2.27.so
0xffff90b98000-0xffff90b99000 /lib/aarch64-linux-gnu/libm-2.27.so
0xffff90b99000-0xffff90b9a000 /lib/aarch64-linux-gnu/libm-2.27.so
0xffff90b9a000-0xffff90ba0000 /lib/aarch64-linux-gnu/librt-2.27.so
0xffff90ba0000-0xffff90baf000 /lib/aarch64-linux-gnu/librt-2.27.so
0xffff90baf000-0xffff90bb0000 /lib/aarch64-linux-gnu/librt-2.27.so
0xffff90bb0000-0xffff90bb1000 /lib/aarch64-linux-gnu/librt-2.27.so
0xffff90bb1000-0xffff90bc8000 /lib/aarch64-linux-gnu/libpthread-2.27.so
0xffff90bc8000-0xffff90bd7000 /lib/aarch64-linux-gnu/libpthread-2.27.so
0xffff90bd7000-0xffff90bd8000 /lib/aarch64-linux-gnu/libpthread-2.27.so
0xffff90bd8000-0xffff90bd9000 /lib/aarch64-linux-gnu/libpthread-2.27.so
0xffff90bd9000-0xffff90bdd000
0xffff90bdf000-0xffff90bf3000
0xffff90bf3000-0xffff90c10000 /lib/aarch64-linux-gnu/ld-2.27.so
0xffff90c10000-0xffff90c1d000
0xffff90c1d000-0xffff90c1e000 [vvar]
0xffff90c1e000-0xffff90c1f000 [vdso]
0xffff90c1f000-0xffff90c20000 /lib/aarch64-linux-gnu/ld-2.27.so
0xffff90c20000-0xffff90c22000 /lib/aarch64-linux-gnu/ld-2.27.so
0xffffeace2000-0xffffead12000 [stack]
==20194==End of process memory map.

What did you see instead?

pass

[zhangfannie]

The below are some findings.
<1>. Refer to the implementation of msan mapping (line 82 of https://github.com/llvm-mirror/compiler-rt/blob/6e2190f8eba3e327f19d687d609e7dda56c75736/lib/msan/msan.h
), the error is caused by the binary(0x00000041d7a0) is loaded to the invalid segment.
<2>. cat test.cc

#include <stdio.h>
#include <malloc.h>

int main(int argc, char** argv) {
  int* a = (int*)malloc(sizeof(int)*10);
  a[5] = 0;
  if (a[argc])
    printf("xx\n");
  return 0;
}

Run the command "clang -fsanitize=memory -fno-omit-frame-pointer -g -O2 -v test.cc", the log shows the program is linked with pie mode.
<3>. cat msan.go

package main

/*
#include <stdint.h>

void f(int32_t *p, int n) {
  int i;

  for (i = 0; i < n; i++) {
    p[i] = (int32_t)i;
  }
}
*/
import "C"

import (
        "fmt"
        "os"
        "unsafe"
)

func main() {
        a := make([]int32, 10)
        C.f((*C.int32_t)(unsafe.Pointer(&a[0])), C.int(len(a)))
        for i, v := range a {
                if i != int(v) {
                        fmt.Println("bad %d: %v\n", i, a)
                        os.Exit(1)
                }
        }
}

If I used PIE mode to build the msan program, the program can pass on arm64.
CC=clang go build -buildmode=pie -msan msan.go
./msan

I will post a fixing CL, using PIE link mode when using MSAN.

[ianlancetaylor]

Likely the fix should be to buildModeInit in cmd/go/internal/work/init.go.

[zhangfannie]

@ianlancetaylor Yes, I did that too. Thank you. 🙂

[gopherbot]

Change https://golang.org/cl/190482 mentions this issue: cmd/go/internal/work: use pie link mode when using MSAN on arm64