cmd/dist: misc/cgo/test: incorrect pie options used for ppc64x

[laboger]

What version of Go are you using (go version)?
go version go1.9 linux/ppc64le and go upstream

Does this issue reproduce with the latest release?
Yes, but the failures only happen on Unbuntu 16.04 and no other distro we've tried.

What operating system and processor architecture are you using (go env)?
Failures have only been seen on Ubuntu 16.04.
ppc64le

When building golang with ./all.bash, failures occur in some tests in misc/cgo/test with errors like this:
error while loading shared libraries: R_PPC64_ADDR16_HA re136cd776c for symbol `' out of range

After digging through cmd/dist/test.go I found that the failures occurred only when the tests were built with the external linker when passing -pie as a flag to the external linker. For example in misc/cgo/nocgo I can build the test by hand:
go test -ldflags '-linkmode=external "-extldflags=-pie"' -c
./nocgo.test
./nocgo.test: error while loading shared libraries: R_PPC64_ADDR16_HA re136cd776c for symbol `' out of range

According to the linker documentation -pie should only be used with position independent code, but for ppc64x, it is not PIC by default. Either these tests should not be built and run on ppc64x with these options, or different options should be used. It seems like -buildmode=pie is what should be used here because that will build PIC but I tried that and that resulted in different build errors which I haven't investigated yet.

[ianlancetaylor]

Why would the failures only occur on Ubuntu? Is there some difference in the default system linker?

[ianlancetaylor]

In any case I agree that this test seems a bit dubious and that it should probably be using -buildmode=pie (which did not exist when the test was added).

[ianlancetaylor]

(The -pie tests were added in https://golang.org/cl/6280).

[laboger]

If I try to build these tests with -buildmode=pie it does build the test and it says it passes but I get this warning. Not sure if it matters:

searching for runtime.a in $WORK/_/home/boger/golang/go1.9/go/misc/cgo/nocgo/test/runtime.a
searching for runtime.a in $WORK/runtime.a
searching for runtime/cgo.a in $WORK//home/boger/golang/go1.9/go/misc/cgo/nocgo/_test/runtime/cgo.a
searching for runtime/cgo.a in $WORK/runtime/cgo.a
searching for runtime/cgo.a in /home/boger/golang/go1.9/go/pkg/linux_ppc64le_shared/runtime/cgo.a
warning: unable to find runtime/cgo.a

My toolchain doesn't have a pkg/linux_ppc64le_shared directory and I'm not sure what causes it to get built. I went looking for that directory on various systems and golang builds I've done and see that it is there sometimes but not others and I'm not aware of what I might have done to cause it to be created and populated.

[laboger]

I don't know why this failure only happens on Ubuntu 16.04. I built on a 17.04 as well as earlier Ubuntu versions, and some RHEL 7.x systems and they all worked. 16.04 has 2.26.1, 17.04 has 2.28. I believe the message comes from the dynamic linker at runtime so maybe the glibc version is a factor. Once I noticed that the testscase is not using valid linker options for ppc64le, I no longer thought it was important why it failed, especially since it did not seem easy for me to figure that out.

[gopherbot]

Change https://golang.org/cl/66870 mentions this issue: cmd/dist,misg/cgo/test: fix pie options for tests on ppc64x

[mkumatag]

with latest code, following tests are passed:

##### ../misc/cgo/test
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/test	1.706s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/test	1.731s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/testtls	0.003s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/testtls	0.003s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/test	1.692s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/testtls	0.004s
PASS
ok  	_/root/workspace/test-golang-upstream/go/misc/cgo/nocgo	0.006s

#####

But breaking later in the code:

##### ../misc/cgo/testcarchive
--- FAIL: TestCompileWithoutShared (1.26s)
	carchive_test.go:600: [go build -buildmode=c-archive -gcflags=-shared=false -o libgo2.a libgo2]
	carchive_test.go:602: 
	carchive_test.go:609: [gcc -fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build472192193=/tmp/go-build -gno-record-gcc-switches -I pkg/linux_ppc64le_shared -o ./testnoshared main5.c libgo2.a]
	carchive_test.go:611: 
	carchive_test.go:618: [./testnoshared 3]
	carchive_test.go:620: ./testnoshared: error while loading shared libraries: R_PPC64_ADDR16_HA re1028a6ca8 for symbol `' out of range
	carchive_test.go:361: got exit status 127; expected signal broken pipe
FAIL

[mkumatag]

created #22126 to track above issue.

[gopherbot]

Change https://golang.org/cl/73970 mentions this issue: runtime: use -buildmode=pie in testCgoPprofPIE instead of -extldflags=-pie