cmd/go: cross-compiling for macOS causes net.a to be compiled from sources

What version of Go are you using (go version)?

$ go version
go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes. This is a go1.10 - 1.11 regression.

What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/workspace"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build338071509=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ cat dns/main.go                                                                                          
package main

import "net"

func main() {
	net.LookupIP("google.com")
}

$ ls -la /usr/local/go/pkg/darwin_amd64/net.a
-rw-r--r-- 1 root root 1241304 Jun  6 23:51 /usr/local/go/pkg/darwin_amd64/net.a

$ go clean -cache

$ go build -x -o dns-test ./dns
WORK=/tmp/go-build098597944
mkdir -p $WORK/b002/
cat >$WORK/b002/importcfg << 'EOF' # internal
# import config
importmap golang_org/x/net/route=vendor/golang_org/x/net/route
packagefile context=/usr/local/go/pkg/darwin_amd64/context.a
packagefile errors=/usr/local/go/pkg/darwin_amd64/errors.a
packagefile vendor/golang_org/x/net/route=/usr/local/go/pkg/darwin_amd64/vendor/golang_org/x/net/route.a
packagefile internal/nettrace=/usr/local/go/pkg/darwin_amd64/internal/nettrace.a
packagefile internal/poll=/usr/local/go/pkg/darwin_amd64/internal/poll.a
packagefile internal/singleflight=/usr/local/go/pkg/darwin_amd64/internal/singleflight.a
packagefile io=/usr/local/go/pkg/darwin_amd64/io.a
packagefile math/rand=/usr/local/go/pkg/darwin_amd64/math/rand.a
packagefile os=/usr/local/go/pkg/darwin_amd64/os.a
packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a
packagefile sort=/usr/local/go/pkg/darwin_amd64/sort.a
packagefile sync=/usr/local/go/pkg/darwin_amd64/sync.a
packagefile sync/atomic=/usr/local/go/pkg/darwin_amd64/sync/atomic.a
packagefile syscall=/usr/local/go/pkg/darwin_amd64/syscall.a
packagefile time=/usr/local/go/pkg/darwin_amd64/time.a
EOF
cd /usr/local/go/src/net
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b002/_pkg_.a -trimpath $WORK/b002 -p net -std -buildid _tpOUnWlUki4sCfaeRYL/_tpOUnWlUki4sCfaeRYL -goversion go1.10.3 -D "" -importcfg $WORK/b002/importcfg -pack -c=4 ./addrselect.go ./cgo_stub.go ./conf.go ./dial.go ./dnsclient.go ./dnsclient_unix.go ./dnsconfig_unix.go ./dnsmsg.go ./error_posix.go ./fd_unix.go ./file.go ./file_unix.go ./hook.go ./hook_unix.go ./hosts.go ./interface.go ./interface_bsd.go ./interface_darwin.go ./ip.go ./iprawsock.go ./iprawsock_posix.go ./ipsock.go ./ipsock_posix.go ./lookup.go ./lookup_unix.go ./mac.go ./net.go ./nss.go ./parse.go ./pipe.go ./port.go ./port_unix.go ./rawconn.go ./sendfile_stub.go ./sock_bsd.go ./sock_posix.go ./sockopt_bsd.go ./sockopt_posix.go ./sockoptip_bsdvar.go ./sockoptip_posix.go ./sys_cloexec.go ./tcpsock.go ./tcpsock_posix.go ./tcpsockopt_darwin.go ./tcpsockopt_posix.go ./udpsock.go ./udpsock_posix.go ./unixsock.go ./unixsock_posix.go ./writev_unix.go
/usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b002/_pkg_.a # internal
cp $WORK/b002/_pkg_.a /root/.cache/go-build/71/71d5652836fbf72132d2864101b08cfb7dc5657beaa4cd0804e7d893646a3d60-d # internal
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg << 'EOF' # internal
# import config
packagefile net=$WORK/b002/_pkg_.a
packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a
EOF
cd /workspace/src/go.example.com/dns
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -p main -complete -buildid t_6XipuZQQ3V_Rlugib8/t_6XipuZQQ3V_Rlugib8 -goversion go1.10.3 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 ./main.go
/usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/_pkg_.a # internal
cp $WORK/b001/_pkg_.a /root/.cache/go-build/35/35e87cb5622549aaa8a434a32334706a5b3862ff7d49ce9daf3b29ba88094f41-d # internal
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile go.example.com/dns=$WORK/b001/_pkg_.a
packagefile net=$WORK/b002/_pkg_.a
packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a
packagefile context=/usr/local/go/pkg/darwin_amd64/context.a
packagefile errors=/usr/local/go/pkg/darwin_amd64/errors.a
packagefile vendor/golang_org/x/net/route=/usr/local/go/pkg/darwin_amd64/vendor/golang_org/x/net/route.a
packagefile internal/nettrace=/usr/local/go/pkg/darwin_amd64/internal/nettrace.a
packagefile internal/poll=/usr/local/go/pkg/darwin_amd64/internal/poll.a
packagefile internal/singleflight=/usr/local/go/pkg/darwin_amd64/internal/singleflight.a
packagefile io=/usr/local/go/pkg/darwin_amd64/io.a
packagefile math/rand=/usr/local/go/pkg/darwin_amd64/math/rand.a
packagefile os=/usr/local/go/pkg/darwin_amd64/os.a
packagefile sort=/usr/local/go/pkg/darwin_amd64/sort.a
packagefile sync=/usr/local/go/pkg/darwin_amd64/sync.a
packagefile sync/atomic=/usr/local/go/pkg/darwin_amd64/sync/atomic.a
packagefile syscall=/usr/local/go/pkg/darwin_amd64/syscall.a
packagefile time=/usr/local/go/pkg/darwin_amd64/time.a
packagefile runtime/internal/atomic=/usr/local/go/pkg/darwin_amd64/runtime/internal/atomic.a
packagefile runtime/internal/sys=/usr/local/go/pkg/darwin_amd64/runtime/internal/sys.a
packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a
packagefile reflect=/usr/local/go/pkg/darwin_amd64/reflect.a
packagefile math=/usr/local/go/pkg/darwin_amd64/math.a
packagefile internal/testlog=/usr/local/go/pkg/darwin_amd64/internal/testlog.a
packagefile internal/race=/usr/local/go/pkg/darwin_amd64/internal/race.a
packagefile strconv=/usr/local/go/pkg/darwin_amd64/strconv.a
packagefile unicode/utf8=/usr/local/go/pkg/darwin_amd64/unicode/utf8.a
packagefile unicode=/usr/local/go/pkg/darwin_amd64/unicode.a
packagefile internal/cpu=/usr/local/go/pkg/darwin_amd64/internal/cpu.a
EOF
mkdir -p $WORK/b001/exe/
cd .
/usr/local/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=6-oHCWY8Z57CMNCfUB-n/t_6XipuZQQ3V_Rlugib8/9dtyWN9RKHU5XwbNrmpr/6-oHCWY8Z57CMNCfUB-n -extld=gcc $WORK/b001/_pkg_.a
/usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal
cp $WORK/b001/exe/a.out dns-test
rm -r $WORK/b001/

What did you expect to see?

I expected net.a from the macOS distribution to be used.

What did you see instead?

The package net.a was compiled from sources instead.

[davecheney]

How did you get the darwin/amd64 .a files on your Linux/amd64 system? Did they come in the tar ball From golang.org? Did you build them from source?

…

On 5 Sep 2018, at 15:14, juliengric ***@***.***> wrote: What version of Go are you using (go version)? $ go version go version go1.10.3 linux/amd64 Does this issue reproduce with the latest release? I haven't tried with the latest release. What operating system and processor architecture are you using (go env)? $ go env GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/workspace" GORACE="" GOROOT="/usr/local/go" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" CXX="g++" CGO_ENABLED="1" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build338071509=/tmp/go-build -gno-record-gcc-switches" What did you do? $ cat dns/main.go package main import "net" func main() { net.LookupIP("google.com") } $ ls -la /usr/local/go/pkg/darwin_amd64/net.a -rw-r--r-- 1 root root 1241304 Jun 6 23:51 /usr/local/go/pkg/darwin_amd64/net.a $ go clean -cache $ go build -x -o dns-test ./dns WORK=/tmp/go-build098597944 mkdir -p $WORK/b002/ cat >$WORK/b002/importcfg << 'EOF' # internal # import config importmap golang_org/x/net/route=vendor/golang_org/x/net/route packagefile context=/usr/local/go/pkg/darwin_amd64/context.a packagefile errors=/usr/local/go/pkg/darwin_amd64/errors.a packagefile vendor/golang_org/x/net/route=/usr/local/go/pkg/darwin_amd64/vendor/golang_org/x/net/route.a packagefile internal/nettrace=/usr/local/go/pkg/darwin_amd64/internal/nettrace.a packagefile internal/poll=/usr/local/go/pkg/darwin_amd64/internal/poll.a packagefile internal/singleflight=/usr/local/go/pkg/darwin_amd64/internal/singleflight.a packagefile io=/usr/local/go/pkg/darwin_amd64/io.a packagefile math/rand=/usr/local/go/pkg/darwin_amd64/math/rand.a packagefile os=/usr/local/go/pkg/darwin_amd64/os.a packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a packagefile sort=/usr/local/go/pkg/darwin_amd64/sort.a packagefile sync=/usr/local/go/pkg/darwin_amd64/sync.a packagefile sync/atomic=/usr/local/go/pkg/darwin_amd64/sync/atomic.a packagefile syscall=/usr/local/go/pkg/darwin_amd64/syscall.a packagefile time=/usr/local/go/pkg/darwin_amd64/time.a EOF cd /usr/local/go/src/net /usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b002/_pkg_.a -trimpath $WORK/b002 -p net -std -buildid _tpOUnWlUki4sCfaeRYL/_tpOUnWlUki4sCfaeRYL -goversion go1.10.3 -D "" -importcfg $WORK/b002/importcfg -pack -c=4 ./addrselect.go ./cgo_stub.go ./conf.go ./dial.go ./dnsclient.go ./dnsclient_unix.go ./dnsconfig_unix.go ./dnsmsg.go ./error_posix.go ./fd_unix.go ./file.go ./file_unix.go ./hook.go ./hook_unix.go ./hosts.go ./interface.go ./interface_bsd.go ./interface_darwin.go ./ip.go ./iprawsock.go ./iprawsock_posix.go ./ipsock.go ./ipsock_posix.go ./lookup.go ./lookup_unix.go ./mac.go ./net.go ./nss.go ./parse.go ./pipe.go ./port.go ./port_unix.go ./rawconn.go ./sendfile_stub.go ./sock_bsd.go ./sock_posix.go ./sockopt_bsd.go ./sockopt_posix.go ./sockoptip_bsdvar.go ./sockoptip_posix.go ./sys_cloexec.go ./tcpsock.go ./tcpsock_posix.go ./tcpsockopt_darwin.go ./tcpsockopt_posix.go ./udpsock.go ./udpsock_posix.go ./unixsock.go ./unixsock_posix.go ./writev_unix.go /usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b002/_pkg_.a # internal cp $WORK/b002/_pkg_.a /root/.cache/go-build/71/71d5652836fbf72132d2864101b08cfb7dc5657beaa4cd0804e7d893646a3d60-d # internal mkdir -p $WORK/b001/ cat >$WORK/b001/importcfg << 'EOF' # internal # import config packagefile net=$WORK/b002/_pkg_.a packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a EOF cd /workspace/src/go.example.com/dns /usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -p main -complete -buildid t_6XipuZQQ3V_Rlugib8/t_6XipuZQQ3V_Rlugib8 -goversion go1.10.3 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 ./main.go /usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/_pkg_.a # internal cp $WORK/b001/_pkg_.a /root/.cache/go-build/35/35e87cb5622549aaa8a434a32334706a5b3862ff7d49ce9daf3b29ba88094f41-d # internal cat >$WORK/b001/importcfg.link << 'EOF' # internal packagefile go.example.com/dns=$WORK/b001/_pkg_.a packagefile net=$WORK/b002/_pkg_.a packagefile runtime=/usr/local/go/pkg/darwin_amd64/runtime.a packagefile context=/usr/local/go/pkg/darwin_amd64/context.a packagefile errors=/usr/local/go/pkg/darwin_amd64/errors.a packagefile vendor/golang_org/x/net/route=/usr/local/go/pkg/darwin_amd64/vendor/golang_org/x/net/route.a packagefile internal/nettrace=/usr/local/go/pkg/darwin_amd64/internal/nettrace.a packagefile internal/poll=/usr/local/go/pkg/darwin_amd64/internal/poll.a packagefile internal/singleflight=/usr/local/go/pkg/darwin_amd64/internal/singleflight.a packagefile io=/usr/local/go/pkg/darwin_amd64/io.a packagefile math/rand=/usr/local/go/pkg/darwin_amd64/math/rand.a packagefile os=/usr/local/go/pkg/darwin_amd64/os.a packagefile sort=/usr/local/go/pkg/darwin_amd64/sort.a packagefile sync=/usr/local/go/pkg/darwin_amd64/sync.a packagefile sync/atomic=/usr/local/go/pkg/darwin_amd64/sync/atomic.a packagefile syscall=/usr/local/go/pkg/darwin_amd64/syscall.a packagefile time=/usr/local/go/pkg/darwin_amd64/time.a packagefile runtime/internal/atomic=/usr/local/go/pkg/darwin_amd64/runtime/internal/atomic.a packagefile runtime/internal/sys=/usr/local/go/pkg/darwin_amd64/runtime/internal/sys.a packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a packagefile reflect=/usr/local/go/pkg/darwin_amd64/reflect.a packagefile math=/usr/local/go/pkg/darwin_amd64/math.a packagefile internal/testlog=/usr/local/go/pkg/darwin_amd64/internal/testlog.a packagefile internal/race=/usr/local/go/pkg/darwin_amd64/internal/race.a packagefile strconv=/usr/local/go/pkg/darwin_amd64/strconv.a packagefile unicode/utf8=/usr/local/go/pkg/darwin_amd64/unicode/utf8.a packagefile unicode=/usr/local/go/pkg/darwin_amd64/unicode.a packagefile internal/cpu=/usr/local/go/pkg/darwin_amd64/internal/cpu.a EOF mkdir -p $WORK/b001/exe/ cd . /usr/local/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=6-oHCWY8Z57CMNCfUB-n/t_6XipuZQQ3V_Rlugib8/9dtyWN9RKHU5XwbNrmpr/6-oHCWY8Z57CMNCfUB-n -extld=gcc $WORK/b001/_pkg_.a /usr/local/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal cp $WORK/b001/exe/a.out dns-test rm -r $WORK/b001/ What did you expect to see? I expected net.a from the macOS distribution to be used. What did you see instead? The package net.a was compiled from sources instead. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

This is how I add the Linux and macOS Go packages to the Docker image:

ENV GO_VERSION 1.10.3

RUN wget -nv https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz && \
    tar -xzf go${GO_VERSION}.linux-amd64.tar.gz -C /usr/local && \
    rm -rf go${GO_VERSION}.linux-amd64.tar.gz

RUN wget -nv https://storage.googleapis.com/golang/go${GO_VERSION}.darwin-amd64.tar.gz && \
    tar -xzf go${GO_VERSION}.darwin-amd64.tar.gz -o -C /usr/local go/pkg && \
    rm -rf go${GO_VERSION}.darwin-amd64.tar.gz

[davecheney]

I don’t think that is going to work because a, the time stamps won’t match b, the hash of the arguments used to build the Darwin/amd64 .a’s won’t match because cgo is disabled when cross compiling. c, you don’t need to do that Just let the compile cross compile the bits it needs when building from Linux to Darwin. This is well tested and known to work for several years.

…

On 6 Sep 2018, at 05:43, juliengric ***@***.***> wrote: This is how I add the Linux and macOS Go packages to the Docker image: ENV GO_VERSION 1.10.3 RUN wget -nv https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz && \ tar -xzf go${GO_VERSION}.linux-amd64.tar.gz -C /usr/local && \ rm -rf go${GO_VERSION}.linux-amd64.tar.gz RUN wget -nv https://storage.googleapis.com/golang/go${GO_VERSION}.darwin-amd64.tar.gz && \ tar -xzf go${GO_VERSION}.darwin-amd64.tar.gz -o -C /usr/local go/pkg && \ rm -rf go${GO_VERSION}.darwin-amd64.tar.gz — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

a. Timestamps don't seem to be an issue since the other macOS packages are used as-is. The issue is only with the net.a package.

b. The hash difference sounds like the issue I'm hitting since Cgo is indeed disabled. However, based on the following I was assuming this would work:

From @minux in this comment:

cross compilation with cgo-enabled net package is not that hard.
You can reuse the package contained in binary distribution and
force internal linking.

From the cgo documentation (l.796 - l.811):

An important exception is builds using a pre-compiled copy of the
standard library. In particular, package net uses cgo on most systems,
and we want to preserve the ability to compile pure Go code that
imports net without requiring gcc to be present at link time. (In this
case, the dynamic library requirement is less significant, because the
only library involved is libc.so, which can usually be assumed
present.)

This conflict between functionality and the gcc requirement means we
must support both internal and external linking, depending on the
circumstances: if net is the only cgo-using package, then internal
linking is probably fine, but if other packages are involved, so that there
are dependencies on libraries beyond libc, external linking is likely
to work better. The compilation of a package records the relevant
information to support both linking modes, leaving the decision
to be made when linking the final binary.

c. My intent is to cross-compile for macOS with the Cgo DNS resolver enabled and to do it without using a macOS toolchain as this would cause the produced binary to depend on a specific macOS version.

This works as expected using go1.9. This sounds like a regression introduced by the content-based staleness changes made in go1.10.

From Linux:

$ go version
go version go1.9.7 linux/amd64

$ cd /code && CGO_ENABLED=1 GOOS='darwin' GOARCH='amd64' ./_contrib/build/wrapper.sh make
+ mkdir -p /workspace/src/go.apple.com/pie
+ ln -fs /code /workspace/src/go.apple.com/pie/secrets
+ cd /workspace/src/go.apple.com/pie/secrets
+ exec make
go build -x -o dns-test ./dns
WORK=/tmp/go-build748583215
mkdir -p $WORK/go.apple.com/pie/secrets/dns/_obj/
mkdir -p $WORK/go.apple.com/pie/secrets/dns/_obj/exe/
cd /workspace/src/go.apple.com/pie/secrets/dns
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/go.apple.com/pie/secrets/dns.a -trimpath $WORK -goversion go1.9.7 -p main -complete -buildid 19c268e8e4bdd6263d300e0fb239a8ec2fc547fe -D _/workspace/src/go.apple.com/pie/secrets/dns -I $WORK -pack ./main.go
cd .
/usr/local/go/pkg/tool/linux_amd64/link -o $WORK/go.apple.com/pie/secrets/dns/_obj/exe/a.out -L $WORK -extld=gcc -buildmode=exe -buildid=19c268e8e4bdd6263d300e0fb239a8ec2fc547fe $WORK/go.apple.com/pie/secrets/dns.a
cp $WORK/go.apple.com/pie/secrets/dns/_obj/exe/a.out dns-test

From macOS:

❯ file dns-test
dns-test: Mach-O 64-bit executable x86_64

❯ otool -L ./dns-test        
./dns-test:
	/usr/lib/libSystem.B.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 0.0.0, current version 0.0.0)

❯ GODEBUG=netdns=2 ./dns-test
go package net: using cgo DNS resolver
go package net: hostLookupOrder(google.com) = cgo

[davecheney]

/cc @bcmills @ianlancetaylor

[bcmills]

Sorry, but we're moving away from GOPATH/pkg as a repository of unversioned, prebuilt packages (see #4719 and #28152). I don't see a way that this can work going forward.

There might be some other way to address the underlying use-case of building macOS binaries without a macOS toolchain. I'm not sure what form that would take, or if it's even plausible, but if you can figure out a plausible design and want to pursue it, please file a separate proposal.