Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: CGO arm builds fail on Go 1.10beta1 #23288

Closed
karalabe opened this issue Dec 30, 2017 · 10 comments

Comments

Projects
None yet
3 participants
@karalabe
Copy link
Contributor

commented Dec 30, 2017

What version of Go are you using (go version)?

go1.10beta1

What operating system and processor architecture are you using (go env)?

Travis CI, Linux

What did you do?

Tried to cross compile go-ethereum to ARM5/6/7/64, which builds fine on all previous versions of Go.

What did you expect to see?

Successful build.

What did you see instead?

arm-linux-gnueabi-gcc: error: unrecognized command line option '-m64'

Full build logs at https://travis-ci.org/ethereum/go-ethereum/jobs/323227026

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

You assume we know how to build (or how you built) go-ethereum. How can we reproduce this to debug? I'm sure it's in that travis log somewhere, but a concise repro in the bug would be nice.

/cc @ianlancetaylor

@bradfitz bradfitz added this to the Go1.10 milestone Dec 30, 2017

@karalabe

This comment has been minimized.

Copy link
Contributor Author

commented Dec 30, 2017

Fair enough, I'll try to cook up a repro for you.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

Maybe your GOARM=5 CC=arm-linux-gnueabi-gcc go run build/ci.go install -arch arm should be CC_FOR_TARGET instead.

@karalabe

This comment has been minimized.

Copy link
Contributor Author

commented Dec 30, 2017

Hmm, the issue seems to come from go run build/ci.go (not from the cross compilation invoked within), which for some reason wants to build runtime/cgo, so my CC env var makes that fail:

$ CC=aarch64-linux-gnu-gcc go run -x -v build/ci.go install -arch arm64
WORK=/tmp/go-build716618928
runtime/cgo
mkdir -p $WORK/b034/
cd /opt/google/go/src/runtime/cgo
CGO_LDFLAGS='"-g" "-O2" "-lpthread"' /opt/google/go/pkg/tool/linux_amd64/cgo -objdir $WORK/b034/ -importpath runtime/cgo -import_runtime_cgo=false -import_syscall=false -- -I $WORK/b034/ -g -O2 -Wall -Werror ./cgo.go
# runtime/cgo
aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'

Now, I could understand why this would fail if ci.go would depend on cgo, but it does not, rather it (and all its dependency tree) is pure Go, so go run shouldn't try to build runtime/cgo, yet it does.


Note, I can fix my issue by not setting CC, rather perhaps passing the compiler as a flag to ci.go and have that set it as an env var internally before invoking the cross compilation. However, it would be nice to know why go run wants to touch CGO, without a dependency for it.

Emphasis that this only happens with Go 1.10. None of the previous versions of Go touch runtime/cgo when building my build/ci.go file.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

You can inspect dependencies with go list -json to see why there's a path to runtime/cgo.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

(If ci.go uses os/user or net, it'll probably depend on cgo by default.)

@karalabe

This comment has been minimized.

Copy link
Contributor Author

commented Dec 30, 2017

Hmm, indeed, runtime/cgo is indeed listed as some dependency:

$ go list -json ./build/ci.go 
{
	"Dir": "/work/src/github.com/ethereum/go-ethereum/build",
	"ImportPath": "command-line-arguments",
	"Name": "main",
	"Doc": "The ci command is called from Continuous Integration scripts.",
	"Root": "/work",
	"Stale": true,
	"StaleReason": "stale dependency: github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest",
	"GoFiles": [
		"ci.go"
	],
	"Imports": [
		"bufio",
		"bytes",
		"encoding/base64",
		"flag",
		"fmt",
		"github.com/ethereum/go-ethereum/internal/build",
		"go/parser",
		"go/token",
		"io/ioutil",
		"log",
		"os",
		"os/exec",
		"path/filepath",
		"regexp",
		"runtime",
		"strings",
		"time"
	],
	"Deps": [
		"archive/tar",
		"archive/zip",
		"bufio",
		"bytes",
		"compress/bzip2",
		"compress/flate",
		"compress/gzip",
		"compress/zlib",
		"container/list",
		"context",
		"crypto",
		"crypto/aes",
		"crypto/cipher",
		"crypto/des",
		"crypto/dsa",
		"crypto/ecdsa",
		"crypto/elliptic",
		"crypto/hmac",
		"crypto/internal/cipherhw",
		"crypto/md5",
		"crypto/rand",
		"crypto/rc4",
		"crypto/rsa",
		"crypto/sha1",
		"crypto/sha256",
		"crypto/sha512",
		"crypto/subtle",
		"crypto/tls",
		"crypto/x509",
		"crypto/x509/pkix",
		"encoding",
		"encoding/asn1",
		"encoding/base64",
		"encoding/binary",
		"encoding/hex",
		"encoding/json",
		"encoding/pem",
		"encoding/xml",
		"errors",
		"flag",
		"fmt",
		"github.com/ethereum/go-ethereum/internal/build",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/azure-storage-go",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest/azure",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest/date",
		"github.com/ethereum/go-ethereum/vendor/github.com/dgrijalva/jwt-go",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/cast5",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/armor",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/elgamal",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/errors",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/packet",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/s2k",
		"go/ast",
		"go/parser",
		"go/scanner",
		"go/token",
		"hash",
		"hash/adler32",
		"hash/crc32",
		"image",
		"image/color",
		"image/internal/imageutil",
		"image/jpeg",
		"internal/cpu",
		"internal/nettrace",
		"internal/poll",
		"internal/race",
		"internal/singleflight",
		"internal/syscall/unix",
		"io",
		"io/ioutil",
		"log",
		"math",
		"math/big",
		"math/bits",
		"math/rand",
		"mime",
		"mime/multipart",
		"mime/quotedprintable",
		"net",
		"net/http",
		"net/http/cookiejar",
		"net/http/httptrace",
		"net/http/internal",
		"net/textproto",
		"net/url",
		"os",
		"os/exec",
		"os/user",
		"path",
		"path/filepath",
		"reflect",
		"regexp",
		"regexp/syntax",
		"runtime",
		"runtime/cgo",
		"runtime/internal/atomic",
		"runtime/internal/sys",
		"sort",
		"strconv",
		"strings",
		"sync",
		"sync/atomic",
		"syscall",
		"text/template",
		"text/template/parse",
		"time",
		"unicode",
		"unicode/utf16",
		"unicode/utf8",
		"unsafe",
		"vendor/golang_org/x/crypto/chacha20poly1305",
		"vendor/golang_org/x/crypto/chacha20poly1305/internal/chacha20",
		"vendor/golang_org/x/crypto/cryptobyte",
		"vendor/golang_org/x/crypto/cryptobyte/asn1",
		"vendor/golang_org/x/crypto/curve25519",
		"vendor/golang_org/x/crypto/poly1305",
		"vendor/golang_org/x/net/http2/hpack",
		"vendor/golang_org/x/net/idna",
		"vendor/golang_org/x/net/lex/httplex",
		"vendor/golang_org/x/net/proxy",
		"vendor/golang_org/x/text/secure/bidirule",
		"vendor/golang_org/x/text/transform",
		"vendor/golang_org/x/text/unicode/bidi",
		"vendor/golang_org/x/text/unicode/norm"
	]
}

But why does Go 1.10beta1 want to rebuild runtime/cgo, but previous versions of Go didn't want to do that?

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

But why does Go 1.10beta1 want to rebuild runtime/cgo, but previous versions of Go didn't want to do that?

Because you got lucky before. Go 1.10 has proper build caching based on file contents and all relevant environment variables. Go 1.9 and earlier depended heavily on modtimes of files only (not contents) and ignored many important environment variables.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Dec 30, 2017

That is, in Go 1.9 and before, your runtime/cgo had already been built, so your CC=foo go run ci.go wasn't rebuilding it. But in Go 1.10, it assumes the new $CC value invalidates the cached results (whose cache key includes a different $CC) and rebuilds it.

@karalabe

This comment has been minimized.

Copy link
Contributor Author

commented Dec 30, 2017

Sweet, that's what I was also guessing based on Florin's Advent post, just wanted to confirm. Thank you very much for your time on this, and sorry for the false alarm!

@karalabe karalabe closed this Dec 30, 2017

@golang golang locked and limited conversation to collaborators Dec 30, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.