Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: allow configuration of dsymutil and strip for darwin/mach-o linking #47316

Closed
chancila opened this issue Jul 21, 2021 · 8 comments
Closed

Comments

@chancila
Copy link

@chancila chancila commented Jul 21, 2021

What version of Go are you using (go version)?

1.16.6

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/chancila/.cache/go-build"
GOENV="/home/chancila/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/chancila/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/chancila/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/7954"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/7954/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1637780390=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am cross-compiling a darwin executable from linux with cgo enabled. I have successfully compiled everything but the link step fails due to xcrun missing since the go link command assumes xcrun exists if targeting darwin/mach when it really should probably only assume that if targeting darwin and host is darwin. I'm currently planning to work around this by creating a shim xcrun however having the ability to directly set these tools would be cleaner and easier to integrate with other build systems (namely bazel which I'm using in this case). The underlying tools have equivalents in open source llvm so they are not exclusive to macos hosts.

The code in question https://github.com/golang/go/blob/master/src/cmd/link/internal/ld/lib.go#L1643 should allow configuration of the tool in some way so that this use-case can be covered.

What did you expect to see?

Successful build

What did you see instead?

Linker error from ld/lib.go: Exitf("%s: running dsymutil failed: %v\n%s", os.Args[0], err, out) due to missing xcrun

@thanm thanm added this to the Backlog milestone Jul 21, 2021
@thanm
Copy link
Contributor

@thanm thanm commented Jul 21, 2021

Loading

@thanm
Copy link
Contributor

@thanm thanm commented Jul 21, 2021

If as you say you are planning on writing a shim xcrun, would this not take care of the problems you describe?

Putting additoinal knobs/controls into the linker would mainly just make it more complicated and hard to understand (for example, for the 99% of folks not doing cross compilation, they might think that they need to specify a dsymutil path, or something to this effect).

Loading

@chancila
Copy link
Author

@chancila chancila commented Jul 21, 2021

I mean it would solve the issue in a very annoying way, in general this is a somewhat odd inconsistency given how the rest of the external c toolchain tools are configurable and hardcoding these tools kind of makes the toolchain non-hermetic, or at least requires some very non-obvious work to make hermetic, even if host is also darwin.

Loading

@chancila
Copy link
Author

@chancila chancila commented Jul 22, 2021

trying to force xcrun in the path, I'm now running into another issue... #43783

due to how bazel works pretty much everything is relative to a execution root...so the go invocation sees that the xcrun which is added in the c++ toolchain is relative and complains...

Loading

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 22, 2021

@chancila could you share more information about your C cross toolchain? What tools are included and how are they named? Thanks.

We used to just run dsymutil and strip from PATH. Some user reported that they installed GNU tools to PATH and it doesn't work.

Maybe we could use $CC --print-prog-name dsymutil. Would that work for you?

Loading

@cherrymui cherrymui changed the title allow configuration of dsymutil and strip for darwin/mach-o linking cmd/link: allow configuration of dsymutil and strip for darwin/mach-o linking Jul 22, 2021
@chancila
Copy link
Author

@chancila chancila commented Jul 22, 2021

@cherrymui

  • hosted on linux (ubuntu 20.04)
  • targeting macos (amd64 and arm64)
  • plain llvm (12) release (from official llvm releases on github)
  • linux hosted apple ld64 (from https://github.com/tpoechtrager/cctools-port)
  • apple macos sysroot from xcode 12.1 (and 12.5 for arm64)

I'm exercising this whole thing through bazel and rules_go...the fundamental issue is that the go linker tries to use xcrun when targeting mach binaries but it probably needs a fallback path when the linker/toolchain is hosted on non-macos systems rather than blindly using xcrun.

I think --print-prog-name would work fine actually, verifying with my toolchain it does the right thing.

In this scenario the failures are xcrun dsymutil and xcrun strip

I have successfully managed to cgo cross compile for both arm64 and amd64 by adding a shim xcrun wrapper in my system to /usr/bin globally as a hack, I would prefer to not have to do this as the toolchain is otherwise hermetic to the build and handled by bazel without modifying the system tools at all.

Loading

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Jul 22, 2021

@chancila thanks! I think I'd go with --print-prog-name.

Loading

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 22, 2021

Change https://golang.org/cl/336769 mentions this issue: cmd/link: use "CC --print-prog-name" to locate tools

Loading

@gopherbot gopherbot closed this in eb9f090 Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants