New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: some Go executables broken with macOS 10.12.4 / Xcode 8.3 but fine with Xcode 8.2 #19734

Closed
dmaclach opened this Issue Mar 28, 2017 · 66 comments

Comments

Projects
None yet
@dmaclach

dmaclach commented Mar 28, 2017

See couchbase/sync_gateway#2417 for repro steps.

System:

MacBook Pro (x86_64)
macOS 10.12.3 (Sierra)
go version go1.8 darwin/amd64
Xcode 8.3; clang -v
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.4.0
Thread model: posix

What did you do?

See description at
couchbase/sync_gateway#2417

I also found this with some code I am unable to share.

My code does work fine with Xcode 8.2:
clang -v
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.4.0
Thread model: posix

and from reading the bug linked above it works fine there as well.

@favadi

This comment has been minimized.

favadi commented Mar 28, 2017

I have the same problem as OP but only for some binaries, not all. When trying to start, it is killed instantly.

$ ./test
Killed: 9

I can't figure out the common pattern for broken programs.

@ALTree ALTree changed the title from Some Go executables broken with macOS 10.12.4 / Xcode 8.3 but fine with Xcode 8.2 to runtime: some Go executables broken with macOS 10.12.4 / Xcode 8.3 but fine with Xcode 8.2 Mar 28, 2017

@rasky

This comment has been minimized.

Member

rasky commented Mar 28, 2017

Reproduction:

package main

import (
	"fmt"

	_ "github.com/veandco/go-sdl2/sdl"
)

func init() {
	fmt.Println("init")
}

func main() {
	fmt.Println("Hello, world")
}
 $ ./testx
 fish: './testx' terminated by signal SIGKILL (Forced quit)

Importing go-sdl2 is enough to make it crash, the prints are just to show that the runtime is unable to get to main.init. go-sdl2 has a single init() function, but commenting it out doesn't change anything, it still crashes right away.

I installed SDL itself via homebrew, the bottled version, a few weeks ago.

@rasky

This comment has been minimized.

Member

rasky commented Mar 28, 2017

Basically, just using cgo and linking against sdl2 causes the crash:

package sdl

// #cgo linux freebsd darwin pkg-config: sdl2
import "C"

If you make the above file the whole go-sdl package, the binary will still crash.

@rasky

This comment has been minimized.

Member

rasky commented Mar 28, 2017

Uhm it looks like lldb is even unable to spawn the binary:

(lldb) run
error: error: ::posix_spawnp ( pid => 24893, path = '/Users/rasky/Sources/go/src/ndsemu/testx/testx', file_actions = 0x7fff555225e8, attr = 0x7fff555225f8, argv = 0x7fd1e92a8150, envp = 0x7fd1e92ad520 ) err = Cannot allocate memory (0x0000000c)

so it seems like the binary is being linked in a way that the OS refuses to run it.

/cc @ianlancetaylor

@fighterlyt

This comment has been minimized.

fighterlyt commented Mar 28, 2017

just the same, cgo make it crash

@crazytyper

This comment has been minimized.

crazytyper commented Mar 28, 2017

same here but with github.com/shirou/gopsutil/cpu instead of sdl2.

trying to run the binary with dtruss gives Unknown error.

@rgeronimi

This comment has been minimized.

rgeronimi commented Mar 28, 2017

Same here, very simple to reproduce:

$ go test github.com/blevesearch/cld2
signal: killed
FAIL	github.com/blevesearch/cld2	0.003s

Investigating the MacOS system logs, it seems to be an issue with taskgated:
taskgated MacOS error: -67062
taskgated no signature for pid=4766 (cannot make code: UNIX[No such process])

Maybe a code signature issue around cgo...

@rasky

This comment has been minimized.

Member

rasky commented Mar 28, 2017

I'll notice that all Go binaries (including working ones) generate the -67062 error, but I think that's normal because they are not codesigned. The errors I get only when running these crashing cgo binaries are:

default	14:32:47.099750 +0200	taskgated	UNIX error exception: 3
default	14:32:47.099853 +0200	taskgated	no signature for pid=28180 (cannot make code: UNIX[No such process])
@rgeronimi

This comment has been minimized.

rgeronimi commented Mar 28, 2017

@rasky
On my environment (MacOS 10.12.14, Xcode8.3), non-cgo go programs do not generate -67062 errors. In my tests I performed, this error was strictly correlated to (1) crashes with the "signal: killed" message (2) cgo programs.
Additionally, I also get the "UNIX error exception: 3" message in the system logs

@margerum

This comment has been minimized.

margerum commented Mar 28, 2017

getting signal: killed when trying to use go run *.go

go build works, but then i get "Killed: 9" when I try to run the app

@aprilchild

This comment has been minimized.

aprilchild commented Mar 28, 2017

same here, dowloaded Xcode 8.2.1, replaced in /Applications and it works again..

@bradfitz bradfitz added this to the Go1.8.1 milestone Mar 28, 2017

@repustate

This comment has been minimized.

repustate commented Mar 28, 2017

For everyone running into this issue, I've been able to build binaries by adding -ldflags -s during the go build phase.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Mar 28, 2017

By default on Darwin the linker will invoke dsymutil to put the debug info directly into the executable. This invocation of dsymutil is disabled by the linker's -s option. Perhaps this is really a bug in dsymutil, or in how dsymutil deals with files created by the Go linker. Could somebody with an affected system please try modifying the code in Link.hostlink in cmd/link/internal/ld/lib.go to skip invoking dsymutil, and see if that fixes the problem? Thanks.

@josharian

This comment has been minimized.

Contributor

josharian commented Mar 28, 2017

@ianlancetaylor that does fix the problem. Tested on 1.8.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Mar 28, 2017

The call to dsymutil was added to fix #8973. I suppose the next question is: if we don't call dsymutil, what happens when you try to debug a Go program built with cgo?

@cosnicolaou

This comment has been minimized.

Contributor

cosnicolaou commented Mar 28, 2017

I'm seeing this without using cgo, what's the suggested work around - use xcode 8.2?

@bradfitz

This comment has been minimized.

Member

bradfitz commented Mar 28, 2017

@cosnicolaou, see above (#19734 (comment)). Does that not work for you?

@cosnicolaou

This comment has been minimized.

Contributor

cosnicolaou commented Mar 28, 2017

thanks, it works, from reading the comments I thought this was a cgo only problem/fix. It's also painful for our build/automation for obvious reasons.

@travissanderson-wf

This comment has been minimized.

travissanderson-wf commented Mar 28, 2017

would adding these flags to build scripts have any negative repercussions that you're aware of? performance implications or anything like that?

kcking added a commit to kcking/homebrew-kr that referenced this issue Mar 28, 2017

@cosnicolaou

This comment has been minimized.

Contributor

cosnicolaou commented Mar 28, 2017

no, it's just a pain to find all of the invocations etc.

@travissanderson-wf

This comment has been minimized.

travissanderson-wf commented Mar 28, 2017

agreed :) thanks for verifying they are safe to add though

@bradfitz

This comment has been minimized.

Member

bradfitz commented Apr 7, 2017

Er, build, not run. Maybe that's okay.

@rsc

This comment has been minimized.

Contributor

rsc commented Apr 7, 2017

Right, build, and even then only build with newer Xcode. It's not like the gettimeofday crashes.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Apr 7, 2017

This may impact their point release builds, but I'm not sure how Kubernetes does their releases (on an actual Mac, or cross-compiled).

Running file on https://storage.googleapis.com/kubernetes-release/release/v1.6.1/bin/darwin/amd64/kubectl just says kubectl: Mach-O 64-bit executable x86_64. (no ldd... I don't know Mac tools)

@rsc

This comment has been minimized.

Contributor

rsc commented Apr 7, 2017

They can always add -ldflags=-s or update to Go 1.8.1 like everyone else. Also, kubectl at least doesn't appear to have any cgo code in it (otool -l kubectl | grep nundefsym).

@abh

This comment has been minimized.

abh commented Apr 8, 2017

Yeah, kubectl seems to be cross compiled (it doesn't use the macOS DNS resolver).

@vyorkin

This comment has been minimized.

vyorkin commented Apr 11, 2017

none of the above workarounds worked for me :(

mac os x sierra 10.12.4
xcode 8.2 (8C38)
command line tools: http://adcdownload.apple.com/Developer_Tools/Command_Line_Tools_macOS_10.12_for_Xcode_8.2/Command_Line_Tools_macOS_10.12_for_Xcode_8.2.dmg

before installing command line tools I've removed /Library/Developer/CommandLineTools directory

UPD: it works now! after uninstalling the previous version with brew uninstall ethereum and installing a new one from the develop branch with brew install ethereum --devel

@pawelkowalak

This comment has been minimized.

pawelkowalak commented Apr 11, 2017

Update to Go 1.8.1, it is fixed now.

@rodrigo-brito

This comment has been minimized.

rodrigo-brito commented Apr 11, 2017

Go 1.8.1 Works! 🎉

lparth added a commit to lparth/go that referenced this issue Apr 13, 2017

cmd/link: disable mach-o dwarf munging with -w (in addition to -s)
Might as well provide a way around the mach-o munging
that doesn't require stripping all symbols.
After all, -w does mean no DWARF.

For golang#11887, golang#19734, and anyone else that needs to disable
this code path without losing the symbol table.

Change-Id: I254b7539f97fb9211fa90f446264b383e7f3980f
Reviewed-on: https://go-review.googlesource.com/38853
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

lparth added a commit to lparth/go that referenced this issue Apr 13, 2017

cmd/link: make mach-o dwarf segment properly aligned
Without this, the load fails during kernel exec, which results in the
mysterious and completely uninformative "Killed: 9" error.

It appears that the stars (or at least the inputs) were properly aligned
with earlier versions of Xcode so that this happened accidentally.
Make it happen on purpose.

Gregory Man bisected the breakage to this change in LLVM,
which fits the theory nicely:
llvm-mirror/llvm@9a41e59

Fixes golang#19734.

Change-Id: Ice67a09af2de29d3c0d5e3fcde6a769580897c95
Reviewed-on: https://go-review.googlesource.com/38854
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

ns-codereview pushed a commit to couchbase/indexing that referenced this issue Apr 20, 2017

MB-23969: Temporary workaround to allow 1.7.3 to work on OSX 10.12.4
OSX 10.12.4 debug symbols breaks go 1.7.3 binaries. Temporarily add
-s flag to strip debug info.

Revert this commit when we move to 1.8.1

See golang/go#19734

Change-Id: I8d777c82841c75c6b1b28f6b7d69690b93e5fb87

harshavardhana added a commit to harshavardhana/mc that referenced this issue Apr 20, 2017

harshavardhana added a commit to harshavardhana/mc that referenced this issue Apr 20, 2017

vadmeste added a commit to minio/mc that referenced this issue Apr 20, 2017

@bobzoller bobzoller referenced this issue Apr 21, 2017

Closed

log cron failures #2166

0 of 4 tasks complete

@amrfaissal amrfaissal referenced this issue Apr 22, 2017

Closed

Bee Killed #2584

@OSMeteor

This comment has been minimized.

OSMeteor commented Apr 24, 2017

use go1.8.1 fixed it good

@golang golang locked and limited conversation to collaborators Apr 24, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.