cmd/link: darwin_amd64: running dsymutil failed: signal: segmentation fault

[pcman312]

What version of Go are you using (go version)?

$ go version
go version go1.10.3 darwin/amd64

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G1114

$ clang --version
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Does this issue reproduce with the latest release?

Yes. I was originally on 10.2 but upgraded to 10.3 to see if a fix already existed and the issue persisted.

What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE=[snip]
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH=[snip]
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1n/x7xttf9n7ylbl8vn50830dsh6qmzqb/T/go-build029428003=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do? / What did you expect to see? / What did you see instead?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

We have a REST service that accesses an elasticsearch database. In the main it imports a package in the service that accesses ES (.../db/elastic). That sub-package imports ES "gopkg.in/olivere/elastic.v3". I'm refactoring some things around and pulled that library into the main in addition to the sub-package.

We format our imports like this:

[stdlib]

[not-our code - this is where the olivere library is]

[our code - both this service as well as any internal libraries we are using]

When I pulled in the olivere library and compiled it, I got this error:

/usr/local/go/pkg/tool/darwin_amd64/link: /usr/local/go/pkg/tool/darwin_amd64/link: running dsymutil failed: signal: segmentation fault

I've been playing around with it to try to narrow down what's causing it and so far I've been able to narrow my suspicions down to the order of imports.

This import successfully builds:

[other external imports]

ES "gopkg.in/olivere/elastic.v3"
".../db/elastic"
[other service/internal imports]

This import segfaults as above:

[other external imports]
ES "gopkg.in/olivere/elastic.v3"

".../db/elastic"
[other service/internal imports]

If I put the olivere import in a separate block (newlines around it), it seems to have to be strictly after the service import .../db/elastic:

Succeeds:

[other imports]

".../db/elastic"

ES "gopkg.in/olivere/elastic.v3"

[other imports]

Also succeeds:

[other imports]

// The order doesn't matter if they don't have newlines between them
ES "gopkg.in/olivere/elastic.v3"
".../db/elastic"

[other imports]

Fails:

[other imports]

ES "gopkg.in/olivere/elastic.v3"

".../db/elastic"

[other imports]

I've been trying to build a standalone program that I can reproduce this in, but so far I have been unsuccessful. When I pull out the code in question to a sandbox program, I am unable to reproduce it. It's possible that some other import is playing some sort of role, but I haven't been able to narrow it down because it does not seem to matter where I put the two imports as long as they are in the same block (without newlines between them) or are in a specific order.

Also worth noting: when I do a cross compile to linux, this succeeds regardless of how I organize the imports.

The behavior seems very similar to #23374 but I don't know if it's related beyond using the same compiler.

[ianlancetaylor]

CC @thanm

Can you check whether this is fixed in the 1.11beta1 beta release? Thanks.

[thanm]

Hi @pcman312 -- I'd like to take a closer look at this, but I'm not sure there is much I can do without a stand-alone reproducer. Perhaps you could share an example with the same imports even if it doesn't cause the crash?

Also worth mentioning: folks have seen DWARF problems with older versions of Xcode, for example in https://github.com/golang/go/issues/25392. For your system, could you pls share the output of:

$ xcodebuild -version

Thanks.

[pcman312]

@thanm

$ xcodebuild
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

I'm not sure how much of this I can share since it's proprietary code dealing with our database unfortunately. I should be able to put something together that has the same dependency tree, just no functional contents if that works.

[thanm]

same dependency tree, just no functional contents

This is very unlikely to help, sorry. DWARF problems of this sort can be tricky to reproduce.

[pcman312]

No kidding. I was just looking through it to try to narrow it down to a smaller program that I could share and discovered a couple other dependencies that when removed makes the compile error disappear. Do you have any recommendations for things I can do to try to reproduce this for you?

[pcman312]

I have a version of this that I can share that reproduces the error. I haven't been able to get this to reduce any further:
https://github.com/pcman312/darwin_test

I confirmed this by having a coworker try to compile it and he got the same error.

When I mess around with the imports in main.go or with any of the handling after the Do() call (https://github.com/pcman312/darwin_test/blob/master/es/esclient.go#L21-L26) it successfully compiles.

[pcman312]

Interesting further investigation: I had another engineer who did not have Go set up on his computer try building that test and his succeeded. That suggests there is some difference between my local version of either of the two libraries in question. I will upload copies of my local versions to the darwin_test package. They will not be in a vendor directory, just in appropriately named folders.

[thanm]

Thanks for creating the reproducer. So far I can't reproduce the dsymutil crash on my mac with either 1.10.3 or the 1.11 beta candidate... I'll keep poking around at it to see what I can figure out.

[pcman312]

I've uploaded copies of those two libraries to the darwin_test project. I'll see if that engineer I mentioned previously can reproduce it with those versions.