cmd/link: error with -buildmode=shared: "cannot implicitly include runtime/cgo in a shared library"

[ghost]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.7.1

What operating system and processor architecture are you using (go env)?

[olegs@localhost tests]$ GOROOT=/home/olegs/.go/go GOPATH=/home/olegs/projects/tests ~/.go/go/bin/go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/olegs/projects/tests"
GORACE=""
GOROOT="/home/olegs/.go/go"
GOTOOLDIR="/home/olegs/.go/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build144208683=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

It's too long, I attached the log to this ticket.
In a nutshell, made a hello-world example with shared libraries. It compiles with Go 1.6.1, but not with Go 1.7.1

What did you expect to see?

I expected app to compile

What did you see instead?

/home/olegs/.go/go/pkg/tool/linux_amd64/link: cannot implicitly include runtime/cgo in a shared library
build-log.txt

[ianlancetaylor]

CC @mwhudson

[mwhudson]

Does it work if you don't pass -a on the offending 'go build' invocation?

I admit that I run into this error sometimes and don't really understand why, and given that I implemented this it's probably a sign that something needs to be fixed...

[ghost]

But having -a would mean using some results from old build / possibly going through some caching, where this check isn't performed, wouldn't it? And yes, third time the charm: it did compile after two unsuccessful compilations without -a.

I tried to follow the linker's code around this error message, and I'm confused: I never wanted CGO, and in this particular place iscgo is false, yet for some reason it is followed by loadinternal("runtime/cgo").

Since I'm interested in any sort of solution to this: is there any way to completely avoid having to deal with CGO, or are some bits of it mandatory? If it is possible, then what do I need to do to exclude it?

[mwhudson]

On 22 September 2016 at 23:16, least-olegs notifications@github.com wrote:

But having -a would mean using some results from old build / possibly
going through some caching, where this check isn't performed, wouldn't it?
And yes, third time the charm: it did compile after two unsuccessful
compilations without -a.

I tried to follow the linker's code around this error message, and I'm
confused: I never wanted CGO, and in this particular place iscgo is
false, yet for some reason it is followed by loadinternal("runtime/cgo").

Since I'm interested in any sort of solution to this: is there any way to
completely avoid having to deal with CGO, or are some bits of it mandatory?
If it is possible, then what do I need to do to exclude it?

cgo is unavoidable with -buildmode=shared. I suspect the real fix for this
issue is in cmd/go, not cmd/link.

[mwhudson]

Ah so what's going on is that when the linker looks for the runtime package, it finds $WORK/runtime.a but there is no $WORK/runtime.shlibname file to indicate that it is part of a shared library as the shlibname files are only created as part of installation. When the shared library is linked as a separate step, the runtime.a is next to a runtime.shlibname, so the link loads libstd.so, including the runtime/cgo package and so the later search for runtime/cgo finds that it has already been loaded.

I guess a fix would be to have the shlibname files created after the library is built and just copy them for the install step. @ianlancetaylor does that make sense to you?

[ianlancetaylor]

That seems to make sense, yes.

[mwhudson]

I tried this for a while and got horribly confused. Will try again in a bit.

The same issue results in a more insidious bug though: if you go build -o exe $path where exe links against a stale shared library, the stale shared library is re-build and re-installed, but the generated exe does not link against it!

Relatedly although the shared library will be re-built and re-installed, the .a files that make up the shared library are not re-installed. The naive fix for that results in loops in the action graph though :/

[bradfitz]

This looks like a cmd/go caching bug, which will be addressed in Go 1.10 with @rsc's cmd/go work, so closing this. Other bugs track cmd/go caching.