Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: spurious error message when external linking a pure Go program #31544

Open
cherrymui opened this issue Apr 18, 2019 · 12 comments
Open

cmd/go: spurious error message when external linking a pure Go program #31544

cherrymui opened this issue Apr 18, 2019 · 12 comments
Milestone

Comments

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Apr 18, 2019

What version of Go are you using (go version)?

tip (3235f7c)

Does this issue reproduce with the latest release?

Yes, happens with Go 1.11, 1.12, and tip, but not with Go 1.10.

What operating system and processor architecture are you using (go env)?

linux/amd64

What did you do?

$ mkdir -p /tmp/src/hello
$ cat >hello.go
package main
func main() { println("hello") }
$ GO111MODULE=off GOPATH=/tmp go build -ldflags=-linkmode=external
# hello
loadinternal: cannot find runtime/cgo
$ echo $?
0
$ ./hello
hello

When using external linking for a pure Go program, the linker prints loadinternal: cannot find runtime/cgo message. The linking actually succeeded, and the generated binary works fine.

What did you expect to see?

No error message, like with Go 1.10.

$ GOPATH=/tmp go1.10 build -ldflags=-linkmode=external 
$ ./hello 
hello

What did you see instead?

Spurious loadinternal: cannot find runtime/cgo error message.

@cherrymui

This comment has been minimized.

Copy link
Contributor Author

@cherrymui cherrymui commented Apr 18, 2019

This also happens in module mode:

$ go mod init hello
go: creating new go.mod: module hello
$ GO111MODULE=on go build -ldflags=-linkmode=external 
# hello
loadinternal: cannot find runtime/cgo

But it does not happen if the file name (hello.go) is given explicitly.

$ go build -ldflags=-linkmode=external hello.go 
$
@julieqiu julieqiu changed the title cmd/go or cmd/link: spurious error message when external linking a pure Go program cmd/go: spurious error message when external linking a pure Go program Apr 22, 2019
@julieqiu julieqiu added this to the Go1.13 milestone Apr 22, 2019
@julieqiu

This comment has been minimized.

Copy link

@julieqiu julieqiu commented Apr 22, 2019

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented May 8, 2019

I can reproduce the failure. I'm bisecting it now to try to get some idea of how it broke.

@bcmills bcmills self-assigned this May 8, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented May 8, 2019

git bisect points to CL 129059 (CC @rsc @alandonovan).

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented May 8, 2019

This is a result of the -ldflags=-linkmode=external flag now (correctly) being passed when compiling the hello package, which reveals a typestate bug in load.PackagesAndErrors.

The decision in the LinkerDeps function about whether to add the runtime/cgo dependency is based on whether the -ldflags=-linkmode=external flag is set for the package, which itself depends on whether CmdlinePkg flag is set:

match := func(p *Package) bool { return p.Internal.CmdlinePkg || p.Internal.CmdlineFiles } // default predicate with no pattern

That field is set after the call to loadImport:

p := loadImport(pre, pkg, base.Cwd, nil, &stk, nil, 0)
p.Match = append(p.Match, m.Pattern)
p.Internal.CmdlinePkg = true

However, by that point loadImport has already called load on the package:

p.load(stk, bp, err)

...and load is what's supposed to figure out the implicit linker dependencies:

for _, dep := range LinkerDeps(p) {

@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented May 9, 2019

@cherrymui, unless this is important for you for some reason, we'd like to postpone the fix to Go 1.14. It's subtle code.

/cc @bcmills

@bcmills bcmills modified the milestones: Go1.13, Go1.14 May 9, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented May 9, 2019

Change https://golang.org/cl/176217 mentions this issue: cmd/go/internal/work: set CmdlinePkg in loadImport instead of after

@cherrymui

This comment has been minimized.

Copy link
Contributor Author

@cherrymui cherrymui commented May 9, 2019

@cherrymui, unless this is important for you for some reason, we'd like to postpone the fix to Go 1.14. It's subtle code.

Sure, it is fine to postpone.

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@michael-obermueller

This comment has been minimized.

Copy link

@michael-obermueller michael-obermueller commented Dec 9, 2019

We would like to propose to reconsider the priority fixing this issue. The implication of it is, that such Go binaries have no runtime/cgo dependency which means they do not create threads using pthread_create but instead the clone system call.

Preloaded (LD_PRELOAD) shared objects may rely on the fact that Go creates POSIX threads.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Dec 10, 2019

I certainly think this should be fixed, but I'm not following the concern with LD_PRELOAD shared objects. I don't see how it would be possible to use an LD_PRELOAD with a pure Go program.

@bcmills bcmills modified the milestones: Backlog, Go1.15 Dec 10, 2019
@bcmills bcmills removed their assignment Dec 10, 2019
@Hollerberg

This comment has been minimized.

Copy link

@Hollerberg Hollerberg commented Dec 10, 2019

@ianlancetaylor: The scenario is very specific, but has a serious impact on existing applications.

A shared object of our (application performance monitoring) product is pre-loaded into Go processes to hook certain Go functions. Customers use -ldflags=-linkmode=external in some cases to enforce dynamically built applications.

As the issue causes iscgo set to false, Go runtime will create threads with clone and not with pthread_create. Thus, pre-loaded C code executed in Go runtime thread context (invoked from hooked Go functions) will crash the application accessing e.g. TLS.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Dec 10, 2019

OK, that seems not only specific but completely unsupported.

Again, I think this bug should be fixed, but the Go team can't take any responsibility for keeping that kind of pre-loaded library working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.