New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: using -linkshared dies if compile not passed -p option #16632

Open
mwhudson opened this Issue Aug 7, 2016 · 16 comments

Comments

Projects
None yet
8 participants
@mwhudson
Contributor

mwhudson commented Aug 7, 2016

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

tip

  1. What operating system and processor architecture are you using (go env)?

Ubuntu 16.04.

  1. What did you do?

./make.bash
go install -buildmode=shared std
cd ../test
go run run.go -linkshared method4.go

  1. What did you expect to see?

no output

  1. What did you see instead?
# go run run.go -- method4.go
exit status 1
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x4b4275]

goroutine 1 [running]:
panic(0x5a0bc0, 0xc420010180)
    /opt/opensource/go/src/runtime/panic.go:500 +0x1a1
cmd/link/internal/ld.relocsym(0xc4204c8690)
    /opt/opensource/go/src/cmd/link/internal/ld/data.go:530 +0x1425
cmd/link/internal/ld.reloc()
    /opt/opensource/go/src/cmd/link/internal/ld/data.go:658 +0xe7
cmd/link/internal/ld.Ldmain()
    /opt/opensource/go/src/cmd/link/internal/ld/pobj.go:213 +0x10ff
cmd/link/internal/amd64.Main()
    /opt/opensource/go/src/cmd/link/internal/amd64/obj.go:45 +0x19
main.main()
    /opt/opensource/go/src/cmd/link/main.go:28 +0x27d

FAIL    method4.go  0.693s
exit status 1
@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Aug 8, 2016

r.Sym.Sect is nil in:

        case obj.R_ADDROFF:
            o = Symaddr(r.Sym) - int64(r.Sym.Sect.Vaddr) + r.Add

r.Sym is a type symbol ("type..NEZkhXSX") from libstd.so.

Is noone else seeing this?

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Aug 8, 2016

Ping @crawshaw

@mwhudson mwhudson added this to the Go1.7 milestone Aug 8, 2016

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Aug 8, 2016

If I patch ldshlibsyms to fill out r.Sym.Addr and r.Sym.Sect.Vaddr then the test fails like this:

runtime: nameOff 0x6f698 out of range 0x6039c0 - 0x604468
fatal error: runtime: name offset out of range

which suggests an offset into the some section in libstd.so is being compared against the location of a section in the executable?

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Aug 8, 2016

Oh phew, this is a test bogosity as well, it's because run.go does not pass -p to go tool compile. Some nasty hacks to do that make all the tests (apart from goprint) pass, so not a 1.7 blocker imo.

@mwhudson mwhudson modified the milestones: Go1.8, Go1.7 Aug 8, 2016

@mwhudson mwhudson changed the title from cmd/link: crashes running "go run run.go -linkshared method4.go" to test: does not pass -p to go tool compile when compiling packages Aug 8, 2016

@quentinmit quentinmit added the NeedsFix label Oct 10, 2016

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 18, 2016

compile -p is an option not a requirement. Whatever is broken here should work without it.

@rsc rsc changed the title from test: does not pass -p to go tool compile when compiling packages to cmd/link: using -linkshared dies if compile not passed -p option Oct 18, 2016

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Oct 18, 2016

(Sorry I missed your original ping @mwhudson, the first email github sent was when @rsc commented.)

I don't understand the connection between -p and this bug. Do you understand how setting -p is fixing it?

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented Oct 18, 2016

No, I don't remember any more, would have to start investigating again.

sent from my phone, please excuse brevity

On 18 Oct 2016 23:51, "David Crawshaw" notifications@github.com wrote:

(Sorry I missed your original ping @mwhudson https://github.com/mwhudson,
the first email github sent was when @rsc https://github.com/rsc
commented.)

I don't understand the connection between -p and this bug. Do you
understand how setting -p is fixing it?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16632 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApBFrveZrGOTw89J1UcGvPSLaP2ylSjks5q1KRngaJpZM4JenUI
.

@rsc rsc modified the milestones: Go1.9, Go1.8 Nov 11, 2016

@bradfitz

This comment has been minimized.

Member

bradfitz commented Jun 7, 2017

@mwhudson, is this still a problem?

@bradfitz bradfitz modified the milestones: Go1.10, Go1.9 Jun 7, 2017

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Aug 9, 2017

I just ran across the same problem. With tip this test still fails:

$ go run run.go -linkshared method4.go
# go run run.go -- method4.go
exit status 1
cannot parse package data of prog.o for hash generation, no newline found
cannot parse package data of method4a.o for hash generation, no newline found
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x50491e]

goroutine 1 [running]:
cmd/link/internal/ld.relocsym(0xc420512000, 0xc42053d5c0)
        /home/b00375952/Sources/go/go/src/cmd/link/internal/ld/data.go:639 +0x115e
cmd/link/internal/ld.(*Link).reloc(0xc420512000)
        /home/b00375952/Sources/go/go/src/cmd/link/internal/ld/data.go:772 +0xbe
cmd/link/internal/ld.Main()
        /home/b00375952/Sources/go/go/src/cmd/link/internal/ld/main.go:221 +0xa20
main.main()
        /home/b00375952/Sources/go/go/src/cmd/link/main.go:58 +0xac

FAIL    method4.go      0.244s
exit status 1

I can reproduce the same panic backtrace when building my own code in shared mode.

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Aug 15, 2017

I have gathered some additional information after playing with this on tip. type..NEZkhXSX is the hashed name of the symbol type..importpath.main.. This symbol is defined in runtime.a and hence also in libstd.so. Consider the following three cases:

  1. prog.o is compiled without -dynlink. The symbol name is never hashed. It is referenced in prog.o but never defined. prog.o is linked with method4a.o, runtime.a, sys.a and atomic.a. The reference in prog.o is relocated to the definition in runtime.a, and an executable is generated correctly.

  2. prog.o is compiled with -dynlink -p=main. Due to dynamic linking, the symbol name is hashed by link to save space. type..NEZkhXSX is referenced and also defined in prog.o. Thus, by the time libstd.so is loaded in ldshlibsyms, the symbol already has type SRODATA and presumably a valid pointer in its Sect field. link doesn't crash in ld.relocsym but eventually fails because it can't parse package data for hash generation (as indicated in my previous comment), which seems to be a separate, new issue.

  3. prog.o is compiled with only -dynlink. type..NEZkhXSX is referenced in prog.o but never defined. When ldshlibsyms encounters the definition in libstd.so, it sees that the symbol still has type Sxxx, so it changes its Type to SDYNIMPORT and its File to $GOROOT/pkg/linux_amd64_dynlink/libstd.so, and leaves its Sect field as nil. This causes the crash because the symbol reference is a R_ADDROFF relocation in another type symbol, specifically type.*main.T2, and the relocation must be relative to start address of a known section.

I understand why type..importpath."". isn't defined in prog.o, but why is type..importpath.main. defined in runtime.a? It seems that the problem could be avoided by having link generate a section that defines type..importpath.main. on the fly (if it is linking an executable), before ldshlibsyms is invoked.

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Aug 15, 2017

Answering my own question, the import path for package main is created in runtime.a by this line of code in dumpbasictypes:

dimportpath(types.NewPkg("main", ""))

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Aug 20, 2017

I have patched cmd/link/internal/gc/main.go to add a definition for type..NEZkhXSX (when linking an executable in linkshared mode), and that fixes the test case. But the problem affects more than just relocations for type..import.main. In my situation, I am trying to build a shared library from code that uses runtime symbols via //go:linkname. Doing so leads to R_CALL and R_GOTPCREL relocations for runtime symbols, whose pkgPath are not defined in the section where the relocation occurs.

The problem started with commit c165988, when the type of uncommonType.pkgPath was changed from name to nameOff, to reduce code size. Without undoing that change, I think there are two potential approaches to make this work:

  1. Teach link to insert symbol definitions for all relocated pkgPath names whose symbols are SDYNIMPORT.

  2. Create a new type of nameOff for the pkgPath field, which provides two methods to resolve the desired name, depending on the type of the symbol. If the symbol is defined in the binary being built, this field remains a 32-bit offset from the current section. If the symbols is SDYNIMPORT and undefined, this field works like a GOT-relative address.

@rsc

This comment has been minimized.

Contributor

rsc commented Nov 22, 2017

Removing release-blocker since this has been happening since Go 1.8.

@rsc rsc modified the milestones: Go1.10, Go1.11 Nov 22, 2017

@rsc rsc removed the release-blocker label Nov 22, 2017

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Nov 22, 2017

I have written a fix for this problem on 1.9, but tip now gives a different error with this test case:

$ go run run.go -linkshared method4.go
# go run run.go -- method4.go
exit status 1
/home/bryanpkc/Sources/go/go/pkg/tool/linux_amd64/link: prog.o is not an archive file

FAIL    method4.go      0.178s
exit status 1
@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Nov 22, 2017

run.go needs to be updated to run go tool compile with -pack so that intermediate output files are proper archives.

@bryanpkc

This comment has been minimized.

Contributor

bryanpkc commented Nov 22, 2017

Actually my fix is incomplete. It only handles the specific case I care about (method offsets to methods defined in shared libraries), not the general case of section-relative offsets to types and strings in shared libraries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment