New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin: json - gcc failed - version node not found for symbol type ...@... #19529

Closed
fsenart opened this Issue Mar 13, 2017 · 4 comments

Comments

Projects
None yet
6 participants
@fsenart

fsenart commented Mar 13, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/fsenart/projects"
GORACE=""
GOROOT="/home/fsenart/tools/go1.8"
GOTOOLDIR="/home/fsenart/tools/go1.8/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build992249354=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

package main

type Foo struct {
	Bar string `json:"Bar@baz,omitempty"`
}

func Dummy() {
	print(&Foo{})
}

What did you expect to see?

I expect to be able to build the above code as a plugin:

go build -buildmode=plugin -o issue.so

What did you see instead?

# issue_json
/home/fsenart/tools/go1.8/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: $WORK/issue_json/_obj/exe/a.out.so: version node not found for symbol type..namedata.Bar.json:"Bar@baz,omitempty"
/usr/bin/ld: failed to set dynamic section sizes: Bad value
collect2: error: ld returned 1 exit status

Observations

  • The error happens only when using the @ sign in the json tag
  • The error does not happen when building the exact same program in regular mode.

@ALTree ALTree added this to the Go1.9 milestone Mar 13, 2017

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Mar 13, 2017

Contributor

This must be some kind of interaction with the way that the GNU linker uses '@' in a symbol name to separate the name from the version.

Contributor

ianlancetaylor commented Mar 13, 2017

This must be some kind of interaction with the way that the GNU linker uses '@' in a symbol name to separate the name from the version.

@bradfitz bradfitz modified the milestones: Go1.9, Go1.10 Jun 7, 2017

@crawshaw

This comment has been minimized.

Show comment
Hide comment
@crawshaw

crawshaw Sep 29, 2017

Contributor

-buildmode=shared does not have this problem because it base64 encodes all the type symbol names to save space. It sounds easy to do the same thing for plugins, but it turns out to be a tricky initialization problem. The symbol rename is done as the linker reads in object files. Problem is, we want the symbol names in a -buildmode=plugin to match the host program, where DynlinkingGo() is true in the linker. But we set that to true for plugin host programs by scanning the symbols for the "plugin" package, which we don't know yet when we are reading in the symbols for the first time. So to do this we would have to move that symbol renaming logic after all symbols are loaded.

Oddly, we do not stumble over the GNU linker with this symbol name in a binary:

$ cat ~/junk.go 
package main

import (
	_ "plugin" // triggers DynlinkingGo()
	"reflect"
)

type Foo struct {
	Bar string `json:"Bar@baz,omitempty"`
}

func main() {
	println(reflect.TypeOf(Foo{}).Field(0).Tag)
}

$ go build junk.go
$ nm junk | grep json
00000000004e79a6 R type..namedata.Bar.json:"Bar@baz,omitempty"
$

Hmm.

Contributor

crawshaw commented Sep 29, 2017

-buildmode=shared does not have this problem because it base64 encodes all the type symbol names to save space. It sounds easy to do the same thing for plugins, but it turns out to be a tricky initialization problem. The symbol rename is done as the linker reads in object files. Problem is, we want the symbol names in a -buildmode=plugin to match the host program, where DynlinkingGo() is true in the linker. But we set that to true for plugin host programs by scanning the symbols for the "plugin" package, which we don't know yet when we are reading in the symbols for the first time. So to do this we would have to move that symbol renaming logic after all symbols are loaded.

Oddly, we do not stumble over the GNU linker with this symbol name in a binary:

$ cat ~/junk.go 
package main

import (
	_ "plugin" // triggers DynlinkingGo()
	"reflect"
)

type Foo struct {
	Bar string `json:"Bar@baz,omitempty"`
}

func main() {
	println(reflect.TypeOf(Foo{}).Field(0).Tag)
}

$ go build junk.go
$ nm junk | grep json
00000000004e79a6 R type..namedata.Bar.json:"Bar@baz,omitempty"
$

Hmm.

@crawshaw

This comment has been minimized.

Show comment
Hide comment
@crawshaw

crawshaw Sep 29, 2017

Contributor

I spent a while trying to move the symbol renaming logic later in cmd/link (specifically, after all symbols were loaded). Turns out it is very complicated.

Symbols are partially de-duplicated as they are loaded. See the objReader.readSym method. If the renaming happens later, then a symbol that has previously been renamed will not be de-duplicated. For example, type.unsafe.Pointer in libstd.so is loaded as type.cZzVjlwW, but from the .a file being linked it comes in as type.unsafe.Pointer. Only one of these contains the contents necessary for the Go type decoding that the linker does later.

So to move the renaming later, all the de-duplication logic would have to be untangled from the object file reading. This is probably a worthy project, some already has been done (see dupTextSyms, which moves text symbol de-duplication to after loading), but it is a significant project.

Contributor

crawshaw commented Sep 29, 2017

I spent a while trying to move the symbol renaming logic later in cmd/link (specifically, after all symbols were loaded). Turns out it is very complicated.

Symbols are partially de-duplicated as they are loaded. See the objReader.readSym method. If the renaming happens later, then a symbol that has previously been renamed will not be de-duplicated. For example, type.unsafe.Pointer in libstd.so is loaded as type.cZzVjlwW, but from the .a file being linked it comes in as type.unsafe.Pointer. Only one of these contains the contents necessary for the Go type decoding that the linker does later.

So to move the renaming later, all the de-duplication logic would have to be untangled from the object file reading. This is probably a worthy project, some already has been done (see dupTextSyms, which moves text symbol de-duplication to after loading), but it is a significant project.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot Sep 30, 2017

Change https://golang.org/cl/67312 mentions this issue: cmd/link: type symbol name mangling for plugins

gopherbot commented Sep 30, 2017

Change https://golang.org/cl/67312 mentions this issue: cmd/link: type symbol name mangling for plugins

@gopherbot gopherbot closed this in 24e4a12 Oct 5, 2017

@s12v s12v referenced this issue Oct 31, 2017

Closed

Amazon Elasticsearch support #5466

0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment