Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: ABI hash for package in shared library should not include file path of dependency #30531

Closed
eternal-flame-AD opened this issue Mar 2, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@eternal-flame-AD
Copy link

commented Mar 2, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ef/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ef/code/go/"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build320733553=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Write a main package with GO111MODULE, which opens another package as a plugin.
  2. Write another main package with GO111MODULE and a common dependency (the version of the dependency is in equal to that of the first package), which is built as plugin and loaded by the previous package.
  3. Build both packages by go build [-buildmode=plugin] and test the compatibility between them.
  4. Build both packages by go mod vendor && go build -mod=vendor [-buildmode=plugin] and test the compatibility between them.

What did you expect to see?

In both cases the program runs successfully.

What did you see instead?

The package built by go mod vendor is rejected due to different ABI hashes of the common dependency. I dug into this and found that the difference in the ABI hash was caused by that the path to the package on the filesystem was recorded and hashed, as go mod vendor vendors the common depencency into different directories, the final ABI hash is different.

@eternal-flame-AD eternal-flame-AD changed the title cmd/link: ABI hash for package in shared library should not include cmd/link: ABI hash for package in shared library should not include file path of dependency Mar 2, 2019

@bauerm97

This comment has been minimized.

Copy link

commented Mar 4, 2019

I think this is related to #16860 , perhaps even a duplicate of the issue. It looks like there are plans in place for Go 1.13 to run go build in a "reproducible" mode, stripping the prefix of the source directory from the final binary. In the mean time, you can use this magic incantation of go build to fix this:

go build -gcflags=all=-trimpath=$(PREFIX) -asmflags=all=-trimpath=$(PREFIX) $(PACKAGE)

That's how we are solving this issue in the meantime while we wait for Go 1.13 to provide a fix. You can see that it works in this example:

mibauer@ubuntu:~/modules-plugins/main$ pwd
/home/mibauer/modules-plugins/main
mibauer@ubuntu:~/modules-plugins/main$ go clean -cache
mibauer@ubuntu:~/modules-plugins/main$ go build -buildmode=plugin -gcflags=all=-trimpath=/home/mibauer/modules-plugins -asmflags=all=-trimpath=/home/mibauer/modules-plugins p1
mibauer@ubuntu:~/modules-plugins/main$ go build -buildmode=plugin -gcflags=all=-trimpath=/home/mibauer/modules-plugins -asmflags=all=-trimpath=/home/mibauer/modules-plugins p2
mibauer@ubuntu:~/modules-plugins/main$ ls
build-vx  dumpdep  go.mod  go.sum  main.go  p1.so  p2.so  pkg  tools  vendor
mibauer@ubuntu:~/modules-plugins/main$ cp p1.so ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins/main$ cp p2.so ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins/main$ cd ~/modules-plugins-path2/main/
mibauer@ubuntu:~/modules-plugins-path2/main$ go build .
mibauer@ubuntu:~/modules-plugins-path2/main$ ./main 
panic: plugin.Open("p1"): plugin was built with a different version of package gopkg.in/yaml.v2

goroutine 1 [running]:
main.main()
    /home/mibauer/modules-plugins-path2/main/main.go:13 +0x229
mibauer@ubuntu:~/modules-plugins-path2/main$ rm main
mibauer@ubuntu:~/modules-plugins-path2/main$ go build -gcflags=all=-trimpath=/home/mibauer/modules-plugins-path2 -asmflags=all=-trimpath=/home/mibauer/modules-plugins-path2 .
mibauer@ubuntu:~/modules-plugins-path2/main$ ./main 
p1: Type: yaml.Encoder
p2: Type: yaml.Encoder
yaml.Encoder
mibauer@ubuntu:~/modules-plugins-path2/main$
@eternal-flame-AD

This comment has been minimized.

Copy link
Author

commented Mar 5, 2019

Thanks very much for the workaround:)

I took a look at the issue you mentioned, it was talking about build path being included in the linking stage while I was talking about in the compiling stage. Never mind, I think the "reproducible" build mode would solve both:)

@julieqiu julieqiu closed this Mar 11, 2019

mem added a commit to mem/singularity that referenced this issue Apr 5, 2019

Do not pass -mod=vendor when building singularity
This interferes with plugin loading. See:

- golang/go#31278
- golang/go#30531
- golang/go#28983

Signed-off-by: Marcelo E. Magallon <marcelo@sylabs.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.