plugin: loading on machine with different GOPATH fails

[suoigwg]

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes?

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/blablabla/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build657274199=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I build a go plugin from a docker image and copy it to a server. They have essentially the same go env except for the go path. (One was /go the other /home/username/go)

When I tried to load the plugin from other server, I always get

panic: plugin.Open("./parser"): plugin was built with a different version of package xxxx

It can only be fixed by setting the gopath the same as that in the docker image I used to build the plugin

I don't know if this is a bug or it is just how go plugin works.

Thanks.

[randall77]

I don't think that's intentional.
Can you run go version on your two different Go installations and report what they print?

The version mismatch is not between the plugin and the Go installation, it is between the Go installation that built the plugin and the Go installation that built the main program. Where was the main binary built? (You don't need the Go installation on the server at all, at least to run Go binaries and plugins.)

I can install two identical go instances at different paths, build the plugin with one and the main program with another, and the plugin loads just fine.

[suoigwg]

@randall77
They run exactly the same version of go(1.10.3). The only difference in go env is the go path.

Yes the main file is build by a different installation of go on a different machine(which only differs in gopath)

Basically what I am doing is building a plugin that references some model.go, and serve the model and plugin to the client so they can write program of their own.

that is

package model

type PublicModel struct{
	Msg string
}

package main

import "go-playground/model"

type api struct {}

func (i api)PublicApi() model.PublicModel {
	md := model.PublicModel{Msg: "Secret implementation"}
	return md
}


var Endpoint api

build a plugin.so and serve the so and model to user, the user may write a program on their machine like

package main

import (
	"plugin"
		"go-playground/model"
)

type endPoint interface {
	PublicApi() model.PublicModel
}
func main()  {
	plgin, err := plugin.Open("./plugin.so")
	if err != nil {
		panic(err)
	}
	symbol, err:= plgin.Lookup("Endpoint")
	if err != nil {
		panic(err)
	}
	b, ok := symbol.(endPoint)
	if !ok{
		panic("Type assertion fail")
	}
	md := b.PublicApi()
	println(md.Msg)
}

Essentially I need to hide the implementation of the PublicApi, I tried to build the plugin in a docker container(go path /go) and test it on a vm (go path /home/username/go). By testing I mean copy copying plugin.so ./model and stub.go to the corresponding directory and of course build the stub.go there, I always get that panic unless I change the go path to the same

[suoigwg]

@randall77
docker file to reproduce
environment 1

FROM ubuntu:16.04

ENV DEBIAN_FRONTEND noninteractive
ENV INITRD No
ENV LANG en_US.UTF-8
ENV GOVERSION 1.10.3
ENV GOROOT /opt/go
ENV GOPATH /go

RUN  apt-get update \
  && apt-get install -y wget \
  && apt-get install -y git \
  && apt-get install -y gcc \
  && rm -rf /var/lib/apt/lists/*

RUN cd /opt && wget https://storage.googleapis.com/golang/go${GOVERSION}.linux-amd64.tar.gz && \
    tar zxf go${GOVERSION}.linux-amd64.tar.gz && rm go${GOVERSION}.linux-amd64.tar.gz && \
    ln -s /opt/go/bin/go /usr/bin/ && \
    mkdir $GOPATH

build plugin

from env:1.0

WORKDIR /go/src/go-playground
COPY  ./ /go/src/go-playground
RUN  go build --buildmode=plugin -o plugin.so

copy the plugin from docker container

environment 2(differ only in gopath)

FROM ubuntu:16.04

ENV DEBIAN_FRONTEND noninteractive
ENV INITRD No
ENV LANG en_US.UTF-8
ENV GOVERSION 1.10.3
ENV GOROOT /opt/go
ENV GOPATH /another-go

RUN  apt-get update \
  && apt-get install -y wget \
  && apt-get install -y git \
  && apt-get install -y gcc \
  && rm -rf /var/lib/apt/lists/*

RUN cd /opt && wget https://storage.googleapis.com/golang/go${GOVERSION}.linux-amd64.tar.gz && \
    tar zxf go${GOVERSION}.linux-amd64.tar.gz && rm go${GOVERSION}.linux-amd64.tar.gz && \
    ln -s /opt/go/bin/go /usr/bin/ && \
    mkdir $GOPATH

use plugin in env2

from env:2.0
WORKDIR /another-go/src/go-playground
COPY ./stub.go ./
COPY ./plugin.so ./
COPY /model/ /another-go/src/go-playground/model

root@808cfd262c8d:/another-go/src/go-playground# go run stub.go
panic: plugin.Open("./plugin"): plugin was built with a different version of package go-playground/model

goroutine 1 [running]:
main.main()
/another-go/src/go-playground/stub.go:14 +0x259
exit status 2

[suoigwg]

It will work by using plugin in env1 in the same way

[randall77]

I still can't reproduce your issue. Your plugin runs fine for me.
Please run go version on all the Go installations involved and report the results. Maybe your symlinks aren't putting the go binary sufficiently in front of a default Go installation in the path?

I've never used Docker, unfortunately, so I don't know what things like "use plugin in env2" mean.

[suoigwg]

go version from machine 1

go version go1.10.3 linux/amd64

go env in case you need more info

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build029805510=/tmp/go-build -gno-record-gcc-switches"

go version from machine 2

go version go1.10.3 linux/amd64

go env from machine 2

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/another-go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build463889578=/tmp/go-build -gno-record-gcc-switches"

[randall77]

Ok, my bad. I was thinking GOROOT, not GOPATH. I am now able to reproduce.
With the following directory structure:

./gopathA
./gopathA/src
./gopathA/src/model
./gopathA/src/model/model.go
./gopathB
./gopathB/src
./gopathB/src/model
./gopathB/src/model/model.go
./main.go
./plugin.go

If you do:

$ GOPATH=`pwd`/gopathA go build -buildmode=plugin  -o plugin.so plugin.go
$ GOPATH=`pwd`/gopathB go run main.go
panic: plugin.Open("./plugin"): plugin was built with a different version of package model

The version check is using the hash of some header information in the exported information from the plugin package. That exported information has the full path in it, not just the path from GOPATH. As such, it's different for the two otherwise identical plugin packages.

The full path just appears in the filenames. I'm not sure that's relevant information to be hashing. For source code from GOROOT, the path starts at $GOROOT/src/... but for source from GOPATH the full path is used. I can understand why, as GOPATH can have colon-separated paths and it would not be clear which one was the source of the file in question.

Seems related to #9206 and #16860.

[juhwany]

@randall77
It still happens in Go 1.11. Is there any progress about this issue?

[randall77]

No progress. I think I understand the problem but I have no idea what the fix might be.

[BryceDFisher]

This is the same problem that I opened with #27062, except in my case the "vendor" directory was being included in the full file path, which messed up the package tracking from the plugin. Should I close my original issue to consolidate here?

[randall77]

I don't think they are quite the same issue. This one is the exact same import path in both the plugin and main build. #27062 has paths differ by the vendor prefix.
Just the issue cross-reference should be fine. They are related, and may be fixed together, but they don't need consolidation.

[rsc]

Once we do #16860, we can make it kick in for plugins automatically. For now, nothing to do for Go 1.12.

[juhwany]

@rsc

Regarding #16860, It seems that Go 1.13 will support reproducible builds.

Will this loading plugin issue be solved at Go 1.13 ?

[mytototo]

Any news regarding this issue? I am struggling distributing plugins to third-party users. I have tried as many build options as possible but still no success.