plugin: fails to correctly reference type from vendor

[BryceDFisher]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

yes, 1.10.3

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN="/home/fisherb/go/bin"
GOCACHE="/home/fisherb/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/fisherb/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build861164804=/tmp/go-build -gno-record-gcc-switches"

What did you do?

When using plugins, in a typical github vendoring approach: $GOPATH/src/github.com/author/repo/package with resources in $GOPATH/src/github.com/author/repo/package/vendor/github.com/author/repoa/package, if the plugin was compiled using a separate package, that is ultimately imported into the main package via vendoring, the types don't match and casting is not possible.

This tree works (rooted at $GOPATH/src/github.com:

plbody
└── pltypes
    ├── plbody
    │   └── plbody.go
    └── pltypes.go
plmain
└── plmain.go
plplugin
└── plplugin
    └── plplugin.go

This tree does not work (rooted at $GOPATH/src/github.com)

plbody
└── pltypes
    ├── plbody
    │   └── plbody.go
    └── pltypes.go
plmain
├── plmain.go
└── vendor
    └── github.com
        └── plbody
            └── pltypes
                ├── plbody
                │   └── plbody.go
                └── pltypes.go
plplugin
└── plplugin
    ├── plplugin.go
    └── vendor
        └── github.com
            └── plbody
                └── pltypes
                    ├── plbody
                    │   └── plbody.go
                    └── pltypes.go

In both the working and not working cases, the imports are the same, just the directory structure is different.
plbody.go:

package plbody

import (
        "log"
        "plugin"

        "github.com/plbody/pltypes"
)

//Test is the test function
var Test func(a pltypes.A, s string) error

func init() {
        pl, err := plugin.Open("plplugin.so")
        if err != nil {
                log.Fatalf("Unable to find plugin: %s", err.Error())
        }
        obj, err := pl.Lookup("Test")
        if err != nil {
                log.Fatalf("Unable to find Test in plugin: %s", err.Error())
        }

        t := obj.(pltypes.Tester)
        Test = t.Test
}

pltypes.go:

package pltypes

//Tester contains a test function
type Tester interface {
        Test(A, string) error
}

//A A is a custom data type
type A struct {
        Data string
}

plmain.go:

package main

import (
        "log"

        "github.com/plbody/pltypes"
        "github.com/plbody/pltypes/plbody"
)

func main() {
        log.Printf("%v", plbody.Test(pltypes.A{}, ""))
}

plplugin.go:

package main

import "github.com/plbody/pltypes"

type test struct{}

//Test is the exported test struct
var Test test

func main() {}

func (t test) Test(a pltypes.A, s string) error {
        return nil
}

What did you expect to see?

2018/08/17 15:03:34 <nil>

What did you see instead?

The types are not castable to each other:

panic: interface conversion: *main.test is not pltypes.Tester: missing method Test

goroutine 1 [running]:
github.com/plmain/vendor/github.com/plbody/pltypes/plbody.init.0()
        /home/fisherb/go/src/github.com/plmain/vendor/github.com/plbody/pltypes/plbody/plbody.go:23 +0xcf

[andybons]

@ianlancetaylor

[BryceDFisher]

Upon further inspection, it appears the reflection does not view the type A as the same type in the function definition, despite the fact that reflect.TypeOf(A) returns the same PkgPath ("") and Name ("A"). If you remove A from the function definition, everything works as expected.

[BryceDFisher]

It looks like the discussion from https://github.com/golang/proposal/blob/master/design/25719-go15vendor.md and the comment in

go/src/go/build/build.go

Line 360 in 3acd257

// By default, Import searches vendor directories

ensure that plugins built with vendor directories will never see types as used above as the same type at run time.

Does anyone have a better method that I could use in my example to load the plugin with the custom type pltypes.A? Or should I assume that plugins will not support this behavior for the foreseeable future and therefore I should move on to other options.

[mcandre]

Recommendations:

• Encourage Context key lookup by value rather than reference.
• Drop vendor/ from reflection labels.

[mcandre]

My workaround for this bug, is to store the go get's and refs in a reusable script:

$ cat deps.sh
#!/bin/sh
unset IFS
set -euf

go get github.com/TykTechnologies/tyk
sh -c "cd $GOPATH/src/github.com/TykTechnologies/tyk && git checkout v2.9.4.3"

# Re-enable Tyk plugin feature
sh -c "cd $GOPATH/src/github.com/TykTechnologies/tyk && go install -a -tags 'goplugin'"

$ ./deps.sh

This is not ideal for Windows, but it basically works in any UNIX environment, such as WSL.

[seankhliao]

I believe the solution is to use modules (and thus not use nested vendor directories).

[mcandre]

I do use modules. Vendoring is an option in go modules. I use this option to mitigate leftpad (actually logrus and moby) problems.

[ianlancetaylor]

I think the issue here is the use of plugins with vendor directories.

[seankhliao]

I'm not sure how this applies to building with modules, even with vendoring. The original report was for the main and plugin package each vendoring a shared dependency in their own subtrees, the identity of the dependency was different due to the different build paths.

in module mode: if the main and plugin package are in the same module, then it shares the vendor directory, and thus same identity for the dependency.
if the main and plugin package are in different modules, then -trimpath is mandatory, and then it works?

I think we would need a modern reproducer if we want to keep this open.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)