go generate fails to mockgen on go 1.14

[Callisto13]

Actual behavior.

On go 1.14.

Running go generate ./... when the vendor dir is present fails with:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

(I am guessing this is because mockgen depends on some runtime thing being there which would not be included when vendoring.)

We saw this appear after bumping to go 1.14. On go 1.13.x go generate ran fine.

Expected behavior

My mocks should be generated and no error should occur.

To Reproduce

I have set up a dumb example repo which mimics how we generate mocks in our codebase.

Silly interfaces found in dinosaur/dinosaur.go. Mocks should be generated in mocks/mock_dino.go.

To create mocks we keep a doc.go file in mocks/ and set the mocks to be generated in that package using the format:

//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go

1. Ensure you are running go 1.14
2. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
3. Note that the vendor dir is not present.
4. Observe that go generate ./... works happily.
5. go mod vendor
6. Observe that go generate ./... fails with the error noted above.
7. Remove the vendor dir and run through again for lolz.
8. Do the exact same with go 1.13.x and see that everything works fine with the vendor dir present.

The behaviour is seen on all my team's machines.

Additional Information

• gomock mode (reflect or source): Source
• gomock version or git ref: v1.4.3
• golang version: 1.14

Triage Notes for the Maintainers

[codyoss]

Please update your generate command to look like this: //go:generate go run github.com/golang/mock/mockgen --build_flags=--mod=vendor -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go

Notable added: --build_flags=--mod=vendor.

That did the trick for me /w the latest mockgen and 1.14 😄

[Callisto13]

Afraid that doesn't work for me @codyoss, running go generate ./... (after go mod vendor) still results in:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

sorry, I should have mentioned that --build_flags=--mod=vendor was the first thing I tried.

[maelvls]

Thank you for the minimal working example @Callisto13!! Helped me a lot when I tried to reproduce the bug.

TL;DR: Go 1.13 and below would still pick up from the pkg/mod cache even when vendor/ was there. With Go 1.14, any Go command automatically uses vendor/. And since github.com/golang/mock/mockgen is not imported anywhere, go mod vendor doesn't vendor it and go generate ./... fails 😔 A workaround is to add this somewhere:

import (
	_ "github.com/golang/mock/mockgen/model"
)

---

After some investigation, I think that what happens is that when you run go mod vendor it doesn't vendor github.com/golang/mock/mockgen since this import path isn't imported anywhere.

Let's see step by step

1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
2. go generate ./... succeeds
3. now let's vendor stuff

   % go mod vendor
   % git status
   Untracked files:
     (use "git add <file>..." to include in what will be committed)
        vendor/github.com/golang/mock/AUTHORS
        vendor/github.com/golang/mock/CONTRIBUTORS
        vendor/github.com/golang/mock/LICENSE
        vendor/github.com/golang/mock/gomock/call.go
        vendor/github.com/golang/mock/gomock/callset.go
        vendor/github.com/golang/mock/gomock/controller.go
        vendor/github.com/golang/mock/gomock/matchers.go
        vendor/modules.txt

4. at this point go generate ./... fails:

   % go generate ./...
   package github.com/golang/mock/mockgen: cannot find package "." in:
         /Users/mvalais/code/ori/cretaceous/vendor/github.com/golang/mock/mockgen
   mocks/doc.go:1: running "go": exit status 1

5. now let's add a fake import so that github.com/golang/mock/mockgen gets vendored:

   diff --git a/mocks/doc.go b/mocks/doc.go
   index ac383c5..c30949b 100644
   --- a/mocks/doc.go
   +++ b/mocks/doc.go
   @@ -1,3 +1,7 @@
    //go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
    
    package mocks
   +
   +import (
   +	_ "github.com/golang/mock/mockgen"
   +)

6. let's vendor and see what gets added:

   % go mod vendor
   % git status
   Untracked files:
     (use "git add <file>..." to include in what will be committed)
        vendor/github.com/golang/mock/AUTHORS
        vendor/github.com/golang/mock/CONTRIBUTORS
        vendor/github.com/golang/mock/LICENSE
        vendor/github.com/golang/mock/gomock/call.go
        vendor/github.com/golang/mock/gomock/callset.go
        vendor/github.com/golang/mock/gomock/controller.go
        vendor/github.com/golang/mock/gomock/matchers.go
        vendor/github.com/golang/mock/mockgen/mockgen.go
        vendor/github.com/golang/mock/mockgen/model/model.go
        vendor/github.com/golang/mock/mockgen/parse.go
        vendor/github.com/golang/mock/mockgen/reflect.go
        vendor/github.com/golang/mock/mockgen/version.1.11.go
        vendor/github.com/golang/mock/mockgen/version.1.12.go
        vendor/golang.org/x/tools/AUTHORS
        vendor/golang.org/x/tools/CONTRIBUTORS
        vendor/golang.org/x/tools/LICENSE
        vendor/golang.org/x/tools/PATENTS
        vendor/golang.org/x/tools/go/gcexportdata/gcexportdata.go
        vendor/golang.org/x/tools/go/gcexportdata/importer.go
        vendor/golang.org/x/tools/go/internal/gcimporter/bexport.go
        vendor/golang.org/x/tools/go/internal/gcimporter/bimport.go
        vendor/golang.org/x/tools/go/internal/gcimporter/exportdata.go
        vendor/golang.org/x/tools/go/internal/gcimporter/gcimporter.go
        vendor/golang.org/x/tools/go/internal/gcimporter/iexport.go
        vendor/golang.org/x/tools/go/internal/gcimporter/iimport.go
        vendor/golang.org/x/tools/go/internal/gcimporter/newInterface10.go
        vendor/golang.org/x/tools/go/internal/gcimporter/newInterface11.go
        vendor/golang.org/x/tools/go/internal/packagesdriver/sizes.go
        vendor/golang.org/x/tools/go/packages/doc.go
        vendor/golang.org/x/tools/go/packages/external.go
        vendor/golang.org/x/tools/go/packages/golist.go
        vendor/golang.org/x/tools/go/packages/golist_overlay.go
        vendor/golang.org/x/tools/go/packages/packages.go
        vendor/golang.org/x/tools/go/packages/visit.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_fileno.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_ino.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_bsd.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_linux.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_portable.go
        vendor/golang.org/x/tools/internal/fastwalk/fastwalk_unix.go
        vendor/golang.org/x/tools/internal/gopathwalk/walk.go
        vendor/golang.org/x/tools/internal/semver/semver.go
        vendor/modules.txt

7. now it works but compiling fails (see next comment from @Callisto13)

   % go generate ./...
   # Works!
   % go test ./...
   mocks/doc.go:6:2: import "github.com/golang/mock/mockgen" is a program, not an importable package

8. as @Callisto13 proposed in the next comment, we can import mockgen/model instead (which is the only non-main module!)

   import (
   	_ "github.com/golang/mock/mockgen/model"
   )

Quite an unpleasant workaround I must admit 🙄

[Callisto13]

Thanks @maelvls!

I experimented with this last week and came to almost the same conclusion.

Unfortunately importing _ "github.com/golang/mock/mockgen" leads to a compilation failure:

import "github.com/golang/mock/mockgen" is a program, not an importable package

Having any fake import is a rather unpleasant hack so I am hoping the folks at gomock can suggest something less awful.

[codyoss]

I am not sure I have a great solution for this. To get the 1.13 behavior I think you have use a -mod=mod but I have not had a chance to test this: https://golang.org/doc/go1.14#go-command

You have to pick if you want the go tooling to properly respect the vendor folder, only on partially. I am thinking the former is what most people will want.

With modules I think people are going to find having something like a tool.go file, where there are some underscore imports quite helpful, for reproducible builds. I don't think this is a bad pattern to use. This is nice to make sure things like goimports is in your CI system. Here is an example from another project I work on: https://github.com/googleapis/google-cloud-go/blob/master/tools.go

[maelvls]

I just tried the -mod=mod solution, it works! For anyone to try,

1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
2. apply this diff

   diff --git a/mocks/doc.go b/mocks/doc.go
   index ac383c5..204c82e 100644
   --- a/mocks/doc.go
   +++ b/mocks/doc.go
   @@ -1,3 +1,3 @@
   -//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
   +//go:generate go run -mod=mod github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go -build_flags=-mod=mod

   Note that there are two -mod=mod laying around: one for the go run, one passed with -build_flags that is used for subsequent go calls that mockgen does
3. go mod vendor
4. go generate ./... should work!

[Callisto13]

🎉 thanks @codyoss and @maelvls