Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go generate fails to mockgen on go 1.14 #415

Closed
Callisto13 opened this issue Mar 19, 2020 · 7 comments
Closed

go generate fails to mockgen on go 1.14 #415

Callisto13 opened this issue Mar 19, 2020 · 7 comments

Comments

@Callisto13
Copy link

Callisto13 commented Mar 19, 2020

Actual behavior.

On go 1.14.

Running go generate ./... when the vendor dir is present fails with:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

(I am guessing this is because mockgen depends on some runtime thing being there which would not be included when vendoring.)

We saw this appear after bumping to go 1.14. On go 1.13.x go generate ran fine.

Expected behavior

My mocks should be generated and no error should occur.

To Reproduce

I have set up a dumb example repo which mimics how we generate mocks in our codebase.

Silly interfaces found in dinosaur/dinosaur.go. Mocks should be generated in mocks/mock_dino.go.

To create mocks we keep a doc.go file in mocks/ and set the mocks to be generated in that package using the format:

//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
  1. Ensure you are running go 1.14
  2. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
  3. Note that the vendor dir is not present.
  4. Observe that go generate ./... works happily.
  5. go mod vendor
  6. Observe that go generate ./... fails with the error noted above.
  7. Remove the vendor dir and run through again for lolz.
  8. Do the exact same with go 1.13.x and see that everything works fine with the vendor dir present.

The behaviour is seen on all my team's machines.

Additional Information

  • gomock mode (reflect or source): Source
  • gomock version or git ref: v1.4.3
  • golang version: 1.14

Triage Notes for the Maintainers

@Callisto13 Callisto13 changed the title go generate fails to mockgen when vendor dir present on go 1.14 go generate fails to mockgen on go 1.14 Mar 20, 2020
@codyoss
Copy link
Member

codyoss commented Mar 22, 2020

Please update your generate command to look like this: //go:generate go run github.com/golang/mock/mockgen --build_flags=--mod=vendor -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go

Notable added: --build_flags=--mod=vendor.

That did the trick for me /w the latest mockgen and 1.14 😄

@Callisto13
Copy link
Author

Callisto13 commented Mar 23, 2020

Afraid that doesn't work for me @codyoss, running go generate ./... (after go mod vendor) still results in:

package github.com/golang/mock/mockgen: cannot find package "." in:
        /Users/callisto/workspace/cretaceous/vendor/github.com/golang/mock/mockgen
mocks/doc.go:1: running "go": exit status 1

sorry, I should have mentioned that --build_flags=--mod=vendor was the first thing I tried.

@maelvls
Copy link

maelvls commented Mar 23, 2020

Thank you for the minimal working example @Callisto13!! Helped me a lot when I tried to reproduce the bug.

TL;DR: Go 1.13 and below would still pick up from the pkg/mod cache even when vendor/ was there. With Go 1.14, any Go command automatically uses vendor/. And since github.com/golang/mock/mockgen is not imported anywhere, go mod vendor doesn't vendor it and go generate ./... fails 😔 A workaround is to add this somewhere:

import (
	_ "github.com/golang/mock/mockgen/model"
)

After some investigation, I think that what happens is that when you run go mod vendor it doesn't vendor github.com/golang/mock/mockgen since this import path isn't imported anywhere.

Let's see step by step

  1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
  2. go generate ./... succeeds
  3. now let's vendor stuff
    % go mod vendor
    % git status
    Untracked files:
      (use "git add <file>..." to include in what will be committed)
         vendor/github.com/golang/mock/AUTHORS
         vendor/github.com/golang/mock/CONTRIBUTORS
         vendor/github.com/golang/mock/LICENSE
         vendor/github.com/golang/mock/gomock/call.go
         vendor/github.com/golang/mock/gomock/callset.go
         vendor/github.com/golang/mock/gomock/controller.go
         vendor/github.com/golang/mock/gomock/matchers.go
         vendor/modules.txt
  1. at this point go generate ./... fails:

    % go generate ./...
    package github.com/golang/mock/mockgen: cannot find package "." in:
          /Users/mvalais/code/ori/cretaceous/vendor/github.com/golang/mock/mockgen
    mocks/doc.go:1: running "go": exit status 1
  2. now let's add a fake import so that github.com/golang/mock/mockgen gets vendored:

    diff --git a/mocks/doc.go b/mocks/doc.go
    index ac383c5..c30949b 100644
    --- a/mocks/doc.go
    +++ b/mocks/doc.go
    @@ -1,3 +1,7 @@
     //go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
     
     package mocks
    +
    +import (
    +	_ "github.com/golang/mock/mockgen"
    +)
  3. let's vendor and see what gets added:

    % go mod vendor
    % git status
    Untracked files:
      (use "git add <file>..." to include in what will be committed)
         vendor/github.com/golang/mock/AUTHORS
         vendor/github.com/golang/mock/CONTRIBUTORS
         vendor/github.com/golang/mock/LICENSE
         vendor/github.com/golang/mock/gomock/call.go
         vendor/github.com/golang/mock/gomock/callset.go
         vendor/github.com/golang/mock/gomock/controller.go
         vendor/github.com/golang/mock/gomock/matchers.go
         vendor/github.com/golang/mock/mockgen/mockgen.go
         vendor/github.com/golang/mock/mockgen/model/model.go
         vendor/github.com/golang/mock/mockgen/parse.go
         vendor/github.com/golang/mock/mockgen/reflect.go
         vendor/github.com/golang/mock/mockgen/version.1.11.go
         vendor/github.com/golang/mock/mockgen/version.1.12.go
         vendor/golang.org/x/tools/AUTHORS
         vendor/golang.org/x/tools/CONTRIBUTORS
         vendor/golang.org/x/tools/LICENSE
         vendor/golang.org/x/tools/PATENTS
         vendor/golang.org/x/tools/go/gcexportdata/gcexportdata.go
         vendor/golang.org/x/tools/go/gcexportdata/importer.go
         vendor/golang.org/x/tools/go/internal/gcimporter/bexport.go
         vendor/golang.org/x/tools/go/internal/gcimporter/bimport.go
         vendor/golang.org/x/tools/go/internal/gcimporter/exportdata.go
         vendor/golang.org/x/tools/go/internal/gcimporter/gcimporter.go
         vendor/golang.org/x/tools/go/internal/gcimporter/iexport.go
         vendor/golang.org/x/tools/go/internal/gcimporter/iimport.go
         vendor/golang.org/x/tools/go/internal/gcimporter/newInterface10.go
         vendor/golang.org/x/tools/go/internal/gcimporter/newInterface11.go
         vendor/golang.org/x/tools/go/internal/packagesdriver/sizes.go
         vendor/golang.org/x/tools/go/packages/doc.go
         vendor/golang.org/x/tools/go/packages/external.go
         vendor/golang.org/x/tools/go/packages/golist.go
         vendor/golang.org/x/tools/go/packages/golist_overlay.go
         vendor/golang.org/x/tools/go/packages/packages.go
         vendor/golang.org/x/tools/go/packages/visit.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_fileno.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_ino.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_bsd.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_dirent_namlen_linux.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_portable.go
         vendor/golang.org/x/tools/internal/fastwalk/fastwalk_unix.go
         vendor/golang.org/x/tools/internal/gopathwalk/walk.go
         vendor/golang.org/x/tools/internal/semver/semver.go
         vendor/modules.txt
  4. now it works but compiling fails (see next comment from @Callisto13)

    % go generate ./...
    # Works!
    % go test ./...
    mocks/doc.go:6:2: import "github.com/golang/mock/mockgen" is a program, not an importable package
  5. as @Callisto13 proposed in the next comment, we can import mockgen/model instead (which is the only non-main module!)

    import (
    	_ "github.com/golang/mock/mockgen/model"
    )

Quite an unpleasant workaround I must admit 🙄

@Callisto13
Copy link
Author

Callisto13 commented Mar 23, 2020

Thanks @maelvls!

I experimented with this last week and came to almost the same conclusion.

Unfortunately importing _ "github.com/golang/mock/mockgen" leads to a compilation failure:

import "github.com/golang/mock/mockgen" is a program, not an importable package

Having any fake import is a rather unpleasant hack so I am hoping the folks at gomock can suggest something less awful.

@codyoss codyoss reopened this Mar 23, 2020
@codyoss
Copy link
Member

codyoss commented Mar 23, 2020

I am not sure I have a great solution for this. To get the 1.13 behavior I think you have use a -mod=mod but I have not had a chance to test this: https://golang.org/doc/go1.14#go-command

You have to pick if you want the go tooling to properly respect the vendor folder, only on partially. I am thinking the former is what most people will want.

With modules I think people are going to find having something like a tool.go file, where there are some underscore imports quite helpful, for reproducible builds. I don't think this is a bad pattern to use. This is nice to make sure things like goimports is in your CI system. Here is an example from another project I work on: https://github.com/googleapis/google-cloud-go/blob/master/tools.go

@maelvls
Copy link

maelvls commented Mar 23, 2020

I just tried the -mod=mod solution, it works! For anyone to try,

  1. git clone https://github.com/Callisto13/cretaceous && cd cretaceous
  2. apply this diff
    diff --git a/mocks/doc.go b/mocks/doc.go
    index ac383c5..204c82e 100644
    --- a/mocks/doc.go
    +++ b/mocks/doc.go
    @@ -1,3 +1,3 @@
    -//go:generate go run github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go
    +//go:generate go run -mod=mod github.com/golang/mock/mockgen -package mocks -destination=./mock_dinos.go -source=../dinosaur/dinosaur.go -build_flags=-mod=mod
    Note that there are two -mod=mod laying around: one for the go run, one passed with -build_flags that is used for subsequent go calls that mockgen does
  3. go mod vendor
  4. go generate ./... should work!

@Callisto13
Copy link
Author

🎉 thanks @codyoss and @maelvls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants