Imports of vendored packages add the vendor prefix to the import path #18

johanbrandhorst · 2017-07-11T09:53:33Z

I recently updated moq (imports, yay!) and tried regenerating one of my files and got the following imports generated:

package mocks

// AUTOGENERATED BY MOQ
// github.com/matryer/moq

import (
	"sync"
	"<myrepo>/vendor/golang.org/x/net/context"
)

This obviously doesn't work.

johanbrandhorst · 2017-07-11T16:12:13Z

I'm also seeing a lot of weird errors when trying to use moq with a vendor folder:

file.go:36:37: cannot use bla.Bla (of type github.com/group/bla.Bla) as <myrepo>/vendor/github.com/group/bla.Bla value in argument to something.Something

These files used to work just fine with Moq until recently.

johanbrandhorst · 2017-07-11T16:20:34Z

Almost certainly introduced by 1bd8336

matryer · 2017-07-11T18:42:55Z

This one is surprisingly difficult to test

matryer · 2017-07-11T19:18:23Z

@johanbrandhorst Could you quickly grab branch vendoring-18 and see if that user.go file works for you?

matryer · 2017-07-11T19:19:07Z

For some reason, the vendoring stuff just isn't working here.

matryer · 2017-07-11T19:27:21Z

OK it's working (turns out testdata in the path screws things up)... now I have a failing test, which proves this bug.

matryer · 2017-07-11T19:44:55Z

@johanbrandhorst I think I fixed it... can you test it on your code once I merge #19 in?

matryer · 2017-07-11T20:33:38Z

p.s. @johanbrandhorst you should do a fresh install... I've also changed the way it captures the calls etc. and it's a breaking change.

johanbrandhorst · 2017-07-11T22:01:30Z

It has not fixed the 2nd problem, I'm still seeing things like

file.go:36:37: cannot use bla.Bla (of type github.com/group/bla.Bla) as <myrepo>/vendor/github.com/group/bla.Bla value in argument to something.Something

preventing me from testing this quickly without having to comment out all instances moq finds offensive.

johanbrandhorst · 2017-07-11T22:05:48Z

Does not seem to have fixed the other vendoring error either, sorry.

johanbrandhorst · 2017-08-02T14:56:55Z

This is still happening...

matryer · 2017-08-02T14:58:35Z

I wonder if you can submit a PR that proves it with a failing test? I couldn't recreate it very easily. It'll probably be an easy fix once we know the actual problem.

johanbrandhorst · 2017-08-02T15:00:12Z

I'll see if I can make a minimal test case, thanks

johanbrandhorst · 2017-08-02T20:02:42Z

https://github.com/johanbrandhorst/moq-vendor-bug/tree/master. This is exhibiting the incorrect import bug but (not yet?) the actual runtime error.

ataliadvanstep · 2017-09-02T05:37:10Z

I have same issue

I pulled the code from master but it seems the bug is still there

jayd3e · 2017-09-17T20:14:20Z

I'm unfortunately seeing this issue as well. I'm getting import errors whenever I try to generate an interface. moq doesn't seem to be able to find my project-specific packages, when I'm using a vendor/ folder. For example let's say I have all of my packages under /go/src/github.com/org/project/{PACKAGE_NAME}, and I also have a vendor subfolder at /go/src/github.com/org/project/vendor, I get an error message like so:

file.go:8:2: could not import github.com/org/project/core (can't find import: github.com/org/project/core)
moq [flags] destination interface [interface2 [interface3 [...]]]
  -out string
    	output file (default stdout)
  -pkg string
    	package name (default will infer)
service/account.go:13: running "moq": exit status 1
make: *** [generate] Error 1

jayd3e · 2017-09-19T03:58:22Z

If there is a hack or short-term fix for this, it would be much appreciated. I'm not a fan of any of the other mocking libraries besides moq. Is there a commit that we know the vendor/ directory was working for?

matryer · 2017-09-19T07:52:33Z

I wonder if someone could make a PR simulating this as a failing test? I cannot reproduce it. It's probably an easy fix, but not if I can't see the error.

Perhaps @jayd3e?

jayd3e · 2017-09-20T08:02:58Z

I'll try to create an isolated test to show this behavior. Will get back to you in the next couple days.

jayd3e · 2017-09-22T03:07:25Z

@matryer I just created a repo around this issue, showing what I'm running into. Hope this helps:

https://github.com/jayd3e/moq-issue

There are notes in the README.

johanbrandhorst · 2017-09-22T08:21:16Z

I made a repo which reproduces the error cause in August. I think Mat wants a test case for the tests, not a repo.

jayd3e · 2017-09-26T16:47:32Z

@johanbrandhorst just to confirm, are you seeing the same issues as I did in that repo? In other words, does my issue appear to be the same one that prompted you creating this thread?

jayd3e · 2017-09-26T16:52:53Z

I can try and create a test, but it would basically just be the same thing as that repo except programatic.

johanbrandhorst · 2017-09-26T18:19:36Z

I've described the two different errors I've seen, the repo I submitted shows the first error (the vendor path pretended to the imported package). I think a programmatic test that reproduces the error is exactly what we want, ideally both of them.

jayd3e · 2017-10-04T08:12:46Z

I'm not able to write a test for this. It might be an environment issue for me. If you guys are able to use moq for a large project, can you walk me through how you setup your environment successfully to use moq? I really want to use this lib, but I'm not quite sure where to go from here.

johanbrandhorst · 2017-10-04T10:39:03Z

I have had a suspicion that it might be an environment issue as well... I don't know if it depends on your $GOPATH or the contents of vendor, but I think it might be where the problem lies.

jayd3e · 2017-10-09T22:29:50Z

Alright, I'll try to do some more debugging on this tonight using my environment.

matryer · 2017-10-10T07:24:21Z

I’d love to get this fixed, but it is pretty tough without a a falling test and I cannot reproduce it.

…

On 9 Oct 2017, at 23:29, Joseph Dallago ***@***.***> wrote: Alright, I'll try to do some more debugging on this tonight using my environment. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGNG3uLqtULOb_jYvHkZq84CaaX1FKyks5sqp5fgaJpZM4OT_nY>.

jayd3e · 2017-10-10T22:34:10Z

@matryer are you able to use moq on a project with >10 dependencies using master? If so, I would love to hear how you usually setup your environment. In other words, what you set your GOPATH to and where you put your dependencies(vendor or in the standard GOPATH).

johnchildren · 2017-10-12T15:07:28Z

Hi, I work with @johanbrandhorst and after changing a few too many import paths I found a way to reproduce this issue. My test is in this commit: johnchildren@aa63789 .

Obviously it's not a great test but it seems to provide a starting point at least.

The problem seems to be with go generate (as moq works perfectly without it) which is I suppose why other mocking tools have the same problem, though it looks like a workaround might be possible based on some of the issues on MockGen (golang/mock#28 golang/mock#99).

benma · 2017-12-06T14:45:37Z

I just had the same issue. Is there any movement on this? This is practically a deal breaker.

matryer · 2017-12-06T15:03:29Z

I just brought this test into the project and guess what? The test didn't fail.

The package it's importing is vendorerd (see it here).

It produced this:

// Code generated by moq; DO NOT EDIT
// github.com/matryer/moq

package user

import (
	"github.com/matryer/somerepo"
	"sync"
)

var (
	lockServiceMockDoSomething sync.RWMutex
)

// ServiceMock is a mock implementation of Service.
//
//     func TestSomethingThatUsesService(t *testing.T) {
//
//         // make and configure a mocked Service
//         mockedService := &ServiceMock{
//             DoSomethingFunc: func(in1 somerepo.SomeType) error {
// 	               panic("TODO: mock out the DoSomething method")
//             },
//         }
//
//         // TODO: use mockedService in code that requires Service
//         //       and then make assertions.
//
//     }
type ServiceMock struct {
	// DoSomethingFunc mocks the DoSomething method.
	DoSomethingFunc func(in1 somerepo.SomeType) error

	// calls tracks calls to the methods.
	calls struct {
		// DoSomething holds details about calls to the DoSomething method.
		DoSomething []struct {
			// In1 is the in1 argument value.
			In1 somerepo.SomeType
		}
	}
}

// DoSomething calls DoSomethingFunc.
func (mock *ServiceMock) DoSomething(in1 somerepo.SomeType) error {
	if mock.DoSomethingFunc == nil {
		panic("moq: ServiceMock.DoSomethingFunc is nil but Service.DoSomething was just called")
	}
	callInfo := struct {
		In1 somerepo.SomeType
	}{
		In1: in1,
	}
	lockServiceMockDoSomething.Lock()
	mock.calls.DoSomething = append(mock.calls.DoSomething, callInfo)
	lockServiceMockDoSomething.Unlock()
	return mock.DoSomethingFunc(in1)
}

// DoSomethingCalls gets all the calls that were made to DoSomething.
// Check the length with:
//     len(mockedService.DoSomethingCalls())
func (mock *ServiceMock) DoSomethingCalls() []struct {
	In1 somerepo.SomeType
} {
	var calls []struct {
		In1 somerepo.SomeType
	}
	lockServiceMockDoSomething.RLock()
	calls = mock.calls.DoSomething
	lockServiceMockDoSomething.RUnlock()
	return calls
}

benma · 2017-12-06T15:26:46Z

Thanks for the quick response.

I double checked, and it seems it works, but not all of the time. After deleting my go/pkg folder, I got this error:

myfile.go:6:2: could not import github.com/btcsuite/btcutil (could not import github.com/btcsuite/btcd/btcec (/home/user/go/src/github.com/my/repo/vendor/github.com/btcsuite/btcd/btcec/genprecomps.go:10:1: package main; expected btcec))

The issue seems to be related to // +build: ignore being ignored. Any ideas?
https://github.com/btcsuite/btcd/blob/2e60448ffcc6bf78332d1fe590260095f554dd78/btcec/genprecomps.go#L8

johnchildren · 2017-12-06T15:51:30Z

Strange, it's passing for me as well now. I have no idea what has changed, but even backdating my Go to 1.8 and using earlier versions of moq seem to have no effect.

However the good news is that moq seems to be working elsewhere as well while mockgen is still failing.

coderanger · 2019-02-13T00:40:34Z

This is back to being broken after #87

$ moq -pkg edit_test ../../../vendor/github.com/aws/aws-sdk-go/service/kms/kmsiface KMSAPI | head 
// Code generated by moq; DO NOT EDIT.
// github.com/matryer/moq

package edit_test

import (
        "context"
        "github.com/Ridecell/ridectl/vendor/github.com/aws/aws-sdk-go/service/kms/kmsiface"
        "github.com/aws/aws-sdk-go/aws/request"
        "github.com/aws/aws-sdk-go/service/kms"
        "sync"
)

matryer · 2019-02-13T07:22:12Z

Any idea why that would break after #87?

…

On 13 Feb 2019, at 00:40, Noah Kantrowitz ***@***.***> wrote: This is back to being broken after #87 $ moq -pkg edit_test ../../../vendor/github.com/aws/aws-sdk-go/service/kms/kmsiface KMSAPI | head // Code generated by moq; DO NOT EDIT. // github.com/matryer/moq package edit_test import ( "context" "github.com/Ridecell/ridectl/vendor/github.com/aws/aws-sdk-go/service/kms/kmsiface" "github.com/aws/aws-sdk-go/aws/request" "github.com/aws/aws-sdk-go/service/kms" "sync" ) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

coderanger · 2019-02-13T07:29:58Z

@matryer I'm guessing #87 used underlying data that was never fixed, so we should track down what caused this to be fixed some of the time and make it use that data instead? Or maybe it was never fixed and the cause is environmental?

johanbrandhorst · 2019-02-13T08:09:19Z

It affected me as well when testing the branch. #87 (comment)

matryer mentioned this issue Jul 11, 2017

Vendoring 18 #19

Merged

matryer closed this as completed in #19 Jul 11, 2017

matryer reopened this Aug 2, 2017

kwoodhouse93 mentioned this issue Nov 10, 2017

Custom importer may be causing errors #34

Closed

sudo-suhas mentioned this issue Jan 17, 2020

Handle target interface inside vendor with other pkg name #122

Merged

sudo-suhas closed this as completed in #122 Jan 20, 2020

kohkimakimoto mentioned this issue Jan 6, 2021

Fix import paths that are in vendor rjeczalik/interfaces#26

Merged

Imports of vendored packages add the vendor prefix to the import path #18

Imports of vendored packages add the vendor prefix to the import path #18

Comments

johanbrandhorst commented Jul 11, 2017

johanbrandhorst commented Jul 11, 2017

johanbrandhorst commented Jul 11, 2017

matryer commented Jul 11, 2017

matryer commented Jul 11, 2017 • edited

matryer commented Jul 11, 2017

matryer commented Jul 11, 2017

matryer commented Jul 11, 2017

matryer commented Jul 11, 2017

johanbrandhorst commented Jul 11, 2017

johanbrandhorst commented Jul 11, 2017

johanbrandhorst commented Aug 2, 2017

matryer commented Aug 2, 2017

johanbrandhorst commented Aug 2, 2017

johanbrandhorst commented Aug 2, 2017

ataliadvanstep commented Sep 2, 2017 • edited

jayd3e commented Sep 17, 2017 • edited

jayd3e commented Sep 19, 2017 • edited

matryer commented Sep 19, 2017 • edited

jayd3e commented Sep 20, 2017

jayd3e commented Sep 22, 2017 • edited

johanbrandhorst commented Sep 22, 2017

jayd3e commented Sep 26, 2017 • edited

jayd3e commented Sep 26, 2017

johanbrandhorst commented Sep 26, 2017

jayd3e commented Oct 4, 2017

johanbrandhorst commented Oct 4, 2017

jayd3e commented Oct 9, 2017

matryer commented Oct 10, 2017 via email

jayd3e commented Oct 10, 2017

johnchildren commented Oct 12, 2017

benma commented Dec 6, 2017

matryer commented Dec 6, 2017 • edited

benma commented Dec 6, 2017

johnchildren commented Dec 6, 2017 • edited

coderanger commented Feb 13, 2019

matryer commented Feb 13, 2019 via email

coderanger commented Feb 13, 2019

johanbrandhorst commented Feb 13, 2019

matryer commented Jul 11, 2017 •

edited

ataliadvanstep commented Sep 2, 2017 •

edited

jayd3e commented Sep 17, 2017 •

edited

jayd3e commented Sep 19, 2017 •

edited

matryer commented Sep 19, 2017 •

edited

jayd3e commented Sep 22, 2017 •

edited

jayd3e commented Sep 26, 2017 •

edited

matryer commented Dec 6, 2017 •

edited

johnchildren commented Dec 6, 2017 •

edited