Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "unexpected module path" error lacks enough detail to find underlying problem (docker Sirupsen/logrus mismatch) #28489

Open
jayschwa opened this issue Oct 30, 2018 · 25 comments

Comments

Projects
None yet
@jayschwa
Copy link
Contributor

commented Oct 30, 2018

What version of Go are you using (go version)?

Go 1.11.1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Linux, amd64

Problem

Our go.mod has the following dependency, along with many other dependencies:

github.com/sirupsen/logrus v1.1.1

When running go mod vendor, I get the following error:

go: github.com/Sirupsen/logrus@v1.1.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"

I suspect that some deeper indirect dependency is also using logrus, but with the incorrect (old) capital S. However, the error message isn't giving me enough detail to track it down. It would be nice if the tool gave me a list of the modules or packages that are using the conflicting import path.

Example

https://github.com/jayschwa/go28489

@agnivade

This comment has been minimized.

Copy link
Member

commented Oct 31, 2018

Have you tried go mod why ?

@myitcv

This comment has been minimized.

Copy link
Member

commented Oct 31, 2018

The underlying cause here is almost certainly the same as #26208, although per the description, not obviously so. Not least because time (and releases) have passed since that issue was reported.

There are a number of issues/factors at play here:

  • github.com/docker/docker does not follow semver so we need to do a separate go get for the latest version of that
  • github.com/docker/docker@v17.05.0-ce (for example) is not a module
  • github.com/docker/docker@v17.05.0-ce's dependencies are managed via vendor.conf and are sufficiently esoteric (dependencies not using semver, etc) that we need to go mod init and "merge" those dependencies into our main module's go.mod
  • github.com/Sirupsen/logrus vs github.com/sirupsen/logrus is a long standing issue. GitHub redirects github.com/Sirupsen/logrus to github.com/sirupsen/logrus
  • As of v1.1.0, github.com/sirupsen/logrus/github.com/Sirupsen/logrus is now a module, github.com/sirupsen/logrus. Hence in module mode:
$ go get github.com/Sirupsen/logrus
go: github.com/Sirupsen/logrus@v1.1.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
  • github.com/Sirupsen/logrus is used as such consistently in github.com/docker/docker@v17.05.0-ce and all its dependencies. This means that any use of github.com/sirupsen/logrus in module mode is a problem (case-insensitive import collision: "github.com/sirupsen/logrus" and "github.com/Sirupsen/logrus") but also that we need to use github.com/Sirupsen/logrus (sic) prior to v1.1.0 when it became the module github.com/sirupsen/logrus

Hence we end up needing to do something like the following (notice we get the pre v1.1.0 version of
github.com/Sirupsen/logrus/github.com/sirupsen/logrus via the gomodmerge):

https://gist.github.com/myitcv/f7270ab81ab45aa286f496264f034b56

To my mind, the v1.1.0 module-enabled release of github.com/Sirupsen/logrus/github.com/sirupsen/logrus is a breaking change; because an import path of github.com/Sirupsen/logrus now no longer works when in module mode (the irony). Hence I think the module release of github.com/Sirupsen/logrus/github.com/sirupsen/logrus should in fact have been a v2 release.

So one potential way forward here is for github.com/Sirupsen/logrus/github.com/sirupsen/logrus to create a new v1 release that reverts the conversion to Go modules and instead release github.com/sirupsen/logrus/v2 as a module.

cc @bcmills and @rsc for any additional thoughts here.

@jayschwa

This comment has been minimized.

Copy link
Contributor Author

commented Oct 31, 2018

Have you tried go mod why?

It produces the same error with no additional info.

github.com/docker/docker does not follow semver so we need to do a separate go get for the latest version of that

I should note that I am using a very recent pseudo-version of docker in my example, and the problem still occurs: github.com/docker/docker v0.7.3-0.20181027010111-b8e87cfdad8d

Ideally, docker should start using go.mod and tag its repo correctly, but in the meantime, it would be nice to get more details from the error message so these sorts of problems are easier to diagnose. Nothing in the error message indicates that the conflict is coming from docker or one of its dependencies.

@myitcv

This comment has been minimized.

Copy link
Member

commented Oct 31, 2018

I should note that I am using a very recent pseudo-version of docker in my example, and the problem still occurs: github.com/docker/docker v0.7.3-0.20181027010111-b8e87cfdad8d

Indeed, but the issues I listed above are independent of the version of Docker (unless, at some point recently or in the future, Docker and all its dependencies switch to using github.com/sirupsen/logrus as the import path).

Ideally, docker should start using go.mod and tag its repo correctly,

This will help certainly with the "dance" required to get sensible versions of Docker's dependencies, yes.

But that leaves the github.com/Sirupsen/logrus/github.com/sirupsen/logrus issue:

it would be nice to get more details from the error message so these sorts of problems are easier to diagnose

I think this is a particularly subtle edge case. Whilst in general I absolutely agree with the sentiment, I don't know if there's much we can do to generally improve what is likely to be an extremely unlikely confluence of events. But that's just my two cents.

@bcmills

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

To my mind, the v1.1.0 module-enabled release of github.com/Sirupsen/logrus / github.com/sirupsen/logrus is a breaking change; because an import path of github.com/Sirupsen/logrus now no longer works when in module mode (the irony).

If you believe that github.com/Sirupsen/logrus was ever a valid import path, then that is correct. 🙂

(At some point we should consider whether we can make import paths case-insensitive, but that's definitely not happening in 1.12.)

A fix for #26904 (comment) would allow you to at least replace github.com/Sirupsen/logrus => github.com/sirupsen/logrus, but that opens a whole other can of case-sensitivity worms.

@bcmills

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

So one potential way forward here is for github.com/Sirupsen/logrus/github.com/sirupsen/logrus to create a new v1 release that reverts the conversion to Go modules and instead release github.com/sirupsen/logrus/v2 as a module.

I think it's actually fine to keep the change in v1. Code that uses the “wrong” import path for v1 in is already broken in general anyway: if you try to import the same module with two different cases, you'll get a case-sensitive import collision (#26208 (comment)), so if we allow modules that depend on each path individually build successfully, they still can't be combined: they aren't really “modular”.

@maguro

This comment has been minimized.

Copy link

commented Dec 9, 2018

It looks like people are working hard to get this fixed correctly. Until then, is there a TL;DR workaround I can use?

@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 10, 2018

@maguro, a TL;DR workaround to what? The issue reported here is a less-than-helpful error message, but presumably the thing you want to work around is some error itself⸮

@keyolk

This comment has been minimized.

Copy link

commented Dec 11, 2018

@maguro
in my case I rename all "Sirupsen" to "sirupsen" from my dependencies

@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 21, 2018

I think the problem here is that we're emitting the error in the wrong place.

If we fetch a module with some casing of its path, and the module declaration indicates a different casing, that's not a defect in the module — it's a defect in whatever package is trying to import from that module, and should be diagnosed within that package.

@thepudds

This comment has been minimized.

Copy link

commented Dec 22, 2018

@bcmills

If we fetch a module with some casing of its path, and the module declaration indicates a different casing, that's not a defect in the module — it's a defect in whatever package is trying to import from that module, and should be diagnosed within that package.

I think a better error would go a long way.

I think this is the same issue with docker/libcompose, where go mod tidy and go mod why were both erroring out prior to saying where the conflict came from. In other words, this error (from using docker/libcompose) doesn't tell you the real culprit:

go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"

It seems like there is already a reproduction here, but here is a simple one showing go mod tidy fail for a libcompose client:

mkdir -p /tmp/scratchpad/docker-simple-libcompose-client
cd /tmp/scratchpad/docker-simple-libcompose-client

go1.12beta1 mod init github.com/me/client

cat <<EOF > main.go

package main

import (
        "golang.org/x/net/context"

        "github.com/docker/libcompose/docker"
        "github.com/docker/libcompose/docker/ctx"
        "github.com/docker/libcompose/project"
        "github.com/docker/libcompose/project/options"
)

func main() {
        project, err := docker.NewProject(&ctx.Context{Context: project.Context{ComposeFiles: []string{"docker-compose.yml"},
                        ProjectName:  "my-compose",}, }, nil)

        project.Up(context.Background(), options.Up{})
}
EOF

# run our `go mod tidy` test, which should fail, but error is not as helpful as it could be.

$ go1.12beta1 mod tidy
go: finding github.com/docker/libcompose/project/options latest
go: finding github.com/docker/libcompose/docker/ctx latest
go: finding github.com/docker/libcompose/docker latest
go: finding github.com/docker/libcompose/project latest
go: finding golang.org/x/net/context latest
go: finding github.com/docker/libcompose v0.4.0
go: finding golang.org/x/net latest
go: downloading golang.org/x/net v0.0.0-20181220203305-927f97764cc3
go: downloading github.com/docker/libcompose v0.4.0
go: extracting golang.org/x/net v0.0.0-20181220203305-927f97764cc3
go: extracting github.com/docker/libcompose v0.4.0
go: finding github.com/docker/go-connections/nat latest
go: finding gopkg.in/yaml.v2 v2.2.2
go: finding github.com/docker/docker/api/types latest
go: finding github.com/docker/docker/pkg/jsonmessage latest
go: finding github.com/docker/docker/client latest
go: finding github.com/docker/docker/registry latest
go: finding github.com/docker/docker/reference latest
go: finding github.com/docker/docker/cliconfig/configfile latest
go: finding github.com/docker/docker/pkg/streamformatter latest
go: finding github.com/stretchr/testify/assert latest
go: finding github.com/docker/go-connections v0.4.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/docker/go-connections v0.4.0
go: extracting gopkg.in/yaml.v2 v2.2.2
go: finding github.com/docker/docker/cliconfig latest
go: finding github.com/stretchr/testify v1.2.2
go: extracting github.com/docker/go-connections v0.4.0
go: downloading github.com/stretchr/testify v1.2.2
go: finding github.com/docker/docker/pkg/stdcopy latest
go: extracting github.com/stretchr/testify v1.2.2
go: finding github.com/docker/docker/builder/dockerignore latest
go: finding github.com/docker/docker/pkg latest
go: finding github.com/docker/docker v1.13.1
go: finding github.com/docker/docker/api latest
go: finding github.com/docker/docker/builder latest
go: downloading github.com/docker/docker v1.13.1
go: extracting github.com/docker/docker v1.13.1
go: finding github.com/docker/go-connections/tlsconfig latest
go: finding github.com/docker/docker/api/types/filters latest
go: finding github.com/docker/docker/pkg/archive latest
go: finding github.com/docker/docker/pkg/term latest
go: finding github.com/docker/docker/api/types/strslice latest
go: finding github.com/docker/go-connections/sockets latest
go: finding github.com/docker/docker/pkg/fileutils latest
go: finding github.com/docker/docker/pkg/promise latest
go: finding github.com/docker/docker/runconfig/opts latest
go: finding github.com/docker/docker/pkg/homedir latest
go: finding github.com/docker/docker/api/types/container latest
go: finding github.com/xeipuuv/gojsonschema latest
go: finding github.com/flynn/go-shlex latest
go: finding github.com/docker/docker/runconfig latest
go: finding github.com/docker/docker/pkg/urlutil latest
go: finding github.com/docker/docker/pkg/progress latest
go: finding github.com/docker/go-units v0.3.3
go: downloading github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: downloading github.com/xeipuuv/gojsonschema v0.0.0-20181112162635-ac52e6811b56
go: finding github.com/docker/docker/api/types/network latest
go: extracting github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
go: extracting github.com/xeipuuv/gojsonschema v0.0.0-20181112162635-ac52e6811b56
go: downloading github.com/docker/go-units v0.3.3
go: finding github.com/Sirupsen/logrus v1.2.0
go: finding github.com/docker/docker/api/types/events latest
go: finding github.com/docker/docker/api/types/registry latest
go: extracting github.com/docker/go-units v0.3.3
go: downloading github.com/Sirupsen/logrus v1.2.0
go: extracting github.com/Sirupsen/logrus v1.2.0
go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
go: error loading module requirements

CC @vito-c

@thepudds

This comment has been minimized.

Copy link

commented Jan 16, 2019

In #28489 (comment) above, @myitcv has a very nice explanation of the somewhat common issue one can hit with docker with a uppercase/lowercase mismatch between github.com/Sirupsen/logrus vs. github.com/sirupsen/logrus, and he also has a worked example there.

The starting point for that worked example above happens to be a docker version from ~18 months ago (in part because of the way docker has been tagging their releases and moving repos around, as touched upon above), and the worked example above ends up with github.com/Sirupsen/logrus (the uppercase version) because that worked example shows finding a consistent set of older versions.

Here is a worked example using a more recent docker version, and ends up with github.com/sirupsen/logrus (the lowercase version), using a simple docker/libcompose client that initially fails with:

go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: 
unexpected module path "github.com/sirupsen/logrus"

The following FAQ covers the technique in a bit more detail:

"FAQ: I have a problem with a complex dependency that has not opted in to modules. Can I use information from its current dependency manager?"

First, we clone libcompose in order to run go mod init to convert from the vendor.conf from libcompose to a temporary go.mod, and then use those results to populate the require directives for our client. Here is a worked example of doing that (with a 2018 version of libcompose and docker):

Worked Example: Steps and Results
# ----------------------------------------------------------------------------------------------
# 1. Create a simple libcompose client, which initially fails 'go mod tidy' with error:
#        go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: 
#        unexpected module path "github.com/sirupsen/logrus"
# For why this is expected to fail, see discussion in: https://github.com/golang/go/issues/28489
# ----------------------------------------------------------------------------------------------

mkdir -p /tmp/scratchpad/docker-simple-libcompose-client
cd /tmp/scratchpad/docker-simple-libcompose-client

go mod init github.com/my/client

cat <<EOF > main.go

package main

import (
        "golang.org/x/net/context"

        "github.com/docker/libcompose/docker"
        "github.com/docker/libcompose/docker/ctx"
        "github.com/docker/libcompose/project"
        "github.com/docker/libcompose/project/options"
)

func main() {
        project, _ := docker.NewProject(&ctx.Context{Context: project.Context{ComposeFiles: []string{"docker-compose.yml"},
                        ProjectName:  "my-compose",}, }, nil)

        project.Up(context.Background(), options.Up{})
}
EOF

# run our `go mod tidy` test, which we expect to fail (due to Sirupsen/logrus vs. sirupsen/logrus mismatch)
go mod tidy

# Outputs:
# go: finding golang.org/x/net/context latest
# go: finding github.com/docker/libcompose/docker/ctx latest
# ...
# go: downloading github.com/docker/go-units v0.3.3
# go: github.com/Sirupsen/logrus@v1.2.0: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
# go: error loading module requirements


# -------------------------------------------------------------------------------------------
# 2. Clone libcompose in order to run `go mod init` to convert from `vendor.conf` 
#     to a temporary `go.mod`.
# For why this is a useful technique, see FAQ https://github.com/golang/go/wiki/Modules#i-have-a-problem-with-a-complex-dependency-that-has-not-opted-in-to-modules-can-i-use-information-from-its-current-dependency-manager
# -------------------------------------------------------------------------------------------


git clone https://github.com/docker/libcompose.git /tmp/scratchpad/libcompose-clone
cd /tmp/scratchpad/libcompose-clone

# Latest commit is 213509a from Oct 19 2018 (as of Dec 24 2018)
git checkout -q 213509a

# Initialize a temporary 'go.mod' file here, which will convert the version information from libcompose's vendor.conf
go mod init

# outputs:
# go: creating new go.mod: module github.com/docker/libcompose
# go: copying requirements from vendor.conf

cat go.mod

# -------------------------------------------------------------------------------------------
# 3. Switch back to our libcompose client, and specify the same version of libcompose as we just cloned.
# -------------------------------------------------------------------------------------------

cd /tmp/scratchpad/docker-simple-libcompose-client/
go get github.com/docker/libcompose@213509a 

# -------------------------------------------------------------------------------------------
# 4. Move the requires from the temporary go.mod in the libcompose clone 
#     to our "real" go.mod in our actual libcompose client (deleting any duplicates first from our client go.mod).  
#
#     We could do this manually, but here we will do it by just appending the full set of requires (using 'tail' to skip the 'module' line), 
#     and then let 'go mod tidy' and friends deal with formatting.

tail -n +2 /tmp/scratchpad/libcompose-clone/go.mod >> /tmp/scratchpad/docker-simple-libcompose-client/go.mod

# -------------------------------------------------------------------------------------------
# 5. 'go mod tidy' and 'go build' now succeed for our simple libcompose client.
# -------------------------------------------------------------------------------------------

cd /tmp/scratchpad/docker-simple-libcompose-client/
go mod tidy
go build

# Finally, show the resulting working 'go.mod' file. The first require is from our 'go get', and 
# the remaining require versions were the ones we converted from .../libcompose-clone/vender.conf via 'go mod init'
# and then updated via 'go mod tidy'.

$ cat /tmp/scratchpad/docker-simple-libcompose-client/go.mod

module github.com/my/client

require github.com/docker/libcompose v0.4.1-0.20181019154650-213509acef0f

require (
        github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 // indirect
        github.com/Microsoft/go-winio v0.3.8 // indirect
        github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
        github.com/containerd/continuity v0.0.0-20170913164642-35d55c5e8dd2 // indirect
        github.com/davecgh/go-spew v0.0.0-20160907170601-6d212800a42e // indirect
        github.com/docker/cli v0.0.0-20171020201719-3352c0e137e8 // indirect
        github.com/docker/distribution v0.0.0-20170726174610-edc3ab29cdff // indirect
        github.com/docker/docker v0.0.0-20180221164450-0ede01237c9a // indirect
        github.com/docker/docker-credential-helpers v0.0.0-20170816090621-3c90bd29a46b // indirect
        github.com/docker/go-connections v0.3.0 // indirect
        github.com/docker/go-units v0.0.0-20170127094116-9e638d38cf69 // indirect
        github.com/docker/libtrust v0.0.0-20150526203908-9cbd2a1374f4 // indirect
        github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 // indirect
        github.com/gogo/protobuf v0.0.0-20170307180453-100ba4e88506 // indirect
        github.com/google/go-cmp v0.2.0 // indirect
        github.com/gorilla/context v0.0.0-20160226214623-1ea25387ff6f // indirect
        github.com/gorilla/mux v0.0.0-20160317213430-0eeaf8392f5b // indirect
        github.com/gotestyourself/gotestyourself v2.2.0+incompatible // indirect
        github.com/onsi/ginkgo v1.7.0 // indirect
        github.com/onsi/gomega v1.4.3 // indirect
        github.com/opencontainers/go-digest v0.0.0-20170106003457-a6d0ee40d420 // indirect
        github.com/opencontainers/image-spec v0.0.0-20170515205857-f03dbe35d449 // indirect
        github.com/opencontainers/runc v0.0.0-20161109192122-51371867a01c // indirect
        github.com/opencontainers/selinux v1.0.0 // indirect
        github.com/pkg/errors v0.0.0-20161002052512-839d9e913e06 // indirect
        github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0 // indirect
        github.com/sirupsen/logrus v1.0.3 // indirect
        github.com/stretchr/testify v0.0.0-20151202184152-a1f97990ddc1 // indirect
        github.com/xeipuuv/gojsonpointer v0.0.0-20151027082146-e0fe6f683076 // indirect
        github.com/xeipuuv/gojsonreference v0.0.0-20150808065054-e02fc20de94c // indirect
        github.com/xeipuuv/gojsonschema v0.0.0-20160323030313-93e72a773fad // indirect
        golang.org/x/crypto v0.0.0-20160408163010-3fbbcd23f1cb // indirect
        golang.org/x/net v0.0.0-20180906233101-161cd47e91fd
        golang.org/x/time v0.0.0-20160202183820-a4bde1265759 // indirect
        gopkg.in/airbrake/gobrake.v2 v2.0.9 // indirect
        gopkg.in/check.v1 v1.0.0-20150729080431-11d3bc7aa68e // indirect
        gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2 // indirect
        gotest.tools v2.2.0+incompatible // indirect
)

@thepudds thepudds changed the title cmd/go: "unexpected module path" error lacks enough detail to find underlying problem cmd/go: "unexpected module path" error lacks enough detail to find underlying problem (docker Sirupsen/logrus mismatch) Jan 16, 2019

@thepudds

This comment has been minimized.

Copy link

commented Jan 16, 2019

There is some discussion above about making the error friendlier, including to output the parent of the improperly specified dependency (in order to reduce need to then hunt with go mod graph, or similar).

In addition to that, the current error likely could do a better job describing the issue.

As an example, the repo history of docker is a bit confusing, but if you look at github/moby/moby for example, there is a deliberately added "import comment" to enforce people always use a single, canonical import starting with github/docker/docker (and not moby/moby). Sample example here:
https://github.com/moby/moby/blame/master/client/client.go#L42

Here is current / pre-modules error you get if you try to go get github.com/moby/moby/client:

$ mkdir -p $GOPATH/example.com/scratchpad/mobytest
$ cd $GOPATH/example.com/scratchpad/mobytest
$ go get github.com/moby/moby/client

package github.com/moby/moby/client: code in directory 
...\go\src\github.com\moby\moby\client expects import "github.com/docker/docker/client"

(That's the "old" error triggered by the enforcement of the import comment).

Here's a "simulated" error message IF they had enforced the same thing with a go.mod:

go: github.com/moby/moby@latest: parsing go.mod: 
unexpected module path "github.com/docker/docker"

I think the old error is simpler to understand, including the fact that it states what the code "expects" (probably making it simpler to understand than something that is just "unexpected" in the newer error).

@gopherbot

This comment has been minimized.

Copy link

commented Jan 18, 2019

Change https://golang.org/cl/158477 mentions this issue: cmd/go: update unexpected module import error to be more actionable

@theckman

This comment has been minimized.

Copy link
Contributor

commented Jan 18, 2019

I just raised the above CL to hopefully improve this error message. I'm looking to become one of the maintainers of logrus, so improving this is of personal interest to me because of the repo migration we're going to try and do to accomplish this[1].

[1] sirupsen/logrus#799 (comment)

@theckman

This comment has been minimized.

Copy link
Contributor

commented Jan 18, 2019

@bcmills to your earlier comment about what the real issue may be[1], I don't think my CL is addressing that as the fix. It's only focused on the value of the error string, and not the location it's being emitted.

I marked my CL as a fix for this issue, only because to me it felt like reworking where the error was emitted was maybe a separate (larger) task. This could be a way to improve the error for 1.12, and we could file a separate issue (or rename this one) to speak of a more complete fix.

[1] #28489 (comment)

@jayschwa

This comment has been minimized.

Copy link
Contributor Author

commented Jan 18, 2019

I marked my CL as a fix for this issue

@theckman, I think your CL makes the error message clearer, and I hope it gets merged. However, discovering where the mismatch is happening in the dependency graph was kind of the crux of this issue when I created it. I don't think it would be accurate to call the CL a fix in its current form. It's also not clear to me what is gained by re-purposing this issue to fit the CL, closing it, and then creating a new issue for the same thing.

@theckman

This comment has been minimized.

Copy link
Contributor

commented Jan 18, 2019

@jayschwa ta; adjusted commit to align with that.

goFrendiAsgard added a commit to goFrendiAsgard/go-skeleton that referenced this issue Feb 7, 2019

@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 12, 2019

mbyczkowski added a commit to square/keysync that referenced this issue Feb 27, 2019

Update Sirupsen/logrus path to match current name (sirupsen/logrus)
This should clean things up and mitigate a potential future issue when upgrading to
Go Modules (as seen in golang/go#28489).
@thepudds

This comment has been minimized.

Copy link

commented Mar 15, 2019

@jayconrod Is the thinking that your WIP https://golang.org/cl/166984 and #30661 would make the error message here much more actionable by making it easy to see what piece of code is importing via the "wrong" import path (e.g., who is importing github.com/Sirupsen/logrus when the logrus go.mod declares itself to be github.com/sirupsen/logrus, or who is misusing github.com/golang/lint vs. golang.org/x/lint, etc.)?

Side note: when you have something functional, I suspect your colleagues would be very interested in trying it out and giving you early feedback as they work through these types of issues. (CC @dmitshur @broady @jadekler).

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Mar 15, 2019

CL 166984 should help with build and list commands. It produces output like this (from the test):

go: example.com/badchain/a@v1.0.0
	-> example.com/badchain/b@v1.0.0
	-> example.com/badchain/c@v1.0.0: parsing go.mod: unexpected module path "example.com/badchain/wrong"

That at least tells you which module has the unexpected import.

Unfortunately, this doesn't produce intelligible output for go get -u because we don't preserve the relationships between modules when upgrading the build list. More refactoring needed there.

@directionless

This comment has been minimized.

Copy link

commented Apr 2, 2019

I spent much of this morning being confused by this. I was eventually untangled by #modules on slack, but I think my experience represents an underlying problem in the complexity and required knowledge to use modules. (And huge shout outs to @thepudds and @johanbrandhorst for their patience and help)

I maintain https://github.com/kolide/launcher/ it's a mid sized project that has some dependancies that ultimately require [sS]irupsen/logrus. Parts of the dependency chain hadn't been updated in a while, some some of the underlying case stuff had been resolved, but not in my go.mod. I also have several internal projects that use launcher. https://github.com/directionless/go-issue-28489 is a fine example.

When I went to update launcher, I used go get -u github.com/kolide/launcher. This resulted in an error about

go: github.com/Sirupsen/logrus@v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"

But I could not tell where that dependency came from. From that example repo, it's not in the go mod why output, and the go mod graph output seems misleading WRT indirect dependancies.

After a fair bit of frustrating, and coaxing from slack I discovered the following:

  1. From launcher, go mod why will show the dependency chain
  2. launcher has a dependency on github.com/docker/distribution, which was so old it had the TitleCase
  3. Using go get github.com/kolide/launcher would have avoided all of this.

I feel like there are several issues.

The most obvious one is that I didn't need the -u. I can upgrade my dependency with go get github.com/kolide/launcher. I'm not sure why I internalized that -u is needed. Probably because I think of u for upgrade. Possible akin to #26581 there's a big footprint of tooling to learn.

The second one, is that if I'm running a command that tells me about an issue with a module, I expect to be able to understand why that module is in play. In retrospect, it's obviously because of the -u, but I didn't realize that. If the error message could explicitly mention that it was being upgraded because of -u that would be a big hint.

The third one, is that unexpected module path isn't really clear. I did notice the case difference, but it took some googling before I realized it was significant. And my initial attempts at fixing it with go get or editing the go.mod file didn't help.

For general searching, here's a somewhat simulated transcript:

$ go get -u github.com/kolide/launcher
go: finding github.com/kolide/launcher latest
go: finding github.com/hailocab/go-hostpool latest
go: finding github.com/bmizerany/assert latest
go: finding github.com/bitly/go-hostpool latest
go: finding github.com/Shopify/logrus-bugsnag latest
go: finding github.com/kolide/osquery-go latest
go: finding github.com/kolide/updater latest
go: finding github.com/jinzhu/inflection latest
go: finding github.com/mixer/clock latest
go: finding github.com/kolide/kit latest
go: finding golang.org/x/sync latest
go: finding golang.org/x/time latest
go: finding golang.org/x/image latest
go: finding golang.org/x/sys latest
go: finding github.com/alexkohler/nakedret latest
go: finding github.com/WatchBeam/clock latest
go: finding github.com/cloudflare/cfssl latest
go: finding github.com/kr/logfmt latest
go: finding github.com/agl/ed25519 latest
go: github.com/Sirupsen/logrus@v1.4.1: parsing go.mod: unexpected module path "github.com/sirupsen/logrus"
go: finding github.com/serenize/snaker latest
go: finding github.com/groob/plist latest
go: finding golang.org/x/net latest
go get: error loading module requirements

# where is this coming from?
$ go mod why github.com/Sirupsen/logrus
# github.com/Sirupsen/logrus
(main module does not need package github.com/Sirupsen/logrus)

# where is this coming from?
$ go mod why github.com/sirupsen/logrus
# github.com/sirupsen/logrus
(main module does not need package github.com/sirupsen/logrus)

# WTF? Why is this problem?
# get slack help

$ go get github.com/kolide/launcher
go: finding github.com/kolide/launcher latest

(Along with an associated go get github.com/docker/distribution in my launcher repo to fix the underlying dependency)

directionless added a commit to kolide/launcher that referenced this issue Apr 2, 2019

Update several mods (#460)
In response to golang/go#28489 (comment)
We need to update `go get -u github.com/docker/distribution`

In response to golang/go#30831 I started trying to track down and upgrade things. I don't think I got them all.
@directionless

This comment has been minimized.

Copy link

commented Apr 2, 2019

Related, as I work through the golang/lint rename (#30831) I feel similarly.

I want to be able to run go get -u and have it update stuff. But, that results in complaints about lint. So I'm chasing down individual dependancies.

@Liamsi

This comment has been minimized.

Copy link

commented May 8, 2019

Agree that the error message should be more clear and tell where it originates from. In my case, I get:

go: github.com/dedis/protobuf@v1.0.6: parsing go.mod: unexpected module path "go.dedis.ch/protobuf"

But when I do go mod graph, I get:

Click for output

github.com/tendermint/tendermint github.com/beorn7/perks@v0.0.0-20180321164747-3a771d992973
github.com/tendermint/tendermint github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d
github.com/tendermint/tendermint github.com/btcsuite/btcutil@v0.0.0-20180706230648-ab6388e0c60a
github.com/tendermint/tendermint github.com/davecgh/go-spew@v1.1.1
github.com/tendermint/tendermint github.com/etcd-io/bbolt@v1.3.2
github.com/tendermint/tendermint github.com/fortytw2/leaktest@v1.2.0
github.com/tendermint/tendermint github.com/fsnotify/fsnotify@v1.4.7
github.com/tendermint/tendermint github.com/go-kit/kit@v0.6.0
github.com/tendermint/tendermint github.com/go-logfmt/logfmt@v0.3.0
github.com/tendermint/tendermint github.com/go-stack/stack@v1.8.0
github.com/tendermint/tendermint github.com/gogo/protobuf@v1.2.1
github.com/tendermint/tendermint github.com/golang/protobuf@v1.3.0
github.com/tendermint/tendermint github.com/golang/snappy@v0.0.0-20180518054509-2e65f85255db
github.com/tendermint/tendermint github.com/gorilla/websocket@v1.2.0
github.com/tendermint/tendermint github.com/hashicorp/hcl@v1.0.0
github.com/tendermint/tendermint github.com/inconshreveable/mousetrap@v1.0.0
github.com/tendermint/tendermint github.com/jmhodges/levigo@v1.0.0
github.com/tendermint/tendermint github.com/kr/logfmt@v0.0.0-20140226030751-b84e30acd515
github.com/tendermint/tendermint github.com/magiconair/properties@v1.8.0
github.com/tendermint/tendermint github.com/matttproud/golang_protobuf_extensions@v1.0.1
github.com/tendermint/tendermint github.com/mitchellh/mapstructure@v1.1.2
github.com/tendermint/tendermint github.com/pelletier/go-toml@v1.2.0
github.com/tendermint/tendermint github.com/pkg/errors@v0.8.0
github.com/tendermint/tendermint github.com/pmezard/go-difflib@v1.0.0
github.com/tendermint/tendermint github.com/prometheus/client_golang@v0.9.1
github.com/tendermint/tendermint github.com/prometheus/client_model@v0.0.0-20180712105110-5c3871d89910
github.com/tendermint/tendermint github.com/prometheus/common@v0.0.0-20181020173914-7e9e6cabbd39
github.com/tendermint/tendermint github.com/prometheus/procfs@v0.0.0-20181005140218-185b4288413d
github.com/tendermint/tendermint github.com/rcrowley/go-metrics@v0.0.0-20180503174638-e2704e165165
github.com/tendermint/tendermint github.com/rs/cors@v1.6.0
github.com/tendermint/tendermint github.com/spf13/afero@v1.1.2
github.com/tendermint/tendermint github.com/spf13/cast@v1.3.0
github.com/tendermint/tendermint github.com/spf13/cobra@v0.0.1
github.com/tendermint/tendermint github.com/spf13/jwalterweatherman@v1.0.0
github.com/tendermint/tendermint github.com/spf13/pflag@v1.0.3
github.com/tendermint/tendermint github.com/spf13/viper@v1.0.0
github.com/tendermint/tendermint github.com/stretchr/testify@v1.2.2
github.com/tendermint/tendermint github.com/syndtr/goleveldb@v0.0.0-20181012014443-6b91fda63f2e
github.com/tendermint/tendermint github.com/tendermint/go-amino@v0.14.1
github.com/tendermint/tendermint golang.org/x/crypto@v0.0.0-20190228161510-8dd112bcdc25
github.com/tendermint/tendermint golang.org/x/net@v0.0.0-20180906233101-161cd47e91fd
github.com/tendermint/tendermint golang.org/x/sys@v0.0.0-20190215142949-d0b11bdaac8a
github.com/tendermint/tendermint golang.org/x/text@v0.3.0
github.com/tendermint/tendermint google.golang.org/genproto@v0.0.0-20181029155118-b69ba1387ce2
github.com/tendermint/tendermint google.golang.org/grpc@v1.13.0
github.com/tendermint/tendermint gopkg.in/yaml.v2@v2.2.1
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/aead/siphash@v1.0.1
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/btclog@v0.0.0-20170628155309-84c8d2346e9f
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/btcutil@v0.0.0-20180706230648-ab6388e0c60a
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/go-socks@v0.0.0-20170105172521-4720035b7bfd
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/goleveldb@v0.0.0-20160330041536-7834afc9e8cd
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/snappy-go@v0.0.0-20151229074030-0bdef8d06723
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/websocket@v0.0.0-20150119174127-31079b680792
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/btcsuite/winsvc@v1.0.0
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/davecgh/go-spew@v0.0.0-20171005155431-ecdeabc65495
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/jessevdk/go-flags@v0.0.0-20141203071132-1679536dcc89
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/jrick/logrotate@v1.0.0
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/kkdai/bstream@v0.0.0-20161212061736-f391b8402d23
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/onsi/ginkgo@v1.7.0
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d github.com/onsi/gomega@v1.4.3
github.com/btcsuite/btcd@v0.0.0-20190115013929-ed77733ec07d golang.org/x/crypto@v0.0.0-20170930174604-9419663f5a44
github.com/hashicorp/hcl@v1.0.0 github.com/davecgh/go-spew@v1.1.1
github.com/onsi/gomega@v1.4.3 github.com/fsnotify/fsnotify@v1.4.7
github.com/onsi/gomega@v1.4.3 github.com/golang/protobuf@v1.2.0
github.com/onsi/gomega@v1.4.3 github.com/hpcloud/tail@v1.0.0
github.com/onsi/gomega@v1.4.3 github.com/onsi/ginkgo@v1.6.0
github.com/onsi/gomega@v1.4.3 golang.org/x/net@v0.0.0-20180906233101-161cd47e91fd
github.com/onsi/gomega@v1.4.3 golang.org/x/sync@v0.0.0-20180314180146-1d60e4601c6f
github.com/onsi/gomega@v1.4.3 golang.org/x/sys@v0.0.0-20180909124046-d0be0721c37e
github.com/onsi/gomega@v1.4.3 golang.org/x/text@v0.3.0
github.com/onsi/gomega@v1.4.3 gopkg.in/fsnotify.v1@v1.4.7
github.com/onsi/gomega@v1.4.3 gopkg.in/tomb.v1@v1.0.0-20141024135613-dd632973f1e7
github.com/onsi/gomega@v1.4.3 gopkg.in/yaml.v2@v2.2.1
github.com/spf13/cast@v1.3.0 github.com/davecgh/go-spew@v1.1.1
github.com/spf13/cast@v1.3.0 github.com/pmezard/go-difflib@v1.0.0
github.com/spf13/cast@v1.3.0 github.com/stretchr/testify@v1.2.2
github.com/gogo/protobuf@v1.2.1 github.com/kisielk/errcheck@v1.1.0
golang.org/x/crypto@v0.0.0-20190228161510-8dd112bcdc25 golang.org/x/sys@v0.0.0-20190215142949-d0b11bdaac8a
gopkg.in/yaml.v2@v2.2.1 gopkg.in/check.v1@v0.0.0-20161208181325-20d25e280405
github.com/golang/protobuf@v1.3.0 golang.org/x/net@v0.0.0-20180906233101-161cd47e91fd
github.com/golang/protobuf@v1.3.0 golang.org/x/sync@v0.0.0-20180314180146-1d60e4601c6f
github.com/golang/protobuf@v1.3.0 google.golang.org/genproto@v0.0.0-20180831171423-11092d34479b
github.com/kisielk/errcheck@v1.1.0 github.com/kisielk/gotool@v1.0.0
github.com/kisielk/errcheck@v1.1.0 golang.org/x/tools@v0.0.0-20180221164845-07fd8470d635

As you can see, the mentioned module isn't even a dependency... Sure, I played around with that module somewhere else but not in combination with that repo at all. (It doesn't show up on go list -u -m all either). It leaves me with no clue what I can do about the error and why it shows up.
ref: tendermint/tendermint#3613

(go version go1.12.5 darwin/amd64)

@dmitshur

This comment has been minimized.

Copy link
Member

commented May 8, 2019

@Liamsi There has been a change in Go tip to make the output better, see #28489 (comment) above. You can try Go tip via go get golang.org/dl/gotip. (h/t to @FiloSottile for making it!)

@Liamsi

This comment has been minimized.

Copy link

commented May 8, 2019

Awesome, will check that out! Thank you! Update: That change actually resolved the issue I had. Also, thanks for the reminder for gotip.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.