Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename of repository not handled well #34

Open
matthijskooijman opened this Issue Apr 11, 2019 · 13 comments

Comments

Projects
None yet
5 participants
@matthijskooijman
Copy link

commented Apr 11, 2019

This might actually be a bug in go, but I'm reporting it here anyway, since this breaks things here for now.

Recently, this library movied from sourcegraph to github. However, using the old sourcegraph.com name is now broken:

$ go get -d sourcegraph.com/sourcegraph/go-diff
go: sourcegraph.com/sourcegraph/go-diff@v0.5.1: parsing go.mod: unexpected module path "github.com/sourcegraph/go-diff"
go: error loading module requirements

The obvious fix is to just use the new github.com import path, but this is not always trivial. In particular, when go-diff is pulled in by a dependency that has the old url, running go get -u to update dependencies fails with the same error.

I guess that the problem is that go does not gracefully support updating to a new version in this case and updating across a rename (so 0.5.0 -> 0.5.1) needs changing of the import path. I can't see an easy way to fix this on the go side of things, so perhaps the fix here is to have sourcegraph.com not publish the 0.5.1 release (only 0.5.0), though that would also mean that people might never realize they're using an outdated version...

(also, I couldn't actually find the code on sourcegraph.com anymore, http://sourcegraph.com/sourcegraph/go-diff gives a 404, but apparently go get still manages to access the files)

@matthijskooijman

This comment has been minimized.

Copy link
Author

commented Apr 11, 2019

As a workaround, adding this to my go.mod helps:

replace sourcegraph.com/sourcegraph/go-diff v0.5.1 => github.com/sourcegraph/go-diff v0.5.1

However, that only works for this particular version. On any subsequent update, the line must be updated again...

@sqs

This comment has been minimized.

Copy link
Member

commented Apr 11, 2019

Sorry for the trouble with this rename. @dmitshur do you have any thoughts here?

@jadekler

This comment has been minimized.

Copy link

commented Apr 11, 2019

Relevant: golang/go#30831

@matthijskooijman Could you post a GO111MODULE=on go mod graph so that we can see which library is bringing in the old sourcegraph.com/sourcegraph/go-diff?

@oz

This comment has been minimized.

Copy link

commented Apr 12, 2019

@jadekler does this help?

$ go mod graph | grep go-diff@v0.5  
golang.org/x/build@v0.0.0-20190201181641-63986c177d1f sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190111050920-041ab4dc3f9d sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190131192101-cfd41133b82b sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190208213738-77d92a99a537 sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190130193844-d3ddf67ac5c8 sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190124033707-36bd60da6b75 sourcegraph.com/sourcegraph/go-diff@v0.5.0
golang.org/x/build@v0.0.0-20190212192241-098540515740 sourcegraph.com/sourcegraph/go-diff@v0.5.0
sourcegraph.com/sourcegraph/go-diff@v0.5.0 github.com/shurcooL/go-goon@v0.0.0-20170922171312-37c2f522c041
sourcegraph.com/sourcegraph/go-diff@v0.5.0 github.com/shurcooL/go@v0.0.0-20180423040247-9e1955d9fb6e
sourcegraph.com/sourcegraph/go-diff@v0.5.0 github.com/gogo/protobuf@v1.1.1
sourcegraph.com/sourcegraph/go-diff@v0.5.0 sourcegraph.com/sqs/pbtypes@v0.0.0-20180604144634-d3ebe8f20ae4

@oz

This comment has been minimized.

Copy link

commented Apr 12, 2019

FWIW, There is a similar issue with the golang/lint module golang/lint#446

@jadekler

This comment has been minimized.

Copy link

commented Apr 12, 2019

That's useful but unfortunately is not showing me who is pulling in the old version of golang.org/x/build (which itself is pulling in the old sourcegraph.com/sourcegraph/go-diff. Could you post the full go mod graph?

@matthijskooijman

This comment has been minimized.

Copy link
Author

commented Apr 15, 2019

before running the upgrade, it seems I have not dependencies on sourcegraph/go-diff at all (only sergi/go-diff):

$ GO111MODULE=on go mod graph |grep sourcegraph.go-diff
$ GO111MODULE=on go mod graph |grep go-diff@
github.com/gobuffalo/github_flavored_markdown@v1.0.7 github.com/sergi/go-diff@v1.0.0
github.com/gobuffalo/github_flavored_markdown@v1.0.5 github.com/sergi/go-diff@v1.0.0
github.com/gobuffalo/github_flavored_markdown@v1.0.4 github.com/sergi/go-diff@v1.0.0

after running the upgrade (with the workaround from #34 (comment)), it seems that for me, it is also x/build that references the wrong one. Tracing it back, this seems to come from cloud.google.com/go@v0.36.0:

$ GO111MODULE=on go mod graph |grep \ sourcegraph.com.sourcegraph.go-diff
golang.org/x/build@v0.0.0-20190111050920-041ab4dc3f9d sourcegraph.com/sourcegraph/go-diff@v0.5.0
$ GO111MODULE=on go mod graph |grep \ golang.org.x.build
cloud.google.com/go@v0.36.0 golang.org/x/build@v0.0.0-20190111050920-041ab4dc3f9d
$ GO111MODULE=on go mod graph |grep \ cloud.google.com.go@v0.36.0
golang.org/x/tools@v0.0.0-20190219185102-9394956cfdc5 cloud.google.com/go@v0.36.0

However, there is a more recent version available that no longer depends on x/build since version 0.37.3 (Which makes some other changes to (fix issues around lint](golang/go#30833 (comment)) as well). I'm not quite sure why my mod graph still includes 0.36.0 rather than the latest version (since I did run go get -u -m just now). I'm not entirely sure how the graph is built at all (since I also see cloud.google.com/go@v0.31.0 as a dependency on the right side, but never on the left side, so it is not a full graph in the sense that I think it picks one version for each package rather than all of them).

I can trace this a bit further:

$ GO111MODULE=on go mod graph |grep \ cloud.google.com.go@v0.36.0                                                                                       
golang.org/x/tools@v0.0.0-20190219185102-9394956cfdc5 cloud.google.com/go@v0.36.0
$ GO111MODULE=on go mod graph |grep \ golang.org/x/tools@v0.0.0-20190219185102-9394956cfdc5                                                             
github.com/gobuffalo/gogen@v0.0.0-20190219194924-d32a17ad9761 golang.org/x/tools@v0.0.0-20190219185102-9394956cfdc5
$ GO111MODULE=on go mod graph |grep \ github.com/gobuffalo/gogen@v0.0.0-20190219194924-d32a17ad9761                                                     
github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342 github.com/gobuffalo/gogen@v0.0.0-20190219194924-d32a17ad9761
$ GO111MODULE=on go mod graph |grep \ github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342                                                     
github.com/gobuffalo/release@v1.2.5 github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342
github.com/gobuffalo/packr/v2@v2.0.2 github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342
github.com/gobuffalo/gogen@v0.0.0-20190224213239-1c6076128bbc github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342

I'm not sure how to quickly see the golang.org/x/tools source, but the next step in the list is github.com/gobuffalo/gogen which does not seem to have a go.mod file. AFAIU, this means that an implicit one using the latest versions is used and made explicit in my project's go.mod file (AFAIK these are the // indirect lines?). However, go.mod has a newer version of x/tools than in the graph:

$ grep x.tools go.mod
        golang.org/x/tools v0.0.0-20190411180116-681f9ce8ac52 // indirect

I'm still confused. I put up the full go mod graph output (both before and after the upgrade), perhaps anyone else can make more sense of this? https://gist.github.com/matthijskooijman/54fb7e36f578a4e83211537a77920f12

For completeness:

$ go version
go version go1.12.2 linux/amd64

(Also, AFAIU the underlying problem in go is that it looks at older versions of dependencies too closely, which is the subject of golang/go#30831. Until that is fixed, the workaround is to remove problematic versions out of your mod graph completely, which is what we're trying to do there I suppose...)

@dmitshur

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

@matthijskooijman Looks like the latest version of github.com/gobuffalo/buffalo-plugins (v1.14.0) module needs to update its dependencies. Here's the chain of modules being required:

go get: github.com/gobuffalo/buffalo-plugins@v1.9.3 ->
	github.com/gobuffalo/buffalo-plugins@v1.14.0 ->
	github.com/gobuffalo/release@v1.2.5 ->
	github.com/gobuffalo/genny@v0.0.0-20190219203444-c95082806342 ->
	github.com/gobuffalo/gogen@v0.0.0-20190219194924-d32a17ad9761 ->
	golang.org/x/tools@v0.0.0-20190219185102-9394956cfdc5 ->
	cloud.google.com/go@v0.36.0 ->
	golang.org/x/build@v0.0.0-20190111050920-041ab4dc3f9d ->
	sourcegraph.com/sourcegraph/go-diff@v0.5.0 ->
	sourcegraph.com/sourcegraph/go-diff@v0.5.1: parsing go.mod: unexpected module path "github.com/sourcegraph/go-diff"

golang.org/x/build@v0.0.0-20190111050920-041ab4dc3f9d is an old version of x/build, the latest does not have a requirement on sourcegraph.com/sourcegraph/go-diff.

That go get output is from Go tip (1.13 in development).

@dmitshur

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

The obvious fix is to just use the new github.com import path, but this is not always trivial. In particular, when go-diff is pulled in by a dependency that has the old url, running go get -u to update dependencies fails with the same error.

I guess that the problem is that go does not gracefully support updating to a new version in this case and updating across a rename (so 0.5.0 -> 0.5.1) needs changing of the import path. I can't see an easy way to fix this on the go side of things, so perhaps the fix here is to have sourcegraph.com not publish the 0.5.1 release (only 0.5.0),

Agree. However, this may not be easy to achieve.

(also, I couldn't actually find the code on sourcegraph.com anymore, http://sourcegraph.com/sourcegraph/go-diff gives a 404, but apparently go get still manages to access the files)

The go get command uses ?go-get=1 query parameter when resolving vanity import paths, as documented here. https://sourcegraph.com/sourcegraph/go-diff?go-get=1 gives 200 OK.

@matthijskooijman

This comment has been minimized.

Copy link
Author

commented Apr 15, 2019

github.com/gobuffalo/gogen which does not seem to have a go.mod file. AFAIU, this means that an implicit one using the latest versions is used

Ah, just noticed that the latest version of gogen has no go.mod file, but an earlier versions has one (and references an older version of x/tools).

@matthijskooijman Looks like the latest version of github.com/gobuffalo/buffalo-plugins (v1.14.0) module needs to update its dependencies. Here's the chain of modules being required:

Thanks. It indeed seems that the rest of the chain is already updated (or has its go.mod file removed, which should also work I guess). However, I would expect that adding this to my go.mod would then fix things:

replace github.com/gobuffalo/release => github.com/gobuffalo/release v1.4.0

(which is the dependency update that would need to happen in side buffalo-plugins).

However, this does not seem to help, I still get the same error on go get -m -u. Adding the previous workaround still works, but afterwards sourcegraph/go-diff has entirely disappeared from my dep tree. I'm building gotip now to see if I can figure if there is any other problematic path.

@matthijskooijman

This comment has been minimized.

Copy link
Author

commented Apr 15, 2019

Yup, there were a few more paths. I opened gobuffalo/buffalo-plugins#18 and @markbates has responded like lightning and published a fixed version. With that, I can upgrade as expected again (though I did have to manually upgrade buffalo-plugins once before running go get -u -m).

matthijskooijman added a commit to 3devo/DvConnector that referenced this issue Apr 16, 2019

BUILD: Update all dependencies
This updates all (indirect) dependencies to their latest versions. There
is a problem in go (hopefully will be fixed in 1.13).  This problem
causes problems when modules are renamed or moved and manifested with
the go-diff and lint packages.

The problem has been worked around in various packages, but this
requires manually updating some packages before doing a full update.
This commit was generated using:

  go get -u -m github.com/gobuffalo/buffalo-plugins
  go get -u -m
  go mod tidy

See also:
 - golang/go#30831
 - sourcegraph/go-diff#34
 - golang/lint#446
@oz

This comment has been minimized.

Copy link

commented Apr 16, 2019

That's useful but unfortunately is not showing me who is pulling in the old version of golang.org/x/build (which itself is pulling in the old sourcegraph.com/sourcegraph/go-diff. Could you post the full go mod graph?

Sorry for the delayed reply. I've checked the dependency graph that referenced the old version of go-diff. In my case, it is related to the lileio/pubsub module, which depends on an old version of Google's GRPC, which (...) depends on an outdated version of go-diff.

I'm re-evaluating some of the dependencies of my little project, to avoid such a graph. 😵

@matthijskooijman

This comment has been minimized.

Copy link
Author

commented Apr 24, 2019

For anyone looking to solve this problem, @jadekler has written (somewhat extensively) about this issue and how to fix it at https://golang/go/wiki/Resolving-Problems-From-Modified-Module-Path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.