Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow replacement modules to alias other active modules #26904

Open
mwf opened this issue Aug 9, 2018 · 42 comments
Open

cmd/go: allow replacement modules to alias other active modules #26904

mwf opened this issue Aug 9, 2018 · 42 comments
Assignees
Labels
modules NeedsInvestigation
Milestone

Comments

@mwf
Copy link

@mwf mwf commented Aug 9, 2018

What version of Go are you using (go version)?

Go tip:
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes (it is not reproduced with go version go1.11beta2 darwin/amd64)

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ikorolev/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/gopath"
GOPROXY=""
GORACE=""
GOROOT="/Users/ikorolev/.gvm/gos/go1.11beta3"
GOTMPDIR=""
GOTOOLDIR="/Users/ikorolev/.gvm/gos/go1.11beta3/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/vgo-a-user/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/go-build138999780=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Sorry, no standalone reproduction, since issue is connected with repository forking

Assume we have a repository A: https://github.com/mwf/vgo-a with the only feature:

package a

var A = "A"

Than we have a fork1 https://github.com/mwf/vgo-a-fork1, adding a feature B :

package a

var B = "B is a new feature in a-fork1"

Unfortunately fork1 will never be merged to the upstream, just because a author don't like this feature.

It's important to note, that both a and a-fork1 don't have go.mod, they are too conservative for that 😄

Then we got a happy user, using both projects in his repo.
go.mod:

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

main.go

package main

import (
	"fmt"

	"github.com/mwf/vgo-a"
	a_fork "github.com/mwf/vgo-a-fork1"
)

func main() {
	fmt.Printf("A: %q\n", a.A)
	fmt.Printf("B: %q\n", a_fork.B)
}

All just works fine:

$ go run .
A: "A"
B: "B is a new feature in a-fork1"

Here appears fork2 https://github.com/mwf/vgo-a-fork2, forked from fork1, and fixing some bugs both in the upstream and in fork1.

We use the fork2 with replace in our main repo: https://github.com/mwf/vgo-a-user/blob/master/go.mod

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.2.1

replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1

What did you expect to see?

Building this with go1.11beta2 works just fine:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version go1.11beta2 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: finding github.com/mwf/vgo-a v0.1.0
go: finding github.com/mwf/vgo-a-fork1 v0.2.0
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

What did you see instead?

Building with the tip (and beta3) returns an error:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

More comments

I understand that this case is very specific and arguable - this should not ever happen ideally, but we have the real case here:
https://github.com/utrack/clay/blob/master/integration/binding_with_body_and_response/go.mod

There is a little workaround, to define go.mod at fork2 and make a replace upstream -> fork2_with_go.mod, but it's too dirty :)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.3.0 // version with go.mod
replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1 // no go.mod

It works with tip and beta3:

$ go version && go run .
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

@mwf
Copy link
Author

@mwf mwf commented Aug 9, 2018

And this crazy stuff could happen only in non-go.mod repositories, because you can't use both upstream and fork at the same time if they have go.mod inside:

module github.com/mwf/vgo-a-user

require (
    github.com/mwf/vgo-a-mod v0.1.0
    github.com/mwf/vgo-a-mod-fork1 v0.2.0
)
$ go run .
go: finding github.com/mwf/vgo-a-mod-fork1 v0.2.0
go: github.com/mwf/vgo-a-mod-fork1@v0.2.0: parsing go.mod: unexpected module path "github.com/mwf/vgo-a-mod"
go: finding github.com/mwf/vgo-a-mod v0.1.0
go: error loading module requirements

You are forced either to use your fork as replace argument, or to change the module in fork's go.mod and never assume it as a fork again :) And this is for the best 👍

So no one will ever hit such an issue in the brave new world of go modules 😄

@bcmills
Copy link
Member

@bcmills bcmills commented Aug 9, 2018

Ideally, I think the long-term solution will be to treat replacements as rewriting the import paths rather than the source code, to allow for precisely this kind of fork-unification behavior.

(For an example of how this can go wrong otherwise, see the code in #26607.)

@bcmills bcmills added NeedsInvestigation modules labels Aug 9, 2018
@bcmills bcmills added this to the Go1.12 milestone Aug 9, 2018
@mwf
Copy link
Author

@mwf mwf commented Aug 9, 2018

@bcmills you set "Go1.12" milestone for the issue.

Maybe we could at least fix the error string in Go1.11 to eliminate the confusion?

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

@bcmills
Copy link
Member

@bcmills bcmills commented Aug 9, 2018

Oh, yeah, that's an easy fix.

@gopherbot
Copy link

@gopherbot gopherbot commented Aug 9, 2018

Change https://golang.org/cl/128878 mentions this issue: cmd/go/internal/modload: emit correct module in duplication error

@F21
Copy link

@F21 F21 commented Jan 17, 2019

I am also seeing this problem when using go mod why. Since testcontainers/testcontainer-go was renamed to testcontainers/testcontainers-go recently, modules break for me since it is being used by one of my dependencies (not sure how deep).

I added:

replace github.com/testcontainers/testcontainer-go => github.com/testcontainers/testcontainers-go v0.0.0-20190108154635-47c0da630f72

to go.mod in order to get go get -u and go mod tidy to work correctly.

However, if I run go mod why github.com/testcontainers/testcontainers-go, I get:

go: finding github.com/testcontainers/testcontainers-go latest
go: github.com/testcontainers/testcontainers-go@v0.0.0-20190108154635-47c0da630f72 used for two different module paths (github.com/testcontainers/testcontainer-go and github.com/testcontainers/testcontainers-go)

@ianlancetaylor ianlancetaylor added this to the Backlog milestone Apr 19, 2021
@bcmills

This comment has been minimized.

@bcmills bcmills removed this from the Backlog milestone Apr 20, 2021
@bcmills bcmills added this to the Go1.17 milestone Apr 20, 2021
@ianlancetaylor

This comment has been minimized.

@bcmills

This comment has been minimized.

@bcmills bcmills removed this from the Go1.17 milestone Apr 20, 2021
@bcmills bcmills added this to the Go1.18 milestone Apr 20, 2021
@ianlancetaylor

This comment has been minimized.

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 22, 2021

@docmerlin mentioned an interesting use-case for path-rewriting replacements.

Suppose that a module previously existed at a path like github.com/foo/bar, and contained a package like github.com/foo/bar/baz/bag. Then the author moved the package out to its own repo, github.com/foo/bag. A user might want to replace github.com/foo/bar/baz/bag with github.com/foo/bag, even though the former path is not really a module at all.

To make matters worse, if the author of github.com/foo/bar didn't actually remove the old ./baz/bag package (perhaps the repo is archived and unmaintained!), then just injecting a new module at github.com/foo/bar/baz/bag would produce an ambiguous import error instead of replacing the packages as intended.

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 2, 2021

Change https://golang.org/cl/332571 mentions this issue: cmd/go/internal/modload: remove ImportMap and PackageDir

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 5, 2021

With everything else happening in Go 1.18, I think this is going to slip to 1.19.

@bcmills bcmills removed this from the Go1.18 milestone Oct 5, 2021
@bcmills bcmills added this to the Go1.19 milestone Oct 5, 2021
@tandr
Copy link

@tandr tandr commented Oct 8, 2021

Looks like there are more than one thing slipping to 1.19 because of the same reasons. May I suggest to make 1.18 with all these items, and then make 1.19 to go with generics (maybe shortly after)?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 8, 2021

@tandr That could have been a good discussion six months ago, but at this point we only have one month left to the 1.18 feature freeze. We're not going to be able to significantly rearrange our priorities now. Sorry.

@tandr
Copy link

@tandr tandr commented Oct 8, 2021

Thanks Ian, I understand.

piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 21, 2022
piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 22, 2022
piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 22, 2022
* update jwt-go

* update replace

why? golang/go#26904
kohrVid added a commit to kohrVid/pg-cli that referenced this issue May 17, 2022
This commit reverts changes made in a previous PR [1] which attempted to
use relative imports within the module. I'm quite new to go modules and
unfortunately that change was equivalent to renaming the module -
something that golang doesn't support.[2] This commit reverts the
changes and adds warnings to previous releases that were published in
error.

[1]: e05c9d2
[2]: golang/go#26904
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
modules NeedsInvestigation
Projects
None yet
Development

No branches or pull requests