Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go mod download' fails with xyz+incompatible replacement targets while 'go mod verifies' works #34254

Closed
dmitris opened this issue Sep 12, 2019 · 11 comments
Assignees
Milestone

Comments

@dmitris
Copy link
Contributor

@dmitris dmitris commented Sep 12, 2019

What version of Go are you using (go version)?

go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOHOSTARCH="amd64"
GOHOSTOS="darwin"

What did you do?

my go.mod file contains replacement targets like:

github.com/pkg/errors => git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible

I ran go mod verify and go mod download.

What did you expect to see?

Since the project used to build with go1.12 and still builds / tests run fine, I expect both commands to work without errors.

What did you see instead?

go mod verify shows all modules verified (as expected), but go mod download fails with:

$ go mod download
go: finding git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible
go: finding git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible
git.ourcompany.com/mirror-github/pkg--errors@v0.8.1+incompatible: invalid version: +incompatible suffix not allowed: major version v0 is compatible
git.ourcompany.com/mirror-github/go-yaml--yaml@v2.2.2+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

This seems related to #34217; on slack @bcmills said that it is likely a bug.

would argue that the acceptable versions should be:
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2
(because […]/v2 should be able to replace gopkg.in/[…].v2, bug notwithstanding) or
gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2
(because the go.mod file should say gopkg.in/yaml.v2 explicitly, and that should be good enough for validation).

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 12, 2019

To clarify, the symptom you saw that I think is clearly a bug is the one when using

replace gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2

which you reported to produce the error:

go: gopkg.in/yaml.v2@v2.2.2: go.mod has non-.../v2 module path "gopkg.in/yaml.v2" (and .../v2/go.mod does not exist) at revision v2.2.2
@imjuanleonard
Copy link

@imjuanleonard imjuanleonard commented Oct 31, 2019

Is there any work around on this?
I have the same problem

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 13, 2019

The first entry here:

	github.com/pkg/errors => git.ourcompany.com/mirror-github/pkg--errors v0.8.1+incompatible

really is incorrect, and the go command is correct to flag it as such. The version v0.8.1 is compatible with the module paths on both sides, so it very clearly should not have a +incompatible suffix.

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 13, 2019

Change https://golang.org/cl/206761 mentions this issue: cmd/go: allow a fork with path […]/vN to replace gopkg.in/[…].vN

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 13, 2019

The second entry here:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml v2.2.2+incompatible

is also incorrect, because the module has a go.mod file: the module path should have a /v2 suffix, or a gopkg.in path and a .v2 suffix.

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 13, 2019

What will work, as of CL 206761, is:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2
@gopherbot gopherbot closed this in 3bea90d Nov 15, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Nov 15, 2019

This should be fixed at head.

The fix likely patches cleanly onto Go 1.13, but I do not believe that it meets our backporting criteria: I have only been able to find two affected modules (gopkg.in/yaml.v2 and gopkg.in/src-d/go-git.v4), and there is at least one possible workaround.

One possible workaround is to check out the replacement source into a local directory, and target the replace directive to the local directory rather than the git-hosted repo. There may be others.

@imjuanleonard
Copy link

@imjuanleonard imjuanleonard commented Nov 16, 2019

Work for me after removing the +incompatible suffix and adding /v2 suffix to company repository, thanks @bcmills

Example import on go mod:

require(
    ...
    "company.repo.com/group/project/v2 v2.7.0"
    ...
)

Go version 1.13.4

GOPRIVATE="company.repo.com"
GOPROXY="https://company.golang.artifact.com,direct"
GOSUMDB="https://sum.golang.org"

@dmitris
Copy link
Contributor Author

@dmitris dmitris commented Nov 18, 2019

works for me as well with the tip Go with this replace:

	gopkg.in/yaml.v2 => git.ourcompany.com/mirror-github/go-yaml--yaml/v2 v2.2.2

The fix likely patches cleanly onto Go 1.13, but I do not believe that it meets our backporting criteria

that's sad (for those of us affected 😄 ) - maybe an exception could be considered since gopkg.in/yaml.v2 is the most popular yaml parsing library? It is the first hit in the package search results for "yaml" - https://pkg.go.dev/search?q=yaml, shows 8654 known imports and likely there are many ones in the corporate codebases as well. @bcmills

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 18, 2019

maybe an exception could be considered since gopkg.in/yaml.v2 is the most popular yaml parsing library?

We've literally had only the two reports of this issue. (I suspect that most people are not carrying their own patches to gopkg.in/yaml.v2, and thus can use a proxy rather than a replace directive.)

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 19, 2019

Change https://golang.org/cl/212105 mentions this issue: cmd/go: relax validation for replacements for gopkg.in paths

gopherbot pushed a commit that referenced this issue Dec 19, 2019
The 'go' command normally requires the 'go.mod' files for replacement
modules to have a major version compatible with the module they are
replacing.

However, prior to CL 206761, the 'go' command erroneously allowed
unversioned paths (which imply major version 0 or 1) to replace
'gopkg.in' paths with any major-version suffix.

An analysis of proxy.golang.org suggests that these replacements,
while uncommon, are not unheard-of. Rather than breaking the modules
that rely on them, we will continue to allow the erroneous replacement
paths for this particular pairing.

Updates #34254

Change-Id: Icb4e745981803edaa96060f17a8720a058219ab1
Reviewed-on: https://go-review.googlesource.com/c/go/+/212105
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants
You can’t perform that action at this time.