cmd/go/internal/modfetch: erroneously resolves a `v2.…+incompatible` version when a `v2/go.mod` file exists

[bcmills]

CL 378400 appears to have introduced a regression for modules that contain a v2/go.mod file and not go.mod file at the repo root.

github.com/sacloud/libsacloud at commit b491c1c2c6bd6e18fd0b55ef6770ecd1e66f1419 is one such example.
At that revision, it contains a v2/go.mod file that correctly declares module github.com/sacloud/libsacloud/v2, and no go.mod file whatsoever at the module root. The corresponding v2 pseudo-version should therefore apply only to the v2 module (github.com/sacloud/libsacloud/v2) — it should not also be valid as a +incompatible version for the module at the root of the repo.

Go 1.17 as originally released correctly rejected that version for the repo root. go1.18rc1 does not.

$ export GOPRIVATE=github.com/sacloud/libsacloud

$ go clean -modcache

$ gotip list -m github.com/sacloud/libsacloud@v2.25.2-0.20210930043551-b491c1c2c6bd
github.com/sacloud/libsacloud v2.25.2-0.20210930043551-b491c1c2c6bd+incompatible

$ go clean -modcache

$ go1.17 list -m github.com/sacloud/libsacloud@v2.25.2-0.20210930043551-b491c1c2c6bd
go list -m: github.com/sacloud/libsacloud@v2.25.2-0.20210930043551-b491c1c2c6bd: invalid version: should be v0 or v1, not v2

(Found by @heschi during Go 1.18 pre-release testing.)

CC @matloob

[bcmills]

One more data point: apparently go1.17 did, in fact, allow a +incompatible suffix on the root module if it was requested explicitly:

$ go clean -modcache

$ go1.17 list -m github.com/sacloud/libsacloud@v2.25.2-0.20210930043551-b491c1c2c6bd+incompatible
github.com/sacloud/libsacloud v2.25.2-0.20210930043551-b491c1c2c6bd+incompatible

So the recent change in behavior is just that we now resolve the +incompatible version automatically instead of erring out.

[bcmills]

@gopherbot, please backport to Go 1.16 and Go 1.17. This is a regression introduced in a security release (for #35671), and can cause confusion if a Go user accidentally passes the wrong module path to go get.

[gopherbot]

Backport issue(s) opened: #51331 (for 1.16), #51332 (for 1.17).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/387675 mentions this issue: cmd/go: avoid +incompatible major versions if a go.mod file exists in a subdirectory for that version

[gopherbot]

Change https://go.dev/cl/387917 mentions this issue: cmd/go/internal/modfetch: simplify handling of weird version tags

[bcmills]

Reopening to track 1.18 cherry-pick.