Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go mod init' should validate module paths for v2+ dependencies #31549

Open
thepudds opened this issue Apr 18, 2019 · 1 comment

Comments

@thepudds
Copy link

commented Apr 18, 2019

What version of Go are you using (go version)?

go 1.12.4

Does this issue reproduce with the latest release?

Yes, including tip.

What did you do?

Observed problems in modules that were ultimately traced back to losing information in the go mod init process.

What did you expect to see?

go mod init setting proper module paths for v2+ dependencies when converting from a prior dependency manager.

What did you see instead?

go mod init never seems to set the /vN as part of the module path for a require statement

Example from scratch

This can happen in different ways in the wild, but here is an example from scratch.

$ export GO111MODULE=off
$ go get github.com/gobuffalo/packr/v2
$ mkdir -p $GOPATH/src/example.com/hello
$ cd $GOPATH/src/example.com/hello
$ cat <<EOF > hello.go
package hello
import _ "github.com/gobuffalo/packr/v2"
EOF

$ dep init
  ...
  Using ^2.2.0 as constraint for direct dep github.com/gobuffalo/packr
  ... 

$ export GO111MODULE=on
$ gotip mod init
go: creating new go.mod: module example.com/scratchpad/hello
go: copying requirements from Gopkg.lock

$ grep packr go.mod
        github.com/gobuffalo/packr v0.0.0-20190416161152-cda4ac255773

cda4ac255773 corresponds to the v2.2.0 tag, but the require in the created go.mod is missing the required /v2 after packr.

In other words, go mod init created an incorrect go.mod.

Gopkg.lock and Gopkg.toml created by this example
$ cat Gopkg.lock
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.


[[projects]]
  digest = "1:d29ee5ef14a7e0253facd0bcebe6a69a7a4e02a67eb24d2aacd8ccb4a7cea6fc"
  name = "github.com/gobuffalo/envy"
  packages = ["."]
  pruneopts = "UT"
  revision = "043cb4b8af871b49563291e32c66bb84378a60ac"
  version = "v1.7.0"

[[projects]]
  branch = "master"
  digest = "1:5cc0801073158de62986111e35292e498e5d762a45da812bac744986fbe11daa"
  name = "github.com/gobuffalo/genny"
  packages = ["."]
  pruneopts = "UT"
  revision = "3ca520ef0d9ea4981534aae98bf9a003d9b18678"

[[projects]]
  branch = "master"
  digest = "1:7779ec172ea5898029020425bf58586c68c341fe4c3c23c2508605767de3ce47"
  name = "github.com/gobuffalo/gogen"
  packages = [
    ".",
    "goimports",
    "gomods",
  ]
  pruneopts = "UT"
  revision = "8f38393713f59beb43221f903c07f58f0377a063"

[[projects]]
  branch = "master"
  digest = "1:f391538f4166f7c0aa50943372af196304488b039fe663c134d42f064e1ed92c"
  name = "github.com/gobuffalo/logger"
  packages = ["."]
  pruneopts = "UT"
  revision = "86e12af44bc273e0063fad5b6ad588890b8cfe6b"

[[projects]]
  digest = "1:a3259d2a6c245b62e40232800e3ebb4d7d5840940638de0e4754cb13cdd37790"
  name = "github.com/gobuffalo/mapi"
  packages = ["."]
  pruneopts = "UT"
  revision = "0bb5e840be332d4280e40f2e6c50777c615bbac5"
  version = "v1.0.2"

[[projects]]
  branch = "master"
  digest = "1:8561bb77169b6b26a39bcf2945f582a10ccbe3533d644ada72094bd5fc86b4d5"
  name = "github.com/gobuffalo/packd"
  packages = ["."]
  pruneopts = "UT"
  revision = "a385830c7fc0495b810318788a043c7655da337d"

[[projects]]
  digest = "1:0fac441be66684180b031233313b8315644c0734fd73944c3e156e3cf5fbe203"
  name = "github.com/gobuffalo/packr"
  packages = [
    "v2",
    "v2/file",
    "v2/file/resolver",
    "v2/file/resolver/encoding/hex",
    "v2/jam/parser",
    "v2/plog",
  ]
  pruneopts = "UT"
  revision = "cda4ac25577350dac682c8715dc6659327232596"
  version = "v2.2.0"

[[projects]]
  branch = "master"
  digest = "1:b90ac64448d67ef218124185a8524f3c665fb7f51f096d5dd51ce950a1775d89"
  name = "github.com/gobuffalo/syncx"
  packages = ["."]
  pruneopts = "UT"
  revision = "33c29581e754bd354236e977dfe426e55331c45d"

[[projects]]
  digest = "1:ecd9aa82687cf31d1585d4ac61d0ba180e42e8a6182b85bd785fcca8dfeefc1b"
  name = "github.com/joho/godotenv"
  packages = ["."]
  pruneopts = "UT"
  revision = "23d116af351c84513e1946b527c88823e476be13"
  version = "v1.3.0"

[[projects]]
  digest = "1:38d2e838817528fd7d1ccdf7fa6700ee0f1727b3c484d13cab54bfbdaab12b0e"
  name = "github.com/karrick/godirwalk"
  packages = ["."]
  pruneopts = "UT"
  revision = "6d1c7760ec857d2984abe0a23fa263877f50d3f0"
  version = "v1.8.0"

[[projects]]
  digest = "1:31e761d97c76151dde79e9d28964a812c46efc5baee4085b86f68f0c654450de"
  name = "github.com/konsorten/go-windows-terminal-sequences"
  packages = ["."]
  pruneopts = "UT"
  revision = "f55edac94c9bbba5d6182a4be46d86a2c9b5b50e"
  version = "v1.0.2"

[[projects]]
  branch = "master"
  digest = "1:6e2ed1bdbf1d14b4d0be58bcd3f1c3000c1e226964354457b8e6ca69e83a1cbb"
  name = "github.com/markbates/oncer"
  packages = ["."]
  pruneopts = "UT"
  revision = "bf2de49a0be218916e69a11d22866e6cd0a560f2"

[[projects]]
  digest = "1:28687e854cec240942c103259668b132a8450b05a6cc677eea2282b26ae29310"
  name = "github.com/markbates/safe"
  packages = ["."]
  pruneopts = "UT"
  revision = "6fea05a5732486546a4836b7a1d596c5ec687b98"
  version = "v1.0.1"

[[projects]]
  digest = "1:cf31692c14422fa27c83a05292eb5cbe0fb2775972e8f1f8446a71549bd8980b"
  name = "github.com/pkg/errors"
  packages = ["."]
  pruneopts = "UT"
  revision = "ba968bfe8b2f7e042a574c888954fccecfa385b4"
  version = "v0.8.1"

[[projects]]
  digest = "1:e09ada96a5a41deda4748b1659cc8953961799e798aea557257b56baee4ecaf3"
  name = "github.com/rogpeppe/go-internal"
  packages = [
    "modfile",
    "module",
    "semver",
  ]
  pruneopts = "UT"
  revision = "438578804ca6f31be148c27683afc419ce47c06e"
  version = "v1.3.0"

[[projects]]
  digest = "1:fd61cf4ae1953d55df708acb6b91492d538f49c305b364a014049914495db426"
  name = "github.com/sirupsen/logrus"
  packages = ["."]
  pruneopts = "UT"
  revision = "8bdbc7bcc01dcbb8ec23dc8a28e332258d25251f"
  version = "v1.4.1"

[[projects]]
  branch = "master"
  digest = "1:bbe51412d9915d64ffaa96b51d409e070665efc5194fcf145c4a27d4133107a4"
  name = "golang.org/x/crypto"
  packages = ["ssh/terminal"]
  pruneopts = "UT"
  revision = "b43e412143f90fca62516c457cae5a8dc1595586"

[[projects]]
  branch = "master"
  digest = "1:e18a5437c48ea1bd6dcd960efdd634d05ae5c2dd033d2d17f73cc20be92895ac"
  name = "golang.org/x/sys"
  packages = [
    "unix",
    "windows",
  ]
  pruneopts = "UT"
  revision = "f0ce4c0180bef7e9c51babed693a6e47fdd8962f"

[[projects]]
  branch = "master"
  digest = "1:be1ab6d2b333b1d487c01f1328aef9dc76cee4ff4f780775a552d2a1653f0207"
  name = "golang.org/x/tools"
  packages = [
    "go/ast/astutil",
    "go/gcexportdata",
    "go/internal/gcimporter",
    "go/internal/packagesdriver",
    "go/packages",
    "go/types/typeutil",
    "imports",
    "internal/fastwalk",
    "internal/gopathwalk",
    "internal/module",
    "internal/semver",
  ]
  pruneopts = "UT"
  revision = "6732636ccdfd99c4301d1d1ac2307f091331f767"

[solve-meta]
  analyzer-name = "dep"
  analyzer-version = 1
  input-imports = ["github.com/gobuffalo/packr/v2"]
  solver-name = "gps-cdcl"
  solver-version = 1

$ cat Gopkg.toml

# Gopkg.toml example
#
# Refer to https://golang.github.io/dep/docs/Gopkg.toml.html
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
#   name = "github.com/user/project"
#   version = "1.0.0"
#
# [[constraint]]
#   name = "github.com/user/project2"
#   branch = "dev"
#   source = "github.com/myfork/project2"
#
# [[override]]
#   name = "github.com/x/y"
#   version = "2.4.0"
#
# [prune]
#   non-go = false
#   go-tests = true
#   unused-packages = true


[[constraint]]
  name = "github.com/gobuffalo/packr"
  version = "2.2.0"

[prune]
  go-tests = true
  unused-packages = true

Additional comments

This issue is not about how to resolve this manually, which I know how to do. Rather, it would be better to not to have to resolve manually, but more importantly would better if others did not encounter follow-on issues due to this behavior. Also, in some cases go mod tidy or similar can let someone be "lucky" and not notice this, but in some cases the version information converted by go mod init is important to avoid follow-on issues.

This was a less significant problem early in the history of modules because not many v2+ packages had adopted modules yet.

However, more v2+ dependencies are adopting go.mod files themselves as modules move towards being on by default in 1.13.

I have seen this behavior be the root cause of hard-to-diagnose problems in the wild, and the rate of those problems likely would increase with more v2+ modules in the ecosystem. (And then later, this problem rate of course would decrease, given it is transitional in nature).

This is related to #30161, but that suggested an alternative solution of dropping the dependency during the conversion process, and that issue was ultimately closed by the reporter after the current behavior was explained and the reporter understood how to manually work around. However, ideally go mod init would record the right information here without the need for someone to read documentation or manually work around.

@thepudds thepudds added the modules label Apr 18, 2019

@dmitshur dmitshur added this to the Go1.13 milestone Apr 18, 2019

@bcmills bcmills changed the title cmd/go: 'go mod init' should record correct module paths for v2+ dependencies cmd/go: 'go mod init' should validate module paths for v2+ dependencies Apr 19, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

Somewhat related to #29814.

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.