/go

x/vgo: decide how much to support gopkg.in #24099

Closed
rogpeppe opened this Issue Feb 24, 2018 · 12 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
  vgo
@rogpeppe
Contributor

rogpeppe commented Feb 24, 2018

What version of Go are you using (go version)?

go version devel +104445e Wed Feb 7 19:22:09 2018 +0000 linux/amd64 vgo:2018-02-20.1

What did you do?

% cat go.mod
module "github.com/rogpeppe/xx"

require "gopkg.in/yaml.v2" v2.1.0
% vgo build
vgo: errors parsing go.mod:
/tmp/xx/go.mod:3: invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.1.0)

If I change the module name in the requirements to "gopkg.in/yaml/v2", I see this output:

% vgo build
vgo: finding gopkg.in/yaml/v2 v2.1.0
vgo: gopkg.in/yaml/v2: invalid gopkg.in/ path: "gopkg.in/yaml/v2"
vgo: finding gopkg.in/yaml/v2 v2.1.0
vgo: gopkg.in/yaml/v2: invalid gopkg.in/ path: "gopkg.in/yaml/v2"
vgo: invalid gopkg.in/ path: "gopkg.in/yaml/v2"

@gopherbot gopherbot added this to the vgo milestone Feb 24, 2018

@rogpeppe

This comment has been minimized.

Show comment
Hide comment
@rogpeppe

rogpeppe Feb 24, 2018

Contributor

OK, I see what's happening now. It's not grokking the major-version-in-path of gopkg.in.
As an example, if I run vgo build on this program:

package main // import "github.com/rogpeppe/xx"

import (
	_ "gopkg.in/juju/charm.v4"
	_ "gopkg.in/juju/charm.v5"
	_ "gopkg.in/juju/charm.v6"
)

func main() {
}

I get this go.mod file:

module "github.com/rogpeppe/xx"

require (
	"github.com/juju/names" v0.0.0-20180129205841-f9b5b8b7614d
	"gopkg.in/juju/charm.v4" v0.0.0-20150410205938-af4be7b079ac
	"gopkg.in/juju/charm.v5" v0.0.0-20161109084014-949f52b3948c
	"gopkg.in/juju/charm.v6" v0.0.0-20180119231043-96786fc3f869
	"gopkg.in/yaml.v1" v0.0.0-20140924161607-9f9df34309c0
)

So I titled this issue wrong; it is possible to use gopkg.in with a major version > 1, although the behaviour is still not correct.

Interestingly, the go.mod file does not contain all the dependencies that I'd expect. For example, it contains gopkg.in/yaml.v1 (used by gopkg.in/juju/charm.v4) but not gopkg.in/yaml.v2 (used by gopkg.in/juju/charm.v6); it contains github.com/juju/names but not gopkg.in/juju/names.v2).
Contributor

rogpeppe commented Feb 24, 2018

OK, I see what's happening now. It's not grokking the major-version-in-path of gopkg.in.
As an example, if I run vgo build on this program:

package main // import "github.com/rogpeppe/xx"

import (
	_ "gopkg.in/juju/charm.v4"
	_ "gopkg.in/juju/charm.v5"
	_ "gopkg.in/juju/charm.v6"
)

func main() {
}

I get this go.mod file:

module "github.com/rogpeppe/xx"

require (
	"github.com/juju/names" v0.0.0-20180129205841-f9b5b8b7614d
	"gopkg.in/juju/charm.v4" v0.0.0-20150410205938-af4be7b079ac
	"gopkg.in/juju/charm.v5" v0.0.0-20161109084014-949f52b3948c
	"gopkg.in/juju/charm.v6" v0.0.0-20180119231043-96786fc3f869
	"gopkg.in/yaml.v1" v0.0.0-20140924161607-9f9df34309c0
)

So I titled this issue wrong; it is possible to use gopkg.in with a major version > 1, although the behaviour is still not correct.

Interestingly, the go.mod file does not contain all the dependencies that I'd expect. For example, it contains gopkg.in/yaml.v1 (used by gopkg.in/juju/charm.v4) but not gopkg.in/yaml.v2 (used by gopkg.in/juju/charm.v6); it contains github.com/juju/names but not gopkg.in/juju/names.v2).

@rogpeppe rogpeppe changed the title from x/vgo: cannot use gopkg.in with major version > 1 to x/vgo: support gopkg.in major versions properly Feb 24, 2018

@tgulacsi

This comment has been minimized.

Show comment
Hide comment
@tgulacsi

tgulacsi Feb 24, 2018

require "gopkg.in/yaml.v2/v2"

works, sort of - it still includes the "gopkg.in/yaml.v2 v1.1.0-gopkg-v2.1.0" - see #24094

tgulacsi commented Feb 24, 2018

require "gopkg.in/yaml.v2/v2"

works, sort of - it still includes the "gopkg.in/yaml.v2 v1.1.0-gopkg-v2.1.0" - see #24094
@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Mar 30, 2018

Contributor

support gopkg.in major versions properly

"Properly" is a loaded word, but it should work at least a little better than it does right now. In the original case the v2.1.0 should probably just be rewritten transparently to v1.1.0-gopkgin-v2.1.0, because we know this is gopkg.in.

I'm going to leave this bug open to track deciding what level of gopkg.in support we want. Maybe we should actually just allow v2.1.0 instead of the funny pseudo-version.

Interestingly, the go.mod file does not contain all the dependencies that I'd expect.

It contains a simplified list of dependencies, omitting those implied by the ones already listed. Because charm.v6's dependencies.tsv lists the exact version of gopkg.in/yaml.v2 that the build chose, it lists charm.v6 in the go.mod and stops there.
Contributor

rsc commented Mar 30, 2018

support gopkg.in major versions properly

"Properly" is a loaded word, but it should work at least a little better than it does right now. In the original case the v2.1.0 should probably just be rewritten transparently to v1.1.0-gopkgin-v2.1.0, because we know this is gopkg.in.

I'm going to leave this bug open to track deciding what level of gopkg.in support we want. Maybe we should actually just allow v2.1.0 instead of the funny pseudo-version.

Interestingly, the go.mod file does not contain all the dependencies that I'd expect.

It contains a simplified list of dependencies, omitting those implied by the ones already listed. Because charm.v6's dependencies.tsv lists the exact version of gopkg.in/yaml.v2 that the build chose, it lists charm.v6 in the go.mod and stops there.

@rsc rsc changed the title from x/vgo: support gopkg.in major versions properly to x/vgo: decide how much to support gopkg.in Mar 30, 2018

@rsc rsc added the NeedsDecision label Mar 30, 2018

@rsc rsc referenced this issue Apr 2, 2018

Open

x/vgo: does not recognize some valid gopkg.in paths #23989

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot Apr 18, 2018

Change https://golang.org/cl/107658 mentions this issue: cmd/go/internal/modfetch: make gopkg.in support more seamless

gopherbot commented Apr 18, 2018

Change https://golang.org/cl/107658 mentions this issue: cmd/go/internal/modfetch: make gopkg.in support more seamless

@gopherbot gopherbot closed this in golang/vgo@84430ca Apr 25, 2018

@rsc rsc referenced this issue Apr 25, 2018

Open

cmd/go: add package version support to Go toolchain #24301

@kron4eg

This comment has been minimized.

Show comment
Hide comment
@kron4eg

kron4eg Apr 25, 2018

After initial vgo build and go.mod (from empty state) generation, I get

gopkg.in/yaml.v2 v2.2.1

in go.mod file (which is expected now), on the second vgo build try error is reported:

invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.2.1)

kron4eg commented Apr 25, 2018

After initial vgo build and go.mod (from empty state) generation, I get

gopkg.in/yaml.v2 v2.2.1

in go.mod file (which is expected now), on the second vgo build try error is reported:

invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.2.1)

@client9 client9 referenced this issue Apr 27, 2018

Closed

x/vgo: unclear error message with gopkg.in #25121

@primalmotion

This comment has been minimized.

Show comment
Hide comment
@primalmotion

primalmotion Apr 29, 2018

this is still happening to me

vgo build ./...
...
vgo: parsing downloaded go.mod: go.mod:22: invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.0.0-20160301204022-a83829b6f129)

primalmotion commented Apr 29, 2018 •
edited
Edited 1 time
  • @primalmotion primalmotion edited Apr 29, 2018 (most recent)

this is still happening to me

vgo build ./...
...
vgo: parsing downloaded go.mod: go.mod:22: invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.0.0-20160301204022-a83829b6f129)
@markbates

This comment has been minimized.

Show comment
Hide comment
@markbates

markbates Apr 29, 2018

Agreed still doesn't work for me either

module github.com/gobuffalo/pop

require (
	github.com/cockroachdb/cockroach-go v0.0.0-20180212155653-59c0560478b7
	github.com/fatih/color v1.6.0
	github.com/go-sql-driver/mysql v1.0.3
	github.com/gobuffalo/envy v1.6.2
	github.com/gobuffalo/packr v1.11.0
	github.com/gobuffalo/uuid v1.2.0
	github.com/gobuffalo/validate v1.0.0
	github.com/jmoiron/sqlx v0.0.0-20180406164412-2aeb6a910c2b
	github.com/joho/godotenv v1.2.0
	github.com/lib/pq v0.0.0-20180327071824-d34b9ff171c2
	github.com/markbates/going v0.0.0-20170113214412-0576708c56ce
	github.com/markbates/inflect v0.0.0-20180405204719-fbc6b23ce49e
	github.com/mattn/anko v0.0.3
	github.com/pkg/errors v0.8.0
	golang.org/x/net v0.0.0-20180420171651-5f9ae10d9af5
	golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
	gopkg.in/yaml.v2 v2.2.1
)

vgo: errors parsing go.mod:
$GOPATH/src/github.com/gobuffalo/pop/go.mod:20: invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.2.1)

markbates commented Apr 29, 2018

Agreed still doesn't work for me either

module github.com/gobuffalo/pop

require (
	github.com/cockroachdb/cockroach-go v0.0.0-20180212155653-59c0560478b7
	github.com/fatih/color v1.6.0
	github.com/go-sql-driver/mysql v1.0.3
	github.com/gobuffalo/envy v1.6.2
	github.com/gobuffalo/packr v1.11.0
	github.com/gobuffalo/uuid v1.2.0
	github.com/gobuffalo/validate v1.0.0
	github.com/jmoiron/sqlx v0.0.0-20180406164412-2aeb6a910c2b
	github.com/joho/godotenv v1.2.0
	github.com/lib/pq v0.0.0-20180327071824-d34b9ff171c2
	github.com/markbates/going v0.0.0-20170113214412-0576708c56ce
	github.com/markbates/inflect v0.0.0-20180405204719-fbc6b23ce49e
	github.com/mattn/anko v0.0.3
	github.com/pkg/errors v0.8.0
	golang.org/x/net v0.0.0-20180420171651-5f9ae10d9af5
	golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
	gopkg.in/yaml.v2 v2.2.1
)

vgo: errors parsing go.mod:
$GOPATH/src/github.com/gobuffalo/pop/go.mod:20: invalid module: gopkg.in/yaml.v2 should be v1, not v2 (v2.2.1)
@cmars

This comment has been minimized.

Show comment
Hide comment
@cmars

cmars Apr 30, 2018

Contributor

Can we reopen this issue? I'm seeing the same issue as @kron4eg above.
Contributor

cmars commented Apr 30, 2018

Can we reopen this issue? I'm seeing the same issue as @kron4eg above.

@ALTree ALTree reopened this Apr 30, 2018

@ALTree ALTree added NeedsInvestigation and removed NeedsDecision labels Apr 30, 2018

@bpicode bpicode referenced this issue in bpicode/fritzctl May 1, 2018

Open

vgo adoption? #135

@keltia

This comment has been minimized.

Show comment
Hide comment
@keltia

keltia May 2, 2018

I get the same with yaml.v2; the initial go.mod file was generated by vgo build but after that, it fails with the above error. yaml.v2 has no tags, does not use semver or anything.

keltia commented May 2, 2018

I get the same with yaml.v2; the initial go.mod file was generated by vgo build but after that, it fails with the above error. yaml.v2 has no tags, does not use semver or anything.

@coyle coyle referenced this issue in storj/storj May 2, 2018

Merged

Clean lint #20

@lelvisl

This comment has been minimized.

Show comment
Hide comment
@lelvisl

lelvisl May 4, 2018

same problem with elastic

module modulename

require (
	github.com/BurntSushi/toml v0.3.0
	github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973
	github.com/garyburd/redigo v1.6.0
	github.com/go-redis/redis v0.0.0-20160126235308-877867d2845fbaf86798befe410b6ceb6f5c29a3
.....
	gopkg.in/olivere/elastic.v3 v1.0.73-gopkgin-v3.0.73
)

→ vgo build
vgo: errors parsing go.mod:
.../App/go.mod:26: invalid module: gopkg.in/olivere/elastic.v3 should be v1, not v3 (v3.0.73)

And if I remove line with elastic from go.mod it builds.

lelvisl commented May 4, 2018

same problem with elastic

module modulename

require (
	github.com/BurntSushi/toml v0.3.0
	github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973
	github.com/garyburd/redigo v1.6.0
	github.com/go-redis/redis v0.0.0-20160126235308-877867d2845fbaf86798befe410b6ceb6f5c29a3
.....
	gopkg.in/olivere/elastic.v3 v1.0.73-gopkgin-v3.0.73
)

→ vgo build
vgo: errors parsing go.mod:
.../App/go.mod:26: invalid module: gopkg.in/olivere/elastic.v3 should be v1, not v3 (v3.0.73)

And if I remove line with elastic from go.mod it builds.

@docwhat docwhat referenced this issue May 8, 2018

Closed

x/vgo: get gopkg.in/yaml.v2 succeeds, but produces invalid go.mod #25243

@illegalnumbers

This comment has been minimized.

Show comment
Hide comment
@illegalnumbers

illegalnumbers May 8, 2018

I also see this issue.

illegalnumbers commented May 8, 2018

I also see this issue.

@joshuarubin joshuarubin referenced this issue in golang/vgo May 9, 2018

Closed

x/vgo: Fix parsing of go.mod with packages from gopkg.in #4

joshuarubin added a commit to joshuarubin/vgo that referenced this issue May 9, 2018

@joshuarubin
x/vgo: fix parsing of go.mod with packages from gopkg.in 
This addresses an issue where Parsing of `go.mod` with modules from
`gopkg.in` fails. The `go.mod` file is produced automatically, so the
first time something like `vgo build` is run (before `go.mod` is
created), it succeeds, but the second time it is run, it fails.

This fixes the latest issues described in
golang/go#24099

The funcs `parseGopkgIn` and `dotV` were pulled in (and slightly
modified/simplified) from `vendor/cmd/go/internal/modfetch/gopkgin.go`.

I'm sure that this is not the best way to handle the special case of
`gopkg.in` (especially compared to the effort put into creating the
`VersionFixer`), but it's a starting point that has test cases that
prove the autogenerated `go.mod` file can't be parsed.

Fixes #24099

Signed-off-by: Joshua Rubin <joshua@rubixconsulting.com>
Verified
This commit was signed with a verified signature.
@joshuarubin joshuarubin Joshua Rubin
GPG key ID: 7E86D83E7AD27F82 Learn about signing commits
07de619

joshuarubin added a commit to joshuarubin/vgo that referenced this issue May 9, 2018

@joshuarubin
x/vgo: fix parsing of go.mod with packages from gopkg.in 
This addresses an issue where Parsing of `go.mod` with modules from
`gopkg.in` fails. The `go.mod` file is produced automatically, so the
first time something like `vgo build` is run (before `go.mod` is
created), it succeeds, but the second time it is run, it fails.

This fixes the latest issues described in
golang/go#24099

The funcs `parseGopkgIn` and `dotV` were pulled in (and slightly
modified/simplified) from `vendor/cmd/go/internal/modfetch/gopkgin.go`.

I'm sure that this is not the best way to handle the special case of
`gopkg.in` (especially compared to the effort put into creating the
`VersionFixer`), but it's a starting point that has test cases that
prove the autogenerated `go.mod` file can't be parsed.

Fixes #24099

Signed-off-by: Joshua Rubin <joshua@rubixconsulting.com>
Verified
This commit was signed with a verified signature.
@joshuarubin joshuarubin Joshua Rubin
GPG key ID: 7E86D83E7AD27F82 Learn about signing commits
f608544

gopherbot pushed a commit to golang/vgo that referenced this issue May 10, 2018

@joshuarubin @rsc
x/vgo: Fix parsing of go.mod with packages from gopkg.in 
This addresses an issue where Parsing of `go.mod` with modules from `gopkg.in` fails. The `go.mod` file is produced automatically, so the first time something like `vgo build` is run (before `go.mod` is created), it succeeds, but the second time it is run, it fails.

This fixes the latest issues described in golang/go#24099

The funcs `parseGopkgIn` and `dotV` were pulled in (and slightly modified/simplified) from `vendor/cmd/go/internal/modfetch/gopkgin.go`.

I'm sure that this is not the best way to handle the special case of `gopkg.in` (especially compared to the effort put into creating the `VersionFixer`), but it's a starting point that has test cases that prove the autogenerated `go.mod` file can't be parsed.

Fixes #24099

Change-Id: Ie0bee2a50b330a9f85c1459ff6eb0bfa43058bf1
GitHub-Last-Rev: f608544
GitHub-Pull-Request: golang/vgo#4
Reviewed-on: https://go-review.googlesource.com/112277
Reviewed-by: Russ Cox <rsc@golang.org>
5643497

@rsc rsc referenced this issue May 10, 2018

Open

x/build/cmd/gerritbot: does not respect codereview.cfg for fixing commit messages #25337

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc May 10, 2018

Contributor

Fixed by recent CL.
Contributor

rsc commented May 10, 2018

Fixed by recent CL.

@rsc rsc closed this May 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment