Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go mod tidy, vendor hide module load errors #27063

Open
rsc opened this issue Aug 17, 2018 · 10 comments

Comments

Projects
None yet
10 participants
@rsc
Copy link
Contributor

commented Aug 17, 2018

The module loader leaves some problems for the standard loader to find, like not being able to resolve imports. But 'go mod tidy' and 'go mod vendor' do not run the standard loader, so those problems go undiagnosed. Maybe we should look for them explicitly.

For example make a file x.go that says import "nonexist". go mod tidy succeeds quietly.

@rsc rsc added this to the Go1.12 milestone Aug 17, 2018

@rsc rsc added the modules label Aug 17, 2018

@rsc rsc changed the title cmd/go: go mod tidy hides module load errors cmd/go: go mod tidy, vendor hide module load errors Aug 18, 2018

@kevinburke

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2018

#27868 is possibly a duplicate of this issue; I'm not positive.

@jsternberg

This comment has been minimized.

Copy link

commented Oct 9, 2018

We have run into this issue where using go mod tidy on a machine that doesn't have bzr, but that has bzr dependencies, will just not mention that it didn't work correctly.

We caught it because our CI system is now running go mod tidy and it has bzr installed.

@gopherbot

This comment has been minimized.

Copy link

commented Dec 10, 2018

Change https://golang.org/cl/153459 mentions this issue: cmd/go: fix 'go test' and 'go fmt' with files outside a module

gopherbot pushed a commit that referenced this issue Dec 11, 2018

cmd/go: fix 'go test' and 'go fmt' with files outside a module
Use the actual loader result in findModule instead of making
assumptions about nesting in the build list.
As a side-effect, this produces much clearer error messages for
packages that (for one reason or another) failed to load.

Adjust the package and module path outside a module to
"command-line-arguments". That string already appears in the output of
a number of (module-mode and GOPATH-mode) commands for file arguments,
and as far as I can tell operation outside a module is currently the
only case in which the module path of a package is not actually a
prefix of the import path.

Fixes #28011
Fixes #27099
Fixes #28943
Updates #27102
Updates #28459
Updates #27063

Change-Id: I61d5556df7b1b7d1efdaffa892f0e3e95b612d87
Reviewed-on: https://go-review.googlesource.com/c/153459
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

@bcmills bcmills modified the milestones: Go1.13, Go1.12 Dec 12, 2018

@myitcv

This comment has been minimized.

Copy link
Member

commented Jan 18, 2019

This might be another instance of this, I'm not 100% sure:

The following:

cd $(mktemp -d)
go mod init mod
cat <<EOD > main.go
package x

import (
        _ "github.com/hashicorp/vault/api"
)

func main() {
}
EOD
go mod tidy
go mod graph | grep labix

gives no results.

But if you then do:

go list all
go mod tidy
go mod graph | grep labix

you get:

mod labix.org/v2/mgo@v0.0.0-20140701140051-000000000287

which I think is as expected because:

$ go mod why -m labix.org/v2/mgo
# labix.org/v2/mgo
mod
github.com/hashicorp/vault/api
github.com/hashicorp/vault/api.test
github.com/hashicorp/vault/helper/builtinplugins
github.com/hashicorp/vault/builtin/logical/nomad
github.com/hashicorp/nomad/api
github.com/hashicorp/nomad/nomad/structs
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/go-msgpack/codec.test
labix.org/v2/mgo/bson
@leitzler

This comment has been minimized.

Copy link
Contributor

commented May 6, 2019

@bcmills this issue is tagged with NeedsInvestigation, are there any outstanding questions that need answers before solving this one?

As context I think this is a quite important issue to solve before go1.13 as it adds a lot of confusion for users.

I've seen several cases the thing that happens when bzr isn't installed (as described above by Paul and Jonathan), as well as when indirect dependencies have dependencies that gets removed/broken.

@thepudds

This comment has been minimized.

Copy link

commented May 6, 2019

Hi @leitzler or @jsternberg, I think launchpad.net/gocheck and labix.org/v2/mgo might be examples, but do have a simple set of "from scratch" steps that are an example this with bzr where go mod tidy does not complain? Could be helpful to make sure the ultimate fix here covers that case as well.

@leitzler

This comment has been minimized.

Copy link
Contributor

commented May 6, 2019

Sure! So far I have three different cases that I think is directly caused by this issue.

  1. A Bazaar dependency when the bazaar client bzr isn't installed.
$ docker run -it --rm golang sh
# go version
go version go1.12.4 linux/amd64
# cd $(mktemp -d)
# rm $(which bzr)
# go mod init x
go: creating new go.mod: module x
# cat > main.go
package x

import _ "launchpad.net/gocheck"
# go mod tidy
# cat go.mod
module x

go 1.12
# go build
main.go:3:8: unknown import path "launchpad.net/gocheck": bzr branch --use-existing-dir https://launchpad.net/~niemeyer/gocheck/trunk . in /go/pkg/mod/cache/vcs/f46ce2ae80d31f9b0a29099baa203e3b6d269dace4e5357a2cf74bd109e13339: exec: "bzr": executable file not found in $PATH
  1. An direct (or indirect) import can't be found:
$ cd $(mktemp -d) && go mod init x
go: creating new go.mod: module x
$ cat > main.go
package x

import _ "github.com/modsdemo/brokenimport"
$ go mod tidy
go: finding github.com/modsdemo/brokenimport latest
$ cat go.mod
module x

go 1.12

require github.com/modsdemo/brokenimport v0.0.0-20190506193535-03bc7b9ad6c8
$ go build
/home/leitzler/go/pkg/mod/github.com/modsdemo/brokenimport@v0.0.0-20190506193535-03bc7b9ad6c8/brokenimport.go:3:8: unknown import path "github.com/modsdemo/havebeenrenamed": cannot find module providing package github.com/modsdemo/havebeenrenamed           
  1. Using invalid GOPROXY setting
$ go mod init x
go: creating new go.mod: module x
$ cat > main.go
package x

import _ "github.com/modsdemo/notags"
$ GOPROXY=proxy.golang.org go mod tidy           # Missing https://, gives no error msg
$ cat go.mod
module x

go 1.12
$ GOPROXY=proxy.golang.org go build              # Missing https://
main.go:3:8: unknown import path "github.com/modsdemo/notags": cannot find module providing package github.com/modsdemo/notags                                  
$ go mod tidy                                    # Works as expected 
go: finding github.com/modsdemo/notags latest
$ go build                                       # Works as expected
$
@thepudds

This comment has been minimized.

Copy link

commented May 6, 2019

@leitzler That's great! Thank you for taking the time to do that.

@bcmills

This comment has been minimized.

Copy link
Member

commented May 28, 2019

@leitzler

are there any outstanding questions that need answers before solving this one?

We need to figure out which paths are missing the needed error checks, and how to add them without breaking things like go list -e.

@rogpeppe

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

FWIW I just ran across what I think was this issue when using go mod vendor and it took me ages to work out what the problem was. There was an "ambiguous import" issue in one module; running go mod vendor appeared to succeed but actually didn't vendor the modules that there was a problem with, so we just saw an error about a missing module when CI ran.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.