Problems updating a package using go get with GO111MODULE=on

[carlca]

FAO: @rsc

What version of Go are you using (go version)?

go version go1.11rc1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="darwin"

What did you do?

I'm running with GO111MODULE=on.

I used go get -u github.com/dave/wasmgo to install a package for the first time yesterday. I didn't have to use go mod init at all, and it successfully installed to ~/code/go/pkg/mod/github.com/dave/wasmgo@v0.0.0-20180816095012-16077efc512b.

The author has made changes to the package, so I ran go get -u github.com/dave/wasmgo again, but I got the error go: cannot find main module; see 'go help modules'.

I was able to fix this by running go mod init go in my ~/go folder to create a dummy go.mod file - go get -u github.com/dave/wasmgo then ran successfully.

It then occurred to me that littering my machine with dummy go.mod files was probably not a good idea, so I navigated to the aforementioned ~/code/go/pkg/mod/github.com/dave/wasmgo@v0.0.0-20180816095012-16077efc512b folder and ran go mod init github.com/dave/wasmgo. I got the error...

go: creating new go.mod: module github.com/dave/wasmgo
go: open /Users/carlca/code/go/pkg/mod/github.com/dave/wasmgo@v0.0.0-20180816095012-16077efc512b/go.mod: permission denied

Despite the first line of the error message, go.mod has not been created.

How do I fix this? I cannot believe I have to create a go.mod file in a higher level, generic (oops, the "G" word!) folder. Any ideas?

What did you expect to see?

I expected to be able to run go mod init github.com/dave/wasmgo in the ...\v0.0.0-20180816095012-16077efc512b folder

What did you see instead?

Permission denied message as explained above.

[carlca]

UPDATE:
I was able to get go get -u github.com/dave/wasmgo to run by running go mod init github.com/dave, but this seems like an unsatisfactory solution. It lacks intellectual coherence and justification!

[thepudds]

Hi @carlca, I'm not an expert, but a couple quick comments.

One comment is that when you are running a command like go get in module-aware mode, the resulting behavior still depends which module you are currently "in" as the current module, which means it depends on where you are in your filesystem hierarchy when you execute the go get command (given part of a module's definition is based on the location of the go.mod file in the filesystem hierarchy). For example, I believe that is how go get knows which go.mod file to update when you have more than one module on your system.

That's a bit informal, but here is a more formal statement from the HTML documentation on tip.golang.org:

https://tip.golang.org/cmd/go/#hdr-Defining_a_module

A module is defined by a tree of Go source files with a go.mod file in the tree's root directory. The directory containing the go.mod file is called the module root. Typically the module root will also correspond to a source code repository root (but in general it need not). The module is the set of all Go packages in the module root and its subdirectories, but excluding subtrees with their own go.mod files.

Also, I believe ~/code/go/pkg/mod includes an on-disk cache that is more or less transparently managed by the go command, so normally you wouldn't need to navigate there manually.

In case this might be helpful, there's an a quick write-up here, along with some short "how to" sections that follow:
https://github.com/golang/go/wiki/Modules#new-concepts
https://github.com/golang/go/wiki/Modules#how-to-define-a-module

edit: couple small clarity improvements

[thepudds]

Hi @carlca, also, looking at this more, it seems github.com/dave/wasmgo is a command/binary? So it sounds like part of your question then might be how do you install a command/binary via go get when running in module-aware mode?

@gopherbot, please add label modules

[thepudds]

@carlca You might be hitting a variation of #24250.

One snippet from there:

This isn't blocking Go 1.11. The main reason being that if GO111MODULE=auto (the default) then we can't reasonably do anything with modules when outside a go.mod tree. That's the opt-in. And inside a go.mod tree this already works.

...which gets back to some of my earlier comments about the behavior is currently expected to be different depending on where you are in the filesystem when you run go get (including whether or not you are "inside" a module by being inside a filesystem tree that has a go.mod file, etc.)

[carlca]

@carlca You might be hitting a variation of #24250.

It looks like it. Oh well, my solution of a dummy go.mod file in the folder above will have to do for now.

[carlca]

Hi, @thepudds, would you mind having a look at the series of tweets I've just had involving Russ Cox. They are at

Me: https://twitter.com/carlcaulkett/status/1034516890373304320
Me: https://twitter.com/carlcaulkett/status/1034560669234933761
Russ: https://twitter.com/_rsc/status/1034625354172981248
Me: https://twitter.com/carlcaulkett/status/1034712098381746176
Me: https://twitter.com/carlcaulkett/status/1034718306387804160

I cannot work out why Russ is saying that "module based go get works fine for me" unless he is running a beta version of 1.12 and just hasn't mentioned it! Otherwise, I fear it's a bit of a politician's answer!

[bcmills]

If you are running with GO111MODULE=on, then you must be inside a module to use go get until #24250 is addressed.

Even then, you will probably still need to be inside a module if the thing you are getting is not a main binary: otherwise, the result is somewhat nonsensical. (You're telling the go command to add some package or module to the requirements of the current module, but without a current module!)

[bcmills]

I navigated to the aforementioned ~/code/go/pkg/mod/github.com/dave/wasmgo@v0.0.0-20180816095012-16077efc512b folder and ran go mod init github.com/dave/wasmgo.

GOPATH/pkg/mod is a cache of downloaded, immutable modules — it's not a working tree like GOPATH/src. The fact that you can't make changes there yourself is intentional.

[bcmills]

Closing as a duplicate of #24250. You must be in a module to go get a new dependency for that module. You will not need to be in a module to go get a command-line tool, but that's exactly what #24250 tracks.

[thepudds]

Hi, @thepudds, would you mind having a look at the series of tweets I've just had involving Russ Cox. > They are at
Me: https://twitter.com/carlcaulkett/status/1034516890373304320
...
I cannot work out why Russ is saying that "module based go get works fine for me" unless he is running a beta version of 1.12 and just hasn't mentioned it! Otherwise, I fear it's a bit of a politician's answer!

@carlca sorry for the late reply, but given you had addressed that question to me, just wanted to briefly comment that I looked at those tweets, and I don't think Russ was giving a politician's reply. ;-)

go get does work with modules. It just has a some rough edges in 1.11, many of which are due to the opt-in nature of modules and wanting to get some real-world experience with modules prior to finalizing go get's behavior outside of modules (e.g., the #24250 issue Bryan linked to), at least as far as I understand.

I think you've seen this, but some of the options for some of those rough edges are mentioned in some of the FAQs on the modules wiki, including this shell script by rogpeppe.

So in short, go get works in 1.11 with modules, but still has some improving to do.

[carlca]

Okay. Suppose we take a github package rivo/tview, a promising text mode cli package. If you set GO111MODULE=on, go get github.com/rivo/tview does not work.

My question, then, is: if one uses GO111MODULE=auto does one get the semantic versioning benefits of Go Modules? Presumably we wouldn't if we set GO111MODULE=off, but then that would defeat the whole object of the exercise.

Certainly, the github.com repository in question does not seem to have either go.mod or go.sum present, but if one runs go get github.com/rivo/tview with GO111MODULE=auto, the resultant set of files does include go.mod and go.sum, although they show creation dates of 26-Oct 2018, which suggests that they were already part of the repo. Yet the page at https://github.com/rivo/tview does not show these files.

I feel like I am still missing a crucial bit of understanding here...

UPDATE: I've just read the FAQs and looked at the shell script. Had I seen those a few weeks or months ago, then perhaps this whole episode might not have been so wearisome!