Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go directive is insufficiently documented for module authors to be able to make a decision about its value #30791

Open
dmitshur opened this issue Mar 12, 2019 · 14 comments

Comments

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Mar 12, 2019

Go 1.12 has added a go directive to the go.mod file. It's documented in the following places.

At https://golang.org/doc/go1.12#modules:

The go directive in a go.mod file now indicates the version of the language used by the files within that module. It will be set to the current release (go 1.12) if no existing version is present. If the go directive for a module specifies a version newer than the toolchain in use, the go command will attempt to build the packages regardless, and will note the mismatch only if that build fails.

Additionally, a relevant paragraph from https://golang.org/doc/go1.12#compiler:

The compiler now accepts a -lang flag to set the Go language version to use. For example, -lang=go1.8 causes the compiler to emit an error if the program uses type aliases, which were added in Go 1.9. Language changes made before Go 1.12 are not consistently enforced.

At https://golang.org/cmd/go/#hdr-The_go_mod_file:

go, to set the expected language version;

There is additional information available in issues, proposals, commit messages and code review comments, which I did not mention above. However, that information is not readily accessible to users.

It is very common for package authors to aim to ensure their Go packages can be used in multiple versions of Go. Sometimes it's just Go 1.12.x and Go 1.11.x (the current and previous releases). Other times the goal is to support even older versions of Go. This often includes running tests in CI with those versions of Go to ensure that the build and tests are successful. When these package authors add a go.mod file file to their repositories, they should be able to make a sensible decision about what go directive should be included.

Problem

I believe the current documentation is not sufficient for module authors to make a well-informed decision on whether to include the go directive in their go.mod files, and if so, what the value of the go directive should be.

I think we should try to improve documentation to resolve that. But first I want to make sure others agree that it's a problem.

(This is based on looking over the discussions that have happened around various PRs/CLs where the go directive is introduced, often by people who are very familiar with modules. Sometimes this happens more often due to #30790.)

/cc @ianlancetaylor @dsnet @bcmills @julieqiu @heschik @matloob

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Mar 12, 2019

I'm sure we should write better docs.

The basic guideline is fairly simple: a "go" directive 1.N means that the code in that module is permitted to use language features that existed in 1.N even if those features were removed in later releases of the language.

In other words, if the code compiles successfully at version 1.N, then it will continue to compile for all later versions of the language, even if it happens to use language features that were later removed.

To a first approximation, nobody should ever have to worry about the "go" directive. The only likely time you might need to set it manually is if you are copying some existing code to a new module, that code uses some obsolete language feature, and you don't have time to rewrite it. In that case you might want to set the "go" directive (using go mod edit -go) to the version used by the original module, thus permitting your new module to use the obsolete features. When writing new code you will presumably simply avoid the obsolete language features.

@bcmills
Copy link
Member

@bcmills bcmills commented Mar 13, 2019

a "go" directive 1.N means that the code in that module is permitted to use language features that existed in 1.N even if those features were removed in later releases of the language.

My understanding of #28221 is that it should not only permit the older features, but also prohibit newer features. Otherwise, there is little incentive for folks to upgrade to a language version beyond the removed features: they can just set go 1.11 and get every feature.

That understanding is based on this part of the proposal discussion:

suppose we had this feature for the 1.9 release, when type aliases were introduced to the language. Then passing -lang=go1.8 to the compiler would give a syntax error for any code that used a type alias.

That has an interesting interaction with build tags, though: if I set go 1.11 in my go.mod file, but then have a source file somewhere that specifies // +build go1.13, is that file permitted to use language features that were added in 1.13? Is it permitted to use language features that were removed in 1.13? Or should that file be ignored entirely, since the go 1.11 language did not include a go1.13 tag by default?

@bcmills
Copy link
Member

@bcmills bcmills commented Mar 13, 2019

I would argue that a // +build go1.13 file in a go 1.11 module should be ignored, since it would have been ignored when the module was originally tested under Go 1.11.

Or perhaps it should result in an explicit error, since the user may otherwise be confused that it has no effect.

@bcmills
Copy link
Member

@bcmills bcmills commented Mar 13, 2019

(CC @jayconrod)

@dmitshur
Copy link
Member Author

@dmitshur dmitshur commented Mar 14, 2019

Those are good questions. When looking for an answer, we should make sure that authors who wish to create Go modules that work with a wide range of Go versions (e.g., 1.8–1.12 as go-cmp does, 1.2–1.12 as bluemonday does, or all versions of Go as iter does) have a good way of selecting the value of the go directive. /cc @dsnet @buro9 @bradfitz

@dmitshur
Copy link
Member Author

@dmitshur dmitshur commented Mar 14, 2019

That understanding is based on this part of the proposal discussion:

suppose we had this feature for the 1.9 release, when type aliases were introduced to the language. Then passing -lang=go1.8 to the compiler would give a syntax error for any code that used a type alias.

I made similar conclusions based on that type alias example. When I asked Ian about it, his response was:

When I added the -lang option to cmd/compile, I had to have some way
to test the way the go tool handled it. So, I changed cmd/compile to
reject type aliases when using a -lang option before Go 1.9. But for
future backward-compatible changes, like for example the support for
0o number prefixes that we just added, I do not expect that
cmd/compile will be changed to reject the new features when building
for an older version.

[...] it is not intended to be taken as guidance for future work.

Which made me understand that the type alias example can be misleading when thinking about how the go directive value should be chosen.

@rsc rsc self-assigned this Apr 9, 2019
@rsc
Copy link
Contributor

@rsc rsc commented Apr 9, 2019

I will write some docs.

@rsc
Copy link
Contributor

@rsc rsc commented Apr 30, 2019

See #31747 too.

@kortschak
Copy link
Contributor

@kortschak kortschak commented May 21, 2019

I would argue that a // +build go1.13 file in a go 1.11 module should be ignored, since it would have been ignored when the module was originally tested under Go 1.11.

Or perhaps it should result in an explicit error, since the user may otherwise be confused that it has no effect.

On the basis of this, it seems like an odd behaviour for go test to attempt to mutate the go.mod file if there is no go directive line. If I have a go mod file that makes no claims about the language version, I would think under the model quoted, that should be respected.

Real world justification (reaching because of lack of docs): if a module is aimed at a number of go versions, how is that specified?

@thepudds
Copy link

@thepudds thepudds commented Jun 11, 2019

@kortschak, regarding your comment above about the interaction with build tags, also worth reading #31747 (comment) if you haven’t already.

@kortschak
Copy link
Contributor

@kortschak kortschak commented Jun 11, 2019

Thanks, @thepudds.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 12, 2019

Change https://golang.org/cl/181840 mentions this issue: cmd/go: add more docs for go directive in go.mod file

gopherbot pushed a commit that referenced this issue Jun 21, 2019
Updates #30791

Change-Id: I67efd7fd3b3a550428b16518bb27a3d81c178d28
Reviewed-on: https://go-review.googlesource.com/c/go/+/181840
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@Helcaraxan
Copy link
Contributor

@Helcaraxan Helcaraxan commented Sep 5, 2019

Although not entirely within the scope of this issue a related piece of feedback I have seen (and heard many times more) is that the meaning of the go directive, without even speaking of its exact semantics, tends to be poorly understood by gophers that are new to modules.

The overwhelming reaction is that people believe that it will enforce a specific Go toolchain to be used. As in running your local Go 1.13 toolchain on a module with a go 1.11 directive will result in the 1.11 toolchain being downloaded and invoked instead.

The only real documentation that can be found on the web appears to be the 1.12 release notes (already referenced in the issue description) which leaves enough space for people to come to such a conclusion.

Source (one amongst many): golangci/golangci-lint#605 (comment)

@thepudds
Copy link

@thepudds thepudds commented Sep 5, 2019

FWIW, I have seen a fair amount of confusion on this topic as well.

Now that the playground is running Go 1.13, here is a trivial example that illustrates one aspect of the expected behavior. Setting the Go language version to go 1.11 in the go.mod here causes a compilation error if you use signed integer shifts, which is a Go 1.13 language change:

https://play.golang.org/p/Mb7imkDHhoz

The current error is:

invalid operation: x << s (signed shift count type int, only supported as of -lang=go1.13)

In contrast, setting the Go language version in the go.mod to go 1.13 allows the use of signed integer shifts, as expected:

https://play.golang.org/p/VokbN-hHq02

That (I think?) illustrates how the go directive in a go.mod controls the version of the Go language that is used when building files within that module.


One particular aspect of confusion is people end up debating each other on random issues on open source projects regarding what the go value should be in the go.mod. (And it often is a "random" issue in the sense that someone is often sending a PR unrelated to modules that happens to also initially set or update the go version in the go.mod).

I think it would be helpful if there was a small bit of advice in the documentation on that piece, and I'm not sure "whatever cmd/go picks" is what most projects should be doing currently, or at least there is some nuance to the choice.

(If someone asked me, and if that someone does not yet want to use newer language features like signed integer shifts, then I'd probably personally recommend go 1.11 as a reasonable default choice currently for an open source project, given versions earlier than Go 1.11 don't even read the value. Alternatively, a recommendation could be to set it to the oldest version of Go that the open source project already knows it supports (e.g., go 1.8 or whatever). All that said, I'm not sure of the opinion of the core Go team).

erikdubbelboer added a commit to valyala/fasthttp that referenced this issue Sep 28, 2019
I chose 1.11 for the version in the go.mod file based on the information
in this issue: golang/go#30791
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.