This discussion is about backward compatibility, meaning new versions of Go compiling older Go code. For the problem of old versions of Go compiling newer Go code, see this other discussion about forward compatibility.
Go 1 introduced Go's compatibility promise, which says that old programs will by and large continue to run correctly in new versions of Go. There is an exception for security problems and certain other implementation overfitting. For example, code that depends on a given type not implementing a particular interface may change behavior when the type adds a new method, which we are allowed to do.
We now have about ten years of experience with Go 1 compatibility. In general it works very well for the Go team and for users. However, there are also practices we've developed since then that it doesn't capture (specifically GODEBUG settings), and there are still times when users’ programs break. I think it is worth extending our approach to try to break programs even less often, as well as to explicitly codify GODEBUG settings and clarify when they are and are not appropriate.
As background, I've been talking to the Kubernetes team about their experiences with Go. It turns out that Go's been averaging about one Kubernetes-breaking change per year for the past few years. I don't think Kubernetes is an outlier here: I expect most large projects have similar experiences. Once per year is not high, but it's not zero either, and our goal with Go 1 compatibility is zero.
Here are some examples of Kubernetes-breaking changes that we've made:
These kinds of behavioral changes don't only cause pain for Kubernetes developers and users. They also make it impossible to update older, long-term-supported versions of Kubernetes to a newer version of Go. Those older versions don't have the same access to performance improvements and bug fixes. Again, this is not specific to Kubernetes. I am sure lots of projects are in similar situations.
As the examples show, over time we've adopted a practice of being able to opt out of these risky changes using
Other important compatibility-related GODEBUG settings include:
Programs that need one to use these can usually set the GODEBUG variable in
Another problem with the GODEBUGs is that you have to know they exist. If you have a large system written for Go 1.17 and want to update to Go 1.18's toolchain, you need to know which settings to flip to keep as close to Go 1.17 semantics as possible.
I believe that we should make it even easier and safer for large projects like Kubernetes to update to new Go releases. In particular, I think we should probably:
GODEBUG may not be the mechanism I'd design today, but it's what we have and it doesn't seem bad enough to be worth adding a second way, so I'm going to assume it stays. Then the two things we need are (1) a way to set individual GODEBUG defaults in package main, and (2) a way to make the default GODEBUGs match an earlier version of Go.
For (1), I am thinking about something like
in any package main source file. These would be pulled out by the go command and linked into the binary for processing at startup (before any Go code runs). The GODEBUG environment variable would still override these, of course.
For (2), I am thinking about having the go line in the go.mod of package main's module, which already defines the exact language semantics of package main's Go source files, also define the default GODEBUG settings. So if package main's go.mod says
As noted above, some GODEBUG settings will stick around forever (for example, execerrdot=1, netdns=go), while we will want to retire others. When a newer version of Go is compiling code written for an older version, if a GODEBUG has been retired, the behavior will depend on whether it is named explicitly (as in (1)) or implicitly (as in (2)). If a retired GODEBUG is mentioned explicitly, the build should fail. If it is only implied by the earlier Go version, then build should succeed, on the assumption that the vast majority of programs that say “go 1.17” are saying it because they were written in that era, not because they require support for SHA1 certificates.
I think these two changes would go a long way toward making it even easier and safer to update to new Go toolchains, because it separates the update from the riskiest behavior changes and makes those changes easy to temporarily opt out of and also to debug.
Note that this mechanism would be inappropriate to use for new, incompatible features, because the settings in package main are affecting the entire binary, and it is unlikely that all the packages in a large program would agree on which version of a large feature they want. In contrast, for “extra-backwards compatibility shims” like these, especially in the context where you're keeping older code running, affecting the whole binary is appropriate and does not cause problems.
One thing people ask occasionally is whether Go will ever add LTS (long-term support) releases. I've always thought of Go 1 as the LTS release of Go, but subtle, necessary breaking changes like the ones listed above contradicted that idea. Being able to use a new Go toolchain but still get more faithful “old” semantics in these cases brings us much closer to Go 1 as LTS.
There is a question about what to do if go.mod says a newer version of Go than the toolchain being used for the build. In that case the older toolchain does not know what the newer version does differently. I am filing a separate discussion about this forward compatibility problem.
This is a discussion, not a proposal. I haven't implemented this nor even worked out all the implications. I'm curious what people think and what concerns they have. Thanks!
Beta Was this translation helpful? Give feedback.