cmd/dist, x/build/cmd/golangbuild: switch to a devel version format that satisfies go/version.Lang and go/version.IsValid in more cases

[dmitshur]

golangbuild currently generates VERSION files with strings like this, a behavior that the Go build system had for a while:

devel <commit>
devel <change>/<patchset>

Which is broadly based on make.bash's auto-detection logic used to construct a version when a VERSION file specifying it isn't present, which generates version strings in a similar format (the "go1.X-" prefix being new as of CL 264938):

devel go1.X-<short commit> <date>

That format satisfies the property that a "devel" substring is present, but not the property that go/version.IsValid is true and go/version.Lang works on it.

As motivated in #73369 (comment), I suggest we change to the following format that have all of the aforementioned properties:

go1.X-devel_<commit>
go1.X-devel_<change>_<patchset>

Where go1.X corresponds to the language version specified in internal/goversion.Version, something golangbuild has easy access to. Something to consider including .0 like go1.X.0-devel_..., but given the first pre-release version (e.g., go1.25rc1) leaves it out, maybe its fine to leave it out too. Since this is just a development version which isn't tagged or published anywhere, changing it if we find a need to shouldn't be too expensive.

There might be test cases or other code that relies on the current format, such as checking for "devel" prefix (or its absence), that'd need to be updated as part of this.

golangbuild.maybeUpdateVersionFile diff

diff --git a/go/src/infra/experimental/golangbuild/buildmode.go b/go/src/infra/experimental/golangbuild/buildmode.go
index 0c1bcf13f5..0452c72822 100644
--- a/go/src/infra/experimental/golangbuild/buildmode.go
+++ b/go/src/infra/experimental/golangbuild/buildmode.go
@@ -123,8 +123,9 @@ func getGo(ctx context.Context, spec *buildSpec, goName, goroot string, goSrc *s
 //   - If the input property "version_file" is present, it always overwrites
 //     the VERSION file with that value.
 //   - If no VERSION file is present or the VERSION file is empty, then the
-//     VERSION file is written with contents `devel <commit>` or
-//     `devel <change>/<patchset>` (existing behavior).
+//     VERSION file is written with contents `go1.X-devel_<commit>` or
+//     `go1.X-devel_<change>_<patchset>` (this matches the version that
+//     make.bash auto-infers from git, other than <date> being left out).
 //   - If a VERSION file is present AND the first line matches `go1.X.Y`,
 //     then only the first line is kept, and we append `-devel_<commit>` or
 //     `-devel_<change>_<patchset>` to the version.

CC @golang/release, @prattmic.

[gabyhelp]

Related Issues

• proposal: start using semantic versions for Go releases #32450 (closed)
• cmd/go: include Go's base version in its devel version strings #44960 (closed)
• proposal: cmd/go: Use `git describe` output as canonical version string #21209 (closed)
• x/build: builders write a custom VERSION file with a simple "devel {hash}" format when testing Go #42221 (closed)
• go build should remove semantic versioning when determing file name #29643 (closed)

Related Code Changes

• cmd/dist: include "go1.x-" in devel go version strings
• cmd/compile: accept -lang=go1 as -lang=go1.0
• [release-branch.go1.23] go/types, types2: only use fileVersion if 1.21 or greater
• cmd/go: introduce the GOVERSION env variable

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[prattmic]

I am perfectly happy with the go1.X-devel... format, but I feel fairly strongly that golangbuild should generate a version in the same format used by cmd/dist (https://cs.opensource.google/go/go/+/master:src/cmd/dist/build.go;l=439;drc=b16c04f43993436f24b1e4155a4652193eb1b90c) for standard builds [1].

So this change to golangbuild sounds good if we make the same change to cmd/dist, otherwise it seems we should change golangbuild to be in line with dist.

[1] Why does golangbuild need to generate a VERSION file at all?

[dmitshur]

Yes, agreed, we should apply this to cmd/dist too. Retitled to capture that too.

As for why golangbuild (and coordinator, previously) generates a VERSION file, it does it for a few related reasons:

• cmd/dist's findgoversion logic that creates a devel version with the commit hash can only work if 1) .git directory is present, and 2) a git binary is available. This might not be the case for all builders; they might opt to test a GOROOT tree without the .git directory (if that's faster) and some builders may not have a git binary in PATH.

• Next, it's meant to be a (probably small) optimization: the build system already did the work of figuring out what the commit hash (or change ID/patch set number) it's testing, so it can more quickly write that information down into the VERSION file to prevent the somewhat-expensive findgoversion logic from needing to re-compute the same thing.

• Finally, sometimes the build system has more information readily available, like knowing that what the CL number and patch set is for pre-submit runs. Another aspect is that the current release process leaves a stale VERSION file behind on release branches, which would be more misleading if the build system re-used as-is.

[xnox]

There is also need to identify downstream golang toolchains, their vendor, and patch level. In a way that is encoded in the binaries built by those toolchains. And then distributed to another party and be able to link it all back.
See the dismissed proposal at #73194 which talks about that. Partially related.

I wonder, if it is best to keep the top level version as short as possible, and still semver compliant. And have more verbose and structured information about the toolchain in the buildinfo, which propagates to the binaries.

$ go version -m somegobinary 
somegobinary: go1.25-devel
...
	build	toolchain.vcs=git
	build	toolchain.vcs.revision=1c075e12cf123600b57cfb8bb66b9aa107a37e08
	build	toolchain.vcs.time=2025-04-11T19:28:28Z
	build	toolchain.vcs.modified=false
	build	toolchain.stable=false (or .released or .devel or .snapshot or whatever name makes most sense)

Similarly this would be very useful for downstreams to embed toolchain.vendor and toolchain.vendor-version. Very often many parties are involved as to who creates the golang toolchain, who builds the binaries, by whom/where they are deployed. If one uses a linux distribution golang fork with backported CVEs today, builds go binaires, puts them into container, and pushes it to dockerhub or as a UBI, and somebody else deploys them, it is impossible to tell from the go binary alone which golang was used and if it was 1.23.3 from upstream, or a particular linux vendor, which vendor, and if it has backported CVE fixes.

Not too dissimilar with being able to identify snapshot builds, and pre-release builds.

If you can, please consider adding structured data about the toolchain to the buildinfo that persists in binaries beyond the top level "go1.x" string.

[dmitshur]

@xnox Including additional fields about the toolchain and its possible modifications in the build information (in addition to the go version string) like that is an interesting idea. It has some connection with the idea of having go mod verify also verify the toolchain against those that can be downloaded, which came up but I'm not sure if it has a tracking issue. However, I suspect you'll run into limitations of how useful it can be: many installed toolchains will not have VCS information, and if they do, toolchain.vcs.revision doesn't communicate much if the commit hash is an unpublished local commit. There may be more open questions, and it's out of scope for this issue, so I suggest you file a separate issue for it if you'd like to pursue it further.

[gopherbot]

Change https://go.dev/cl/667955 mentions this issue: cmd/dist: add "devel" substring check to isRelease computation

[gopherbot]

Change https://go.dev/cl/668015 mentions this issue: cmd/dist: move "devel" substring in git-inferred development version

[gopherbot]

Change https://go.dev/cl/668355 mentions this issue: cmd/go, cmd/internal/objabi: detect "devel" version by substring

[gopherbot]

Change https://go.dev/cl/669016 mentions this issue: go/analysis/unitchecker: update TestVetStdlib's skip for "devel" move

[mvdan]

I was struggling with this just last week - thanks so much for this change!