Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/debug: document BuildInfo.Main.Version == "(devel)" #29228

Open
mark-rushakoff opened this issue Dec 13, 2018 · 23 comments
Open

runtime/debug: document BuildInfo.Main.Version == "(devel)" #29228

mark-rushakoff opened this issue Dec 13, 2018 · 23 comments

Comments

@mark-rushakoff
Copy link
Contributor

@mark-rushakoff mark-rushakoff commented Dec 13, 2018

What version of Go are you using (go version)?

$ go version
go version devel +571d93e977 Thu Dec 13 15:08:48 2018 +0000 darwin/amd64

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mr/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mr/go"
GOPROXY=""
GORACE=""
GOROOT="/Users/mr/gotip/src/github.com/golang/go"
GOTMPDIR=""
GOTOOLDIR="/Users/mr/gotip/src/github.com/golang/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mr/gomod/debug-module-version-demo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ct/bl4_z3g51ks8239_r2k07v_40000gn/T/go-build638578617=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Repro case is https://github.com/mark-rushakoff/debug-module-version-demo. It's a module, and its main.go contents are a simple use of debug.ReadBuildInfo:

package main

import (
	"fmt"
	"runtime/debug"

	_ "rsc.io/quote"
)

func main() {
	bi, ok := debug.ReadBuildInfo()
	if !ok {
		panic("couldn't read build info")
	}

	fmt.Printf("%s version %s\n", bi.Path, bi.Main.Version)

	for _, d := range bi.Deps {
		fmt.Printf("\tbuilt with %s version %s\n", d.Path, d.Version)
	}
}

What did you expect to see?

I expected to see the first line print github.com/mark-rushakoff/debug-module-version-demo version v0.0.0-20181213... when checked out at an arbitrary commit, or github.com/mark-rushakoff/debug-module-version-demo version v0.0.1 when checked out at tag v0.0.1. I tried both go run . and go build . && ./debug-module-version-demo but both cases printed (devel).

What did you see instead?

github.com/mark-rushakoff/debug-module-version-demo version (devel)
	built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
	built with rsc.io/quote version v1.5.2
	built with rsc.io/sampler version v1.3.0

Based on the behavior I've observed, it looks as though the main module returned by debug.ReadBuildInfo is hardcoded to (devel) for the main module, which I assume is intended behavior. If so, that's unfortunate for use cases like mycmd version to easily print the module version of the software being built; but it should be documented.

The current documentation at https://tip.golang.org/pkg/runtime/debug/#ReadBuildInfo does not mention (devel) in any way, nor does it mention any special behavior of the Main module.

/cc @hyangah since you're on git blame for src/runtime/debug/mod.go.

@mark-rushakoff mark-rushakoff changed the title runtime/debug: document BuildInfo.Main.Version == "devel" runtime/debug: document BuildInfo.Main.Version == "(devel)" Dec 13, 2018
@bcmills
Copy link
Member

@bcmills bcmills commented Dec 13, 2018

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag, although that might be misleading if some other module has a cyclic requirement specifying a higher version than the tag.

@bcmills bcmills added the modules label Dec 13, 2018
@bcmills bcmills added this to the Go1.12 milestone Dec 13, 2018
@stapelberg
Copy link
Contributor

@stapelberg stapelberg commented Dec 22, 2018

+1.

I thought being able to print the program’s main module’s version was the whole point of embedding build information into the binary.

@mark-rushakoff
Copy link
Contributor Author

@mark-rushakoff mark-rushakoff commented Jan 7, 2019

#26404 (comment) is some separate uncertainty about when and why (devel) shows up. There are some linked issues there that seem to paint a fuller picture, but I don't have time to go through them right now.

I think this issue is still fully separate from #26404, as this is just about documenting (devel).

@mark-rushakoff
Copy link
Contributor Author

@mark-rushakoff mark-rushakoff commented Jan 24, 2019

It looks like one way to get the version to show up, is to make a new module referring to the one you actually want to build; and then build the target module.

bash-3.2$ gotip mod init ignore
go: creating new go.mod: module ignore

bash-3.2$ gotip get github.com/mark-rushakoff/debug-module-version-demo
go: finding github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: downloading github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: extracting github.com/mark-rushakoff/debug-module-version-demo v0.0.1

bash-3.2$ gotip build !$
gotip build github.com/mark-rushakoff/debug-module-version-demo

bash-3.2$ ./debug-module-version-demo
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
	built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
	built with rsc.io/quote version v1.5.2
	built with rsc.io/sampler version v1.3.0
@mholt
Copy link

@mholt mholt commented Apr 19, 2019

@bcmills

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag,

Or even just any of the current VCS state. For Caddy's builds, we use tag if clean, or if not clean: nearest tag, commit SHA, and build date (although I can understand why build date is a bad idea if seeking reproducible builds).

For now, @mark-rushakoff's workaround is quite handy.

+1 to this though, some version info on the main module would be immensely helpful.

@hyangah
Copy link
Contributor

@hyangah hyangah commented Aug 29, 2019

As observed in #33926, currently it's possible to end up building a binary with the same main module version but with different dependency versions. People who intend to use the main module version as the binary version should be careful.

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 3, 2019

People who intend to use the main module version as the binary version should be careful.

Using the “main module” version as the binary version (in the sense of https://tip.golang.org/cmd/go/#hdr-The_main_module_and_the_build_list) should be fine as long as the module doesn't have filesystem-local replace directives.

Unfortunately, that is not the version that the Main field indicates today (see #33975).

So I agree that the BuildInfo.Main field itself is not really reliable as an indication of the overall configuration in use, even when that version is not (devel).

@mholt
Copy link

@mholt mholt commented Sep 13, 2019

Will this be a high priority for Go 1.14? It could really ease the burden of build-from-souce workflows. Like, a lot.

mholt added a commit to caddyserver/caddy that referenced this issue Sep 13, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Sep 16, 2019

@mholt, this is unlikely to happen in Go 1.14.

(We have a pretty full slate of issues already, and any time the git command is involved there are thorny security considerations that must be addressed with care.)

@mback2k
Copy link

@mback2k mback2k commented Sep 16, 2019

Is using the git command really a requirement of this feature? Wouldn't it be possible to get the relevant metadata directly from .git/HEAD?

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 16, 2019

@mback2k, .git/HEAD tells us the commit id of the HEAD revision. It does not tell us:

  1. Whether the working directory is even in a Git repo, or perhaps in some other VCS nested within one.
  2. Where to find the .git directory.
  3. Whether the working tree contains ignored files or other differences from the commit indicated by .git/HEAD.
@mark-rushakoff
Copy link
Contributor Author

@mark-rushakoff mark-rushakoff commented Sep 16, 2019

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel). Whether it can be correctly set to a particular git sha seems to be a different topic in my opinion -- perhaps that should be another issue?

@mholt
Copy link

@mholt mholt commented Sep 16, 2019

@mark-rushakoff

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel)

BuildInfo.Main.Version always returns "(devel)".

I oppose documenting this, because I believe this is a bug: BuildInfo.Main.Version always returns "(devel)" even when the main module is versioned at HEAD.

Documenting that it always returns "(devel)" makes it appear as correct behavior and makes it hard or impossible to change later.

Let's fix the actual problem, not document the bug as expected behavior.

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 18, 2019

@mholt, we can document the current behavior in a way that does not promise that behavior for all time.

For example, we can document that Buildinfo.Main.Version is (devel) when the go command does not know the version of the main module, without promising that it will never be changed to discover that version.

@hyangah
Copy link
Contributor

@hyangah hyangah commented Sep 19, 2019

nit:

Buildinfo.Main.Version is (devel) when the go command does not know the version of the main module, without promising that it will never be changed to discover that version.

The intention was that the BuildInfo.Main is the module that contains the main package, not the main module described in https://golang.org/cmd/go/#hdr-The_main_module_and_the_build_list
issue 33975.

I hope correcting the documentation error will help explaining this BuildInfo.Main.Version == "(devel)" case.

@mholt
Copy link

@mholt mholt commented Sep 19, 2019

Yeah, I think what we want is the version for the "main module" - the module that contains the main package.

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
sh0rez added a commit to jsonnet-bundler/jsonnet-bundler that referenced this issue Feb 28, 2020
The go-modules approach is a bit broken: golang/go#29228
To resolve that, we now compute that version using `git describe` and
set it using `ldflags`.

While this is not as nice/builtin as the other approach, it at least
works all the time and across go versions.

Furthermore, I did refactor the Makefile in preparation for an AUR
package that builds from source and for building release artifacts in CI.
@tv42
Copy link

@tv42 tv42 commented Jun 8, 2020

While the main module still shows up as "(devel)" for just go build, the full workaround from #29228 (comment) seems to be no longer necessary. Instead, you can just go get things outside of a current module:

$ GOBIN=$PWD GO111MODULE=on go get github.com/mark-rushakoff/debug-module-version-demo
$ ./debug-module-version-demo 
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
        built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
        built with rsc.io/quote version v1.5.2
        built with rsc.io/sampler version v1.3.0
@ldemailly
Copy link

@ldemailly ldemailly commented Aug 7, 2020

How would one build a binary (go service in docker image for instance) from ci/docker from a tag to get said tag in the binary being built?
(the source tree is already there (from ci source checkout step) and being copied (dockerfile COPY . .), not downloaded)

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Aug 10, 2020

As far as I know, there's no good way to do this (and it appears this issue is somewhat codifying that we'll never know). I'd love to be able to access my current version as it would appear had I created some other module and used @<commit ID> with a tidy, such that I could use that version in other places.

For my docker builds, I pass something in via --build-arg=version=..., then include it by doing things like -ldflags="-X path.to/version.verson=${version}".

@ldemailly
Copy link

@ldemailly ldemailly commented Aug 10, 2020

I use the -ldflags="-X path.to/version.verson=${version}" too but I wanted the 2 versions to be in sync
for now I have a super ugly workaround which is to go get ...@tag but it's quite annoying specially on private repos to redownload something that is already there (it forces some very unsightly experimental docker support for ssh in docker etc)

putting it here in case it helps someone else:

DOCKER_BUILDKIT=1 docker build --ssh default ...

and dockerfile with

# syntax=docker/dockerfile:experimental
FROM golang as build
ARG VERSION
...
RUN git config --global url.ssh://git@somePrivateRepo/.insteadOf https://somePrivateRepo/
RUN mkdir -m 700 /root/.ssh && ssh-keyscan -4 -t rsa somePrivateRepo > /root/.ssh/known_hosts
RUN --mount=type=ssh go get -v -a -ldflags -s ... somePrivateRepo/pkg/path@${VERSION}

it would be so much easier for go build to have the option to check with git and/or some sort of --trust-me-the-version-is xxx flag

alexmv added a commit to alexmv/smokescreen that referenced this issue Aug 31, 2020
The `$Id$` which can be auto-expanded in files via the `ident`
attribute does not function the same as the old CVS `$Id$` keyword.
In CVS, the keyword expansion was updated on every commit, to contain
the current commit id.  In git, it is expanded with the identifier of
the _blob it is found in_.  That is, previous to this commit, the
`$Id$` (and hence the reported "version hash") was
f220e479c5d8d85c7b753e95dc5fe0b67bbfbd38 -- and had been since the
file was changed last, in f386360.

Remove the misleading hash, and attendant git attributes file.  It
will frequently mislead callers that the version has not changed
when, in reality, it has.

Git itself does not have a way to embed "the current commit hash" into
a file in a way that is updated whenever the commit changes.  Nor does
Go natively have a way to embed it into the binary at build time,
though this may change in the future[1].

As alluded to in that ticket, most projects elect to pass in the
build-time commit information via something like:
```
GIT_COMMIT=$(git rev-parse HEAD)
go build -ldflags "-X main.gitCommit=$(GIT_COMMIT)"
```

However, smokescreen does not currently have any build system external
to `go build` which could embed the above logic.  Rather than choose a
build system and introduce a new dependency, remove the misleading
hash entirely.

[1] golang/go#37475 and also
    golang/go#29228
@Fryuni
Copy link

@Fryuni Fryuni commented Jan 6, 2021

Wouldn't adding a -modversion flag on go build solve this problem?
It is way more clear than -ldflags="-X main.version=${version}". Most (id not all) CI services include the tag and the commit hash as an environment variables, so it can be used easily and cleanly for building.

Is there a separate issue for this? I couldn't find it. I implemented it in the middle of writing this comment... here it is

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 6, 2021

Change https://golang.org/cl/281912 mentions this issue: cmd/go: add -modversion flag setting main module version on debug.ReadBuildInfo

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jan 6, 2021

@Fryuni The main module doesn't have a version. If it did, MVS would need to work differently in the case where another module requires a higher version of the main module. It may also be misleading if a binary appears to be built from a published version (authenticated by the sumdb) but was actually built from a local checkout with modifications.

Go 1.16 supports building and installing with go install pkg@version (#40276). When that command is used, there is no main module.

#37475 is also an accepted but unimplemented proposal to automatically stamp VCS information into binaries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.