Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: Make it easier to embed compiled git SHA into binary #22147

Closed
kevinburke opened this Issue Oct 5, 2017 · 10 comments

Comments

Projects
None yet
6 participants
@kevinburke
Copy link
Contributor

kevinburke commented Oct 5, 2017

If I compile Go from source and run go version, I get output like this:

go version devel +475d92ba4d Thu Oct 5 10:50:18 2017 +0000 darwin/amd64

I would love to get the same output in my programs. Specifically, I'm trying to debug a reported error in a program I wrote and it would be a lot easier if I knew which SHA they were running.

It's easy in the Go build tool because it can specify what code runs during the build process. From findgoversion():

	// Otherwise, use Git.
	// What is the current branch?
	branch := chomp(run(goroot, CheckExit, "git", "rev-parse", "--abbrev-ref", "HEAD"))

	// What are the tags along the current branch?
	tag := "devel"
	precise := false

	// If we're on a release branch, use the closest matching tag
	// that is on the release branch (and not on the master branch).
	if strings.HasPrefix(branch, "release-branch.") {
		tag, precise = branchtag(branch)
	}

	if !precise {
		// Tag does not point at HEAD; add hash and date to version.
		tag += chomp(run(goroot, CheckExit, "git", "log", "-n", "1", "--format=format: +%h %cd", "HEAD"))
	}

I have options for replicating this behavior, none very good:

  • Use linker flags: This blog post describes compiling Go with -ldflags -X in such a way that you can embed the SHA at compile time. https://medium.com/@joshroppo/setting-go-1-5-variables-at-compile-time-for-versioning-5b30a965d33e

    I can do this of course, but I'm not sure it's feasible or possible to ask all of my users to remember to do this when they compile my program. I can add a Make target or something to automate the instructions but that's something they have to remember to do too.

    I could release compiled libraries but that's infeasible in a company environment where tip is changing rapidly and users are used to compiling from source.

  • Tag a new release every time I add a commit to the program. I wrote this tool to do that: github.com/Shyp/bump_version. Generally though it's not practical to tag every single commit, the same reason there aren't tags for every commit between Go 1.8.3 and Go 1.9.0.

It would be nice if there was some API for detecting or automatically embedding the git SHA in the binary so I can access it at runtime, and print it out when users of my program run <mybinary> version or <mybinary> env, the same way the Go team found this information useful and found a way to embed it while the program was being compiled.

@mvdan

This comment has been minimized.

Copy link
Member

mvdan commented Oct 5, 2017

I have thought the same in the past, and I agree that the two current workarounds you listed are suboptimal. Let me play the devil's advocate, however:

  • What happens when there is no source control, or when there is but it's not Git?
  • Should this affect all packages, or just main ones?
  • Would this "version hash" be taken into account to know if a package needs rebuilding?
  • Could renaming a tag or removing .git break a reproducible build?
@kevinburke

This comment has been minimized.

Copy link
Contributor Author

kevinburke commented Oct 13, 2017

Here are places where this field would come in handy. I will update this list as I come across more examples.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

ianlancetaylor commented Mar 29, 2018

@kevinburke As you say above, you can of course use -ldflags=-X today. But it seems that you aren't happy with that, because you want it to be automatic. But I really don't think we want any such thing to be automatic; as @mvdan says, what if the build doesn't use git? So it has to be under the control of some option--in which case why not just use -X?

Do you have any suggestions for how this could work in practice?

Also, one of your reasons is so that you can know which git revision a user is running. But their git SHA doesn't necessarily mean anything to you, depending on how the sources have moved around. So what you want to know is actually the git SHA that they started from. You could do that by storing the git SHA in your own repo, somehow, though I agree that that too has problems.

Does the new go.mod file used by vgo help in any way here?

@fd0

This comment has been minimized.

Copy link

fd0 commented Mar 30, 2018

A small note explaining what we do in practice right now for restic: we're using a small program called build.go. It allows compiling restic from a checkout of the repo and from the extracted release tar.gz without having to setup a GOPATH and only depending on the Go compiler, so it's very end-user compatible. It requires vendoring all dependencies though.

In addition (that's why I'm mentioning it here), it embeds the Git version if available and the contents of the VERSION file in the source into the binary via ldflags, so restic version can print some nice output:

restic 0.8.3 (v0.8.3-109-gd49502a5) compiled with go1.10 on linux/amd64

The first string 0.8.3 is the content of the file VERSION, followed by the output of git describe.

It turned out to be very helpful when users report issues and bugs, we request this information in our issue template.

To everybody having a similar requirements, I encourage you to give build.go a try. Just copy it to your project, configure the section at the top and it should mostly work. To all others: please ignore this comment ;)

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 18, 2019

As of Go 1.12, you can get the versions of all modules compiled into the binary (in module mode) using runtime/debug.BuildInfo.

If the version in use is a pseudo-version, it also includes the first dozen or so characters of the VCS commit ID.

@bcmills bcmills closed this Jan 18, 2019

@fd0

This comment has been minimized.

Copy link

fd0 commented Jan 19, 2019

@bcmills how does this work? I've built a small test program, running it with Go 1.12beta2 shows the versions for all dependencies, but not for the main module (when compiled from a checkout of the repo):

$ go version
go version go1.12beta2 linux/amd64

$ unset GOPATH
$ export GO111MODULE=on

$ go build
[...]

$ ./godebugbuildinfo
test program for debug.BuildInfo
path: github.com/fd0/godebugbuildinfo
  main: debug.Module{Path:"github.com/fd0/godebugbuildinfo", Version:"(devel)", Sum:"", Replace:(*debug.Module)(nil)}
  deps:
    &debug.Module{Path:"github.com/fatih/color", Version:"v1.7.0", Sum:"h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=", Replace:(*debug.Module)(nil)}
    &debug.Module{Path:"github.com/mattn/go-colorable", Version:"v0.0.9", Sum:"h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4=", Replace:(*debug.Module)(nil)}
    &debug.Module{Path:"github.com/mattn/go-isatty", Version:"v0.0.4", Sum:"h1:bnP0vzxcAdeI1zdubAl5PjU6zsERjGZb7raWodagDYs=", Replace:(*debug.Module)(nil)}

I can only see (devel) as the version for the main module, even when I'm adding a version tag in the repository (like v1.0.0).

Is there anything I'm missing?

@dominikh

This comment has been minimized.

Copy link
Member

dominikh commented Jan 19, 2019

I think this is affected by #29228. I'll reopen this issue until that's resolved.

@dominikh dominikh reopened this Jan 19, 2019

@fd0

This comment has been minimized.

Copy link

fd0 commented Jan 19, 2019

Thanks!

@mvdan

This comment has been minimized.

Copy link
Member

mvdan commented Jan 19, 2019

I'm pretty sure this is now being tracked under #29814, which brings up all the implementation details. I believe @bcmills was trying to consolidate all the older related issues into a single one with a proposed solution.

@dominikh

This comment has been minimized.

Copy link
Member

dominikh commented Jan 19, 2019

Sounds good enough to me, now that these two issues reference each other.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.