Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: export module information for binaries programmatically #26404

Closed
bcmills opened this issue Jul 16, 2018 · 14 comments

Comments

Projects
None yet
8 participants
@bcmills
Copy link
Member

commented Jul 16, 2018

https://research.swtch.com/vgo-repro says:

When we do integrate versions into the main Go toolchain, we will add APIs to access this information from inside a running binary, just like runtime.Version provides access to the more limited Go version information.

I didn't see an issue filed to track that, so I'm filing it now.

@bcmills bcmills added this to the Go1.12 milestone Jul 16, 2018

@mvdan

This comment has been minimized.

Copy link
Member

commented Jul 16, 2018

This would be great - I think it has been one of the biggest reasons why some projects have been using makefiles and custom build scripts.

@gopherbot

This comment has been minimized.

Copy link

commented Oct 23, 2018

Change https://golang.org/cl/143977 mentions this issue: cmd/go/internal/modload: fix use of //go:linkname

gopherbot pushed a commit that referenced this issue Oct 23, 2018

cmd/go/internal/modload: fix use of //go:linkname
I can't find the exact rule about space before compiler directive
openings from
https://golang.org/cmd/compile/#hdr-Compiler_Directives
but it seems like the compiler doesn't recognize it
as a compiler directive if it is preceded by space.
Removing the space made the //go:linkname in the __gomod__.go file
working as intended.

Manually tested.

Update #26404

Change-Id: I589f7203a628b2fa6238d82878029e0f098091b6
Reviewed-on: https://go-review.googlesource.com/c/143977
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@hyangah

This comment has been minimized.

Copy link
Contributor

commented Oct 23, 2018

The cl/143977 allowed us to retrieve the module information embedded during build. The API will be in the runtime/debug package.

BTW, using this info for version stamping of a binary instead of Makefiles and custom build scripts that utilize '-X' flag, works only when 1) the binary is built in the module enabled mode and 2) the binary is built through the module cache.

If the binary is built with the traditional GOPATH way, the info will not exist. If module is enabled but the binary is built out of checked in or cloned source repository, all dependency version info will be encoded but the main module's version info will remain undetermined (for the reason explained in the issue 28083).

@gopherbot

This comment has been minimized.

Copy link

commented Oct 23, 2018

Change https://golang.org/cl/144220 mentions this issue: runtime/debug: add API to read module info in binary

gopherbot pushed a commit that referenced this issue Nov 13, 2018

runtime/debug: add API to read module info in binary
When module is enabled, the go tool embeds build information
related to the module in the binary including the dependencies
and the replace information (See
src/cmd/go/internal/modload.PackageBuildInfo).

The newly introduced ReadBuildInfo reads the information and
makes it accessible programmatically.

Update #26404

Change-Id: Ide37022d609b4a8fb6b5ce02afabb73f04fbb532
Reviewed-on: https://go-review.googlesource.com/c/144220
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@bcmills

This comment has been minimized.

Copy link
Member Author

commented Nov 15, 2018

@hyangah, I notice that https://golang.org/cl/144220 is merged. Is there anything more to do for this issue, or should we close it?

@leighmcculloch

This comment has been minimized.

Copy link
Contributor

commented Dec 23, 2018

What's the significance of these new functions being in a package named runtime/debug? Are they only available when built in a particular way? The name debug, based on my experience with other languages, signals that it's not available or shouldn't be used in production code. Is that the case?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Dec 24, 2018

@leighmcculloch As it says at https://golang.org/pkg/runtime/debug: "Package debug contains facilities for programs to debug themselves while they are running." It's fine to use the runtime/debug package in production code.

@pwaller

This comment has been minimized.

Copy link
Contributor

commented Jan 3, 2019

As @mvdan said, this would eliminate the need for make in a number of projects I'm involved in, which would be a significant quality of life improvement.

I wrote a small program to try out ReadBuildInfo, but the reported version of the binary is always (devel) at the moment. Even if it is committed and tagged. I am building with go build in with the main package as $PWD. Click to expand details below.

How do I build a binary which knows, for example, the git sha, or the current git tag?

package main

import (
	"log"
	"runtime/debug"

	"github.com/kr/pretty"
)

func main() {
	bi, ok := debug.ReadBuildInfo()
	if !ok {
		log.Fatal("!ok")
	}
	pretty.Println(bi)
}

I get the following output:

&debug.BuildInfo{
    Path: "foo.to/v",
    Main: debug.Module{
        Path:    "foo.to/v",
        Version: "(devel)",
        Sum:     "",
        Replace: (*debug.Module)(nil),
    },
    Deps: {
        &debug.Module{
            Path:    "github.com/kr/pretty",
            Version: "v0.1.0",
            Sum:     "h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=",
            Replace: (*debug.Module)(nil),
        },
        &debug.Module{
            Path:    "github.com/kr/text",
            Version: "v0.1.0",
            Sum:     "h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=",
            Replace: (*debug.Module)(nil),
        },
    },
}
@mvdan

This comment has been minimized.

Copy link
Member

commented Jan 3, 2019

@pwaller I think the issue is what @hyangah mentioned earlier - the information is missing if you've checked out a main module and are building it directly. Which is a shame, as it means we probably can't fix the need for custom build scripts just yet.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Jan 9, 2019

How do I build a binary which knows, for example, the git sha, or the current git tag?

If you build the binary in module mode from outside of a module using 1.12 (#24250), then you'll pick up all of its version information.

We probably should do something more to interrogate the current VCS state, but that's a lot more subtle than just folding in information from the build list.

@dsnet

This comment has been minimized.

Copy link
Member

commented Jan 10, 2019

main module's version info will remain undetermined (for the reason explained in the issue 28083).

I understand why the version string cannot be retrieved. What's the reason why the Sum is empty as well? My understanding is that the Sum is a hash of the input Go sources. If so, that knowledge is available at build time, correct?

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Jan 10, 2019

Yes, in theory the sum is available. However, we ignore certain git transformations when constructing the zip archive for a module, so the sum as computed inside the user's checkout of the module might legitimately differ from the sum calculated from outside.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Jan 10, 2019

(It's also not obvious to me what purpose that would serve...)

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Jan 18, 2019

The fundamentals of this issue are resolved. I'll file a separate one for incorporating VCS information (particularly version information about the same module), since there are significant security and usability implications to consider.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.