Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: factor version reading into x/mod #39301

Open
skelterjohn opened this issue May 28, 2020 · 7 comments
Open

cmd/go: factor version reading into x/mod #39301

skelterjohn opened this issue May 28, 2020 · 7 comments
Labels
Milestone

Comments

@skelterjohn
Copy link
Contributor

@skelterjohn skelterjohn commented May 28, 2020

With go version 1.14 and go modules, we get a nice report from go version -m <binary>. The code backing this command is not available to use outside the binary, as it uses internal packages.

Supply chain analysis tools, which I work on, would benefit greatly from being able to reliably get this information using a package in the standard library. Instead, I'm copying and lightly modifying the code used to run the go version -m command since shelling out to a binary is not always an option.

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented May 28, 2020

To clarify, you're asking for a package to read the BuildInfo from a compiled executable?

@andybons andybons changed the title package to get the runtime/debug.BuildInfo data proposal: package to get the runtime/debug.BuildInfo data May 28, 2020
@gopherbot gopherbot added this to the Proposal milestone May 28, 2020
@gopherbot gopherbot added the Proposal label May 28, 2020
@skelterjohn
Copy link
Contributor Author

@skelterjohn skelterjohn commented May 28, 2020

From the bytes of a compiled executable. Something like func ReadBuildInfo(io.Reader) (debug.BuildInfo, error) would be ideal.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals May 28, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented May 28, 2020

shelling out to a binary is not always an option

Can you clarify why? If you need to inspect Go binaries, having Go installed seems pretty reasonable.

@dmitshur
Copy link
Member

@dmitshur dmitshur commented May 29, 2020

Have you considered the rsc.io/goversion/version package?

@skelterjohn
Copy link
Contributor Author

@skelterjohn skelterjohn commented May 29, 2020

For why I prefer this to shelling out:

The product I work with is GCP's container image vulnerability scanner. It scans XX thousand container images per minute. Each of those container images has hundreds, thousands of files. I want to find Go binaries and report on how they were made so we can do vulnerability analysis, contributor history, etc. Shelling out to go version -m for each one is impractical for a number of reasons. First, in the execution environment I'm in (google's borg), starting a subprocess is an expensive operation that involves lots of extra logging. Second, the overhead to writing a file and running the go tool is far greater than inspecting the bytes as they fly by in our streaming read. Even if I limit it to executable files, there are still probably hundreds to inspect per image. Every cycle counts at this scale.

For consideration of rsc.io/goversion/version:

I did see that, and it does to approximately what I want. However, it does not have the interface I'd like (give it a file path rather than bytes, which increases the overhead per invocation significantly), and it's also not covered by any compatibility guarantees. For instance, the code it uses to load information from the executable and extract the mod string is different than that used by go version -m.

What I have done is copied, with light modifications to make it fit, the code that is used by go version -m to extract the mod string and the code used by runtime/debug.ReadBuildInfo() to parse it. This will work fine for now, but it implies an ongoing maintenance burden to ensure that it keeps up-to-date with any changes in the executable or mod string format. Since go version -m's code and runtime/debug.ReadBuildInfo()'s code imply that burden is being shouldered by the Go distribution as well, I would like to benefit :)

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented May 29, 2020

Thanks @skelterjohn for posting this. Vulnerability scanning is a use case we had in mind when embedding module version information in binaries. This would help complete the picture.

I think it would make sense to do this somewhere in x/mod, probably in a new package, since this doesn't fit in any existing package there.

cc @bcmills @FiloSottile @katiehockman @matloob @rsc

@rsc
Copy link
Contributor

@rsc rsc commented Jun 3, 2020

We discussed this some on #35667, where we decided it was OK to add go version -json to get a JSON-formatted output, and also OK to move the cmd/go version reader somewhere into x/mod. (Please use the one in cmd/go and not the rsc.io version, which is older.)

So this probably doesn't need to be a proposal, since we've already decided to do it.

@rsc rsc changed the title proposal: package to get the runtime/debug.BuildInfo data cmd/go: factor version reading into x/mod Jun 3, 2020
@rsc rsc removed this from Incoming in Proposals Jun 3, 2020
@rsc rsc added NeedsFix and removed Proposal labels Jun 3, 2020
@rsc rsc modified the milestones: Proposal, Backlog Jun 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.