Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: add support for automatically generating .note.gnu.build-id #41004

Open
segevfiner opened this issue Aug 24, 2020 · 9 comments
Open
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@segevfiner
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.15 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build730694799=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I want the ability to automatically generate a .note.gnu.build-id section in Go binaries. The linker currently supports a flag to get one from the outside: -B, but this is supposed to be a CRC that is calculated by the linker rather than something that can reliably be set from the outside as a linker flag.

This is used by delve and other debuggers to find split debug symbols.

What did you expect to see?

I want a flag like the binutils linker has -Wl,-build-id that adds an automatically generated .note.gnu.build-id section.

What did you see instead?

There is only a flag to set it from the outside, but it is meant to be a CRC calculated by the linker to be used reliably to differentiate different debug symbols.

@ALTree ALTree changed the title Add support for automatically generating .note.gnu.build-id cmd/link: add support for automatically generating .note.gnu.build-id Aug 24, 2020
@ALTree ALTree added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. FeatureRequest labels Aug 24, 2020
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
@seankhliao seankhliao added this to the Unplanned milestone Aug 27, 2022
@Foxboron
Copy link
Contributor

Foxboron commented Jan 4, 2023

I hacked a little bit on this by just running sha1 on the output buffer we have in the linker. It gives me a stable sha1 sum and I think(?) it should be good enough? Should we stamp the same build id if we use the -s -w flags maybe?

diff --git a/src/cmd/link/internal/ld/elf.go b/src/cmd/link/internal/ld/elf.go
index a1ae7eab57..3f9f115282 100644
--- a/src/cmd/link/internal/ld/elf.go
+++ b/src/cmd/link/internal/ld/elf.go
@@ -10,6 +10,7 @@ import (
        "cmd/internal/sys"
        "cmd/link/internal/loader"
        "cmd/link/internal/sym"
+       "crypto/sha1"
        "debug/elf"
        "encoding/binary"
        "encoding/hex"
@@ -825,6 +826,11 @@ func elfgobuildid(sh *ElfShdr, startva uint64, resoff uint64) int {
        return elfnote(sh, startva, resoff, n)
 }

+func elfproducebuildinfo(out *OutBuf) {
+       id := sha1.Sum(out.Data())
+       buildinfo = id[:]
+}
+
 func elfwritebuildinfo(out *OutBuf) int {
        sh := elfwritenotehdr(out, ".note.gnu.build-id", ELF_NOTE_BUILDINFO_NAMESZ, uint32(len(buildinfo)), ELF_NOTE_BUILDINFO_TAG)
        if sh == nil {
@@ -1396,9 +1402,7 @@ func (ctxt *Link) doelf() {
        if ctxt.IsFreebsd() {
                shstrtab.Addstring(".note.tag")
        }
-       if len(buildinfo) > 0 {
-               shstrtab.Addstring(".note.gnu.build-id")
-       }
+       shstrtab.Addstring(".note.gnu.build-id")
        if *flagBuildid != "" {
                shstrtab.Addstring(".note.go.buildid")
        }
@@ -1911,6 +1915,9 @@ func asmbElf(ctxt *Link) {
                phsh(pnotei, sh)
        }

+       if len(buildinfo) == 0 {
+               elfproducebuildinfo(ctxt.Out)
+       }
        if len(buildinfo) > 0 {
                sh := elfshname(".note.gnu.build-id")
                resoff -= int64(elfbuildinfo(sh, uint64(startva), uint64(resoff)))
@@ -2285,9 +2292,7 @@ elfobj:
                if ctxt.HeadType == objabi.Hfreebsd {
                        a += int64(elfwritefreebsdsig(ctxt.Out))
                }
-               if len(buildinfo) > 0 {
-                       a += int64(elfwritebuildinfo(ctxt.Out))
-               }
+               a += int64(elfwritebuildinfo(ctxt.Out))
                if *flagBuildid != "" {
                        a += int64(elfwritegobuildid(ctxt.Out))
                }

@Foxboron
Copy link
Contributor

Foxboron commented Feb 17, 2023

@bcmills sorry for the ping here. But has there been any discussions on how to solve this?

Would a sha256 hash over the actionID and/or the .note.go.buildid string be a good enough solution?

Generally this would help debug packages on Linux distros and debuggers like delve do the right thing instead of having to pass the -buildid linker flag explicitly.

@ianlancetaylor
Copy link
Contributor

We already have a Go build ID. We can just write that out as the GNU build ID as well. I think the only question is whether we should always do it. The GNU linker only does it upon request, but on many Linux systems the compiler is configured to request it by default (by passing a --build-id option when invoking the linker).

@Foxboron
Copy link
Contributor

Most (all?) debuggers expect the value for basic functionality and I struggle to see why we would stamp go build id and not the GNU build id along with it?

The only issue is that the GNU build ID needs to either be a md5 or sha1, so I don't think we can pass the Go build id as-is?

Having this as a build option for distributions is non-trivial as it's still hard to override the command line arguments passed to the compiler.

@ianlancetaylor
Copy link
Contributor

To the best of my knowledge the GNU build ID does not have to be an MD5 or SHA1. Those are just common implementations of it. It can be any random relatively unique string.

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Feb 17, 2023

For example, for the gold linker the default value (if you just specify --build-id with no specified style) is neither MD5 nor SHA1, it's a tree hash that can be computed in parallel.

@prattmic
Copy link
Member

Linux's perf tool is one such tool that automatically records the GNU buildid (directly in the kernel during mmap event generation), which can then be used by downstream tooling to find the correct binary for symbolization. For pure Go binaries, no build ID is recorded.

@prattmic
Copy link
Member

prattmic commented Apr 19, 2023

We already have a Go build ID. We can just write that out as the GNU build ID as well. I think the only question is whether we should always do it.

FWIW, Linux perf expects the GNU build ID to be 20 bytes or less (https://elixir.bootlin.com/linux/v6.2/source/lib/buildid.c#L30). I'm not sure if this is specified or a convention, but the Go build ID is significantly longer (83 bytes).

@znkr
Copy link
Contributor

znkr commented Apr 19, 2023

Truncating larger hashes seems to be okay: https://reviews.llvm.org/D121531

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Status: Triage Backlog
Development

No branches or pull requests

8 participants