cmd/go: using `--trimpath` in `go build` removes `-ldflags` from `go version -m` output #63432

wallyqs · 2023-10-07T04:24:02Z

What version of Go are you using (`go version`)?

$ go version 1.21.2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=''

GOARCH='arm64'

GOBIN=''

GOCACHE='/Users/example/Library/Caches/go-build'

GOENV='/Users/example/Library/Application Support/go/env'

GOEXE=''

GOEXPERIMENT=''

GOFLAGS=''

GOHOSTARCH='arm64'

GOHOSTOS='darwin'

GOINSECURE=''

GOMODCACHE='/Users/example/repos/dev/pkg/mod'

GONOPROXY=''

GONOSUMDB=''

GOOS='darwin'

GOPATH='/Users/example/repos/dev'

GOPRIVATE=''

GOPROXY='https://proxy.golang.org,direct'

GOROOT='/opt/homebrew/Cellar/go/1.21.2/libexec'

GOSUMDB='sum.golang.org'

GOTMPDIR=''

GOTOOLCHAIN='auto'

GOTOOLDIR='/opt/homebrew/Cellar/go/1.21.2/libexec/pkg/tool/darwin_arm64'

GOVCS=''

GOVERSION='go1.21.2'

GCCGO='gccgo'

AR='ar'

CC='cc'

CXX='c++'

CGO_ENABLED='1'

GOMOD='/Users/example/repos/dev/src/github.com/nats-io/nats-server/go.mod'

GOWORK=''

CGO_CFLAGS='-O2 -g'

CGO_CPPFLAGS=''

CGO_CXXFLAGS='-O2 -g'

CGO_FFLAGS='-O2 -g'

CGO_LDFLAGS='-O2 -g'

PKG_CONFIG='pkg-config'

GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/kr/7z1nxlm160ng389wq51vpktw0000gn/T/go-build4075377866=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

git clone https://github.com/nats-io/nats-server
go build -ldflags="-X main.version='2.10.2'"  --trimpath .
go version -m ./nats-server | grep ldflags

What did you expect to see?

go version -m ./nats-server | grep ldflags
	build	-ldflags="-X main.version='2.10.2'"
echo $?
0

What did you see instead?

go version -m ./nats-server | grep ldflags
echo $?
1

The text was updated successfully, but these errors were encountered:

cuonglm · 2023-10-07T04:43:40Z

See: #52372

wallyqs · 2023-10-07T05:46:33Z

Thanks ok I see now there are a couple of TODOs so this is a known issue:

go/src/cmd/go/internal/load/pkg.go

Line 2336 in d252fdd

// TODO: since we control cmd/link, in theory we can parse ldflags to

Maybe just whitelisting -X main.version which is what sbom tooling like syft uses to detect a go build version might be sufficient?

dmitshur · 2023-10-09T22:25:27Z

CC @bcmills, @matloob.

bcmills · 2023-10-12T18:16:24Z

@wallyqs if you'd like to make progress on that TODO, you're welcome to send a fix! 🙃
(https://go.dev/doc/contribute)

I don't think we should add a special case for -X main.version, though. This is a general problem and ought to have a general solution.

gopherbot · 2023-10-21T21:09:53Z

Change https://go.dev/cl/536795 mentions this issue: cmd/go: add -ldflags to build metadata when CGO is disabled

dolmen · 2023-11-10T09:37:17Z

What about removing -ldflags only if it contains a path separator (/, \)?

dolmen · 2023-11-10T09:37:22Z

What about having special info in buildinfo to mention that -ldflags was used for build but is omitted there?

bcmills · 2023-11-10T15:15:41Z

@dolmen, I think the right solution here is probably to parse the ldflags and replace absolute references in flags with with environment variables when possible (such as $GOROOT or $GOPATH or $GOMODCACHE).

On the other hand, if we can't identify a specific variable that refers to the path passed in -ldflags then I'm not sure what we should do about it. 🤔

dolmen · 2023-11-13T23:18:37Z

Is go build protected against shell injection via -ldflags?

I'm thinking about the case where a tool that rebuilds binaries would take the raw ldflags value from a binary and pass it to go build...

bcmills · 2023-11-14T16:50:31Z

go build disallows many flags in #cgo directives (search for CGO_CFLAGS_ALLOW in https://pkg.go.dev/cmd/cgo). I don't know whether the -ldflags flag itself has those protections.

* Parse the build info for -ldflags. If included, the flags will be further parsed, and searched for -X sub-flags. These sub-flags are commonly used to set the binary version during build time. A common pattern is to run `go build -ldflags='-X main.version=<semver>'` so that the main package's version variable is replaced with the latest tag. It's not guaranteed that this flag will propogate into the binary. See golang/go#63432 for more info. * Flag parsing reuses the pflags library that's used by the main Trivy binary. This keeps the implementation concise, and a bit more robust than creating a custom flag parser for -X flag commonly passed to -ldflags. * Add simple binary to test for validity of ldflags parsing. The flag * The documentation has been updated to reflect the improved accuracy of the Go binary parser.

Add a new boolean option -trimldflags. Only meaningful when -trimpath is true. Defaults to true for backwards compatibility. Otheriwise when set to false reports ldflags in buildinfo, in spite of -trimpath setting. Also when ldflags are trimmed from the output, leave a reproducible marker that it happened. The difference between `-trimpath` and `-trimpath -trimldflags=false` shown below for a compilation with ldflags set: ```diff $ diff -u default.txt trimldflags\=false.txt --- default.txt 2024-04-26 18:51:15.106892203 +0100 +++ trimldflags=false.txt 2024-04-26 18:50:19.187008488 +0100 @@ -137,7 +137,7 @@ dep sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E= build -buildmode=exe build -compiler=gc - build -trimldflags=true + build -ldflags="-X main.Version=234" build -trimpath=true build DefaultGODEBUG=asynctimerchan=1,gotypesalias=0 build CGO_ENABLED=1 ``` Fixes: golang#63432

gopherbot · 2024-04-26T18:06:12Z

Change https://go.dev/cl/582055 mentions this issue: cmd/go: report trimpath erasing ldflags, and allow override

Add a new boolean option -trimldflags. Only meaningful when -trimpath is true. Defaults to true for backwards compatibility. Otheriwise when set to false reports ldflags in buildinfo, in spite of -trimpath setting. Also when ldflags are trimmed from the output, leave a reproducible marker that it happened. Building with '-trimpath -ldflags="-X main.Version=234"' will now emit: build -trimldflags=true Adding -trimldflags=false to the above will emit ldflags: build -ldflags="-X main.Version=234" Fixes: golang#63432

* Parse the build info for -ldflags. If included, the flags will be further parsed, and searched for -X sub-flags. These sub-flags are commonly used to set the binary version during build time. A common pattern is to run `go build -ldflags='-X main.version=<semver>'` so that the main package's version variable is replaced with the latest tag. It's not guaranteed that this flag will propogate into the binary. See golang/go#63432 for more info. * Flag parsing reuses the pflags library that's used by the main Trivy binary. This keeps the implementation concise, and a bit more robust than creating a custom flag parser for -X flag commonly passed to -ldflags. * Add simple binary to test for validity of ldflags parsing. The flag * The documentation has been updated to reflect the improved accuracy of the Go binary parser.

xnox · 2024-04-30T14:19:43Z

Hi, I am proposing to add a flag to allow override, and emit ldflags despite trimpath active.

Another option is to a heuristic to allow ldflags..... as long as it doesn't have any slashes.

Separately, other commands allow substituting prefixes, so maybe an option to regexp substitute things in ldflags would work too?

@matloob requested to discuss this further, so happy to discuss more.

Note, that my proposal to override ldflags should not result in any detrimental behavour w.r.t. reproducible builds by default.

Add a new boolean option -trimldflags. Only meaningful when -trimpath is true. Defaults to true for backwards compatibility. Otheriwise when set to false reports ldflags in buildinfo, in spite of -trimpath setting. Also when ldflags are trimmed from the output, leave a reproducible marker that it happened. Building with '-trimpath -ldflags="-X main.Version=234"' will now emit: build -trimldflags=true Adding -trimldflags=false to the above will emit ldflags: build -ldflags="-X main.Version=234" Fixes: golang#63432 Change-Id: I47d633ef0e6a90136799bef9701b7ff22c92f095 GitHub-Pull-Request: golang#67072

xnox · 2024-05-02T17:16:08Z

@matloob which one of the above solutions are desired? the currently proposed PR addresses the outstanding TODO.

Even when -trimpath is active, emit full ldflags in the version information ELF note. Vulnerability scanners typically parse ldflags field to detect main package version, thus binaries that are built with -trimpath are currently actively evading vulnerability scanners. Fixes: wolfi-dev#17647 Fixes: golang/go#63432

xnox · 2024-05-22T00:34:17Z

@matloob @michaelmatloob any comments?

matloob · 2024-05-22T20:17:16Z

I think the best option is to do what the TODO suggests and what Bryan suggested in #63432 (comment) and parse the ldflags to see if it contains known paths.

LaurentGoderre · 2024-07-17T15:04:29Z

Agree with @matloob and @bcmills because removing the ldflags when there are slashes would remove them when injecting data into modules like here:

https://www.digitalocean.com/community/tutorials/using-ldflags-to-set-version-information-for-go-applications

xnox · 2024-07-17T16:04:03Z

Was -trimpath ldflags stripping got added to get to reproducible builds of go toolchain itself? in that case the goal is to "preserve as many ldflags as possible" but "without breaking reproducibility of the go toolchain builds itself" +/- any other bugs?

Cause absolute and relative paths are ok, as long as they are stable. It's when they are dynamic the binaries become non reproducible.

Fixes golang#63432

gopherbot · 2024-08-27T22:04:06Z

Change https://go.dev/cl/608818 mentions this issue: cmd/go/internal/load: trim paths from ldflags instead of removing them

dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Oct 9, 2023

dmitshur added this to the Backlog milestone Oct 9, 2023

bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 12, 2023

gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 12, 2023

rittneje mentioned this issue Oct 15, 2023

cmd/go/internal/load: -ldflags should be included in buildinfo when -trimpath is set and CGO_ENABLED=0 #63558

Closed

xnox mentioned this issue Apr 24, 2024

golang -trimpath is hiding information wolfi-dev/os#17647

Closed

oatovar mentioned this issue Apr 26, 2024

feat(go): parse main mod version from build info settings aquasecurity/trivy#6564

Merged

6 tasks

xnox linked a pull request Apr 26, 2024 that will close this issue

cmd/go: report trimpath erasing ldflags, and allow override #67072

Open

xnox mentioned this issue May 2, 2024

go: fix ldfalgs, switch to git-checkout wolfi-dev/os#18454

Merged

wallyqs mentioned this issue Jun 26, 2024

Use the tag name for the version on release nats-io/nats-server#5599

Closed

LaurentGoderre mentioned this issue Jul 17, 2024

Add serverVersion var set via ldflags nats-io/nats-server#5650

Merged

LaurentGoderre linked a pull request Jul 22, 2024 that will close this issue

cmd/go/internal/load: trim ldflags paths instead of removing them #68544

Open

LaurentGoderre added a commit to LaurentGoderre/go that referenced this issue Jul 22, 2024

trim paths from ldflags in BuildInfo instead of removing them

7f6c9d5

Fixes golang#63432

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: using `--trimpath` in `go build` removes `-ldflags` from `go version -m` output #63432

cmd/go: using `--trimpath` in `go build` removes `-ldflags` from `go version -m` output #63432

wallyqs commented Oct 7, 2023

cuonglm commented Oct 7, 2023

wallyqs commented Oct 7, 2023

dmitshur commented Oct 9, 2023

bcmills commented Oct 12, 2023

gopherbot commented Oct 21, 2023

dolmen commented Nov 10, 2023

dolmen commented Nov 10, 2023

bcmills commented Nov 10, 2023

dolmen commented Nov 13, 2023 •

edited

Loading

bcmills commented Nov 14, 2023

gopherbot commented Apr 26, 2024

xnox commented Apr 30, 2024

xnox commented May 2, 2024

xnox commented May 22, 2024

matloob commented May 22, 2024

LaurentGoderre commented Jul 17, 2024

xnox commented Jul 17, 2024

gopherbot commented Aug 27, 2024

cmd/go: using --trimpath in go build removes -ldflags from go version -m output #63432

cmd/go: using --trimpath in go build removes -ldflags from go version -m output #63432

Comments

wallyqs commented Oct 7, 2023

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

cuonglm commented Oct 7, 2023

wallyqs commented Oct 7, 2023

dmitshur commented Oct 9, 2023

bcmills commented Oct 12, 2023

gopherbot commented Oct 21, 2023

dolmen commented Nov 10, 2023

dolmen commented Nov 10, 2023

bcmills commented Nov 10, 2023

dolmen commented Nov 13, 2023 • edited Loading

bcmills commented Nov 14, 2023

gopherbot commented Apr 26, 2024

xnox commented Apr 30, 2024

xnox commented May 2, 2024

xnox commented May 22, 2024

matloob commented May 22, 2024

LaurentGoderre commented Jul 17, 2024

xnox commented Jul 17, 2024

gopherbot commented Aug 27, 2024

cmd/go: using `--trimpath` in `go build` removes `-ldflags` from `go version -m` output #63432

cmd/go: using `--trimpath` in `go build` removes `-ldflags` from `go version -m` output #63432

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

dolmen commented Nov 13, 2023 •

edited

Loading