-ldflags=-X with quotes not working when -literals is also used #492

Ne0nd0g · 2022-03-04T13:45:14Z

What version of Garble and Go are you using?

$ garble version
v0.5.1

$ go version
go version go1.17.5 linux/amd64

What environment are you running Garble on?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rastley/.cache/go-build"
GOENV="/home/rastley/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/rastley/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/rastley/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/rastley/sdk/go1.17.5"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/rastley/sdk/go1.17.5/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.5"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/merlin/Payload_Type/merlin/agent_code/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2615366059=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I compiled Merlin using Garble with the -literals argument and attempted to set verbose variable through the go build ldflags argument. When I ran the agent, there was no output to STDOUT but there should have been if the verbose variable was set.

rastley@ubuntu:/tmp$ garble version
v0.5.1
rastley@ubuntu:/tmp$ git clone https://github.com/MythicAgents/merlin
<SNIP>
rastley@ubuntu:/tmp$ cd merlin/Payload_Type/merlin/agent_code/
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -tiny -literals build -o merlin.bin -ldflags '-X "main.verbose=True"' main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$

What did you expect to see?

I expected to see verbose output to STDOUT because the verbose variable was set to True. The output would look like:

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
[i]Host Information:
[i]	Agent UUID: 1e1c41d6-8511-403b-9f83-0ec9b0a4d71e
[i]	Platform: linux
[i]	Architecture: amd64
[i]	User Name: rastley
[i]	User GUID: 1000
[i]	Integrity Level: 3
[i]	Hostname: ubuntu
[i]	Process: /tmp/merlin/Payload_Type/merlin/agent_code/merlin.bin
[i]	PID: 480365

What did you see instead?

I saw no output, indicating that the verbose variable was not set.

Misc.

I opened main.go and added func init(){ fmt.Printf("Verbose: %s\n", verbose) } to assist with debugging:

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -tiny -literals build -o merlin.bin -ldflags '-X "main.verbose=True"' main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: false

Potentially related to #323

The text was updated successfully, but these errors were encountered:

mvdan · 2022-03-04T13:54:49Z

Hmm, is that kind of nested quoting right? I don't think the linker knows about it, so it just discards the malformed -X argument.

This works:

$ go build -ldflags=-X=main.verbose=true && ./agent_code
verbose: true
$ garble build -ldflags=-X=main.verbose=true && ./agent_code
verbose: true

This does not, whether or not garble is involved:

$ garble build -ldflags '-X="main.verbose=true"' && ./agent_code
verbose: false
$ garble -literals build -ldflags '-X="main.verbose=true"' && ./agent_code
verbose: false

Ne0nd0g · 2022-03-04T14:05:14Z

hmm, I got a different result

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ go build -o merlin.bin -ldflags='-X "main.verbose=True"' main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: True
[i]Host Information:
[i]	Agent UUID: 82f4385e-74f5-471e-b1a7-138146748a03
[i]	Platform: linux
[i]	Architecture: amd64
[i]	User Name: rastley
[i]	User GUID: 1000
[i]	Integrity Level: 3
[i]	Hostname: ubuntu
[i]	Process: /tmp/merlin/Payload_Type/merlin/agent_code/merlin.bin
[i]	PID: 485116
unknown C2 profile: 
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -tiny -literals build -o merlin.bin -ldflags='-X "main.verbose=True"' main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: false
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$

mvdan · 2022-03-04T14:07:42Z

Could you try a garble build without the quoting, like I did?

Also, is it at all possible that you're not actually rebuilding merlin.bin with go build? Try removing the binary and re-running the build with the quoted ldflags.

Ne0nd0g · 2022-03-04T15:10:06Z

Removing the quotes does work. I'll need quotes to set multiple -X arguments and to quote unsafe characters like ( and \.

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -literals build -o merlin.bin -ldflags=-X=main.verbose=true main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: true

Is it possible there is some type of caching that is saving a "good" run that is re-used? I added the -seed random argument to potentially keep it from re-using past work. I noticed you didn't use the -literals argument both times.

rastely@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -literals build -ldflags '-X "main.verbose=True"' -o merlin.bin main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: True
[i]Host Information:
[i]	Agent UUID: 729d4b44-a3cb-4bfc-9d92-d484c2f333d9
[i]	Platform: linux
[i]	Architecture: amd64
[i]	User Name: rastley
[i]	User GUID: 1000
[i]	Integrity Level: 3
[i]	Hostname: ubuntu
[i]	Process: /tmp/merlin/Payload_Type/merlin/agent_code/merlin.bin
[i]	PID: 519506
unknown C2 profile: 
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ rm merlin.bin 
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ garble -literals -seed random build -ldflags '-X "main.verbose=True"' -o merlin.bin main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: false

mvdan · 2022-03-04T15:27:27Z

I'm still not sure why we're seeing inconsistent results, but I'm pretty certain that neither -ldflags nor -X unquote their arguments. But I am pretty sure that -X can be passed multiple times to set multiple variables, e.g. go build -ldflags='-X=main.foo=val1 -X=main.bar=val2, so I would try that.

Ne0nd0g · 2022-03-04T15:39:12Z

That sounds right, I already set multiple -X arguments when not using garble. When using multiple -X arguments, do they need to be wrapped in quotes? I'm assuming yes because I pass in additional ldflags other than -X. One of my -X arguments is main.useragent=Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko. How would you get that string into a -X argument without additional quotes?

mvdan · 2022-03-04T15:49:15Z

You are absolutely right; -ldflags does support quotes when splitting its flags. We happily just split by spaces without looking at quoting, so it's no wonder that it breaks.

And I totally botched my examples earlier, as you point out. I did not use -literals or garble consistently 🤦

mvdan · 2022-03-04T16:09:38Z

I can also confirm there's something funky with build caching. I get the expected results with regards to quoting if I add the -a build flag to rebuild all packages, but if I don't, it somehow does not rebuild correctly.

In particular, using -ldflags with - In particular, a command like: garble -literals build -ldflags='-X "main.foo=foo bar"' would fail, because we would try to use "\"main" as the package name for the -X qualified name, with the leading quote character. This is because we used strings.Split(ldflags, " "). Instead, use the same quoted.Split that cmd/go uses, copied over thanks to x/tools/cmd/bundle and go:generate. Updates burrowers#492.

mvdan · 2022-03-04T16:32:10Z

Please test the PR above when you can :) I'll leave this PR open to investigate the cache issue further.

Ne0nd0g · 2022-03-04T16:51:44Z

I'm not sure which is right? The difference is a = after -X. I personally use the former when compiling outside of go:

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ /tmp/t/garble-fork/garble -literals -seed random build -ldflags '-X "main.verbose=True" -X "main.profile=http" -X "main.payloadID=0e041292-a312-459e-a6b3-b0be1b83751f"' -o merlin.bin main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: True

rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ /tmp/t/garble-fork/garble -literals -seed random build -ldflags '-X="main.verbose=True" -X "main.profile=http" -X "main.payloadID=0e041292-a312-459e-a6b3-b0be1b83751f"' -o merlin.bin main.go
rastley@ubuntu:/tmp/merlin/Payload_Type/merlin/agent_code$ ./merlin.bin 
Verbose: false

mvdan · 2022-03-04T17:50:56Z

I also noticed the quoting does not work with the extra equals sign. This seems to be the same both with and without garble.

Ne0nd0g · 2022-03-04T18:39:19Z

Can confirm that the associate PR enables garble to work with the quoted ldflags arguments as the way that I currently use it when calling go directly.

mvdan · 2022-03-04T20:31:36Z

Thanks for confirming!

In particular, using -ldflags with - In particular, a command like: garble -literals build -ldflags='-X "main.foo=foo bar"' would fail, because we would try to use "\"main" as the package name for the -X qualified name, with the leading quote character. This is because we used strings.Split(ldflags, " "). Instead, use the same quoted.Split that cmd/go uses, copied over thanks to x/tools/cmd/bundle and go:generate. Updates burrowers#492.

In particular, using -ldflags with - In particular, a command like: garble -literals build -ldflags='-X "main.foo=foo bar"' would fail, because we would try to use "\"main" as the package name for the -X qualified name, with the leading quote character. This is because we used strings.Split(ldflags, " "). Instead, use the same quoted.Split that cmd/go uses, copied over thanks to x/tools/cmd/bundle and go:generate. Updates #492.

Noticed while debugging burrowers#492, as adding -debug to a previously run command would unexpectedly rebuild all packages in the build, as if -a was given. While here, remove commented out testscript that were kept in error.

Noticed while debugging #492, as adding -debug to a previously run command would unexpectedly rebuild all packages in the build, as if -a was given. While here, remove commented out testscript that were kept in error.

I spent some time trying to reproduce the bug but couldn't, so for now, make a detailed note for it. We can come back to it if we actually run into it in the future. Fixes burrowers#492.

mvdan · 2022-03-05T14:28:24Z

I couldn't reproduce the cache issue, even though I still believe it exists by inspecting our code. The PR above closes this issue by adding a TODO.

I spent some time trying to reproduce the bug but couldn't, so for now, make a detailed note for it. We can come back to it if we actually run into it in the future. Fixes #492.

kmahyyg · 2022-05-16T03:24:06Z

I currently run into the same issue without quotes here. And stably reproducible in our production code.
garble -literals -tiny -seed=random build -o bin/xorer_${BUILDRAND} -trimpath -ldflags="-s -w -X common.VERSION=$(git describe --always --long --dirty --tags) -X common.EndUserID=${USERID_B} -X common.EndUserNonce=${USERNONCE_B} -X common.EndUserPublicKey=$(cat ~/.lic-root.pub) -X common.EndUserLicenseType=${USERLIC_TYPE_B}" cmd/xorer.go

My fault... I write the package name in a wrong way...

mvdan mentioned this issue Mar 4, 2022

add support for -ldflags using quotes #493

Merged

mvdan changed the title ~~ldflags -X not set when using literals argument~~ -ldflags=-X with quotes not working when -literals is also used Mar 4, 2022

mvdan mentioned this issue Mar 5, 2022

add a TODO about a possible cache bug with -literals and -ldflags #495

Merged

capnspacehook closed this as completed in #495 Mar 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

-ldflags=-X with quotes not working when -literals is also used #492

-ldflags=-X with quotes not working when -literals is also used #492

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022 •

edited

Loading

Ne0nd0g commented Mar 4, 2022 •

edited

Loading

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022 •

edited

Loading

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 5, 2022

kmahyyg commented May 16, 2022 •

edited

Loading

-ldflags=-X with quotes not working when -literals is also used #492

-ldflags=-X with quotes not working when -literals is also used #492

Comments

Ne0nd0g commented Mar 4, 2022

What version of Garble and Go are you using?

What environment are you running Garble on?

What did you do?

What did you expect to see?

What did you see instead?

Misc.

mvdan commented Mar 4, 2022 • edited Loading

Ne0nd0g commented Mar 4, 2022 • edited Loading

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022 • edited Loading

mvdan commented Mar 4, 2022

Ne0nd0g commented Mar 4, 2022

mvdan commented Mar 4, 2022

mvdan commented Mar 5, 2022

kmahyyg commented May 16, 2022 • edited Loading

mvdan commented Mar 4, 2022 •

edited

Loading

Ne0nd0g commented Mar 4, 2022 •

edited

Loading

Ne0nd0g commented Mar 4, 2022 •

edited

Loading

kmahyyg commented May 16, 2022 •

edited

Loading