Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: error if pass a very long args which contains \n character #42295

Closed
lyyyuna opened this issue Oct 30, 2020 · 9 comments
Closed

cmd/link: error if pass a very long args which contains \n character #42295

lyyyuna opened this issue Oct 30, 2020 · 9 comments
Labels
Milestone

Comments

@lyyyuna
Copy link

@lyyyuna lyyyuna commented Oct 30, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15.3 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin

What did you do?

I build with the following command,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a very long string.....which contains \n characters.....\''"

get go link error.

Then I change the above command with a shorter one,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a short string.....which contains \n characters.....\''"

no go link error this time.

What did you expect to see?

No go link error if I pass a very long args with \n character inside.

My analysis

go1.14 and go1.13 have no such issue.

I notice that go has introduced response file to pass long args in Windows: https://go-review.googlesource.com/c/go/+/110395/7/src/cmd/go/internal/work/exec.go#2483

and enabled it for other OS in go1.15:

func useResponseFile(path string, argLen int) bool {

response file use \n to separate args will cause this issue.

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Oct 30, 2020

get go link error.

What error?

Could you share the exact command you run that succeeded with Go 1.14 but failed with Go 1.15? Thanks.

@dmitshur dmitshur changed the title go link error if pass a very long args which contains \n character. cmd/link: error if pass a very long args which contains \n character Oct 30, 2020
@dmitshur dmitshur modified the milestones: Backlog, Go1.16 Oct 30, 2020
@lyyyuna
Copy link
Author

@lyyyuna lyyyuna commented Oct 31, 2020

@cherrymui
just show link's usage message, and the return exit code is 2.

In our case, we need to put a INI config file's content(which contains a lot of '\n') in the -ldflags.
The INI config content is very long which exceeds 32KB, so the -ldflags is written into a response file instead to pass to cmd/link as args.

It's easy to reproduce it with long -ldflags, I think go should not simply use '\n' as a separator in the response file.

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Oct 31, 2020

Can you paste in exact commands you run, and the exact output? Possibly to use go build -x to give extra information.

The command you mentioned above,

go build  -v -ldflags "-X 'qiniu.com/kodotest.regionNb=\'a short string.....which contains \n characters.....\''"

short or long, when I paste it to a shell, never works on my machine with any version of Go.

@lyyyuna
Copy link
Author

@lyyyuna lyyyuna commented Nov 1, 2020

Here are two example config files (config-wrong.ini, config.ini):
https://gist.github.com/lyyyuna/ca5e6c7720260180080689373f5e5d62

config-wrong.ini is longer than config.ini with only one extra line: a=a
machine: darwin

cmd/link error with go1.15 / config-wrong.ini

go build  -ldflags="-X 'qiniu.com/qtest.sample=`cat config-wrong.ini`'" .

get

# xxxxxxxxx
usage: link [options] main.o
  -B note
        add an ELF NT_GNU_BUILD_ID note when using ELF
  -E entry
        set entry symbol name

build success with go1.15 / config.ini

go build -ldflags="-X 'qiniu.com/qtest.sample=`cat config.ini`'" .

I am not sure if link arg is same on all darwin's machines, so you can add extra lines in config-wrong.ini to reproduce it.

@jeremyfaller
Copy link
Contributor

@jeremyfaller jeremyfaller commented Nov 23, 2020

On darwin, in bash, I was able to reproduce this with:

uname -a Darwin jfaller-macbookpro2.roam.corp.google.com 19.5.0 Darwin Kernel Version 19.5.0: Thu Apr 30 18:25:59 PDT 2020; root:xnu-6153.121.1~7/RELEASE_X86_64 x86_6

package main

import "fmt"

var str string

func main() {
        fmt.Println(str)
}

The following fails:

go build -ldflags="-X 'main.str=`openssl rand -base64 22342`'" ./

But, this succeeds:

go build -ldflags="-X 'main.str=`openssl rand -base64 22341`'" ./

And for the people following along at home: getconf ARG_MAX = 262144.

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 24, 2020

Change https://golang.org/cl/272870 mentions this issue: cmd/go: encode backslash and newline in response files

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Nov 30, 2020

@jeremyfaller I understand this issue was meant to be moved to NeedsFix state, is that right? When an issue doesn't have any of NeedsInvestigation, NeedsDecision, or NeedsFix labels, it appears as one that hasn't been triaged.

@jeremyfaller
Copy link
Contributor

@jeremyfaller jeremyfaller commented Nov 30, 2020

Sorry, my mistake for not understanding bug life cycles.

It's definitely triaged, and I have a fix. We just need to make a decision if it goes into the 1.16 release.

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 9, 2020

Change https://golang.org/cl/276555 mentions this issue: cmd/go: encode backslash and newline in response files

@gopherbot gopherbot closed this in 6a64f6d Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants