Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: pass files to compile to compiler in a file instead of on the command line #37768

Open
daojianmahun opened this issue Mar 10, 2020 · 6 comments

Comments

@daojianmahun
Copy link

@daojianmahun daojianmahun commented Mar 10, 2020

go build bin/gameconfig/serverscript/scriptimpl/quest: /usr/local/go/pkg/tool/darwin_amd64/compile: fork/exec /usr/local/go/pkg/tool/darwin_amd64/compile: argument list too long

quest have 2510 files

$ go version  go1.13.8  go1.12.7
MacOS 10.15.3

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mabingtao/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mabingtao/work/golang"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang"
CXX="/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS="-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined"
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/k0/qsyfj5n11pqbm42lh4m72q3r0000gn/T/go-build709987064=/tmp/go-build -gno-record-gcc-switches -fno-common"
@randall77

This comment has been minimized.

Copy link
Contributor

@randall77 randall77 commented Mar 10, 2020

OSX has a limit of 256KB for a command line. I don't see any easy way to change that limit.

We could try to pass the list of files to compile in a file instead of on the command line.

If you're looking for a quick fix, maybe try making each file name shorter. Your names are at least (256<<10) / 2510 = 104 characters long. I'm not sure, but it might also help to build in a short absolute path, like /tmp.

Try passing the -x flag to go build so you can see what it tried to execute.

@randall77 randall77 added this to the Unplanned milestone Mar 10, 2020
@randall77 randall77 changed the title /usr/local/go/pkg/tool/darwin_amd64/compile: argument list too long cmd/go: pass files to compile to compiler in a file instead of on the command line Mar 10, 2020
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Mar 10, 2020

That is a lot of files. I'm assuming this is a generated package, rather than one hand-curated package that large?

If so, #35950 may be relevant.

@daojianmahun

This comment has been minimized.

Copy link
Author

@daojianmahun daojianmahun commented Mar 12, 2020

OSX has a limit of 256KB for a command line. I don't see any easy way to change that limit.

We could try to pass the list of files to compile in a file instead of on the command line.

If you're looking for a quick fix, maybe try making each file name shorter. Your names are at least (256<<10) / 2510 = 104 characters long. I'm not sure, but it might also help to build in a short absolute path, like /tmp.

Try passing the -x flag to go build so you can see what it tried to execute.

but this can compile successfully on os 10.14.3
os 10.14.3 getconf ARG_MAX is 262144

@randall77

This comment has been minimized.

Copy link
Contributor

@randall77 randall77 commented Mar 12, 2020

That's the same as 10.15.2.
I don't understand what the difference might be between 10.14 and 10.15.

Who wrote that it works on 10.14.3? You're quoting that response, but I don't see the original post of that here.

@cjdelisle

This comment has been minimized.

Copy link

@cjdelisle cjdelisle commented Mar 21, 2020

Reproduced, unable to build matterbridge on mac 10.15.3 (19D76):

user@underscore matterbridge % go version
go version go1.13.5 darwin/amd64
user@underscore matterbridge % git log | head -1
commit 6b017b226a9944940158331b7ff51abc6945b119
user@underscore matterbridge % git remote -v
origin  https://github.com/42wim/matterbridge (fetch)
origin  https://github.com/42wim/matterbridge (push)
user@underscore matterbridge % 

Also reproduced with go version go1.14.1 darwin/amd64

@randall77

This comment has been minimized.

Copy link
Contributor

@randall77 randall77 commented Mar 21, 2020

Some one please do go build -x on a failing build and post the non-repetitive parts of the output here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.