cmd/dist: feature request: control over maximum number of CPUs used during build and test

[odysseus9672]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version
go version devel +a31e0a4aac Sat Oct 21 11:32:53 2017 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/sean/go"
GORACE=""
GOROOT="/Users/sean/goLang/go"
GOTOOLDIR="/Users/sean/goLang/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/75/tsmllrjr8xl81b0059bfyyd00000gs/T/go-build923680158=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

Invoke all.bash after updating with git fetch and git pull.

What did you expect to see?

No more than $GOMAXPROCS used during the build and testing process.

What did you see instead?

Every processor available being utilized.

I really like to be able to limit the number of logical processors that the build and test process Go uses. The logical choice would be for the process to respect the environment variable GOMAXPROCS. The reason for wanting to impose this limit is to manage the amount of heat produced on my system during the process.

[ianlancetaylor]

As far as I know the build part will honor GOMAXPROCS. However, some of the tests will intentionally modify GOMAXPROCS and may reset based on the number of CPUs on the machine. Let us know if this does not match your observations.

Since what matters is basically the default value of GOMAXPROCS, on GNU/Linux you could do what you want by running all.bash via the taskset command. Does Darwin have a similar program?

[odysseus9672]

I just tried a run with export GOMAXPROCS=1, and the build process frequently used more than one CPU, though not as often as when it was set higher. I'm unsure of whether this bit of documentation is sufficient to explain that:

The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit. This package's GOMAXPROCS function queries and changes the limit.

from: https://golang.org/pkg/runtime/ .

Also, my attempts to find info on Max OS X cpu affinity reveals this, though it may be dated by now. man cpuset and man taskset both yield nothing.

[odysseus9672]

Maybe GOMAXPROCS isn't the correct way to go, since that controls running programs. Perhaps an argument similar to the maximum number of jobs specifiable as an argument to make (e.g. make -j 2).

[shrx]

Another useful option is passing -gcflags="all=-c=1" to go build.