Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/dist: support configurable number of build jobs #6799

Closed
davecheney opened this issue Nov 20, 2013 · 3 comments
Closed

cmd/dist: support configurable number of build jobs #6799

davecheney opened this issue Nov 20, 2013 · 3 comments
Milestone

Comments

@davecheney
Copy link
Contributor

@davecheney davecheney commented Nov 20, 2013

Problem statement:

Currently cmd/dist uses hard coded defaults for the number of concurrent jobs to execute
during the early stages of ./make.bash; up to the point of go_bootstrap taking over.

This number is 1 on arm machines and 4 everywhere else. For arm specifically this value
used to be correct 3 years ago but now dual and quad core arm host are available this
unnecessarily delays an already slow c compilation phase.

This value can be easily changed after checkout, but as the builders always build from a
clean tree this is not an option.

It may also be for large x86 machines the value of 4 parallel jobs is too conservative,
or to high if operating in a virtualised environment, however this is not the main focus
of this proposal.

Implementation:

I propose to add a new flag cmd/dist which would override the default value. If not set
the current defaults of 4 (or 1 for arm) would apply.

The proposed name of this flag is -j.

This flag would also need to be passed through from the builder command via the existing
GO_DISTFLAGS env var.

Builder owners can set GO_DISTFLAGS to control the number of parallel jobs during the
build allowing them to improve build times.

Once the build arrives at go_bootstrap the current behaviour of running runtime.NumCPU()
jobs would take over. This does not change as part of this proposal.

See also: https://groups.google.com/d/msg/golang-dev/GjhQspYudHk/vNVFsd_49AEJ
@rsc
Copy link
Contributor

@rsc rsc commented Dec 4, 2013

Comment 1:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

@rsc rsc commented Dec 4, 2013

Comment 2:

Labels changed: added repo-main.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@ALTree
Copy link
Member

@ALTree ALTree commented Oct 2, 2019

The bootstrapping process on arm now uses min(maxbg, runtime.NumCPU()), which mostly solves this issue (beefier arm CPUs will now use 2 or 4 jobs).

It seems like there wasn't much support for a new flag, so I'm closing this issue.

@ALTree ALTree closed this Oct 2, 2019
@golang golang locked and limited conversation to collaborators Oct 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.