Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support POSIX jobserver #36868

Open
crawshaw opened this issue Jan 29, 2020 · 1 comment
Open

cmd/go: support POSIX jobserver #36868

crawshaw opened this issue Jan 29, 2020 · 1 comment
Labels
FeatureRequest NeedsInvestigation
Milestone

Comments

@crawshaw
Copy link
Member

@crawshaw crawshaw commented Jan 29, 2020

The automatic parallelism in cmd/go is wonderful when the command is used interactively. However, it interacts poorly when another build system is driving cmd/go. The driving build system cannot safely start multiple Go builds without overloading the system, and it cannot detect when a Go build is doing very little work so it should start other work.

GNU make has an elegant solution for this, the "jobserver." It is much simpler than the name suggests:

On POSIX systems the jobserver is implemented as a simple UNIX pipe. The pipe will be pre-loaded with one single-character token for each available job. To obtain an extra slot you must read a single character from the jobserver pipe; to release a slot you must write a single character back into the jobserver pipe.

https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

I would like to propose that cmd/go parses the environment variable MAKEFLAGS for --jobserver-auth=R,W, and if it finds valid FDs R and W then it uses tokens from the pipe to control its parallelism. This will make cmd/go play nicely with GNU make and other build systems that use the jobserver, like redo.

@bcmills bcmills added FeatureRequest NeedsInvestigation labels Jan 29, 2020
@bcmills bcmills added this to the Backlog milestone Jan 29, 2020
@bcmills
Copy link
Member

@bcmills bcmills commented Jan 29, 2020

CC @jayconrod @matloob

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FeatureRequest NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

2 participants