Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: support POSIX jobserver #36868

Open
crawshaw opened this issue Jan 29, 2020 · 1 comment
Open

cmd/go: support POSIX jobserver #36868

crawshaw opened this issue Jan 29, 2020 · 1 comment

Comments

@crawshaw
Copy link
Contributor

@crawshaw crawshaw commented Jan 29, 2020

The automatic parallelism in cmd/go is wonderful when the command is used interactively. However, it interacts poorly when another build system is driving cmd/go. The driving build system cannot safely start multiple Go builds without overloading the system, and it cannot detect when a Go build is doing very little work so it should start other work.

GNU make has an elegant solution for this, the "jobserver." It is much simpler than the name suggests:

On POSIX systems the jobserver is implemented as a simple UNIX pipe. The pipe will be pre-loaded with one single-character token for each available job. To obtain an extra slot you must read a single character from the jobserver pipe; to release a slot you must write a single character back into the jobserver pipe.

https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

I would like to propose that cmd/go parses the environment variable MAKEFLAGS for --jobserver-auth=R,W, and if it finds valid FDs R and W then it uses tokens from the pipe to control its parallelism. This will make cmd/go play nicely with GNU make and other build systems that use the jobserver, like redo.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jan 29, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.