Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Option for remote concurrency. #7960

Merged
merged 4 commits into from Jun 28, 2019

Conversation

Projects
None yet
4 participants
@hrfuller
Copy link
Contributor

commented Jun 26, 2019

Problem

closes #7918

Currently the process_execution_parallelism flag controls both the remote and local parallelism, depending on the selected, mode. In anticipation of speculative/hybrid execution we will want to control remote and local concurrency separately l.

Solution

Add two new CLI options called remote_execution_parallelism and local_execution_parallelism. Deprecate old flag.

Result

Two new flags added and one flag deprecated. Advice on which version we should deprecate the old flag in would be appreciated.

Show resolved Hide resolved src/python/pants/option/global_options.py Outdated

@stuhood stuhood requested a review from illicitonion Jun 27, 2019

@stuhood
Copy link
Member

left a comment

Thanks! Will need to fix the deprecation.

Show resolved Hide resolved src/python/pants/option/global_options.py Outdated
Show resolved Hide resolved src/python/pants/option/global_options.py Outdated
Show resolved Hide resolved src/rust/engine/src/context.rs Outdated
Show resolved Hide resolved src/python/pants/option/global_options.py Outdated
@stuhood
Copy link
Member

left a comment

Pending the TODO / gRPC pool investigation.

Also note that travis is apocalyptically flaky right now, so will wait to merge this until master is looking a bit healthier.

@hrfuller

This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2019

I can't repro any of the test failures locally.

@stuhood

This comment has been minimized.

Copy link
Member

commented Jun 27, 2019

@hrfuller : The master flakiness should now be resolved. Can merge or rebase.

@hrfuller hrfuller force-pushed the twitter:feat/7918 branch from df3b70f to 4c509ba Jun 27, 2019

@hrfuller

This comment has been minimized.

Copy link
Contributor Author

commented Jun 28, 2019

One shard timed out after 50 minutes 😭

@stuhood

This comment has been minimized.

Copy link
Member

commented Jun 28, 2019

One shard timed out after 50 minutes 😭

Sorry about that... that's was exacerbated by this one: #7919. Will revert in #7967.

@stuhood stuhood merged commit 4d21f6a into pantsbuild:master Jun 28, 2019

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.