Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows #13132

Merged
merged 3 commits into from
May 8, 2019

Conversation

brianquinlan
Copy link
Contributor

@brianquinlan brianquinlan commented May 6, 2019

Lib/test/test_concurrent_futures.py Outdated Show resolved Hide resolved
Co-Authored-By: brianquinlan <brian@sweetapp.com>
else:
if max_workers <= 0:
raise ValueError("max_workers must be greater than 0")
elif (sys.platform == 'win32' and
max_workers > _MAX_WINDOWS_WORKERS):
raise ValueError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider automatically changing max_workers to 61 instead? That would make for a nicer API for people using max_workers=os.cpu_count(), and it would make code like what was in Black before https://github.com/python/black/pull/838/files work unchanged.

On the other hand, it may be unexpected that we silently use fewer workers than the user asked for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought silently using fewer processes could be surprising. I also think that it's pretty unusual in the standard library to silently fix bad argument values.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like raising when too many workers are requested, and doing it quietly when it was "automatic".

@zooba zooba merged commit 3988986 into python:master May 8, 2019
@miss-islington
Copy link
Contributor

Thanks @brianquinlan for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

@zooba
Copy link
Member

zooba commented May 8, 2019

Thanks, Brian!

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 8, 2019
…GH-13132)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
@bedevere-bot
Copy link

GH-13206 is a backport of this pull request to the 3.7 branch.

@miss-islington
Copy link
Contributor

Thanks @brianquinlan for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 29, 2019
…GH-13132)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
@bedevere-bot
Copy link

GH-13643 is a backport of this pull request to the 3.7 branch.

ned-deily pushed a commit that referenced this pull request May 29, 2019
…) (GH-13643)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
mingwandroid pushed a commit to mingwandroid/cpython that referenced this pull request Sep 3, 2020
…GH-13132) (pythonGH-13643)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants