Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More explicitly warn about pre_dispatch = 2*n_jobs with joblib.Parallel #361

Closed
jacksonllee opened this issue Feb 12, 2020 · 0 comments · Fixed by #443
Closed

More explicitly warn about pre_dispatch = 2*n_jobs with joblib.Parallel #361

jacksonllee opened this issue Feb 12, 2020 · 0 comments · Fixed by #443
Assignees

Comments

@jacksonllee
Copy link
Contributor

joblib.Parallel has pre_dispatch default to 2 * n_jobs, which seems unintuitive for a Civis Platform user expecting only n_jobs parallel jobs. The Sphinx docs do mention this behavior and subtly recommend setting n_jobs explicitly:

The default value of pre_dispatch is "2*n_jobs", which will run a maximum of 2 * n_jobs jobs at once in the Civis Platform. Set pre_dispatch="n_jobs" in your :class:~joblib.Parallel call to run at most n_jobs jobs.

Perhaps we should more explicitly warn about this in the documentation, with one of the Sphinx call-out note boxes. In this case, the .. warning:: paragraph directive would seem appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants