Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIVIS-1510] DOC call out joblib.Parallel's pre_dispatch #443

Merged
merged 2 commits into from
Dec 6, 2021

Conversation

jacksonlee-civis
Copy link
Member

@jacksonlee-civis jacksonlee-civis commented Dec 6, 2021

This PR calls out the default value of joblib.Parallel's pre_dispatch to be "2*n_jobs" (not "n_jobs") in the Sphinx docs.

Closes #361

  • (For Civis employees only) Reference to a relevant ticket in the pull request title
  • Changelog entry added to CHANGELOG.md at the repo's root level
  • Description of change in the pull request description
  • The CircleCI builds have all passed

Comment on lines -99 to +105
so it will run ``pre_dispatch`` jobs at once. The default value of
``pre_dispatch`` is "2*n_jobs", which will run a maximum of ``2 * n_jobs`` jobs
at once in the Civis Platform. Set ``pre_dispatch="n_jobs"`` in your
:class:`~joblib.Parallel` call to run at most ``n_jobs`` jobs.
so it will run ``pre_dispatch`` jobs at once.

.. note::
The default value of
``pre_dispatch`` is ``"2*n_jobs"``, which will run a maximum of ``2 * n_jobs`` jobs
at once on Civis Platform. Set ``pre_dispatch="n_jobs"`` in your
:class:`joblib.Parallel` call to run at most ``n_jobs`` jobs.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested using a "warning" when I filed #361, but on second thought a "note" would seem more appropriate actually (a warning would be a bit too severe).

Here's what the compiled docs would look like:

Screen Shot 2021-12-06 at 9 49 43 AM

Copy link
Contributor

@Drew-Pappas Drew-Pappas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jacksonlee-civis jacksonlee-civis merged commit cfe66c2 into master Dec 6, 2021
@jacksonlee-civis jacksonlee-civis deleted the civis-1510-joblib-pre-dispatch branch December 6, 2021 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

More explicitly warn about pre_dispatch = 2*n_jobs with joblib.Parallel
2 participants