New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the parmap calls to use the new parmap arguments #1

Merged
merged 1 commit into from Sep 21, 2017

Conversation

Projects
None yet
2 participants
@zeehio
Copy link
Contributor

zeehio commented Sep 21, 2017

Hi Davide,

I had to make some changes in parmap arguments and now all parmap arguments are prefixed with pm_ (as you can see from the changes in this pull request). Old arguments still work as before, although if they are used they give a deprecation warning. I plan to support both the old and the new arguments for a while, but eventually I will have to break backwards compatibility.

If you want to test this, make sure you have installed the latest parmap version with pip install -U parmap. This last version works with both the new syntax as well as the old one so any code you have will still work.

This pull request updates your parmap calls to the new arguments.

The last parmap version I released allows for keyword arguments to be passed to parallelized functions like this:

parmap.map(yourfunc, your_list, your_arg=your_val)

In order to prevent confusion between parmap arguments and keyword arguments passed to yourfunc, it was necessary to use a common prefix for all parmap parameters.

parmap.map(yourfunc, your_list, your_arg=your_val, pm_parallel=True)

With this new syntax, it is easy for anyone to see that keyword arguments starting with pm_ are for parmap, while other arguments (here your_arg) will be passed to the parallelized function.

That's why I had to change parallel to pm_parallel.

On the other hand, the best way to use parmap with a specific number of CPUs is through the pm_processes argument, instead of creating a multiprocessing pool manually. This has been changed in this commit as well.

Currently the old parmap arguments are still valid, and they give deprecation warnings so users can update the syntax. However it is best to transition to the new arguments, as a future parmap 2.0 version might break this backwards compatibility.

The parmap dependency in setup.py has been updated accordingly as well.

Sorry for the inconvenience 馃槄 and have a nice day

This commit updates the parmap calls to use the new parmap arguments.
The last parmap version allows for keyword arguments to be passed to
parallelized functions like this:

    parmap.map(yourfunc, your_list, your_arg=your_val)

In order to prevent confusion between parmap arguments and keyword
arguments passed to yourfunc, it was necessary to use a common prefix
for all parmap parameters.

    parmap.map(yourfunc, your_list, your_arg=your_val, pm_parallel=True)

With this new syntax, it is easy for anyone to see that keyword arguments
starting with `pm_` are for parmap, while other arguments (here your_arg)
will be passed to the parallelized function.

That's why I had to change `parallel` to `pm_parallel`.

On the other hand, the best way to use parmap with a specific number
of CPUs is through the `pm_processes` argument, instead of creating
a multiprocessing pool manually. This has been changed in this commit
as well.

Currently the old parmap arguments are still valid, and they give
deprecation warnings so users can update the syntax. However it is best
to transition to the new arguments, as a future parmap 2.0 version
might break this backwards compatibility.

The parmap dependency in setup.py has been updated accordingly as well.
@zeehio

This comment has been minimized.

Copy link
Contributor Author

zeehio commented Sep 21, 2017

By the way, parmap now supports showing a progress bar. If you are interested, pass pm_pbar=True and make sure you have the tqdm package (pip install tqdm)

@dgerosa dgerosa merged commit 7bbc638 into dgerosa:master Sep 21, 2017

@dgerosa

This comment has been minimized.

Copy link
Owner

dgerosa commented Sep 21, 2017

Thanks!

@zeehio zeehio deleted the zeehio:update_parmap_args branch Sep 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment