Documentation for multiprocessing - Pool.apply()

[beazley]

BPO	4593
Nosy	@birkenfeld

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-01-22.22:06:46.666>
created_at = <Date 2008-12-08.15:45:19.338>
labels = ['easy', 'docs']
title = 'Documentation for multiprocessing - Pool.apply()'
updated_at = <Date 2009-01-22.22:06:46.665>
user = 'https://bugs.python.org/beazley'

bugs.python.org fields:

activity = <Date 2009-01-22.22:06:46.665>
actor = 'jnoller'
assignee = 'jnoller'
closed = True
closed_date = <Date 2009-01-22.22:06:46.666>
closer = 'jnoller'
components = ['Documentation']
creation = <Date 2008-12-08.15:45:19.338>
creator = 'beazley'
dependencies = []
files = []
hgrepos = []
issue_num = 4593
keywords = ['easy']
message_count = 4.0
messages = ['77312', '77316', '77336', '80385']
nosy_count = 3.0
nosy_names = ['georg.brandl', 'beazley', 'jnoller']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'needs patch'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue4593'
versions = ['Python 2.6', 'Python 3.0', 'Python 2.7']

[beazley]

The documentation for the apply() and apply_async() methods of a Pool
object might emphasize that these operations execute func(*args,**kwargs)
in only one of the pool workers and that func() is not being executed in
parallel on all workers. On first reading, I actually thought it might
be the latter (and had to do some experimentation to figure out what was
actually happening).

[jnoller]

Ah, I see what you're talking about David. Good catch. It wouldn't make
sense to run the apply() in all of the workers, as apply() is a single
function(...) call. map() however is done in parallel amongst all workers.

[beazley]

Actually, you shouldn't discount the potential usefulness of running
apply() in all of the worker nodes. A lot of people coming from
parallel programming know about things like global broadcasts,
reductions, and so forth. For example, if I wanted to perform a global
operation (maybe some kind of configuration) on all workers, I could see
doing some kind of global apply() operation to do it.

That said, I'm not actually asking for any new functionality. I'd just
make it more clear that apply() is not performing a function call on all
pool workers.

Also, given that apply() blocks, I'm not exactly sure how useful it is
in the context of actually performing work in parallel. You might want
to emphasize that apply_async() is better suited for that (the only
other way I could think of to take advantage of apply() in parallel
would be to call it from separate threads in the process that created
the pool).

[jnoller]

fixed in trunk, r68862, merged to 3k in r68863