multiprocessing.Pool windows/linux behaviour difference

[mark-summerfield]

BPO	3598
Nosy	@amauryfa, @pitrou, @mark-summerfield
Files	mtest.py: Simple exercise of multiprocessing.Pool

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-08-19.08:49:37.888>
created_at = <Date 2008-08-19.06:50:56.855>
labels = ['invalid', 'library']
title = 'multiprocessing.Pool windows/linux behaviour difference'
updated_at = <Date 2008-08-19.09:09:58.901>
user = 'https://github.com/mark-summerfield'

bugs.python.org fields:

activity = <Date 2008-08-19.09:09:58.901>
actor = 'mark'
assignee = 'none'
closed = True
closed_date = <Date 2008-08-19.08:49:37.888>
closer = 'amaury.forgeotdarc'
components = ['Library (Lib)']
creation = <Date 2008-08-19.06:50:56.855>
creator = 'mark'
dependencies = []
files = ['11154']
hgrepos = []
issue_num = 3598
keywords = []
message_count = 4.0
messages = ['71408', '71409', '71410', '71411']
nosy_count = 3.0
nosy_names = ['amaury.forgeotdarc', 'pitrou', 'mark']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue3598'
versions = ['Python 3.0']

[mark-summerfield]

When the attached program is run on Linux it runs "instantly" and
outputs one line, e.g.:

$ python3 mtest.py
100 files, 1702627142 bytes

(The number of bytes will vary depending on the system.)

When run on Windows XP there is no output at all; many processes seem to
be created but nothing seems to actually get done.

In both cases I'm using Py30b2.

[amauryfa]

There is a problem in your script.

On Windows, the remote processes have to import the mtest module, in
order to execute the get_size() function.
This in turn calls the main() function, which creates a new pool of
processes, etc... exponential explosion.
Linux forks and does not have this problem.

The solution of course is

if __name__ == '__main__':
    main()

[pitrou]

For what it's worth, this is documented in
http://docs.python.org/dev/library/multiprocessing.html#windows

[mark-summerfield]

On 2008-08-19, Antoine Pitrou wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

For what it's worth, this is documented in
http://docs.python.org/dev/library/multiprocessing.html#windows

Ah yes, sorry I missed that.