Very first multiprocessing example not working on Windows 11

[quattrozhou]

BPO	45914
Nosy	@rhettinger, @terryjreedy, @pfmoore, @tjguk, @zware, @eryksun, @zooba, @quattrozhou

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-11-28.04:37:54.305>
labels = ['3.11', 'type-bug', 'OS-windows', 'docs']
title = 'Very first multiprocessing example not working on Windows 11'
updated_at = <Date 2021-12-04.07:37:31.190>
user = 'https://github.com/quattrozhou'

bugs.python.org fields:

activity = <Date 2021-12-04.07:37:31.190>
actor = 'eryksun'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Windows']
creation = <Date 2021-11-28.04:37:54.305>
creator = 'quattrozhou'
dependencies = []
files = []
hgrepos = []
issue_num = 45914
keywords = []
message_count = 4.0
messages = ['407186', '407189', '407638', '407646']
nosy_count = 9.0
nosy_names = ['rhettinger', 'terry.reedy', 'paul.moore', 'tim.golden', 'docs@python', 'zach.ware', 'eryksun', 'steve.dower', 'quattrozhou']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45914'
versions = ['Python 3.11']

[quattrozhou]

Very first multiprocessing example not working on Windows 11

https://docs.python.org/3/library/multiprocessing.html

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

Tried Python 3.7, 3.8, 3.9, 3.10
Also tried Ubuntu which works fine.
But on Windows (clean installed python):

>>> from multiprocessing import Pool
>>> def f(x):
...     return x*x
...
>>> with Pool(5) as p:
...   print(p.map(f, [1, 2, 3]))
...
Process SpawnPoolWorker-14:
Process SpawnPoolWorker-13:
Traceback (most recent call last):
Process SpawnPoolWorker-15:
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Python39\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 114, in worker
    task = get()
  File "C:\Program Files\Python39\lib\multiprocessing\queues.py", line 368, in get
    return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'f' on <module '__main__' (built-in)>

[eryksun]

AttributeError: Can't get attribute 'f'
on <module '__main__' (built-in)>

The Windows API only supports the spawn method of process creation. In POSIX (except macOS), the default is the fork method, for which the child inherits the interactive main module of the parent. If you switch to the spawn method in Linux via multiprocessing.set_start_method('spawn'), you'll see the same error.

multiprocessing is one package where it's necessary in Windows to test examples using a script. This is implied in the guidelines when it says to "[m]ake sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process)". It's the fact the main module has to be importable by child processes that matters in this case. The behavior is noted with an example at the end of the introduction:

Note
Functionality within this package requires that the __main__ module
be importable by the children. This is covered in Programming 
guidelines however it is worth pointing out here. This means that 
some examples, such as the multiprocessing.pool.Pool examples will 
not work in the interactive interpreter. For example: ...

If it were up to me this note would be at the beginning of the introduction, where everyone would certainly see it. As is, the reader is expected to at least scan over the entire introduction.

[terryjreedy]

I think something should be said right after the example. But the 20 line note seems a bit too much at this location. Perhaps insert

(For reasons given _below_, this will raise error if enter interactively.)

with _below_, say, linked to the 20 line note.

It is not quite clear to me it this only applies on Windows or is general.

[eryksun]

not quite clear to me it this only applies on Windows or is general.

macOS defaults to the spawn method (bpo-33725).