Skip to content

Commit

Permalink
TST,BUG: Us context from spawn to fork in (#22204)
Browse files Browse the repository at this point in the history
Since Python 3.8, the default start method for multiprocessing has been changed from fork to spawn on macOS
The default start method is still fork on other Unix platforms[1], causing inconsistency on memory sharing model
It will cause a memory-sharing problem for the test test_large_zip on macOS as the memory sharing model between spawn and fork is different
The fix

Change the start method for this test back to fork under this testcase context
In this test case context, the bug that caused default start method changed to spawn for macOS will not be triggered
It is context limited, so this change will not affect default start method other than test_large_zip
All platforms have the same memory sharing model now
After the change, test_large_zip is passed on macOS
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Closes gh-22203
  • Loading branch information
vxst authored and charris committed Sep 7, 2022
1 parent e18dd98 commit aadd0c6
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions numpy/lib/tests/test_io.py
Expand Up @@ -13,7 +13,7 @@
from io import BytesIO, StringIO
from datetime import datetime
import locale
from multiprocessing import Process, Value
from multiprocessing import Value, get_context
from ctypes import c_bool

import numpy as np
Expand Down Expand Up @@ -595,7 +595,12 @@ def check_large_zip(memoryerror_raised):
# Use an object in shared memory to re-raise the MemoryError exception
# in our process if needed, see gh-16889
memoryerror_raised = Value(c_bool)
p = Process(target=check_large_zip, args=(memoryerror_raised,))

# Since Python 3.8, the default start method for multiprocessing has
# been changed from 'fork' to 'spawn' on macOS, causing inconsistency
# on memory sharing model, lead to failed test for check_large_zip
ctx = get_context('fork')
p = ctx.Process(target=check_large_zip, args=(memoryerror_raised,))
p.start()
p.join()
if memoryerror_raised.value:
Expand Down

0 comments on commit aadd0c6

Please sign in to comment.