Bug Fix: Plugins Fail To Load on Python 3.8+ #505
Conversation
…wn processing which does not share memory between child processes.
| @@ -55,7 +55,8 @@ def scan_files(self, *filenames: str, num_processors: Optional[int] = None) -> N | |||
| if not num_processors: | |||
| num_processors = mp.cpu_count() | |||
|
|
|||
| with mp.Pool(processes=num_processors) as p: | |||
| # Default to fork multiprocessing. Python 3.8+ defaults to spawn which doesn't share memory. | |||
| with mp.get_context('fork').Pool(processes=num_processors) as p: | |||
There was a problem hiding this comment.
I did find on https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
that it says fork should not be used on macOS, and that it's not available on Windows
so it may be better to resolve this some other way
There was a problem hiding this comment.
Updated to pass the settings object to the child processes.
|
I think it may be better to use the approach with #482 -- essentially, initializing a new settings object per process, rather than sharing memory. It's been a while since I looked at it, but the This way, we'd be less dependent on OS level details since this library is used on multiple platforms. |
Problem
spawnwhere before it wasforkspawndoes not share any memory between the parent and child processes.Solution