Repository navigation
multiprocessing start blocked if start() is in subprocess #7494
Description
Description of the issue
Python multiprocessing buildin module.
By starting a new proccess (subprocess) in a process by proc.start() script is blocked when executed in pyinstaller 5.9. Last working pyinstaller version supports sub-processes seems to be v.5.6.
Context information (for bug reports)
-
Output of
pyinstaller --version:5.9.0) -
Version of Python: 3.11.2
-
Platform: GNU/Linux Debian
-
How you installed Python: apt
-
Did you also try this on another platform? Yes Windows, Does it work there? No
-
try the latest development version, using the following command:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zipThe same
A minimal example program which shows the error
from multiprocessing import Process
def runner2():
print('runner2 was executed')
def runner1():
print('runner1 in proc')
proc2 = Process(target=runner2)
print('try to start subproc')
proc2.start() # BLOCKED HERE
print('start executed')
proc = Process(target=runner1)
proc.start()
Execute
$ dist/run
runner1 in proc
try to start subpr
Stacktrace / full error message
CONTROL+C
^CException ignored in sys.unraisablehook: <built-in function unraisablehook>
KeyboardInterrupt
Process Process-1:
Traceback (most recent call last):
File "multiprocessing/process.py", line 314, in _bootstrap
File "multiprocessing/process.py", line 108, in run
File "client/run.py", line 10, in runner1
proc2.start() # BLOCK HERE
^^^^^^^^^^^^^
File "multiprocessing/process.py", line 121, in start
File "multiprocessing/context.py", line 224, in _Popen
File "multiprocessing/context.py", line 281, in _Popen
File "PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py", line 73, in __init__
KeyboardInterrupt
Any ideas, fixes, workarounds?
Activity
rokm commentedon Mar 15, 2023
At first glance, this looks like regression caused by #7411.
rokm commentedon Mar 15, 2023
And it likely happens because the default (
fork) start method is used, under which the child process inherits the copy of the (now locked) lock, and ends up dead-locked.But since
forkdoes not in fact start the new process from scratch, I wonder if we even need to override theforking.Popento restore_MEIPASS2in the first place...Until this is resolved, you'll need to either use older version of PyInstaller, or manually undo the changes made in 617be5d.