failed pickle hangs python on exit in CMD.exe only

[cmcmarrow]

BPO	42178
Nosy	@pfmoore, @pitrou, @tjguk, @zware, @eryksun, @zooba, @applio, @cmcmarrow, @John-Ted
PRs	bpo-42178: Fix issue causing cmd to hang #23290
Files	mp_hang.py: run mp_hang.py in cmd to get hang

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-27.20:52:33.937>
labels = ['OS-windows', 'type-crash', '3.9']
title = 'failed pickle hangs python on exit in CMD.exe only'
updated_at = <Date 2020-11-16.18:29:48.700>
user = 'https://github.com/cmcmarrow'

bugs.python.org fields:

activity = <Date 2020-11-16.18:29:48.700>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2020-10-27.20:52:33.937>
creator = 'charles.mcmarrow.4'
dependencies = []
files = ['49543']
hgrepos = []
issue_num = 42178
keywords = ['patch']
message_count = 3.0
messages = ['379803', '379816', '381138']
nosy_count = 9.0
nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'davin', 'charles.mcmarrow.4', 'John-Ted']
pr_nums = ['23290']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue42178'
versions = ['Python 3.9']

[cmcmarrow]

This bug only happens for me in Windows 10 when using cmd.exe. This bug does not happen IDLE. It sometimes happens in PowerShell.

I ran mp_hang.py
with
3.9.0 hangs on cmd/powershell
3.8.6 hangs on cmd
3.8.5 hangs on cmd/powershell
Microsoft Windows [Version 10.0.19041.572]

3.8.6 error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\yoshi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 102, in spawn_main
    source_process = _winapi.OpenProcess(
OSError: [WinError 87] The parameter is incorrect

[eryksun]

Due to the exception raised when trying to pickle main.<locals>.dummy in popen_spawn_win32.Popen(), the main process may have already exited by the time the spawned worker calls WinAPI OpenProcess in spawn.spawn_main(). I suppose the reduction of prep_data and process_obj in popen_spawn_win32.Popen() could be dumped to an io.BytesIO instance before WinAPI CreateProcess is called. That way an exception in the pickler would occur before the child process is created.

There's also a race condition that can hang the worker during interpreter startup. When sys.stdin is initialized, the buffered reader does a raw tell(). If the raw layer is an io.FileIO instance (e.g. console input with PYTHONLEGACYWINDOWSSTDIO defined), the raw tell() ultimately calls WinAPI SetFilePointerEx (via _lseeki64). This can hang if the kernel file is already blocked in a synchronous I/O request, which might be the case if the main process has already terminated and its parent (e.g. cmd.exe or powershell.exe) has resumed reading from the console. portable_lseek() in Modules/_io/fileio.c should detect a non-disk file via WinAPI GetFileType and just set self->seekable to 0 in this case.

[eryksun]

Steve or Zach, please review PR 23290, submitted by Teugea Ioan-Teodor. It modifies the startup sequence for worker processes by creating the child process with a suspended thread and pickling process_obj to a BytesIO instance before resuming the thread and writing the pickle data to the pipe. If pickling fails, the child process is terminated before the thread is resumed, since there's no point in allowing the child to execute.

This doesn't directly address the issue with interpreter startup that can cause Python to hang on a synchronous I/O request, but it does avoid the problem.

Alternatively, when there's a pickling error, the parent could close its end of the pipe and wait for the child to exit before propagating the exception, but I'd rather skip executing the child completely.