Windows - Popen (subprocess.py) does not call _handle.Close() at all

[MateuszKlatt]

BPO	28364
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-02-25.12:48:33.735>
created_at = <Date 2016-10-05.14:53:26.542>
labels = ['type-bug', 'library', 'OS-windows']
title = 'Windows - Popen (subprocess.py) does not call _handle.Close() at all'
updated_at = <Date 2021-02-25.12:48:33.734>
user = 'https://bugs.python.org/MateuszKlatt'

bugs.python.org fields:

activity = <Date 2021-02-25.12:48:33.734>
actor = 'eryksun'
assignee = 'none'
closed = True
closed_date = <Date 2021-02-25.12:48:33.735>
closer = 'eryksun'
components = ['Library (Lib)', 'Windows']
creation = <Date 2016-10-05.14:53:26.542>
creator = 'Mateusz Klatt'
dependencies = []
files = []
hgrepos = []
issue_num = 28364
keywords = []
message_count = 2.0
messages = ['278129', '278146']
nosy_count = 6.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'Mateusz Klatt']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28364'
versions = ['Python 2.7']

[MateuszKlatt]

_subprocess.TerminateProcess(self._handle, 1)

is not enough, on windows need to call self._handle.Close() after that

self._handle.Close() should be also called in __del__ - for the process es that ware not killed bu user, but terminated by themselves.

To reproduce... run popen in loop and observe file descriptors usage (SysInternals... handle -s -p <pid>)

[eryksun]

In 2.7, the _handle attribute is a _subprocess_handle object, which automatically calls CloseHandle when deallocated. For example:

>>> p = subprocess.Popen('python -c "import time; time.sleep(120)"')

CreateProcess returns both the process handle and the thread handle. Python doesn't use the thread handle, so it explicitly closes it by calling ht.Close():

Breakpoint 0 hit
KERNELBASE!CloseHandle:
00007ffb`a32fdf70 4883ec28        sub     rsp,28h
0:000> kc 5
Call Site
KERNELBASE!CloseHandle
python27!sp_handle_close
python27!PyCFunction_Call
python27!call_function
python27!PyEval_EvalFrameEx
0:000> g

(IMO, it should skip this step if creationflags contains CREATE_SUSPENDED. The thread handle makes it simpler to call ResumeThread.)

On the other hand, the process handle is deallocated implicitly when it's no longer referenced:

    >>> type(p._handle)
    <type '_subprocess_handle'>
    >>> hex(int(p._handle))
    '0x164L'

>>> p.terminate()
>>> del p

Breakpoint 0 hit
KERNELBASE!CloseHandle:
00007ffb`a32fdf70 4883ec28        sub     rsp,28h
0:000> kc 5
Call Site
KERNELBASE!CloseHandle
python27!sp_handle_dealloc
python27!dict_dealloc
python27!subtype_dealloc
python27!PyDict_DelItem
0:000> r rcx
rcx=0000000000000164

If the process handles aren't being closed in your case, then you're probably keeping a reference to the Popen instances.

P.S. A Windows handle is not a "file descriptor". Kernel handles are user-mode references to kernel objects. They aren't "files" unless the object happens to be a File object. A process handle references a kernel Process object.