Wait subprocess cleanup

[davidlatwe]

On Windows, when using Rez with Python 3.6, OSError: [WinError 6] The handle is invalid raised when start caching package.

Here's the error traceback that I reproduced with Rez installed with Python 3.6

F:\>rez-env big
Traceback (most recent call last):
  File "c:\miniconda3\envs\py36\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\miniconda3\envs\py36\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\davidlatwe.lai\rez\core\Scripts\rez\rez-env.exe\__main__.py", line 7, in <module>
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\cli\_entry_points.py", line 143, in run_rez_env
    return run("env")
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\cli\_main.py", line 170, in run
    returncode = run_cmd()
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\cli\_main.py", line 162, in run_cmd
    return func(opts, opts.parser, extra_arg_groups)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\cli\env.py", line 256, in command
    block=True)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\resolved_context.py", line 958, in _check
    return fn(self, *nargs, **kwargs)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\resolved_context.py", line 1343, in execute_shell
    self._execute(executor)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\utils\memcached.py", line 266, in wrapper
    return func(*nargs, **kwargs)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\resolved_context.py", line 1910, in _execute
    executor.append_system_paths()
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\rex.py", line 1255, in append_system_paths
    paths = sh.get_syspaths()
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rezplugins\shell\cmd.py", line 105, in get_syspaths
    stderr=subprocess.PIPE, shell=True, text=True)
  File "c:\users\davidlatwe.lai\rez\core\lib\site-packages\rez\utils\execution.py", line 84, in __init__
    super(Popen, self).__init__(args, **kwargs)
  File "c:\miniconda3\envs\py36\Lib\subprocess.py", line 616, in __init__
    _cleanup()
  File "c:\miniconda3\envs\py36\Lib\subprocess.py", line 205, in _cleanup
    res = inst._internal_poll(_deadstate=sys.maxsize)
  File "c:\miniconda3\envs\py36\Lib\subprocess.py", line 1055, in _internal_poll
    if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
OSError: [WinError 6] The handle is invalid

This seems to be an won't fix issue of Python<=3.6 on Windows.
Based on issue https://bugs.python.org/issue37380 and the comment https://bugs.python.org/msg346332

[nerdvegas]

won't this raise a TimeoutExpired exception if the rez-pkg-cache daemon runs for longer than 1 sec? (Or am I missing something - this seems highly probable).

[davidlatwe]

Sorry, somehow I got really sloppy on this one, and you are right. It raises TimeoutExpired exception when the process runs longer than 1 sec.

I have changed to use Python 3.7 and didn't revisit this PR since then.
Now that I am back and I found the reason of this issue seems to be related to garbage collect...

Changing _.wait(1) into import gc;gc.collect() solved the problem. (but we shouldn't need this...)

One additional info, I got the idea that the issue might be related to the garbage collection is from this comment, and here's the repro that's been mentioned.

[davidlatwe]

Oh, jsut do del _ before function return or not keeping the reference of the subprocess instance in variable _ solved the problem. :O