Make shell_tools smarter

[balopat]

Description of the issue

There are two main issues with shell_tools:

1. The subprocess API got much better since python 3.7 - run_cmd and run_shell functionality could be simplified and aligned better with the new subprocess.run API (with capture_output=True).
2. There are Windows failures due to missing primitives. This comes up for example when creating a virtual env:

dev_tools\env_tools.py:55: in create_virtual_env
    'virtualenv', None if verbose else '--quiet', '-p', python_path, venv_path, out=sys.stderr
dev_tools\shell_tools.py:196: in run_cmd
    err,
c:\hostedtoolcache\windows\python\3.7.9\x64\lib\asyncio\base_events.py:587: in run_until_complete
    return future.result()
dev_tools\shell_tools.py:120: in _async_wait_for_process
    process = await future_process
c:\hostedtoolcache\windows\python\3.7.9\x64\lib\asyncio\subprocess.py:217: in create_subprocess_exec
    stderr=stderr, **kwds)
c:\hostedtoolcache\windows\python\3.7.9\x64\lib\asyncio\base_events.py:1544: in subprocess_exec
    bufsize, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_WindowsSelectorEventLoop running=False closed=False debug=False>
protocol = <SubprocessStreamProtocol>
args = ('virtualenv', '-p', 'c:\\hostedtoolcache\\windows\\python\\3.7.9\\x64\\python.exe', 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\cirq-pytest\\test_isolated')
shell = False, stdin = None, stdout = -1, stderr = -1, bufsize = 0, extra = None
kwargs = {}

    async def _make_subprocess_transport(self, protocol, args, shell,
                                         stdin, stdout, stderr, bufsize,
                                         extra=None, **kwargs):
        """Create subprocess transport."""
>       raise NotImplementedError
E       NotImplementedError

[pavoljuhas]

I can take care of this one.

[pavoljuhas]

AFAICT, the functionality in run_cmd and run_shell can be fully replaced with the standard subprocess.run.
The functions have a few extra arguments (log_run_to_stderr, abbreviate_non_option_arguments) for outputting the executed command. I intend to add a new function shell_tools.run which will be a thin wrapper to subprocess.run with an extra option to show the executed command; this function will be then used to replace run_cmd and run_shell.

For sake of digestible reviews, the change will be split to several PRs:

• add a new function shell_tools.run
• replace shell_tools.run_cmd with shell_tools.run
• replace shell_tools.run_shell with shell_tools.run
• purge run_cmd, run_shell and any associated local helpers in the shell_tools module