subprocess.Popen.communicate(...) hangs on Windows

[AlexQuinn]

BPO	8631
Nosy	@terryjreedy, @tjguk, @4kir4, @zware, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tjguk'
closed_at = <Date 2017-06-18.19:49:54.199>
created_at = <Date 2010-05-05.21:45:04.639>
labels = ['invalid', 'type-bug', 'library', 'expert-IO', 'OS-windows']
title = 'subprocess.Popen.communicate(...) hangs on Windows'
updated_at = <Date 2017-06-18.19:49:54.197>
user = 'https://bugs.python.org/AlexQuinn'

bugs.python.org fields:

activity = <Date 2017-06-18.19:49:54.197>
actor = 'tim.golden'
assignee = 'tim.golden'
closed = True
closed_date = <Date 2017-06-18.19:49:54.199>
closer = 'tim.golden'
components = ['Library (Lib)', 'Windows', 'IO']
creation = <Date 2010-05-05.21:45:04.639>
creator = 'Alex Quinn'
dependencies = []
files = []
hgrepos = []
issue_num = 8631
keywords = []
message_count = 10.0
messages = ['105088', '113257', '113283', '113419', '122087', '132804', '222053', '222136', '222145', '296291']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'tim.golden', 'Alex Quinn', 'akira', 'BreamoreBoy', 'paroga', 'zach.ware', 'steve.dower', 'Saimadhav.Heblikar']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8631'
versions = ['Python 3.5']

[AlexQuinn]

After using subprocess.Popen(...).communicate(), the session hangs.

c:\>python31
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> Popen(["echo","Hello!"], stdout=PIPE).communicate()[0]
b'Hello!\n'
>>>

At this point, the session stops responding to keyboard input.

=========================================
MORE DETAILS:

Neither Ctrl-C, Ctrl-Z, nor Ctrl-D will break out of it, but Ctrl-Break does break out of it.

The task manager shows no CPU activity for the process.

I've seen problem with Python v3.1.2 and v2.6.1 on Windows 7.

It works fine under Linux:

$ python3.1
Python 3.1 (r31:73572, Mar 22 2010, 14:57:00)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> Popen(["echo","Hello!"], stdout=PIPE).communicate()[0]
b'Hello!\n'
>>> print("No problems under Linux")
No problems under Linux
>>>

Is this bug type "crash" or "behavior"? I went with the former, since it ends the usable Python session.

[BreamoreBoy]

I can't reproduce this using Windows Vista.

[tjguk]

I can't reproduce on W7. Strangely, though, although my banner suggests that I have exactly the same build as you, I get a "\r\n" at the end of the communicate bytestream, not a simple "\n" as you're getting. Do you have any environment variables or Python command-line switches which might be affecting anything?

Could you try with python31/26 -E -S just in case, please?

[AlexQuinn]

I am on Windows 7. I realized the "echo" command I'm piping to belongs to Cygwin. I'll try to make a different example to either support this, or otherwise close the bug.

Thanks!

[briancurtin]

I'm with Tim and Mark - can't reproduce this, so I'm closing the report. If you are able to find another case which can reproduce this, feel free to re-open.

[paroga]

I can reproduce this on WinXP with 2.5, 2.7 and 3.2. On my other box (Win7 64bit) the same code works without problems.
This problem happens only when I make a subprocess.call() to executables from my msysgit installation (e.g. echo, git). A call to 'notepad' or 'svn' works without problems.

After entering the following python does not respond to any input:
>>> import subprocess
>>> subprocess.call('echo')

[terryjreedy]

There are certainly some problems with subprocess on Windows.
Note .../python34> pip pyflakes  installs in a second or two.
...> pyflakes -h  # Windows Command Propmpt WCP
>>> import subprocess as s; s.check_output("pyflakes -h")
console interpreter CI or Idle, all produce help output

WCP> pyflakes c:\programs\python34\lib\turtle.py # or / instead of \
c:\programs\python34\lib\turtle.py:572: local variable 'rgb' is assigned to but never used
... 50+ lines (all < 1/2 second)
c:\programs\python34\lib\turtle.py:4139: undefined name 'exitonclick'

CI>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
pause, so almost think is hanging, then

c:\programs\python34\lib\lib2to3\tests\data\bom.py:2:17: invalid syntax print "BOM BOOM!"
...100 more error for various files.
pyflakes ignores file name given and checks entire stdlib.

ID>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
hangs indefinitely, no output

CI or ID >>> s.check_output("pyflakes c:/programs/python34/lib/turtle.py")  # / instead of \ which was ok in WCP
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "F:\Python\dev\5\py35\lib\subprocess.py", line 627, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command 'pyflakes c:/programs/python34/lib/turtle
.py' returned non-zero exit status 1

Above is CI 3.5 and Idle 3.4. subprocess and pyflakes work fine on linux. We are trying to add a feature to Idle to do above on file in editor, and are stuck here. The actual provisional code is
+ proc = Popen(args, stdout=PIPE, stderr=PIPE)
+ proc.wait()
+ output, error = map(lambda b:b.decode('utf-8'), proc.communicate())
where args is list, which shows the same hang forever behavior.

[4kir4]

ID>>> s.check_output("pyflakes c:\programs\python34\lib\turtle.py")
hangs indefinitely, no output

It might be unrelated to the issue but "\t" is a tab; a raw-string literal should be used instead:

>>> from subprocess import check_output
>>> check_output(r"pyflakes c:\programs\python34\lib\turtle.py")

[terryjreedy]

That has been pointed out to me. I need to repost with the results after correcting the commands (there is still a problem).

[tjguk]

I can't reproduce either on 2.7 or on 3.5 with any of the examples shown. Closing again as not-a-bug.