popen('python -c"...."') tends to hang

[sabren]

BPO	226766
Nosy	@gvanrossum, @tim-one, @mhammond

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/mhammond'
closed_at = <Date 2002-04-03.01:53:35.000>
created_at = <Date 2000-12-24.16:08:03.000>
labels = ['OS-windows']
title = 'popen(\'python -c"...."\') tends to hang'
updated_at = <Date 2002-04-03.01:53:35.000>
user = 'https://bugs.python.org/sabren'

bugs.python.org fields:

activity = <Date 2002-04-03.01:53:35.000>
actor = 'mhammond'
assignee = 'mhammond'
closed = True
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2000-12-24.16:08:03.000>
creator = 'sabren'
dependencies = []
files = []
hgrepos = []
issue_num = 226766
keywords = []
message_count = 8.0
messages = ['2749', '2750', '2751', '2752', '2753', '2754', '2755', '2756']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'tim.peters', 'nobody', 'mhammond', 'sabren', 'rcamesz']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue226766'
versions = []

[sabren]

eg:

import os
os.popen('python -c"x=1;print x"').readlines()

.. On my machine, using popen to call a second instance of python almost always causes python to freeze. No window pops up, but if I press alt-tab, there's an icon for w9xpopen.exe

oddly:

>> os.popen('python -c"print"').readlines()

and

> os.popen('python -c""').readlines()

both work fine.

... This bug is different from bpo-114780 in that it is repeatable and consistent.

It happens on open, so is different from bpo-125891. Eg:

>> proc = os.popen('python -c"x=1; print x"')

will cause the crash.

[sabren]

.. er.. whoops.. It hangs/freezes, not crashes. And in fact, it occasionally returns control to python after several minutes.

[tim-one]

Mark, any idea? The first example also appears to hang for me consistently (W98SE). In a debug build under the debugger, breaking during the hang yields a gibberish disassembly window (i.e., it's not showing code!), so I didn't get anywhere after 5 minutes of thrashing.

[gvanrossum]

Any chance you have Norton AntiVirus 2000 running?

See http://sourceforge.net/bugs/?group_id=5470&func=detailbug&bug_id=114598

I've basically given up on popen for windows. :-(

os.spawn*() works great though -- if you don't need to read the output. :-)

[mhammond]

I have done lots of playing with this over the last month or so. The problem appears to be something to do with "python.exe" used as the target of such a popen command. (Ironically, we tried to use popen to capture remote Python invocations for Komodo - as I guess you are for IDLE)

I have experimented with 3 "different" popen implementations: Python's, one written in Python using the win32 API directly, and one using the Netscape NSPR libraries. They all, however, end up doing the same basic thing with the same basic Windows API functions. They all behace the same WRT reading input.

Python.exe (and one or 2 other exes) appear to hang when they are in an "interactive loop", and the spawning process is trying to read the input pipe. My experiements at breaking into the debugger shows Windows blocked inside the ReadFile() function.

Note that Perl.exe does _not_ appear to provoke this (but Perl doesnt have a builtin interactive loop, so is harder to prove). Also note that "cmd.exe" also does _not_ provoke this - ie, both Perl.exe and cmd.exe both work fine, correctly reading and writing either pipe when the process is "interactive"

So - after all this, my best guess was that there is something weird in the stdio Python uses to read stdin, and that Perl and cmd.exe both avoid this. Random, 100% speculative guess is that the "/MD" stdio is broken, but others wont be.

This is as far as I got - I was going to experiment with /MD, other input techniques etc, but never got to it. It should be simple to reproduce with a simple test .exe, but alas never got to that either. I should have more time over the next few weeks.

[nobody]

um.....

[rcamesz]

Logged In: YES
user_id=313190

Ah, well, it would seem that this is in fact the same bug I
reported much later as bpo-457466: "popenx() argument mangling
hangs python". For some reason I overlooked it when posting
my report, otherwise I'd have added my comments here. Sorry
to add to the confusion. If you need to work around this bug
(it's basically a bug in the commandline parsing), my report
has information on that, as wel as a possible way of fixing
it.

My patched version of popen9x.exe does not hang on Win98,
but
'reports' the error (reformatted to fit the margins):

Python 2.1.1 (#20, Jul 20 2001, 01:19:29) [MSC 32 bit 
(Intel)] on win32 Type "copyright", "credits" or "license" 
for more information.
>>> import os
>>> os.popen('python -c"x=1;print x"').read()
"ARGUMENT PARSE ERROR\nThe way double quotes and spaces were 
used\ncaused popenx () to split the commandline.\n\nArgument 
count = 3\nArgument 0 = 
'C:\\PYTHON21\\w9xpopen.exe'\nArgument 1 = 'C:\\COMMAND.COM 
/c python -cx=1;print'\nArgument 2 = 'x'\n"
>>>

[mhammond]

Logged In: YES
user_id=14198

Dupe of bug 457466.