Windows bug picking up stdin from a pipe

[oeffner]

BPO	16892
Nosy	@tjguk, @zware, @eryksun, @zooba
Files	testsubproc.py: python script demonstrating the problem

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-09-03.22:15:29.657>
created_at = <Date 2013-01-08.13:44:24.548>
labels = ['type-bug', 'invalid', 'OS-windows', 'build']
title = 'Windows bug picking up stdin from a pipe'
updated_at = <Date 2018-09-03.22:15:29.655>
user = 'https://bugs.python.org/oeffner'

bugs.python.org fields:

activity = <Date 2018-09-03.22:15:29.655>
actor = 'zach.ware'
assignee = 'none'
closed = True
closed_date = <Date 2018-09-03.22:15:29.657>
closer = 'zach.ware'
components = ['Build', 'Windows']
creation = <Date 2013-01-08.13:44:24.548>
creator = 'oeffner'
dependencies = []
files = ['28631']
hgrepos = []
issue_num = 16892
keywords = []
message_count = 4.0
messages = ['179352', '223203', '223341', '324543']
nosy_count = 6.0
nosy_names = ['tim.golden', 'BreamoreBoy', 'zach.ware', 'eryksun', 'steve.dower', 'oeffner']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue16892'
versions = ['Python 2.6', 'Python 2.7', 'Python 3.3']

[oeffner]

Hi,

This is a bug that seems to exist on python 2.7, python 3.3 on Windows versions XP, Vista, 7 and 8 and has been around for some years, presumably also in other python versions. It is only recently I have managed to better isolate it although not completely.

My Windows PC is set up with some doskey macros that are loaded at each
instance of the commandline interpreter cmd.exe with the registry key
HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun\mymacros.cmd
Unfortunately this seems to delete stdin when stdin is piped to python scripts that runs another executable invoked with subprocess. In this case the command:

python32 testsubproc.py < mytextfile.txt

will not work. The textfile mytextfile.txt will be ignored. If subprocess is invoked with shell=False the textfile does get piped into stdin. This is however not an option as our porgrams are distributed on other platforms. It would also give an inconsistent experience on the same OS (Windows).

If mymacros.cmd is called interactively from the command prompt the problem does not exist and stdin can be piped into the python script with no problem.

I have attached a working script below which includes a comment section with a small C++ program and the registry key in question to demonstrate the problem.

I would be grateful if there are people who know how to get to the bottom of this bug or even fix it.

Many thanks,

Robert

[BreamoreBoy]

@robert we're sorry about the delay in getting back to you. Sorry Windows gurus this is over my head :(

[eryksun]

Doskey is a command-line interface for a subset of the Win32 console API. Macros are implemented as console input aliases by calling AddConsoleAlias.

http://msdn.microsoft.com/en-us/library/ms681935

You can load macros from a text file that defines aliases for multiple EXEs:

doskey /macrofile=doskey.cfg

Create this file from your current macros:

doskey /macros:all > doskey.cfg

You can also use ctypes to add an alias.

    #!python3
    import ctypes
    import subprocess
    
    AddConsoleAliasW = ctypes.windll.kernel32.AddConsoleAliasW
    AddConsoleAliasW.argtypes = [ctypes.c_wchar_p] * 3
    
    # $db => import builtins;dir(builtins)
    AddConsoleAliasW("$db",
                     "import builtins;dir(builtins)",
                     "python.exe")

    #subprocess.call("doskey")
    subprocess.call("stdintest.exe", shell=True)

The "bug" occurs if I uncomment the line to run doskey.

When stdin is a file, calling sys.stdin.tell() before and after running doskey shows that it's reading standard input, for whatever reason. That's its prerogative, not a bug. Calling sys.stdin.seek(0) after running doskey works, but that's not a possibility when doskey is run via the cmd shell's AutoRun.

By the way, redirection to a file is not a pipe. It actually works for a pipe such as

type mytxtfile.txt | testsubproc.py

The good news is that you only need to load doskey macros once for a given console window. You can use an AutoRun script that tests and sets an environment variable:

@echo off
if "%AUTORUN_DOSKEY%"=="No" goto end
echo %0: setting doskey macros
"%SystemRoot%\System32\doskey.exe" /macrofile="%AppData%\doskey.cfg"
set AUTORUN_DOSKEY=No

:end

This should take care of cases where you're redirecting standard input to a file, assuming the shell wasn't started with the /d option to skip AutoRun.

On Posix systems /bin/sh -c doesn't execute .profile, .bashrc, etc. So maybe on Windows Popen should use %ComSpec% /d /c. You'd still have this problem with os.system, however.

[zware]

Eryk's diagnosis sounds like this is not a bug in Python or something that Python needs to or even should fix.