PyOS_InputHook inconsistency on Windows

[mdehoon]

BPO	1049855
Nosy	@loewis, @kbkaiser
Files	myreadline.c.diff myreadline.c.20050610.diff myreadline.c.20050715.diff myreadline.c.20050728.diff myreadline.c.20050803.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2005-09-25.04:29:26.000>
created_at = <Date 2004-10-19.09:03:42.000>
labels = ['interpreter-core']
title = 'PyOS_InputHook inconsistency on Windows'
updated_at = <Date 2005-09-25.04:29:26.000>
user = 'https://bugs.python.org/mdehoon'

bugs.python.org fields:

activity = <Date 2005-09-25.04:29:26.000>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-10-19.09:03:42.000>
creator = 'mdehoon'
dependencies = []
files = ['6308', '6309', '6310', '6311', '6312']
hgrepos = []
issue_num = 1049855
keywords = ['patch']
message_count = 13.0
messages = ['47091', '47092', '47093', '47094', '47095', '47096', '47097', '47098', '47099', '47100', '47101', '47102', '47103']
nosy_count = 3.0
nosy_names = ['loewis', 'kbk', 'mdehoon']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1049855'
versions = ['Python 2.4']

[mdehoon]

PyOS_InputHook is a pointer to a function to be called
periodically when Python is idle. It is used, for
example, to get messages delivered to graphics windows.
If I compile Python from source (which uses
Modules/readline.c), PyOS_InputHook is called ten times
per second. However, with the Windows-version of
Python, PyOS_InputHook is called only once for each
command typed by the user. It seems that the problem
resides in Parser/myreadline.c (which I presume is used
for the Windows-version of Python):

if (PyOS_InputHook != NULL)
(void)(PyOS_InputHook)();
...
p = fgets(buf, len, fp);

Python idles at "fgets", but PyOS_InputHook is not
being called while Python is idle.

The attached patch solves this problem by using the
"select" function, basically in the same way as what is
in Module/readline.c. I don't know how to compile
Python for Windows, so I wasn't able to test this patch.

[mdehoon]

Logged In: YES
user_id=488897

I have now tested this patch. After fixing one error (I had
forgotton to declare one variable), the patch works
correctly. I have uploaded a fixed patch.
Note that this bug occurs not only on Windows, but on any
Python compiled without readline. (which allowed me to test
this patch by compiling Python without readline support).

[kbkaiser]

Logged In: YES
user_id=149084

The revised patch looks ok to me.

[mdehoon]

Logged In: YES
user_id=488897

Thanks for looking at this patch.

Today I found out that it is possible to compile Python
2.4.1 with the older Microsoft Visual Studio 6.0, which I
happen to have in my office. It turned out that the patch
does not work correctly on Windows, due to the fact that the
"select" function doesn't work with stdin on Windows. I will
fix the patch so it'll work on Windows too.

[kbkaiser]

Logged In: YES
user_id=149084

Yeah, that's why I didn't want to check it in, I can't build
on Windows ithe VC 5.

[mdehoon]

Logged In: YES
user_id=488897

I have rewritten the patch to include Windows support. I
compiled Python on Windows with VC98, and on Linux and Mac
OS X and found no problems with it. The new patch is attached.

[mdehoon]

Logged In: YES
user_id=488897

Thomas Heller sent me this comment:

The PyOS_StdioReadline function calls my_fgets with a file
pointer argument. The my_fgets function in the patch assumes
that STD_INPUT_HANDLE is the handle to use - is this
assumption always correct?

He is right, this assumption is not necessarily correct. I
have made a new patch (labeled 20050610) to solve this
issue. This latest version has been tested on Cygwin, but
not yet on Windows -- I need to dig up a compiler for
Windows first.

[mdehoon]

Logged In: YES
user_id=488897

I have now recompiled Python on Windows to test this patch.
When compiling this patch, I found out that _get_osfhandle
needs to be cast to a HANDLE*. The latest version of this
patch (dated 20050715) includes this cast. I have tested
this patch by running the test suite.
One note: Whereas "python -i inputfile.py" and "python <
inputfile.py" work as expected, "python -i < inputfile.py"
behaves differently with this patch (python waits for the
user to hit enter before proceeding). I am not sure if this
is significant, as I wouldn't know what a user might try to
achieve with "python -i < inputfile.py".
I'd be happy to send a binary for Windows to anybody who
would like to test this patch.

[mdehoon]

Logged In: YES
user_id=488897

I have found a solution for the problem described below
("python -i < inputfile.py" behaves differently). The
solution was to use _isatty to check if stdin is redirected,
and check for keyboard input only if stdin is not
redirected. With the latest version of this patch
(20050728), there are no changes in Python's behaviour on
Windows or Unix (except that PyOS_InputHook is called ten
times per second, which is what this patch intends to
solve). I apologize for the fact that multiple iterations
were needed to converge to the right (I hope) solution for
this bug.

[mdehoon]

Logged In: YES
user_id=488897

One more fix: I noticed that on Windows, the patch causes
the CPU usage to go up to 100%. The reason for this is that
if the users presses a control key (e.g. shift), the
WaitForSingleObject function returns, but _kbhit() returns
false since no character input is waiting in the buffer.
Hence, the loop continues to run, and WaitForSingleObject
will again return because the control key stroke has not
been flushed from the buffer. The fix is in the patch dated
20050803, which is to flush the buffer with
FlushConsoleInputBuffer if a key was hit but no character
input is available.

[loewis]

Logged In: YES
user_id=21627

I think this code is way too fragile to be applied to 2.4.
As for specific errors I could find:

• if the system is not windows, it invokes select
  unconditionally; this should be conditioned with HAVE_SELECT
• the Windows version uses kbhit; this may or may not
  operate on the same handle as hStdIn. In fact, it likely is
  a different handle. See the source code of the crt: kbhit
  uses _coninpfh, which is created from opening CONIN$. Try
  using GetNumberOfConsoleInputEvents/PeekConsoleInput
  instead, checking for a KEY_EVENT event.
• as stdin may be buffered, it may be the case that there is
  already input available, even though the file descriptor is
  not ready.