input() uses sys.__stdout__ instead of sys.stdout for prompt

[KeitaKita]

BPO	24402
Nosy	@pitrou, @taleinat, @berkerpeksag, @vadmium, @serhiy-storchaka, @eryksun, @zhangyangyu
Files	issue24402.fix_only.patch: patch with just a fix (no tests) input-terminal.patch: Now with test case

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vadmium'
closed_at = <Date 2015-10-10.10:04:49.822>
created_at = <Date 2015-06-07.09:30:52.044>
labels = ['interpreter-core', 'type-bug']
title = 'input() uses sys.__stdout__ instead of sys.stdout for prompt'
updated_at = <Date 2015-10-10.10:04:49.821>
user = 'https://bugs.python.org/KeitaKita'

bugs.python.org fields:

activity = <Date 2015-10-10.10:04:49.821>
actor = 'martin.panter'
assignee = 'martin.panter'
closed = True
closed_date = <Date 2015-10-10.10:04:49.822>
closer = 'martin.panter'
components = ['Interpreter Core']
creation = <Date 2015-06-07.09:30:52.044>
creator = 'Keita Kita'
dependencies = []
files = ['39697', '40707']
hgrepos = []
issue_num = 24402
keywords = ['patch']
message_count = 15.0
messages = ['244949', '244957', '245279', '245280', '252436', '252455', '252458', '252676', '252677', '252683', '252686', '252688', '252689', '252690', '252699']
nosy_count = 9.0
nosy_names = ['pitrou', 'taleinat', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'eryksun', 'xiang.zhang', 'Keita Kita']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24402'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

[KeitaKita]

On Python 3.4, input() does not use wrapped sys.stdout. For example, the following script should print "Wrapped stdout" by print() and input() because sys.stdout is replaced with Writer object.

--------

import io
import sys


class Writer:
    def __getattr__(self, name):
        return getattr(sys.__stdout__, name)

    def write(self, data):
        if data != '\n':
            sys.__stdout__.write('Wrapped stdout\n')

    def fileno():
        raise OSError()


sys.stdout = Writer()

print('print')
input('input')

---

But the script does not print "Wrapped stdout" as prompt of input(). The script
prints the following.

-----
Wrapped stdout
input
----

Although, when sys.stdin is also wrapped, input() will use sys.stdout for prompt. For example, the following script prints "Wrapped stdout" by print() and input().

----

import io
import sys


class Writer:
    def __getattr__(self, name):
        return getattr(sys.__stdout__, name)

    def write(self, data):
        if data != '\n':
            sys.__stdout__.write('Wrapped stdout\n')

    def fileno():
        raise OSError()


class Reader:
    def __getattr__(self, name):
        return getattr(sys.__stdin__, name)

    def read(self, size):
        return sys.__stdin__.read(size)

    def fileno():
        raise OSError()


sys.stdout = Writer()
sys.stdin = Reader()

print('print')
input('input')

---

The script prints the following.

-----
Wrapped stdout
Wrapped stdout
----

[eryksun]

This looks like an oversight. In the implementation in bltinmodule.c, if checking sys.stdout.fileno() fails, it clears the error without setting tty = 0. It's the same in 3.5.

    /* We should only use (GNU) readline if Python's sys.stdin and
       sys.stdout are the same as C's stdin and stdout, because we
       need to pass it those. */
    tmp = _PyObject_CallMethodId(fin, &PyId_fileno, "");
    if (tmp == NULL) {
        PyErr_Clear();
        tty = 0;
    }
    else {
        fd = PyLong_AsLong(tmp);
        Py_DECREF(tmp);
        if (fd < 0 && PyErr_Occurred())
            return NULL;
        tty = fd == fileno(stdin) && isatty(fd);
    }
    if (tty) {
        tmp = _PyObject_CallMethodId(fout, &PyId_fileno, "");
        if (tmp == NULL)
            PyErr_Clear();
        else {
            fd = PyLong_AsLong(tmp);
            Py_DECREF(tmp);
            if (fd < 0 && PyErr_Occurred())
                return NULL;
            tty = fd == fileno(stdout) && isatty(fd);
        }
    }

[taleinat]

Okay, so the change is straightforward. With this change the entire test suite passes on my machine.

We'll need a test for this for the test suite, and that's far less straightforward. It seems that test.test_builtin.BuiltinTest.check_input_tty() could be a starting point. Could someone come up with a working test for this?

[taleinat]

Here's a patch with just the required fix (no tests).

[vadmium]

Patch itself looks good.

[vadmium]

This patch includes a test case using pty.fork() to ensure stdin is a terminal, so the bug would be triggered. It is a bit tricky, but hopefully works on Unix platforms. I have only tested it on Linux.

[serhiy-storchaka]

LGTM. See also minor note on Rietveld.

I think it is worth to extract all input() tests to separate class.

[vadmium]

Okay I plan to commit a modified version where I moved the input() test methods that use pseudo terminals to a new PtyTests class. The test_input() method stays in the original class, because it shares a helper method with other tests.

I am resisting dropping that decode() step, because it makes any error message from the forked child hard to read:

AssertionError: 256 != 0 : b'quux\r\nTraceback (most recent call last):\r\n  File "/media/disk/home/proj/python/cpython/Lib/test/test_builtin.py", line 1593, in test_input_no_stdout_fileno\r\n    input("prompt")\r\nTypeError: bad argument type for built-in operation\r\n'

[python-dev]

New changeset 425d81d10b13 by Martin Panter in branch '3.4':
Issue bpo-24402: Fix input() when stdout.fileno() fails; diagnosed by Eryksun
https://hg.python.org/cpython/rev/425d81d10b13

New changeset e97d940a6543 by Martin Panter in branch '3.5':
Issue bpo-24402: Merge input() fix from 3.4 into 3.5
https://hg.python.org/cpython/rev/e97d940a6543

New changeset bcc0f8eb6797 by Martin Panter in branch 'default':
Issue bpo-24402: Merge input() fix from 3.5
https://hg.python.org/cpython/rev/bcc0f8eb6797

[vadmium]

Seems my test hangs the Free BSD 9 buildbot, although Free BSD 10 apparently passed.

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.4/builds/1128/steps/test/logs/stdio

[390/390] test_builtin
Timeout (1:00:00)!
Thread 0x0000000801407400 (most recent call first):
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/test/test_builtin.py", line 1585 in test_input_no_stdout_fileno
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 580 in run
File "/usr/home/buildbot/python/3.4.koobs-freebsd9/build/Lib/unittest/case.py", line 628 in __call__