test_builtin: PtyTests fail when run twice

[vstinner]

BPO	41034
Nosy	@vstinner, @shihai1991
Superseder	bpo-13886: test_builtin.PtyTests fail on non-ASCII characters if the readline module is loaded

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-01-17.12:33:21.372>
created_at = <Date 2020-06-19.10:22:34.521>
labels = ['tests', '3.10']
title = 'test_builtin: PtyTests fail when run twice'
updated_at = <Date 2022-01-17.12:33:21.371>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-01-17.12:33:21.371>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-17.12:33:21.372>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-06-19.10:22:34.521>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41034
keywords = []
message_count = 7.0
messages = ['371871', '371872', '371873', '371874', '371875', '371877', '410771']
nosy_count = 2.0
nosy_names = ['vstinner', 'shihai1991']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '13886'
type = None
url = 'https://bugs.python.org/issue41034'
versions = ['Python 3.10']

[vstinner]

I tried to check for reference leaks, but running test_builtin twice fails:

$ ./python -m test -R 3:3 test_builtin 
0:00:00 load avg: 2.18 Run tests sequentially
0:00:00 load avg: 2.18 [1/1] test_builtin
beginning 6 repetitions
123456
.test test_builtin failed -- multiple errors occurred; run in verbose mode for details
test_builtin failed

== Tests result: FAILURE ==

1 test failed:
test_builtin

Total duration: 1.3 sec
Tests result: FAILURE

The error comes from PtyTests, but the issue is only triggered when builtins.float.hex is also run:

$ ./python -m test test_builtin test_builtin -m test.test_builtin.PtyTests.test_input_tty_non_ascii_unicode_errors -m builtins.float.hex   -v
== CPython 3.10.0a0 (heads/master:310f6aa7db, Jun 19 2020, 12:18:54) [GCC 10.1.1 20200507 (Red Hat 10.1.1-1)]
== Linux-5.6.18-300.fc32.x86_64-x86_64-with-glibc2.31 little-endian
== cwd: /home/vstinner/python/master/build/test_python_122646
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 0.60 Run tests sequentially
0:00:00 load avg: 0.60 [1/2] test_builtin
test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests) ... ok
hex (builtins.float)
Doctest: builtins.float.hex ... ok

---

Ran 2 tests in 0.009s

OK
0:00:00 load avg: 0.63 [2/2] test_builtin
test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests) ... FAIL
hex (builtins.float)
Doctest: builtins.float.hex ... ok

======================================================================
FAIL: test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2105, in test_input_tty_non_ascii_unicode_errors
    self.check_input_tty("prompté", b"quux\xe9", "ascii")
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2092, in check_input_tty
    self.assertEqual(input_result, expected)
AssertionError: 'quux' != 'quux\udce9'
- quux
+ quux\udce9
?     +

---

Ran 2 tests in 0.011s

FAILED (failures=1)
test test_builtin failed
test_builtin failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_builtin

Total duration: 762 ms
Tests result: FAILURE

[vstinner]

test_input_tty_non_ascii_unicode_errors() fails if readline was imported previously. Example:

$ ./python -m test test_readline test_builtin -v 
(...)

======================================================================
FAIL: test_input_tty_non_ascii (test.test_builtin.PtyTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2102, in test_input_tty_non_ascii
    self.check_input_tty("prompté", b"quux\xe9", "utf-8")
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2093, in check_input_tty
    self.assertEqual(input_result, expected)
AssertionError: 'quux' != 'quux\udce9'
- quux
+ quux\udce9
?     +

======================================================================
FAIL: test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2106, in test_input_tty_non_ascii_unicode_errors
    self.check_input_tty("prompté", b"quux\xe9", "ascii")
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2093, in check_input_tty
    self.assertEqual(input_result, expected)
AssertionError: 'quux' != 'quux\udce9'
- quux
+ quux\udce9
?     +

Another example: import readline in test_builtin.

diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py
index 40df7b606a..102b397e66 100644
--- a/Lib/test/test_builtin.py
+++ b/Lib/test/test_builtin.py
@@ -1,3 +1,4 @@
+import readline
 # Python test set -- built-in functions
 
 import ast

$ ./python -m test test_builtin -v -m PtyTests
== CPython 3.10.0a0 (heads/master:310f6aa7db, Jun 19 2020, 12:18:54) [GCC 10.1.1 20200507 (Red Hat 10.1.1-1)]
== Linux-5.6.18-300.fc32.x86_64-x86_64-with-glibc2.31 little-endian
== cwd: /home/vstinner/python/master/build/test_python_123487
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 0.36 Run tests sequentially
0:00:00 load avg: 0.36 [1/1] test_builtin
test_input_no_stdout_fileno (test.test_builtin.PtyTests) ... ok
test_input_tty (test.test_builtin.PtyTests) ... ok
test_input_tty_non_ascii (test.test_builtin.PtyTests) ... FAIL
test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests) ... FAIL

======================================================================
FAIL: test_input_tty_non_ascii (test.test_builtin.PtyTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2102, in test_input_tty_non_ascii
    self.check_input_tty("prompté", b"quux\xe9", "utf-8")
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2093, in check_input_tty
    self.assertEqual(input_result, expected)
AssertionError: 'quux' != 'quux\udce9'
- quux
+ quux\udce9
?     +

======================================================================
FAIL: test_input_tty_non_ascii_unicode_errors (test.test_builtin.PtyTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2106, in test_input_tty_non_ascii_unicode_errors
    self.check_input_tty("prompté", b"quux\xe9", "ascii")
  File "/home/vstinner/python/master/Lib/test/test_builtin.py", line 2093, in check_input_tty
    self.assertEqual(input_result, expected)
AssertionError: 'quux' != 'quux\udce9'
- quux
+ quux\udce9
?     +

---

Ran 4 tests in 0.028s

FAILED (failures=2)
test test_builtin failed
test_builtin failed

== Tests result: FAILURE ==

1 test failed:
test_builtin

Total duration: 446 ms
Tests result: FAILURE

[vstinner]

The two non-ASCII tests are very old:

commit 0d776b1
Author: Antoine Pitrou <solipsis@pitrou.net>
Date: Sun Nov 6 00:34:26 2011 +0100

Issue bpo-13342: input() used to ignore sys.stdin's and sys.stdout's unicode
error handler in interactive mode (when calling into PyOS_Readline()).

The code was refactored later by:

commit c9a6ab5
Author: Martin Panter <vadmium+py@gmail.com>
Date: Sat Oct 10 01:25:38 2015 +0000

Issue bpo-24402: Fix input() when stdout.fileno() fails; diagnosed by Eryksun

Also factored out some test cases into a new PtyTests class.

[vstinner]

I reproduce the issue on Fedora 32 with readline 8.0:

$ ./python -m test.pythoninfo|grep readline
readline._READLINE_LIBRARY_VERSION: 8.0
readline._READLINE_RUNTIME_VERSION: 0x800
readline._READLINE_VERSION: 0x800

[vstinner]

I also reproduce the issue on RHEL 8 with readline 7.0.

But the issue is hidden if stdout is not a TTY. I use "| cat", so Python stdout is a pipe:

$ ./python -m test test_readline test_builtin -v|cat
(...)
Tests result: SUCCESS

[vstinner]

See also:

• bpo-19884: "Importing readline produces erroneous output". Worked around using TERM='xterm-256color env var and by calling rl_variable_bind ("enable-meta-key", "off");
• bpo-11679 "readline interferes with characters beginning with byte \xe9".

[vstinner]

The "./python -m test -R 3:3 test_builtin" command loads the readline module somehow:
---

$ ./python -i -m test -R 3:3 test_builtin 
(...)
SystemExit: 0

>>> import sys
>>> 'readline' in sys.modules
True

---

This bug is a duplicate of bpo-13886. Moreover, I tested manually: my change #74816 fix this issue.