regrtest: use backslashreplace error handler for stdout

[vstinner]

BPO	8533
Nosy	@pitrou, @vstinner, @florentx
Files	regrtest_stdout_backslashreplace.patch regrtest_traceback_stderr-2.patch regrtest_stdout_newline.patch py3k_regrtest_newline.patch py3k_regrtest_newline_more.patch py3k_also_no_unicode_error_on_direct_test_run.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-05-30.21:09:09.043>
created_at = <Date 2010-04-26.10:43:12.726>
labels = ['tests', 'expert-unicode']
title = 'regrtest: use backslashreplace error handler for stdout'
updated_at = <Date 2011-05-30.21:09:09.042>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2011-05-30.21:09:09.042>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2011-05-30.21:09:09.043>
closer = 'vstinner'
components = ['Tests', 'Unicode']
creation = <Date 2010-04-26.10:43:12.726>
creator = 'vstinner'
dependencies = []
files = ['17090', '17152', '17228', '18824', '18825', '19191']
hgrepos = []
issue_num = 8533
keywords = ['patch']
message_count = 21.0
messages = ['104212', '104223', '104229', '104231', '104670', '104773', '104783', '104794', '104824', '105087', '105180', '116003', '116004', '118248', '118253', '118256', '118395', '118396', '123605', '136813', '137330']
nosy_count = 5.0
nosy_names = ['pitrou', 'vstinner', 'ocean-city', 'flox', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue8533'
versions = ['Python 3.1', 'Python 3.2']

[vstinner]

If a test fails, regrtest writes the backtrace to sys.stdout. If the backtrace contains a non-ASCII characters, it's encoded using sys.stdout encoding.

In some conditions, sys.stdout is unable to encode some or all non-ASCII characters. Eg. if there is no locale set (empty environment or at least empty LANG variable value), sys.stdout.encoding="ascii".

If regrtest fails to display a test output (error backtrace), regrtest exits directly (don't execute next tests).

I propose to use backslashreplace error handler in sys.stdout, as done for sys.stderr to avoid this annoying issue.

Attached patch (for py3k) replace sys.stdout by a new file using backslashreplace, just before executing the tests.

I don't know if the issue concerns also Python2.

[pitrou]

A better resolution IMO would be to output tracebacks on stderr instead.

[vstinner]

A better resolution IMO would be to output tracebacks on stderr instead.

Yeah, that sounds easier and safer. Attached patch writes the tracebacks to stderr.

I don't remember how to reproduce this issue :-/ I guess that stdout/stderr encoding should be ASCII (eg. LANG="") and the traceback (eg. the error message) should contain a non-ASCII character.

[vstinner]

Write to stderr instead of stdout might change buildbot output order.

[vstinner]

regrtest_traceback_stderr.patch is not enough: support._run_suite() writes output to sys.stdout instead of sys.stderr. New version of the patch fixes that.

[vstinner]

Ok, let's try sys.stderr solution: commited in r80694 (py3k).

If it breaks buildbot outputs, I will revert it and try the second solution.

[pitrou]

Since it may reorder output, I think it's better revert the patch and try the other solution. However, I don't think you need to replace sys.stdout at all: just output the traceback more carefully.

[vstinner]

As "expected", the patch doesn't work: it randomize the output order :-(

I checked "sparc Ubuntu 3.x": the output order is correct. "test_xxx" lines are written to stdout, "FAIL: ..." + traceback are written to stderr, and the lines are written in the right order.

But it failed on "x86 Tiger 3.x" :
http://www.python.org/dev/buildbot/3.x/builders/x86 Tiger 3.x/builds/130/steps/test/logs/stdio
http://www.python.org/dev/buildbot/3.x/builders/x86 Tiger 3.x/builds/131/steps/test/logs/stdio

When a test is reexecuted in verbose mode, the output is written in red (why not, but it was not the case before my commit), and the stdout and stderr are in a "mixed".

--

antoine> just output the traceback more carefully.

Encode the traceback by hand would be possible, but it's more complex. A possible solution would be to write the output to a StringIO, encode unicode to bytes using the right encoding and backslashreplace error handler, and write the result to stdout. But I don't like buffering the output because the buildbot may hung for different reasons (search in the bug tracker for "test_multiprocessing" or "test_subprocess"...) and the buildbot master may consider the buildbot as dead (no new output since xxx seconds, whereas it's writing to a buffer).

--

I prefer my first simple idea: use backslashreplace error handler for stdout. Let's try: r80703. This commit reverts my previous commit, apply regrtest_stdout_backslashreplace.patch + a fix for multiprocessing mode (regrtest.py -j ...).

[vstinner]

Let's try: r80703

This one looks ok: the output order is kept and I didn't noticed anything special in the buildbot output.

Backported to 3.1 as r80711. Close the issue.

[vstinner]

Reopen: r80703 (and r80711) introduces a new bug: on Windows, there is an empty string between each line. It looks like a newline error.

replace_stdout() should set the newline argument to open(). But how can I get the newline attribute from sys.stdout? sys.stdout._writenl?

regrtest_stdout_newline.patch should fix the newline problem.

[vstinner]

I disabled my patch (replace_stdout function) on Windows until it gets fixed: r80905 (py3k) and r80906 (3.1)

[ocean-city]

Hello. How about this patch? I just mimicked create_stdio() in
Python/pythonrun.c. (Mostly) Newlines looks correct as well as
python2.x. I tested this on windows.

[ocean-city]

And here is more advanced (?) patch. More closer to Python/pythonrun.c
's implementaion.

I tried regrtest_stdout_newline.patch, but it doesn't work.
# AttributeError: '_io.TextIOWrapper' object has no attribute '_writenl'

[ocean-city]

I noticed fd must be dup-ed before passing to io.open.
And I noticed direct test run like "py3k -m test.test_time"
still produces unicode error. (Maybe this is reasonable
limitation)

I can confirm we can suppress the unicode error in such
case with newly attached patch. "py3k_also_no_unicode_error_on_direct_test_run.patch"

[vstinner]

py3k_also_no_unicode_error_on_direct_test_run.patch comes a little bit too late:

$ LANG= ./python Lib/test/regrtest.py -v test_time
== CPython 3.2a2+ (py3k, Oct 8 2010, 01:40:20) [GCC 4.4.5 20100909 (prerelease)]
==   Linux-2.6.32-trunk-686-i686-with-debian-squeeze-sid little-endian
==   Traceback (most recent call last):
  File "Lib/test/regrtest.py", line 1478, in <module>
    main()
  File "Lib/test/regrtest.py", line 454, in main
    print("==  ", os.getcwd())
UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128)

My working directory is /home/haypo/prog/SVN/py3ké.

[ocean-city]

Oops, sorry. I'll withdraw my last patch.

[vstinner]

Oops, sorry. I'll withdraw my last patch.

Why? Your patch is useful to run a single test outside regrtest. But you should not remove the hack on regrtest.py, only keep your patch on unittest/runner.py. There are not exclusive.

[ocean-city]

Thank you. I'll reattach the patch only for Lib/unittest/runner.py

[ocean-city]

Well, can this go into Python3.2?

[python-dev]

New changeset 8be9eaf5829f by Victor Stinner in branch 'default':
Issue bpo-8533: regrtest replaces also sys.stdout on Windows
http://hg.python.org/cpython/rev/8be9eaf5829f

[vstinner]

The original issue (use backslashreplace for stdout in regrtest) is now fixed, and so I closed it.

@Ocean-City: Can you please open a new issue for unittest? (for py3k_also_no_unicode_error_on_direct_test_run.patch)