test_doctest fails when run in verbose mode

[nnorwitz]

BPO	1729305
Nosy	@ezio-melotti, @florentx
Files	doctest.patch: doctest patch that keeps track of encoding test_doctest.patch: test for the bug doctestfail.txt: traceback of the test failure issue1729305_doctest_v2.diff: Patch, apply to trunk issue1729305_doctest_py3k_v2.diff: Patch, apply to py3k

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/florentx'
closed_at = <Date 2010-02-27.14:39:47.543>
created_at = <Date 2007-06-01.05:16:47.000>
labels = ['type-bug', 'tests']
title = 'test_doctest fails when run in verbose mode'
updated_at = <Date 2010-02-27.15:24:20.508>
user = 'https://bugs.python.org/nnorwitz'

bugs.python.org fields:

activity = <Date 2010-02-27.15:24:20.508>
actor = 'flox'
assignee = 'flox'
closed = True
closed_date = <Date 2010-02-27.14:39:47.543>
closer = 'flox'
components = ['Tests']
creation = <Date 2007-06-01.05:16:47.000>
creator = 'nnorwitz'
dependencies = []
files = ['9533', '9534', '15102', '16385', '16387']
hgrepos = []
issue_num = 1729305
keywords = ['patch']
message_count = 10.0
messages = ['32174', '62896', '62897', '93861', '96819', '98234', '98235', '100180', '100184', '100185']
nosy_count = 4.0
nosy_names = ['nnorwitz', 'mbeachy', 'ezio.melotti', 'flox']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1729305'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

[nnorwitz]

This happens on trunk, p3yk branch and 2.5.

Trying:
    doctest.testfile('test_doctest4.txt', encoding='utf-8')
Expecting:
    (0, 4)
test test_doctest crashed -- <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 301-302: ordinal not in range(128)
Traceback (most recent call last):
  File "./Lib/test/regrtest.py", line 557, in runtest_inner
    indirect_test()
  File "/home/neal/python/dev/r25/Lib/test/test_doctest.py", line 2402, in test_main
    test_support.run_doctest(test_doctest, verbosity=True)
  File "/home/neal/python/dev/r25/Lib/test/test_support.py", line 467, in run_doctest
    f, t = doctest.testmod(module, verbose=verbosity)
  File "/home/neal/python/dev/r25/Lib/doctest.py", line 1799, in testmod
    runner.run(test)
  File "/home/neal/python/dev/r25/Lib/doctest.py", line 1345, in run
    return self.__run(test, compileflags, out)
  File "/home/neal/python/dev/r25/Lib/doctest.py", line 1261, in __run
    self.report_failure(out, test, example, got)
  File "/home/neal/python/dev/r25/Lib/doctest.py", line 1125, in report_failure
    self._checker.output_difference(example, got, self.optionflags))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 301-302: ordinal not in range(128)

[mbeachy]

The basic issue here is that running in verbose mode echoes back the
expected values from the file, so the results from non-ascii doctest
files must be encoded before printing.

It looks to me like the DocTestRunner class must grow an '_encoding'
attribute to keep track of how to encode any verbose output. I've
attached a patch that does this.

Happy post bug day!
Mike

[mbeachy]

Here's a patch for test_doctest.py that checks the problem has been fixed.

[ezio-melotti]

This test still fail on Win7 with Py2.6.3rc1 in verbose mode, it works
fine in normal mode.
I attached a file with the traceback.

[florentx]

It still occurs on trunk.

test test_doctest crashed -- <type 'exceptions.UnicodeEncodeError'>:
'ascii' codec can't encode characters in position 343-344:
ordinal not in range(128)

[florentx]

The patch did not work with latest trunk.

Here is an updated version of the patch.

Patch tested with different commands:
~ $ ./python -m test.regrtest test_doctest
~ $ ./python -m test.regrtest -v test_doctest
~ $ ./python -m test.regrtest -v test_doctest |less
~ $ ./python -3 -Wd Lib/test/regrtest.py -v test_doctest |less

[florentx]

And for Python 3. (Slightly different)

[florentx]

This patch is simpler, and it fixes bpo-7667 too.

[florentx]

For Py3k, it is enough to "backslashreplace" the errors on "write".

[florentx]

Fixed in r78488 and r78493.