doctest handling of multiline strings is broken

[filiprem]

BPO	40295
Nosy	@stevendaprano, @serhiy-storchaka, @filiprem
Files	doctest-bugs.py: isolated test case doctest-bugs-2.py: simplified test case to reproduce the bug

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-16.02:30:42.343>
created_at = <Date 2020-04-15.20:37:13.193>
labels = ['type-bug', '3.8', '3.9', 'invalid', '3.7', 'library']
title = 'doctest handling of multiline strings is broken'
updated_at = <Date 2020-04-16.02:46:10.324>
user = 'https://github.com/filiprem'

bugs.python.org fields:

activity = <Date 2020-04-16.02:46:10.324>
actor = 'steven.daprano'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-16.02:30:42.343>
closer = 'steven.daprano'
components = ['Library (Lib)']
creation = <Date 2020-04-15.20:37:13.193>
creator = 'Filip Rembia\xc5\x82kowski'
dependencies = []
files = ['49065', '49066']
hgrepos = []
issue_num = 40295
keywords = []
message_count = 6.0
messages = ['366554', '366560', '366567', '366569', '366574', '366575']
nosy_count = 3.0
nosy_names = ['steven.daprano', 'serhiy.storchaka', 'Filip Rembia\xc5\x82kowski']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40295'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9']

[filiprem]

The doctest module does not compare multiline strings properly, as attached example proves.

Tested on 2.7, 3.6 and 3.9.0a5+. (platform: Ubuntu 18.04).

Related: https://stackoverflow.com/questions/60956015/unexpected-errors-while-testing-python3-code-with-doctest

[serhiy-storchaka]

Did you try to run the tested function in the interactive Python interpreter? Did you get what you expected?

[filiprem]

@serhiy, Thank you for feedback.

Yes the "testme" function (indeed trivial) works as expected - both in interactive Python interpreter and in script file.

If you go to Lib/doctest.py, search for "string-identical" and debug my example there, you will see the problem is real.

I don't see an easy fix for the problem, I only suspect it's related to io.StringIO getvalue() method.

[filiprem]

Actually, the behavior does not depend on leading spaces, and test case can be isolated even further. Sample is attached [doctest-bugs-2.py].

[stevendaprano]

Have you tried calling multiline_output() in the REPL?

It does *not* show your expected output:

# expected
First line
Second line

but the string repr():

# actual
'First line\nSecond line\n'

Change your doctest to either:

    >>> multiline_output()
    'First line\\nSecond line\\n'

(note that you must escape the backslashes) or:

    >>> print(multiline_output())
    First line
    Second line
    <BLANKLINE>

Note that the "<BLANKLINE>" needs to be written literally, as described here:

https://docs.python.org/3/library/doctest.html#doctest.DONT_ACCEPT_BLANKLINE

[stevendaprano]

By the way Filip, you were told on the Stackoverflow page that the output was correct and that you were not using doctest correctly. Serhiy also hinted to you that you should check the output in the REPL and you falsely claimed that it gave the expected output, but it doesn't:

py> multiline_output()
'First line\nSecond line\n'

which is nothing like the output you put in your doctest.

This is not a bug in doctest. The very first line of the doctest documentation says:

The doctest module searches for pieces of text that look like
interactive Python sessions

so this is expected and documented behaviour, not a bug.