pytest -v does not include "Differing Items" with long-enough dicts

[quodlibetor]

in the environment (I also get the error in pytest 2.8.7)

$ uname -v
Darwin Kernel Version 13.4.0
$ pip freeze
py==1.4.31
pytest==2.9.1

given ex.py:

def test_long_assertion():
    long = 'a' * 50
    a = {i: long for i in range(8)}
    b = dict(**a)
    b[7] = 'short'
    assert a == b

Invoking py.test gives a very helpful error message:

$ py.test ex.py
========= test session starts =========
platform darwin -- Python 2.7.5, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /private/tmp, inifile:
plugins: cov-2.2.1
collected 1 items

ex.py F

=================== FAILURES ===================
____________________ test_long_assertion ____________________
    def test_long_assertion():
        long = 'a' * 50
        a = {i: long for i in range(8)}
        b = dict(**a)
        b[7] = 'short'
>       assert a == b
E       assert {0: 'aaaaaaaa...aaaaaaa', ...} == {0: 'aaaaaaaaa...aaaaaaa', ...}
E         Omitting 7 identical items, use -v to show
E         Differing items:
E         {7: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'} != {7: 'short'}
E         Use -v to get the full diff

ex.py:6: AssertionError

However, running with -v breaks the diff (note the lack of "differing items" at the end):

$ py.test -v ex.py
================================== test session starts ===================================
platform darwin -- Python 2.7.5, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Users/bwm/findable/virtualenvs/consumer/bin/python
cachedir: .cache
rootdir: /private/tmp, inifile:
plugins: cov-2.2.1
collected 1 items

ex.py::test_long_assertion FAILED

======================================== FAILURES ========================================
__________________________________ test_long_assertion ___________________________________

    def test_long_assertion():
        long = 'a' * 50
        a = {i: long for i in range(8)}
        b = dict(**a)
        b[7] = 'short'
>       assert a == b
E       assert {0: 'aaaaaaaa...aaaaaaa', ...} == {0: 'aaaaaaaaa...aaaaaaa', ...}
E         Common items:
E         {0: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          1: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          2: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          3: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          4: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          5: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
E          6: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}
E         Differing items:
E         Detailed information truncated (11 more lines), use "-vv" to show

ex.py:6: AssertionError
================================ 1 failed in 0.01 seconds ================================

-vv does include everything (including that diff that is excluded), but it's very verbose when I have dozens of items in my dicts.

This is not an issue when the dict is short -- maybe because we get the full diff instead of "Differing Items":

$ cat ex2.py
def test_long_assertion():
    long = 'a' * 50
    a = {i: i for i in range(8)}
    b = dict(**a)
    b[7] = 'short'
    assert a == b

$ py.test ex2.py -v
================================= test session starts ==================================
platform darwin -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Users/bwm/findable/virtualenvs/tmp-5f9f221206eaa0d8/bin/python2.7
cachedir: .cache
rootdir: /private/tmp, inifile:
collected 1 items

ex2.py::test_long_assertion FAILED

======================================= FAILURES =======================================
_________________________________ test_long_assertion __________________________________

    def test_long_assertion():
        long = 'a' * 50
        a = {i: i for i in range(8)}
        b = dict(**a)
        b[7] = 'short'
>       assert a == b
E       assert {0: 0, 1: 1, 2: 2, 3: 3, ...} == {0: 0, 1: 1, 2: 2, 3: 3, ...}
E         Common items:
E         {0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6}
E         Differing items:
E         {7: 7} != {7: 'short'}
E         Full diff:
E         - {0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7}
E         ?                                               ^
E         + {0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 'short'}
E         ?

[mattduck]

Hey @flub, this is caused because all output that has verbosity < 2 is truncated at 10 lines:

pytest/_pytest/assertion/__init__.py

Line 137 in 8639bf7

show_max = 10

.

The py.test ex.py example above only shows the differing lines because the assertion explanation is short. With py.test -v ex.py, the explanation is more verbose so gets truncated. Pytest isn't explicitly choosing to remove the Differing lines: items, it just looks like it is because Differing lines: is the last line before truncation.

Not sure the best approach here. Do you think it's more useful to just remove the common items output when verbosity=1? The differing items are probably more useful to the user.

We could also change the truncation formatting to make it clearer that all output is truncated.

[flub]

sounds good to both :)

[merwok]

Not sure the best approach here. Do you think it's more useful to just remove the common items
output when verbosity=1? The differing items are probably more useful to the user.

This behaviour would be great.

[mattduck]

hey @merwok, this change has been merged into the features branch, so it should be included in the next minor release (3.1.0).

[merwok]

Good news! ✨

[nicoddemus]

Fixed by #1951