Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve full diff output for lists #5924

Merged
merged 1 commit into from Oct 8, 2019

Conversation

@blueyed
Copy link
Contributor

blueyed commented Oct 7, 2019

Massage text input for difflib when comparing pformat output of
different line lengths.

Also do not strip ndiff output on the left, which currently already
removes indenting for lines with no differences.

Before:

E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E     Right contains 3 more items, first extra item: ' '
E     Full diff:
E     - ['version', 'version_info', 'sys.version', 'sys.version_info']
E     + ['version',
E     +  'version_info',
E     +  'sys.version',
E     +  'sys.version_info',
E     +  ' ',
E     +  'sys.version',
E     +  'sys.version_info']

After:

E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E     Right contains 3 more items, first extra item: ' '
E     Full diff:
E       ['version',
E        'version_info',
E        'sys.version',
E     +  'sys.version_info',
E     +  ' ',
E     +  'sys.version',
E        'sys.version_info'
E       ]

TODO:

  • changelog
  • "improve" / remove asserts
  • more tests (?)
@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 7, 2019

Oh this is awesome, thanks for this!

I think this can go into features... to me that's a bug! 😁

About your other points:

  1. Changelog: I would put your examples on it, even if it makes the entry large; it makes it very clear what the fix is doing.
  2. asserts: I think they could be changed to at least including the offending character: assert closing in [")", "]", "}"], 'unexpected sequence closing {closing} in sequence {left} or something along the lines
  3. Nothing comes to mind at the moment for more tests... you did test for both left/right sides, and I assume we have tests when lists have the same size already.
@blueyed blueyed force-pushed the blueyed:improve-list-diff branch from 647f42b to 5a6e036 Oct 8, 2019
@blueyed

This comment has been minimized.

Copy link
Contributor Author

blueyed commented Oct 8, 2019

Thanks for the review. (Amended since we cannot squash-merge)

Improved the separation of opening/closing parenthesis.
Removed the asserts, but protected with an if - wouldn't be cool to crash there.
Added one more test.

@blueyed

This comment has been minimized.

Copy link
Contributor Author

blueyed commented Oct 8, 2019

I think this might be good then - assuming full coverage and no breaking existing tests.

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 8, 2019

Great refactoring. 👍

How about targeting this on master? Also, only missing the CHANGELOG now, I would suggest in case you don't have it ready yet:

Fixed diff output of sequences of different lengths.

Before::


    E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
    E     Right contains 3 more items, first extra item: ' '
    E     Full diff:
    E     - ['version', 'version_info', 'sys.version', 'sys.version_info']
    E     + ['version',
    E     +  'version_info',
    E     +  'sys.version',
    E     +  'sys.version_info',
    E     +  ' ',
    E     +  'sys.version',
    E     +  'sys.version_info']

After::

    E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
    E     Right contains 3 more items, first extra item: ' '
    E     Full diff:
    E       ['version',
    E        'version_info',
    E        'sys.version',
    E     +  'sys.version_info',
    E     +  ' ',
    E     +  'sys.version',
    E        'sys.version_info'
    E       ]
@blueyed blueyed force-pushed the blueyed:improve-list-diff branch from 5a6e036 to d4569b9 Oct 8, 2019
@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 8, 2019

Ahh you got it already. 😁

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 8, 2019

And how about targeting master?

@blueyed

This comment has been minimized.

Copy link
Contributor Author

blueyed commented Oct 8, 2019

How about targeting this on master?

It's a change in behavior, would rather not do it.

@blueyed

This comment has been minimized.

Copy link
Contributor Author

blueyed commented Oct 8, 2019

Can we check if the changelog renders properly (i.e. with code blocks)?

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 8, 2019

It's a change in behavior, would rather not do it.

I disagree, but OK.

We can, you have to select the ... on the corner of the file and select "View file":

https://github.com/pytest-dev/pytest/blob/d4569b9e17cf7b8804930a1b91d0a7440f4e0ef3/changelog/5924.feature.rst

You are right, this is not rendering correctly... 🤔

Hmm, using .. code-block it renders nicely:

https://gist.github.com/nicoddemus/b61ada17545d59a315d6ac3a233c4c62

Massage text input for difflib when comparing pformat output of
different line lengths.

Also do not strip ndiff output on the left, which currently already
removes indenting for lines with no differences.

Before:

    E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
    E     Right contains 3 more items, first extra item: ' '
    E     Full diff:
    E     - ['version', 'version_info', 'sys.version', 'sys.version_info']
    E     + ['version',
    E     +  'version_info',
    E     +  'sys.version',
    E     +  'sys.version_info',
    E     +  ' ',
    E     +  'sys.version',
    E     +  'sys.version_info']

After:

    E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
    E     Right contains 3 more items, first extra item: ' '
    E     Full diff:
    E       [
    E        'version',
    E        'version_info',
    E        'sys.version',
    E        'sys.version_info',
    E     +  ' ',
    E     +  'sys.version',
    E     +  'sys.version_info',
    E       ]
@blueyed blueyed force-pushed the blueyed:improve-list-diff branch from d4569b9 to 946434c Oct 8, 2019
@blueyed blueyed merged commit fca462c into pytest-dev:features Oct 8, 2019
4 of 6 checks passed
4 of 6 checks passed
codecov/patch 94.73% of diff hit (target 96.35%)
Details
codecov/project 96.35% (-0.01%) compared to 5186635
Details
WIP Ready for review
Details
codecov/changes No unexpected coverage changes found.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
pytest-CI #20191008.5 succeeded
Details
@blueyed blueyed deleted the blueyed:improve-list-diff branch Oct 8, 2019
@blueyed

This comment has been minimized.

Copy link
Contributor Author

blueyed commented Oct 8, 2019

btw: would be nice if it could "scroll" the -v output to the first diff, e.g. with:

E   AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E     Right contains 3 more items, first extra item: ' '
E     Full diff:
E       [
E        'version',
E        'version_info',
E        'sys.version',
E        'sys.version_info',...
E
E     ...Full output truncated (5 lines hidden), use '-vv' to show
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 6, 2019
Follow-up to 946434c (pytest-dev#5924).

Before this patch the test would look like this:

    {'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
    Omitting 1 identical items, use -vv to show
    Right contains 1 more item:
    {'new': 1}
    Full diff:
      {
       'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
    -                  'sub1': {'long_a': 'substring '
    +                  'sub1': {'long_a': 'substring that gets wrapped'}}},
    ?                                                +++++++++++++++++ ++++
    +  'new': 1,
    -                                     'that '
    -                                     'gets '
    -                                     'wrapped'}}},
      }
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 6, 2019
Follow-up to 946434c (pytest-dev#5924).

Before this patch the test would look like this:

    {'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
    Omitting 1 identical items, use -vv to show
    Right contains 1 more item:
    {'new': 1}
    Full diff:
      {
       'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
    -                  'sub1': {'long_a': 'substring '
    +                  'sub1': {'long_a': 'substring that gets wrapped'}}},
    ?                                                +++++++++++++++++ ++++
    +  'new': 1,
    -                                     'that '
    -                                     'gets '
    -                                     'wrapped'}}},
      }
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 6, 2019
Follow-up to 946434c (pytest-dev#5924).

Before this patch the test would look like this:

    {'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
    Omitting 1 identical items, use -vv to show
    Right contains 1 more item:
    {'new': 1}
    Full diff:
      {
       'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
    -                  'sub1': {'long_a': 'substring '
    +                  'sub1': {'long_a': 'substring that gets wrapped'}}},
    ?                                                +++++++++++++++++ ++++
    +  'new': 1,
    -                                     'that '
    -                                     'gets '
    -                                     'wrapped'}}},
      }
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 6, 2019
Follow-up to 946434c (pytest-dev#5924).

Before this patch the test would look like this:

    {'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
    Omitting 1 identical items, use -vv to show
    Right contains 1 more item:
    {'new': 1}
    Full diff:
      {
       'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
    -                  'sub1': {'long_a': 'substring '
    +                  'sub1': {'long_a': 'substring that gets wrapped'}}},
    ?                                                +++++++++++++++++ ++++
    +  'new': 1,
    -                                     'that '
    -                                     'gets '
    -                                     'wrapped'}}},
      }
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 6, 2019
Follow-up to 946434c (pytest-dev#5924).

Before this patch the test would look like this:

    {'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
    Omitting 1 identical items, use -vv to show
    Right contains 1 more item:
    {'new': 1}
    Full diff:
      {
       'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
    -                  'sub1': {'long_a': 'substring '
    +                  'sub1': {'long_a': 'substring that gets wrapped'}}},
    ?                                                +++++++++++++++++ ++++
    +  'new': 1,
    -                                     'that '
    -                                     'gets '
    -                                     'wrapped'}}},
      }

Upstream: pytest-dev#6137
bors bot added a commit to duckinator/bork that referenced this pull request Nov 21, 2019
Merge #79
79: Update pytest to 5.3.0 r=duckinator a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 5.3.0
   ```
   =========================

Deprecations
------------

- `6179 &lt;https://github.com/pytest-dev/pytest/issues/6179&gt;`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given
  that this is the version supported by default in modern tools that manipulate this type of file.

  In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option
  is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``.

  For more information, `see the docs &lt;https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2&gt;`__.



Features
--------

- `4488 &lt;https://github.com/pytest-dev/pytest/issues/4488&gt;`_: The pytest team has created the `pytest-reportlog &lt;https://github.com/pytest-dev/pytest-reportlog&gt;`__
  plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes.

  Each line of the report log contains a self contained JSON object corresponding to a testing event,
  such as a collection or a test result report. The file is guaranteed to be flushed after writing
  each line, so systems can read and process events in real-time.

  The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed
  in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and
  provide feedback.


- `4730 &lt;https://github.com/pytest-dev/pytest/issues/4730&gt;`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.

  This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions.


- `5515 &lt;https://github.com/pytest-dev/pytest/issues/5515&gt;`_: Allow selective auto-indentation of multiline log messages.

  Adds command line option ``--log-auto-indent``, config option
  ``log_auto_indent`` and support for per-entry configuration of
  indentation behavior on calls to ``logging.log()``.

  Alters the default for auto-indention from ``on`` to ``off``. This
  restores the older behavior that existed prior to v4.6.0. This
  reversion to earlier behavior was done because it is better to
  activate new features that may lead to broken tests explicitly
  rather than implicitly.


- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: ``pytester`` learned two new functions, `no_fnmatch_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line&gt;`_ and
  `no_re_match_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line&gt;`_.

  The functions are used to ensure the captured text *does not* match the given
  pattern.

  The previous idiom was to use ``re.match``:

  .. code-block:: python

      assert re.match(pat, result.stdout.str()) is None

  Or the ``in`` operator:

  .. code-block:: python

      assert text in result.stdout.str()

  But the new functions produce best output on failure.


- `6057 &lt;https://github.com/pytest-dev/pytest/issues/6057&gt;`_: Added tolerances to complex values when printing ``pytest.approx``.

  For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle.


- `6061 &lt;https://github.com/pytest-dev/pytest/issues/6061&gt;`_: Added the pluginmanager as an argument to ``pytest_addoption``
  so that hooks can be invoked when setting up command line options. This is
  useful for having one plugin communicate things to another plugin,
  such as default values or which set of command line options to add.



Improvements
------------

- `5061 &lt;https://github.com/pytest-dev/pytest/issues/5061&gt;`_: Use multiple colors with terminal summary statistics.


- `5630 &lt;https://github.com/pytest-dev/pytest/issues/5630&gt;`_: Quitting from debuggers is now properly handled in ``doctest`` items.


- `5924 &lt;https://github.com/pytest-dev/pytest/issues/5924&gt;`_: Improved verbose diff output with sequences.

  Before:

  .. code-block::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E     - [&#39;version&#39;, &#39;version_info&#39;, &#39;sys.version&#39;, &#39;sys.version_info&#39;]
      E     + [&#39;version&#39;,
      E     +  &#39;version_info&#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;]

  After:

  .. code-block::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E       [
      E        &#39;version&#39;,
      E        &#39;version_info&#39;,
      E        &#39;sys.version&#39;,
      E        &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E       ]


- `5936 &lt;https://github.com/pytest-dev/pytest/issues/5936&gt;`_: Display untruncated assertion message with ``-vv``.


- `5990 &lt;https://github.com/pytest-dev/pytest/issues/5990&gt;`_: Fixed plurality mismatch in test summary (e.g. display &quot;1 error&quot; instead of &quot;1 errors&quot;).


- `6008 &lt;https://github.com/pytest-dev/pytest/issues/6008&gt;`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be
  immutable and avoid accidental modifications.


- `6023 &lt;https://github.com/pytest-dev/pytest/issues/6023&gt;`_: ``pytest.main`` now returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still).


- `6026 &lt;https://github.com/pytest-dev/pytest/issues/6026&gt;`_: Align prefixes in output of pytester&#39;s ``LineMatcher``.


- `6059 &lt;https://github.com/pytest-dev/pytest/issues/6059&gt;`_: Collection errors are reported as errors (and not failures like before) in the terminal&#39;s short test summary.


- `6069 &lt;https://github.com/pytest-dev/pytest/issues/6069&gt;`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally.


- `6097 &lt;https://github.com/pytest-dev/pytest/issues/6097&gt;`_: The &quot;[XXX%]&quot; indicator in the test summary is now colored according to the final (new) multi-colored line&#39;s main color.


- `6116 &lt;https://github.com/pytest-dev/pytest/issues/6116&gt;`_: Added ``--co`` as a synonym to ``--collect-only``.


- `6148 &lt;https://github.com/pytest-dev/pytest/issues/6148&gt;`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix.


- `6152 &lt;https://github.com/pytest-dev/pytest/issues/6152&gt;`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes.


- `6176 &lt;https://github.com/pytest-dev/pytest/issues/6176&gt;`_: Improved failure reporting with pytester&#39;s ``Hookrecorder.assertoutcome``.


- `6181 &lt;https://github.com/pytest-dev/pytest/issues/6181&gt;`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary.


- `6206 &lt;https://github.com/pytest-dev/pytest/issues/6206&gt;`_: Improved ``cache.set`` robustness and performance.



Bug Fixes
---------

- `2049 &lt;https://github.com/pytest-dev/pytest/issues/2049&gt;`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes.


- `2548 &lt;https://github.com/pytest-dev/pytest/issues/2548&gt;`_: Fixed line offset mismatch of skipped tests in terminal summary.


- `6039 &lt;https://github.com/pytest-dev/pytest/issues/6039&gt;`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin.

  This is important when used with ``pytester``&#39;s ``runpytest_inprocess``.


- `6047 &lt;https://github.com/pytest-dev/pytest/issues/6047&gt;`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc.


- `6074 &lt;https://github.com/pytest-dev/pytest/issues/6074&gt;`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``.


- `6189 &lt;https://github.com/pytest-dev/pytest/issues/6189&gt;`_: Fixed result of ``getmodpath`` method.



Trivial/Internal Changes
------------------------

- `4901 &lt;https://github.com/pytest-dev/pytest/issues/4901&gt;`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a
  valid ``pytest.ExitCode`` value.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
bors bot added a commit to rehandalal/therapist that referenced this pull request Nov 28, 2019
Merge #105
105: Update pytest to 5.3.1 r=rehandalal a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 5.3.1
   ```
   =========================

Improvements
------------

- `6231 &lt;https://github.com/pytest-dev/pytest/issues/6231&gt;`_: Improve check for misspelling of ``pytest.mark.parametrize``.


- `6257 &lt;https://github.com/pytest-dev/pytest/issues/6257&gt;`_: Handle `exit.Exception` raised in `notify_exception` (via `pytest_internalerror`), e.g. when quitting pdb from post mortem.



Bug Fixes
---------

- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: pytester: fix ``no_fnmatch_line`` when used after positive matching.


- `6082 &lt;https://github.com/pytest-dev/pytest/issues/6082&gt;`_: Fix line detection for doctest samples inside ``property`` docstrings, as a workaround to `bpo-17446 &lt;https://bugs.python.org/issue17446&gt;`__.


- `6254 &lt;https://github.com/pytest-dev/pytest/issues/6254&gt;`_: Fix compatibility with pytest-parallel (regression in pytest 5.3.0).


- `6255 &lt;https://github.com/pytest-dev/pytest/issues/6255&gt;`_: Clear the ``sys.last_traceback``, ``sys.last_type`` and ``sys.last_value``
  attributes by deleting them instead of setting them to ``None``. This better
  matches the behaviour of the Python standard library.
   ```
   
  
  
   ### 5.3.0
   ```
   =========================

Deprecations
------------

- `6179 &lt;https://github.com/pytest-dev/pytest/issues/6179&gt;`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given
  that this is the version supported by default in modern tools that manipulate this type of file.

  In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option
  is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``.

  For more information, `see the docs &lt;https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2&gt;`__.



Features
--------

- `4488 &lt;https://github.com/pytest-dev/pytest/issues/4488&gt;`_: The pytest team has created the `pytest-reportlog &lt;https://github.com/pytest-dev/pytest-reportlog&gt;`__
  plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes.

  Each line of the report log contains a self contained JSON object corresponding to a testing event,
  such as a collection or a test result report. The file is guaranteed to be flushed after writing
  each line, so systems can read and process events in real-time.

  The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed
  in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and
  provide feedback.


- `4730 &lt;https://github.com/pytest-dev/pytest/issues/4730&gt;`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.

  This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions.


- `5515 &lt;https://github.com/pytest-dev/pytest/issues/5515&gt;`_: Allow selective auto-indentation of multiline log messages.

  Adds command line option ``--log-auto-indent``, config option
  ``log_auto_indent`` and support for per-entry configuration of
  indentation behavior on calls to ``logging.log()``.

  Alters the default for auto-indention from ``on`` to ``off``. This
  restores the older behavior that existed prior to v4.6.0. This
  reversion to earlier behavior was done because it is better to
  activate new features that may lead to broken tests explicitly
  rather than implicitly.


- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: `testdir &lt;https://docs.pytest.org/en/latest/reference.htmltestdir&gt;`__ learned two new functions, `no_fnmatch_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line&gt;`_ and
  `no_re_match_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line&gt;`_.

  The functions are used to ensure the captured text *does not* match the given
  pattern.

  The previous idiom was to use ``re.match``:

  .. code-block:: python

      result = testdir.runpytest()
      assert re.match(pat, result.stdout.str()) is None

  Or the ``in`` operator:

  .. code-block:: python

      result = testdir.runpytest()
      assert text in result.stdout.str()

  But the new functions produce best output on failure.


- `6057 &lt;https://github.com/pytest-dev/pytest/issues/6057&gt;`_: Added tolerances to complex values when printing ``pytest.approx``.

  For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle.


- `6061 &lt;https://github.com/pytest-dev/pytest/issues/6061&gt;`_: Added the pluginmanager as an argument to ``pytest_addoption``
  so that hooks can be invoked when setting up command line options. This is
  useful for having one plugin communicate things to another plugin,
  such as default values or which set of command line options to add.



Improvements
------------

- `5061 &lt;https://github.com/pytest-dev/pytest/issues/5061&gt;`_: Use multiple colors with terminal summary statistics.


- `5630 &lt;https://github.com/pytest-dev/pytest/issues/5630&gt;`_: Quitting from debuggers is now properly handled in ``doctest`` items.


- `5924 &lt;https://github.com/pytest-dev/pytest/issues/5924&gt;`_: Improved verbose diff output with sequences.

  Before:

  ::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E     - [&#39;version&#39;, &#39;version_info&#39;, &#39;sys.version&#39;, &#39;sys.version_info&#39;]
      E     + [&#39;version&#39;,
      E     +  &#39;version_info&#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;]

  After:

  ::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E       [
      E        &#39;version&#39;,
      E        &#39;version_info&#39;,
      E        &#39;sys.version&#39;,
      E        &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E       ]


- `5934 &lt;https://github.com/pytest-dev/pytest/issues/5934&gt;`_: ``repr`` of ``ExceptionInfo`` objects has been improved to honor the ``__repr__`` method of the underlying exception.

- `5936 &lt;https://github.com/pytest-dev/pytest/issues/5936&gt;`_: Display untruncated assertion message with ``-vv``.


- `5990 &lt;https://github.com/pytest-dev/pytest/issues/5990&gt;`_: Fixed plurality mismatch in test summary (e.g. display &quot;1 error&quot; instead of &quot;1 errors&quot;).


- `6008 &lt;https://github.com/pytest-dev/pytest/issues/6008&gt;`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be
  immutable and avoid accidental modifications.


- `6023 &lt;https://github.com/pytest-dev/pytest/issues/6023&gt;`_: ``pytest.main`` returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still).


- `6026 &lt;https://github.com/pytest-dev/pytest/issues/6026&gt;`_: Align prefixes in output of pytester&#39;s ``LineMatcher``.


- `6059 &lt;https://github.com/pytest-dev/pytest/issues/6059&gt;`_: Collection errors are reported as errors (and not failures like before) in the terminal&#39;s short test summary.


- `6069 &lt;https://github.com/pytest-dev/pytest/issues/6069&gt;`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally.


- `6097 &lt;https://github.com/pytest-dev/pytest/issues/6097&gt;`_: The &quot;[...%]&quot; indicator in the test summary is now colored according to the final (new) multi-colored line&#39;s main color.


- `6116 &lt;https://github.com/pytest-dev/pytest/issues/6116&gt;`_: Added ``--co`` as a synonym to ``--collect-only``.


- `6148 &lt;https://github.com/pytest-dev/pytest/issues/6148&gt;`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix.


- `6152 &lt;https://github.com/pytest-dev/pytest/issues/6152&gt;`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes.


- `6176 &lt;https://github.com/pytest-dev/pytest/issues/6176&gt;`_: Improved failure reporting with pytester&#39;s ``Hookrecorder.assertoutcome``.


- `6181 &lt;https://github.com/pytest-dev/pytest/issues/6181&gt;`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary.


- `6206 &lt;https://github.com/pytest-dev/pytest/issues/6206&gt;`_: Improved ``cache.set`` robustness and performance.



Bug Fixes
---------

- `2049 &lt;https://github.com/pytest-dev/pytest/issues/2049&gt;`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes.


- `2548 &lt;https://github.com/pytest-dev/pytest/issues/2548&gt;`_: Fixed line offset mismatch of skipped tests in terminal summary.


- `6039 &lt;https://github.com/pytest-dev/pytest/issues/6039&gt;`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin.

  This is important when used with ``pytester``&#39;s ``runpytest_inprocess``.


- `6047 &lt;https://github.com/pytest-dev/pytest/issues/6047&gt;`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc.


- `6074 &lt;https://github.com/pytest-dev/pytest/issues/6074&gt;`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``.


- `6189 &lt;https://github.com/pytest-dev/pytest/issues/6189&gt;`_: Fixed result of ``getmodpath`` method.



Trivial/Internal Changes
------------------------

- `4901 &lt;https://github.com/pytest-dev/pytest/issues/4901&gt;`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a
  valid ``pytest.ExitCode`` value.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.