Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce no_fnmatch_line/no_re_match_line in pytester #5914

Merged
merged 2 commits into from Oct 6, 2019

Conversation

@nicoddemus
Copy link
Member

nicoddemus commented Oct 5, 2019

The current idiom is to use:

  assert re.match(pat, result.stdout.str()) is None

Or

  assert line not in result.stdout.str()

But this does not really give good results when it fails.

Those new functions produce similar output to ther other match lines functions.

@nicoddemus nicoddemus force-pushed the nicoddemus:pytester-no-match branch from 2df6969 to 4163556 Oct 5, 2019
Copy link
Member

RonnyPfannschmidt left a comment

looks good, im not a fan of the method names, but its not easy to find a better one

do we have a own test that could use those, if yes, lets include a change to it

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 5, 2019

do we have a own test that could use those, if yes, lets include a change to it

I was about to do that, but had to stop for lunch. We had this in tons of places, I'm doing a general find/replace to use the new methods.

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 5, 2019

Failures are due to #5902.

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 5, 2019

looks good, im not a fan of the method names, but its not easy to find a better one

Me neither, but I'm open to suggestions. 👍

@nicoddemus nicoddemus force-pushed the nicoddemus:pytester-no-match branch from cf6e5a5 to f0b3e80 Oct 5, 2019
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Oct 5, 2019

a better api would require assertion helper objects

then it would be something like assert not resultstdout.fnmatched_lines("...")

the fact that no good names come up stems from the fact that the api is structurally wrong and scewed

its still better than the stuff we had before

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Oct 6, 2019

Just a thought: might be nice if assert not result.stdout.re_match_lines would work, i.e. the existing methods being aware of being negated.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Oct 6, 2019

nfortunately the existing methods already have inlined assertions, so we cant use thme this way

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 6, 2019

Just a thought: might be nice if assert not result.stdout.re_match_lines would work, i.e. the existing methods being aware of being negated.

It is possible, the methods can be changed to return True when they succeed, so if I want something to not be found, your suggestion would work.

But I'm not sure if it is a good idea, we are mixing automatic assertion check done inside the methods with external assertions when trying to negate the outcome, so users will now thing this is required:

assert result.stdout.fnmatch_liens(...)
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Oct 6, 2019

returning an actual assertion helper that will crash the test if its not at least checked for truth or when it is negated is required
returning plain boolenas in a assertion helper just results in a crocked api, assertion helpers either do asserts, or they return something that can be richly asserted

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 6, 2019

Btw I meant "mixing", not "missing" in the post above.

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Oct 6, 2019

Waiting approve/merge of #5923 to rebase this

nicoddemus added 2 commits Oct 5, 2019
The current idiom is to use:

  assert re.match(pat, result.stdout.str())

Or

  assert line in result.stdout.str()

But this does not really give good results when it fails.

Those new functions produce similar output to ther other match lines functions.
@nicoddemus nicoddemus force-pushed the nicoddemus:pytester-no-match branch from f0b3e80 to 47c2091 Oct 6, 2019
@nicoddemus nicoddemus merged commit 5186635 into pytest-dev:features Oct 6, 2019
6 checks passed
6 checks passed
WIP Ready for review
Details
codecov/changes No unexpected coverage changes found.
Details
codecov/patch 100% of diff hit (target 96.33%)
Details
codecov/project 96.34% (+<.01%) compared to b847d57
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
pytest-CI #20191006.16 succeeded
Details
@nicoddemus nicoddemus deleted the nicoddemus:pytester-no-match branch Oct 6, 2019
@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Oct 7, 2019

My thought was in the direction that fnmatch_lines would be aware if it is being asserted (in general, and specifically with "not" then also), and would still behave like now when not being "asserted".

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Oct 7, 2019

Anyway, great to have this!

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Oct 7, 2019

btw: it might be nice if this could be used in a single function / call, where
order could be tested for then, i.e.

    "a*",
    "!b",
    "c",
    "!d"

To test that there is "a*", then "c" (with no "b" inbetween), and no "d" after
it.

(in general it can be useful to ensure that something does not appear before
something else only).

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Oct 7, 2019

@blueyed there is not general sane/safe way to know whether a code piece is running in a assertion,
even less so if the result is later used in a assertion

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Nov 3, 2019

I am still not really convinced what the benefit of this pattern is in contrast to "'foo' not in bar", especially that it only works for a single line. Might be more useful with re_ though.
Anyway, I've tried to use it and it does not work properly after successful matching.
Since I am not sure if that's intended or not, I've not created a PR already (also it conflicts hard with master, less with features).
See https://github.com/blueyed/pytest/pull/70/files#diff-deec4765976a87a594862ea0e1356ac4R560-R565

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Nov 6, 2019

I am still not really convinced what the benefit of this pattern is in contrast to "'foo' not in bar", especially that it only works for a single line.

My use case is that the str not in bar for many lines of text produces terrible output:

    def test_foo(self, testdir):
        testdir.makepyfile("""
            def test_foo():
                pass        
        
        """)

        result = testdir.runpytest()
        result.stdout.no_fnmatch_line("*1 passed*")

Produces:

__________________________________ TestCustomConftests.test_foo ___________________________________

self = <test_collection.TestCustomConftests object at 0x000002319F946198>
testdir = <Testdir local('C:\\Users\\Bruno\\AppData\\Local\\Temp\\pytest-of-Bruno\\pytest-77\\test_foo0')>

    def test_foo(self, testdir):
        testdir.makepyfile("""
            def test_foo():
                pass

        """)

        result = testdir.runpytest()
>       result.stdout.no_fnmatch_line("*1 passed*")
E       Failed: nomatch: '*1 passed*'
E           and: '======================================= test session starts ======================================='
E           and: 'platform win32 -- Python 3.6.6, pytest-5.2.2.dev30+gc68d50593, py-1.8.0, pluggy-0.13.0'
E           and: "hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('d:\\\\projects\\\\pytest\\\\.hypothesis\\\\examples')"

E           and: 'rootdir: C:\\Users\\Bruno\\AppData\\Local\\Temp\\pytest-of-Bruno\\pytest-77\\test_foo0'
E           and: 'plugins: hypothesis-4.36.0, forked-1.1.1, xdist-1.30.1.dev2+gf7e7d87.d20191104'
E           and: 'collected 1 item'
E           and: ''
E           and: 'test_foo.py .                                                                                [100%]'
E           and: ''
E       fnmatch: '*1 passed*'
E          with: '======================================== 1 passed in 0.02s ========================================'

Changing to assert '1 passed' not in result.stdout.str(), we get this:

__________________________________ TestCustomConftests.test_foo ___________________________________

self = <test_collection.TestCustomConftests object at 0x000001C5BC9CE7B8>
testdir = <Testdir local('C:\\Users\\Bruno\\AppData\\Local\\Temp\\pytest-of-Bruno\\pytest-78\\test_foo0')>

    def test_foo(self, testdir):
        testdir.makepyfile("""
            def test_foo():
                pass

        """)

        result = testdir.runpytest()
        #result.stdout.no_fnmatch_line("*1 passed*")
>       assert '1 passed' not in result.stdout.str()
E       AssertionError: assert '1 passed' not in '======================================= test session starts =======================================\nplatform win32 -...         [100%]\n\n======================================== 1 passed in 0.02s ========================================'
E        +  where '======================================= test session starts =======================================\nplatform win32 -...
    [100%]\n\n======================================== 1 passed in 0.02s ========================================' = <bound method LineMatcher.str of <_pytest.pytester.LineMatcher object at 0x000001C5BD384128>>()
E        +    where <bound method LineMatcher.str of <_pytest.pytester.LineMatcher object at 0x000001C5BD384128>> = <_pytest.pytester.LineMatcher object at 0x000001C5BD384128>.str
E        +      where <_pytest.pytester.LineMatcher object at 0x000001C5BD384128> = <RunResult ret=ExitCode.OK len(stdout.lines)=10 len(stderr.lines)=0 duration=0.11s>.stdout

I think the first example is much easier to read.

Anyway, I've tried to use it and it does not work properly after successful matching.

That does look a bug, thanks for catching it. If you have things in place, could you open a PR against features? That feature has not been released yet AFAIK.

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Nov 7, 2019

My use case is that the str not in bar for many lines of text produces terrible output:

I see.
Maybe improving that then would have been better, since it benefits it in general.

Also keep in mind that the tests are meant to pass normally (and that using no_fnmatch_line adds some overhead to it also).
So when it fails you can investigate using --pdb in case the output is hard / impossible to read.

Note also that this might show that it is not handled correctly via https://github.com/blueyed/pytest/blob/e0fb4de35191b12d22934f8c6ef7eddb173f42da/src/_pytest/assertion/util.py#L447-L453, which should give a "%s is contained here:" message).

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Nov 7, 2019

That does look a bug, thanks for catching it. If you have things in place, could you open a PR against features? That feature has not been released yet AFAIK.

Given the arguments above and since this is not released I am more in the camp of reverting / removing that feature.
If you want to keep it really, the fix is here: aade7ed

@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Nov 7, 2019

Sorry, I'm not convinced... just because I can investigate the output with --pdb doesn't mean the feature is not useful, specially on CI. Overhead I also don't buy, unless we have data to back it up.

But let's hear from others: @asottile @RonnyPfannschmidt @The-Compiler, what's your opinion?

@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Nov 7, 2019

Like I've said it looks more like the reporting needs to be improved/fixed there likely.

With this it looks good:

def test_linematcher_str(LineMatcher):
    lm = LineMatcher(["one", "two", "three"])
    assert "one" not in lm.str()
    def test_linematcher_str(LineMatcher):
        lm = LineMatcher(["one", "two", "three"])
>       assert "one" not in lm.str()
E       AssertionError: assert 'one' not in 'one\ntwo\nthree'
E         'one' is contained here:
E           one
E           two
E           three

testing/test_pytester.py:547: AssertionError
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 20, 2019
This is necessary for when using e.g. `no_fnmatch_line` after it.
Factor it out into `_fail`.

(cherry picked from commit aade7ed)

Ref: pytest-dev#5914 (comment)
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 20, 2019
This is necessary for when using e.g. `no_fnmatch_line` after it.
Factor it out into `_fail`.

(cherry picked from commit aade7ed)

Ref: pytest-dev#5914 (comment)
@blueyed

This comment has been minimized.

Copy link
Contributor

blueyed commented Nov 20, 2019

=> #6237 for the fix.

@merwok

This comment has been minimized.

Copy link

merwok commented Nov 20, 2019

Hi there! I saw this mentioned in the changelog (from a dependency update PR in one of my projects), but I don’t understand if these are new functions meant to be used to write tests, or changes to pytest internals. I don’t know what pytester is; searching the doc for no_fnmatch_line doesn’t tell more. The changelog mentions «the previous idiom», with a result.stdout reference that I’m not familiar with, but not the new idiom. Maybe adding a link or sentence to the doc would help readers like me. Thanks!

@asottile

This comment has been minimized.

Copy link
Member

asottile commented Nov 20, 2019

pytester is mostly used to test pytest and pytest plugins (and doesn't really have much use outside of that)

we should probably link to pytester in the changelog notes 👍

nicoddemus added a commit to nicoddemus/pytest that referenced this pull request Nov 20, 2019
@nicoddemus

This comment has been minimized.

Copy link
Member Author

nicoddemus commented Nov 20, 2019

Thanks @merwok for the nudge, opened #6242 to improve that. 👍

bors bot added a commit to duckinator/bork that referenced this pull request Nov 21, 2019
Merge #79
79: Update pytest to 5.3.0 r=duckinator a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 5.3.0
   ```
   =========================

Deprecations
------------

- `6179 &lt;https://github.com/pytest-dev/pytest/issues/6179&gt;`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given
  that this is the version supported by default in modern tools that manipulate this type of file.

  In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option
  is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``.

  For more information, `see the docs &lt;https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2&gt;`__.



Features
--------

- `4488 &lt;https://github.com/pytest-dev/pytest/issues/4488&gt;`_: The pytest team has created the `pytest-reportlog &lt;https://github.com/pytest-dev/pytest-reportlog&gt;`__
  plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes.

  Each line of the report log contains a self contained JSON object corresponding to a testing event,
  such as a collection or a test result report. The file is guaranteed to be flushed after writing
  each line, so systems can read and process events in real-time.

  The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed
  in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and
  provide feedback.


- `4730 &lt;https://github.com/pytest-dev/pytest/issues/4730&gt;`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.

  This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions.


- `5515 &lt;https://github.com/pytest-dev/pytest/issues/5515&gt;`_: Allow selective auto-indentation of multiline log messages.

  Adds command line option ``--log-auto-indent``, config option
  ``log_auto_indent`` and support for per-entry configuration of
  indentation behavior on calls to ``logging.log()``.

  Alters the default for auto-indention from ``on`` to ``off``. This
  restores the older behavior that existed prior to v4.6.0. This
  reversion to earlier behavior was done because it is better to
  activate new features that may lead to broken tests explicitly
  rather than implicitly.


- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: ``pytester`` learned two new functions, `no_fnmatch_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line&gt;`_ and
  `no_re_match_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line&gt;`_.

  The functions are used to ensure the captured text *does not* match the given
  pattern.

  The previous idiom was to use ``re.match``:

  .. code-block:: python

      assert re.match(pat, result.stdout.str()) is None

  Or the ``in`` operator:

  .. code-block:: python

      assert text in result.stdout.str()

  But the new functions produce best output on failure.


- `6057 &lt;https://github.com/pytest-dev/pytest/issues/6057&gt;`_: Added tolerances to complex values when printing ``pytest.approx``.

  For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle.


- `6061 &lt;https://github.com/pytest-dev/pytest/issues/6061&gt;`_: Added the pluginmanager as an argument to ``pytest_addoption``
  so that hooks can be invoked when setting up command line options. This is
  useful for having one plugin communicate things to another plugin,
  such as default values or which set of command line options to add.



Improvements
------------

- `5061 &lt;https://github.com/pytest-dev/pytest/issues/5061&gt;`_: Use multiple colors with terminal summary statistics.


- `5630 &lt;https://github.com/pytest-dev/pytest/issues/5630&gt;`_: Quitting from debuggers is now properly handled in ``doctest`` items.


- `5924 &lt;https://github.com/pytest-dev/pytest/issues/5924&gt;`_: Improved verbose diff output with sequences.

  Before:

  .. code-block::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E     - [&#39;version&#39;, &#39;version_info&#39;, &#39;sys.version&#39;, &#39;sys.version_info&#39;]
      E     + [&#39;version&#39;,
      E     +  &#39;version_info&#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;]

  After:

  .. code-block::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E       [
      E        &#39;version&#39;,
      E        &#39;version_info&#39;,
      E        &#39;sys.version&#39;,
      E        &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E       ]


- `5936 &lt;https://github.com/pytest-dev/pytest/issues/5936&gt;`_: Display untruncated assertion message with ``-vv``.


- `5990 &lt;https://github.com/pytest-dev/pytest/issues/5990&gt;`_: Fixed plurality mismatch in test summary (e.g. display &quot;1 error&quot; instead of &quot;1 errors&quot;).


- `6008 &lt;https://github.com/pytest-dev/pytest/issues/6008&gt;`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be
  immutable and avoid accidental modifications.


- `6023 &lt;https://github.com/pytest-dev/pytest/issues/6023&gt;`_: ``pytest.main`` now returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still).


- `6026 &lt;https://github.com/pytest-dev/pytest/issues/6026&gt;`_: Align prefixes in output of pytester&#39;s ``LineMatcher``.


- `6059 &lt;https://github.com/pytest-dev/pytest/issues/6059&gt;`_: Collection errors are reported as errors (and not failures like before) in the terminal&#39;s short test summary.


- `6069 &lt;https://github.com/pytest-dev/pytest/issues/6069&gt;`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally.


- `6097 &lt;https://github.com/pytest-dev/pytest/issues/6097&gt;`_: The &quot;[XXX%]&quot; indicator in the test summary is now colored according to the final (new) multi-colored line&#39;s main color.


- `6116 &lt;https://github.com/pytest-dev/pytest/issues/6116&gt;`_: Added ``--co`` as a synonym to ``--collect-only``.


- `6148 &lt;https://github.com/pytest-dev/pytest/issues/6148&gt;`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix.


- `6152 &lt;https://github.com/pytest-dev/pytest/issues/6152&gt;`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes.


- `6176 &lt;https://github.com/pytest-dev/pytest/issues/6176&gt;`_: Improved failure reporting with pytester&#39;s ``Hookrecorder.assertoutcome``.


- `6181 &lt;https://github.com/pytest-dev/pytest/issues/6181&gt;`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary.


- `6206 &lt;https://github.com/pytest-dev/pytest/issues/6206&gt;`_: Improved ``cache.set`` robustness and performance.



Bug Fixes
---------

- `2049 &lt;https://github.com/pytest-dev/pytest/issues/2049&gt;`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes.


- `2548 &lt;https://github.com/pytest-dev/pytest/issues/2548&gt;`_: Fixed line offset mismatch of skipped tests in terminal summary.


- `6039 &lt;https://github.com/pytest-dev/pytest/issues/6039&gt;`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin.

  This is important when used with ``pytester``&#39;s ``runpytest_inprocess``.


- `6047 &lt;https://github.com/pytest-dev/pytest/issues/6047&gt;`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc.


- `6074 &lt;https://github.com/pytest-dev/pytest/issues/6074&gt;`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``.


- `6189 &lt;https://github.com/pytest-dev/pytest/issues/6189&gt;`_: Fixed result of ``getmodpath`` method.



Trivial/Internal Changes
------------------------

- `4901 &lt;https://github.com/pytest-dev/pytest/issues/4901&gt;`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a
  valid ``pytest.ExitCode`` value.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
bors bot added a commit to duckinator/bork that referenced this pull request Nov 28, 2019
Merge #81
81: Update pytest to 5.3.1 r=duckinator a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **5.3.0** to **5.3.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 5.3.1
   ```
   =========================

Improvements
------------

- `6231 &lt;https://github.com/pytest-dev/pytest/issues/6231&gt;`_: Improve check for misspelling of ``pytest.mark.parametrize``.


- `6257 &lt;https://github.com/pytest-dev/pytest/issues/6257&gt;`_: Handle `exit.Exception` raised in `notify_exception` (via `pytest_internalerror`), e.g. when quitting pdb from post mortem.



Bug Fixes
---------

- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: pytester: fix ``no_fnmatch_line`` when used after positive matching.


- `6082 &lt;https://github.com/pytest-dev/pytest/issues/6082&gt;`_: Fix line detection for doctest samples inside ``property`` docstrings, as a workaround to `bpo-17446 &lt;https://bugs.python.org/issue17446&gt;`__.


- `6254 &lt;https://github.com/pytest-dev/pytest/issues/6254&gt;`_: Fix compatibility with pytest-parallel (regression in pytest 5.3.0).


- `6255 &lt;https://github.com/pytest-dev/pytest/issues/6255&gt;`_: Clear the ``sys.last_traceback``, ``sys.last_type`` and ``sys.last_value``
  attributes by deleting them instead of setting them to ``None``. This better
  matches the behaviour of the Python standard library.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
bors bot added a commit to rehandalal/therapist that referenced this pull request Nov 28, 2019
Merge #105
105: Update pytest to 5.3.1 r=rehandalal a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **5.2.4** to **5.3.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 5.3.1
   ```
   =========================

Improvements
------------

- `6231 &lt;https://github.com/pytest-dev/pytest/issues/6231&gt;`_: Improve check for misspelling of ``pytest.mark.parametrize``.


- `6257 &lt;https://github.com/pytest-dev/pytest/issues/6257&gt;`_: Handle `exit.Exception` raised in `notify_exception` (via `pytest_internalerror`), e.g. when quitting pdb from post mortem.



Bug Fixes
---------

- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: pytester: fix ``no_fnmatch_line`` when used after positive matching.


- `6082 &lt;https://github.com/pytest-dev/pytest/issues/6082&gt;`_: Fix line detection for doctest samples inside ``property`` docstrings, as a workaround to `bpo-17446 &lt;https://bugs.python.org/issue17446&gt;`__.


- `6254 &lt;https://github.com/pytest-dev/pytest/issues/6254&gt;`_: Fix compatibility with pytest-parallel (regression in pytest 5.3.0).


- `6255 &lt;https://github.com/pytest-dev/pytest/issues/6255&gt;`_: Clear the ``sys.last_traceback``, ``sys.last_type`` and ``sys.last_value``
  attributes by deleting them instead of setting them to ``None``. This better
  matches the behaviour of the Python standard library.
   ```
   
  
  
   ### 5.3.0
   ```
   =========================

Deprecations
------------

- `6179 &lt;https://github.com/pytest-dev/pytest/issues/6179&gt;`_: The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given
  that this is the version supported by default in modern tools that manipulate this type of file.

  In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option
  is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``.

  For more information, `see the docs &lt;https://docs.pytest.org/en/latest/deprecations.htmljunit-family-default-value-change-to-xunit2&gt;`__.



Features
--------

- `4488 &lt;https://github.com/pytest-dev/pytest/issues/4488&gt;`_: The pytest team has created the `pytest-reportlog &lt;https://github.com/pytest-dev/pytest-reportlog&gt;`__
  plugin, which provides a new ``--report-log=FILE`` option that writes *report logs* into a file as the test session executes.

  Each line of the report log contains a self contained JSON object corresponding to a testing event,
  such as a collection or a test result report. The file is guaranteed to be flushed after writing
  each line, so systems can read and process events in real-time.

  The plugin is meant to replace the ``--resultlog`` option, which is deprecated and meant to be removed
  in a future release. If you use ``--resultlog``, please try out ``pytest-reportlog`` and
  provide feedback.


- `4730 &lt;https://github.com/pytest-dev/pytest/issues/4730&gt;`_: When ``sys.pycache_prefix`` (Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.

  This makes it easier to benefit of cached ``.pyc`` files even on file systems without permissions.


- `5515 &lt;https://github.com/pytest-dev/pytest/issues/5515&gt;`_: Allow selective auto-indentation of multiline log messages.

  Adds command line option ``--log-auto-indent``, config option
  ``log_auto_indent`` and support for per-entry configuration of
  indentation behavior on calls to ``logging.log()``.

  Alters the default for auto-indention from ``on`` to ``off``. This
  restores the older behavior that existed prior to v4.6.0. This
  reversion to earlier behavior was done because it is better to
  activate new features that may lead to broken tests explicitly
  rather than implicitly.


- `5914 &lt;https://github.com/pytest-dev/pytest/issues/5914&gt;`_: `testdir &lt;https://docs.pytest.org/en/latest/reference.htmltestdir&gt;`__ learned two new functions, `no_fnmatch_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_fnmatch_line&gt;`_ and
  `no_re_match_line &lt;https://docs.pytest.org/en/latest/reference.html_pytest.pytester.LineMatcher.no_re_match_line&gt;`_.

  The functions are used to ensure the captured text *does not* match the given
  pattern.

  The previous idiom was to use ``re.match``:

  .. code-block:: python

      result = testdir.runpytest()
      assert re.match(pat, result.stdout.str()) is None

  Or the ``in`` operator:

  .. code-block:: python

      result = testdir.runpytest()
      assert text in result.stdout.str()

  But the new functions produce best output on failure.


- `6057 &lt;https://github.com/pytest-dev/pytest/issues/6057&gt;`_: Added tolerances to complex values when printing ``pytest.approx``.

  For example, ``repr(pytest.approx(3+4j))`` returns ``(3+4j) ± 5e-06 ∠ ±180°``. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. For ``approx`` comparisons to return ``True``, the actual value should fall within this circle.


- `6061 &lt;https://github.com/pytest-dev/pytest/issues/6061&gt;`_: Added the pluginmanager as an argument to ``pytest_addoption``
  so that hooks can be invoked when setting up command line options. This is
  useful for having one plugin communicate things to another plugin,
  such as default values or which set of command line options to add.



Improvements
------------

- `5061 &lt;https://github.com/pytest-dev/pytest/issues/5061&gt;`_: Use multiple colors with terminal summary statistics.


- `5630 &lt;https://github.com/pytest-dev/pytest/issues/5630&gt;`_: Quitting from debuggers is now properly handled in ``doctest`` items.


- `5924 &lt;https://github.com/pytest-dev/pytest/issues/5924&gt;`_: Improved verbose diff output with sequences.

  Before:

  ::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E     - [&#39;version&#39;, &#39;version_info&#39;, &#39;sys.version&#39;, &#39;sys.version_info&#39;]
      E     + [&#39;version&#39;,
      E     +  &#39;version_info&#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;]

  After:

  ::

      E   AssertionError: assert [&#39;version&#39;, &#39;...version_info&#39;] == [&#39;version&#39;, &#39;...version&#39;, ...]
      E     Right contains 3 more items, first extra item: &#39; &#39;
      E     Full diff:
      E       [
      E        &#39;version&#39;,
      E        &#39;version_info&#39;,
      E        &#39;sys.version&#39;,
      E        &#39;sys.version_info&#39;,
      E     +  &#39; &#39;,
      E     +  &#39;sys.version&#39;,
      E     +  &#39;sys.version_info&#39;,
      E       ]


- `5934 &lt;https://github.com/pytest-dev/pytest/issues/5934&gt;`_: ``repr`` of ``ExceptionInfo`` objects has been improved to honor the ``__repr__`` method of the underlying exception.

- `5936 &lt;https://github.com/pytest-dev/pytest/issues/5936&gt;`_: Display untruncated assertion message with ``-vv``.


- `5990 &lt;https://github.com/pytest-dev/pytest/issues/5990&gt;`_: Fixed plurality mismatch in test summary (e.g. display &quot;1 error&quot; instead of &quot;1 errors&quot;).


- `6008 &lt;https://github.com/pytest-dev/pytest/issues/6008&gt;`_: ``Config.InvocationParams.args`` is now always a ``tuple`` to better convey that it should be
  immutable and avoid accidental modifications.


- `6023 &lt;https://github.com/pytest-dev/pytest/issues/6023&gt;`_: ``pytest.main`` returns a ``pytest.ExitCode`` instance now, except for when custom exit codes are used (where it returns ``int`` then still).


- `6026 &lt;https://github.com/pytest-dev/pytest/issues/6026&gt;`_: Align prefixes in output of pytester&#39;s ``LineMatcher``.


- `6059 &lt;https://github.com/pytest-dev/pytest/issues/6059&gt;`_: Collection errors are reported as errors (and not failures like before) in the terminal&#39;s short test summary.


- `6069 &lt;https://github.com/pytest-dev/pytest/issues/6069&gt;`_: ``pytester.spawn`` does not skip/xfail tests on FreeBSD anymore unconditionally.


- `6097 &lt;https://github.com/pytest-dev/pytest/issues/6097&gt;`_: The &quot;[...%]&quot; indicator in the test summary is now colored according to the final (new) multi-colored line&#39;s main color.


- `6116 &lt;https://github.com/pytest-dev/pytest/issues/6116&gt;`_: Added ``--co`` as a synonym to ``--collect-only``.


- `6148 &lt;https://github.com/pytest-dev/pytest/issues/6148&gt;`_: ``atomicwrites`` is now only used on Windows, fixing a performance regression with assertion rewriting on Unix.


- `6152 &lt;https://github.com/pytest-dev/pytest/issues/6152&gt;`_: Now parametrization will use the ``__name__`` attribute of any object for the id, if present. Previously it would only use ``__name__`` for functions and classes.


- `6176 &lt;https://github.com/pytest-dev/pytest/issues/6176&gt;`_: Improved failure reporting with pytester&#39;s ``Hookrecorder.assertoutcome``.


- `6181 &lt;https://github.com/pytest-dev/pytest/issues/6181&gt;`_: The reason for a stopped session, e.g. with ``--maxfail`` / ``-x``, now gets reported in the test summary.


- `6206 &lt;https://github.com/pytest-dev/pytest/issues/6206&gt;`_: Improved ``cache.set`` robustness and performance.



Bug Fixes
---------

- `2049 &lt;https://github.com/pytest-dev/pytest/issues/2049&gt;`_: Fixed ``--setup-plan`` showing inaccurate information about fixture lifetimes.


- `2548 &lt;https://github.com/pytest-dev/pytest/issues/2548&gt;`_: Fixed line offset mismatch of skipped tests in terminal summary.


- `6039 &lt;https://github.com/pytest-dev/pytest/issues/6039&gt;`_: The ``PytestDoctestRunner`` is now properly invalidated when unconfiguring the doctest plugin.

  This is important when used with ``pytester``&#39;s ``runpytest_inprocess``.


- `6047 &lt;https://github.com/pytest-dev/pytest/issues/6047&gt;`_: BaseExceptions are now handled in ``saferepr``, which includes ``pytest.fail.Exception`` etc.


- `6074 &lt;https://github.com/pytest-dev/pytest/issues/6074&gt;`_: pytester: fixed order of arguments in ``rm_rf`` warning when cleaning up temporary directories, and do not emit warnings for errors with ``os.open``.


- `6189 &lt;https://github.com/pytest-dev/pytest/issues/6189&gt;`_: Fixed result of ``getmodpath`` method.



Trivial/Internal Changes
------------------------

- `4901 &lt;https://github.com/pytest-dev/pytest/issues/4901&gt;`_: ``RunResult`` from ``pytester`` now displays the mnemonic of the ``ret`` attribute when it is a
  valid ``pytest.ExitCode`` value.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.