Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: --lf running only a subset of last-failed tests #7758

Closed
jbrockmendel opened this issue Sep 15, 2020 · 16 comments
Closed

BUG: --lf running only a subset of last-failed tests #7758

jbrockmendel opened this issue Sep 15, 2020 · 16 comments

Comments

@jbrockmendel
Copy link

@jbrockmendel jbrockmendel commented Sep 15, 2020

After running the pandas test suite and getting e.g. 20 test failures, running the same command with " --lf" appended should run those 20 tests. In practice I'm seeing only fraction of these run on the subsequent run.

$ pytest pandas/tests --skip-slow --skip-db
[...]
=================================================================== short test summary info ====================================================================
FAILED pandas/tests/resample/test_resampler_grouper.py::test_apply_columns_multilevel - KeyError: Timestamp('2017-01-01 00:00:00', freq='H')
FAILED pandas/tests/groupby/test_function.py::test_median_empty_bins[True] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="0") are different
FAILED pandas/tests/groupby/test_function.py::test_ops_general[count-size] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="0") are different
FAILED pandas/tests/groupby/test_grouping.py::TestGrouping::test_evaluate_with_empty_groups[agg-expected1] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_aggregate_item_by_item - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>0] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>1] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>2] - AssertionError: Attributes of DataFrame.iloc[:, 0] (column ...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-mean-mean] - AssertionError: Attributes of DataFrame.iloc[:, 0] (col...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-median-<lambda>] - AssertionError: Attributes of DataFrame.iloc[:, 0...
FAILED pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-var-<lambda>] - AssertionError: Attributes of DataFrame.iloc[:, 0] (...
FAILED pandas/tests/groupby/aggregate/test_other.py::test_agg_lambda_with_timezone - ValueError: Must produce aggregated value
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[NotImplementedError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[RuntimeError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[KeyError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[IndexError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[OSError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ValueError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ArithmeticError] - AssertionError: Attributes of Series are different
FAILED pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[AttributeError] - AssertionError: Attributes of Series are different
====================================== 20 failed, 5034 passed, 1652 skipped, 374 xfailed, 1 warning in 208.51s (0:03:28) =======================================

$ pytest pandas/tests/resample pandas/tests/groupby pandas/tests/generic/ --skip-slow --lf
[...]
============================================================== 1 failed, 1455 deselected in 0.68s ==============================================================

Running pytest 6.0.2, py3.8.5, OSX 10.15.5

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Sep 16, 2020

Can you show the .pytest_cache/v/cache/lastfailed file after the first run?

@jbrockmendel
Copy link
Author

@jbrockmendel jbrockmendel commented Sep 17, 2020

$ cat pytest_cache-bak/v/cache/lastfailed
{
  "pandas/tests/extension/decimal/test_decimal.py::test_groupby_agg": true,
  "pandas/tests/extension/decimal/test_decimal.py::test_groupby_agg_ea_method": true,
  "pandas/tests/groupby/aggregate/test_aggregate.py::test_aggregate_item_by_item": true,
  "pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>0]": true,
  "pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>1]": true,
  "pandas/tests/groupby/aggregate/test_aggregate.py::test_multiindex_custom_func[<lambda>2]": true,
  "pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-mean-mean]": true,
  "pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-median-<lambda>]": true,
  "pandas/tests/groupby/aggregate/test_cython.py::test_cython_agg_empty_buckets[True-var-<lambda>]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_agg_lambda_with_timezone": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ArithmeticError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[AttributeError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[IndexError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[KeyError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[NotImplementedError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[OSError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[RuntimeError]": true,
  "pandas/tests/groupby/aggregate/test_other.py::test_groupby_agg_err_catching[ValueError]": true,
  "pandas/tests/groupby/test_function.py::test_median_empty_bins[True]": true,
  "pandas/tests/groupby/test_function.py::test_ops_general[count-size]": true,
  "pandas/tests/groupby/test_grouping.py::TestGrouping::test_evaluate_with_empty_groups[agg-expected1]": true,
  "pandas/tests/resample/test_resampler_grouper.py::test_apply_columns_multilevel": true
}%

The output from the second run:

========================================================================= short test summary info ==========================================================================
FAILED pandas/tests/extension/decimal/test_decimal.py::test_groupby_agg - AssertionError: Attributes of Series are different
FAILED pandas/tests/extension/decimal/test_decimal.py::test_groupby_agg_ea_method - AssertionError: Attributes of Series are different
============================================================= 2 failed, 30232 deselected, 2 warnings in 9.07s ==============================================================

@jbrockmendel
Copy link
Author

@jbrockmendel jbrockmendel commented Sep 18, 2020

Possibly relevant: im often running multiple pytest processes in separate directories

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Sep 18, 2020

Possibly relevant: im often running multiple pytest processes in separate directories

At the same time? During each run, the rootdir on the pytest header is the same?

The cache mechanism used by --lf is not multi-process safe, so this can definitely be the reason if you are running long test suites at the same time.

@jbrockmendel
Copy link
Author

@jbrockmendel jbrockmendel commented Sep 18, 2020

During each run, the rootdir on the pytest header is the same?

how do i determine the rootdir?

I have different branches in different git worktree directories, am only running one process at a time for any single branch, always with the cwd in that branch's directory

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Sep 18, 2020

how do i determine the rootdir?

From your answer I assume they are different? 😉

You can use --rootdir, or even better, set the cache_dir configuration option, which will set the cache directory (used by --lf) explicitly.

am only running one process at a time for any single branch,

Ahh OK thanks, so it is not a race condition.

@ojii
Copy link
Contributor

@ojii ojii commented Sep 25, 2020

I'm seeing the same issue:

$ pytest
[...]
4 failed, 1857 passed, 144 skipped
$ pytest --last-failed
[...]
run-last-failure: rerun previous 1 failure first (skipped 24 files)
[...]
1 failed, 870 deselected

is it incorrectly skipping too many files?

We're using pytest 5.4.3, but I upgraded to 6.0.2 and the behavior is the same.

@ornichola
Copy link

@ornichola ornichola commented Sep 29, 2020

I'm facing the same issue. My lastfailed file contains 34 items:

(venv) user@macbook % pytest --cache-show
...
cache/lastfailed contains:
  {
  ...
  }

But running last failed tests select to run only 12 items:

(venv) user@macbook % pytest --lf --collect-only
...
collected 53 items / 41 deselected / 12 selected
...
run-last-failure: rerun previous 12 failures (skipped 55 files)

my environment:

platform darwin -- Python 3.8.1, pytest-5.4.3, py-1.8.1, pluggy-0.13.1
plugins: mock-3.1.1, allure-pytest-2.8.16, flaky-3.6.1, ordering-0.6, forked-1.1.3, xdist-1.32.0
@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Sep 29, 2020

Perhaps something else (hook or plugin) is deselecting them? @ojii @jbrockmendel what plugins do you have installed? Does someone have a minimal, isolated example?

@jbrockmendel
Copy link
Author

@jbrockmendel jbrockmendel commented Sep 29, 2020

what plugins do you have installed?

looks like hypothesis, pytest-xdist, pytest-cov, pytest-asyncio, pytest-forked

Some of these I can't do without, but I can try trimming a few and see if that affects anything.

@The-Compiler
Copy link
Member

@The-Compiler The-Compiler commented Sep 30, 2020

Hmm, I do notice some weird behavior with --lf as well, though it's the opposite for me - pytest runs more than it should, or even thinks nothing failed and runs everything again. I'll see if I can find out what causes it to happen.

@fergushev
Copy link

@fergushev fergushev commented Oct 5, 2020

I'm seeing this as well and the problem (at least in my case) appears to be related to having an __init__.py file in the test directories.

My setup looks something like this:

tests/functional/foo/  10 tests (dir has `__init__.py`)
tests/functional/bar/  10 tests (dir does _not_ have `__init__.py`)
tests/functional/baz/  10 tests (dir has `__init__.py`)

My lastfailed contains all 30 tests from the above directories.
When I run with --lf it only picks up the tests from tests/functional/bar/.
If I delete __init__.py from /foo/ then --lf picks up 20 tests.
If I delete __init__.py from /baz/ then --lf picks up all 30 tests.


{'Python': '3.8.5', 'Platform': 'macOS-10.15.7-x86_64-i386-64bit', 
'Packages': {'pytest': '6.1.1', 'py': '1.9.0', 'pluggy': '0.13.1'}, 
'Plugins': {'metadata': '1.10.0', 'html': '1.20.0', 'xdist': '2.1.0', 'forked': '1.3.0'}}
@bluetech
Copy link
Member

@bluetech bluetech commented Oct 9, 2020

Pretty sure I saw this too, but wasn't in a situation where I could look into it. We should definitely look at what's going wrong here. In the mean time any sort of standalone/consistent reproduction steps would be very helpful.

@bluetech
Copy link
Member

@bluetech bluetech commented Oct 14, 2020

In #7894 @deb17 has provided some more details on this issue.

@bluetech
Copy link
Member

@bluetech bluetech commented Oct 19, 2020

Bisected @deb17's test case to 1b30514 (pr #6448). So this is not a new issue, it regressed in 5.4.0. I think what happens is that the package containing a failing tests gets incorrectly filtered out. I'll try to figure out what goes wrong here.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Oct 19, 2020

The problems with --lf all started with a very simple implementation from me that tried to speed up --lf, and that proved not to be sufficient, so we kept adding more and more complexity to it to the point where the code is really hard to follow/understand and contains subtle bugs.

I'm OK with just dropping all the extra complexity in there, it will make for slightly slower code, but simpler and more importantly, correct.

bors bot added a commit to aragilar/pytest-info-collector that referenced this issue Mar 29, 2021
Merge #14 #17
14: Update sphinx_rtd_theme to 0.5.1 r=aragilar a=pyup-bot


This PR updates [sphinx_rtd_theme](https://pypi.org/project/sphinx_rtd_theme) from **0.5.0** to **0.5.1**.



*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/sphinx-rtd-theme
  - Repo: https://github.com/readthedocs/sphinx_rtd_theme
</details>



17: Update pytest to 6.2.2 r=aragilar a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **6.1.1** to **6.2.2**.



<details>
  <summary>Changelog</summary>
  
  
   ### 6.2.2
   ```
   =========================

Bug Fixes
---------

- `8152 &lt;https://github.com/pytest-dev/pytest/issues/8152&gt;`_: Fixed &quot;(&lt;Skipped instance&gt;)&quot; being shown as a skip reason in the verbose test summary line when the reason is empty.


- `8249 &lt;https://github.com/pytest-dev/pytest/issues/8249&gt;`_: Fix the ``faulthandler`` plugin for occasions when running with ``twisted.logger`` and using ``pytest --capture=no``.
   ```
   
  
  
   ### 6.2.1
   ```
   =========================

Bug Fixes
---------

- `7678 &lt;https://github.com/pytest-dev/pytest/issues/7678&gt;`_: Fixed bug where ``ImportPathMismatchError`` would be raised for files compiled in
  the host and loaded later from an UNC mounted path (Windows).


- `8132 &lt;https://github.com/pytest-dev/pytest/issues/8132&gt;`_: Fixed regression in ``approx``: in 6.2.0 ``approx`` no longer raises
  ``TypeError`` when dealing with non-numeric types, falling back to normal comparison.
  Before 6.2.0, array types like tf.DeviceArray fell through to the scalar case,
  and happened to compare correctly to a scalar if they had only one element.
  After 6.2.0, these types began failing, because they inherited neither from
  standard Python number hierarchy nor from ``numpy.ndarray``.

  ``approx`` now converts arguments to ``numpy.ndarray`` if they expose the array
  protocol and are not scalars. This treats array-like objects like numpy arrays,
  regardless of size.
   ```
   
  
  
   ### 6.2.0
   ```
   =========================

Breaking Changes
----------------

- `7808 &lt;https://github.com/pytest-dev/pytest/issues/7808&gt;`_: pytest now supports python3.6+ only.



Deprecations
------------

- `7469 &lt;https://github.com/pytest-dev/pytest/issues/7469&gt;`_: Directly constructing/calling the following classes/functions is now deprecated:

  - ``_pytest.cacheprovider.Cache``
  - ``_pytest.cacheprovider.Cache.for_config()``
  - ``_pytest.cacheprovider.Cache.clear_cache()``
  - ``_pytest.cacheprovider.Cache.cache_dir_from_config()``
  - ``_pytest.capture.CaptureFixture``
  - ``_pytest.fixtures.FixtureRequest``
  - ``_pytest.fixtures.SubRequest``
  - ``_pytest.logging.LogCaptureFixture``
  - ``_pytest.pytester.Pytester``
  - ``_pytest.pytester.Testdir``
  - ``_pytest.recwarn.WarningsRecorder``
  - ``_pytest.recwarn.WarningsChecker``
  - ``_pytest.tmpdir.TempPathFactory``
  - ``_pytest.tmpdir.TempdirFactory``

  These have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 7.0.0.


- `7530 &lt;https://github.com/pytest-dev/pytest/issues/7530&gt;`_: The ``--strict`` command-line option has been deprecated, use ``--strict-markers`` instead.

  We have plans to maybe in the future to reintroduce ``--strict`` and make it an encompassing flag for all strictness
  related options (``--strict-markers`` and ``--strict-config`` at the moment, more might be introduced in the future).


- `7988 &lt;https://github.com/pytest-dev/pytest/issues/7988&gt;`_: The ``pytest.yield_fixture`` decorator/function is now deprecated. Use :func:`pytest.fixture` instead.

  ``yield_fixture`` has been an alias for ``fixture`` for a very long time, so can be search/replaced safely.



Features
--------

- `5299 &lt;https://github.com/pytest-dev/pytest/issues/5299&gt;`_: pytest now warns about unraisable exceptions and unhandled thread exceptions that occur in tests on Python&gt;=3.8.
  See :ref:`unraisable` for more information.


- `7425 &lt;https://github.com/pytest-dev/pytest/issues/7425&gt;`_: New :fixture:`pytester` fixture, which is identical to :fixture:`testdir` but its methods return :class:`pathlib.Path` when appropriate instead of ``py.path.local``.

  This is part of the movement to use :class:`pathlib.Path` objects internally, in order to remove the dependency to ``py`` in the future.

  Internally, the old :class:`Testdir &lt;_pytest.pytester.Testdir&gt;` is now a thin wrapper around :class:`Pytester &lt;_pytest.pytester.Pytester&gt;`, preserving the old interface.


- `7695 &lt;https://github.com/pytest-dev/pytest/issues/7695&gt;`_: A new hook was added, `pytest_markeval_namespace` which should return a dictionary.
  This dictionary will be used to augment the &quot;global&quot; variables available to evaluate skipif/xfail/xpass markers.

  Pseudo example

  ``conftest.py``:

  .. code-block:: python

     def pytest_markeval_namespace():
         return {&quot;color&quot;: &quot;red&quot;}

  ``test_func.py``:

  .. code-block:: python

     pytest.mark.skipif(&quot;color == &#39;blue&#39;&quot;, reason=&quot;Color is not red&quot;)
     def test_func():
         assert False


- `8006 &lt;https://github.com/pytest-dev/pytest/issues/8006&gt;`_: It is now possible to construct a :class:`~pytest.MonkeyPatch` object directly as ``pytest.MonkeyPatch()``,
  in cases when the :fixture:`monkeypatch` fixture cannot be used. Previously some users imported it
  from the private `_pytest.monkeypatch.MonkeyPatch` namespace.

  Additionally, :meth:`MonkeyPatch.context &lt;pytest.MonkeyPatch.context&gt;` is now a classmethod,
  and can be used as ``with MonkeyPatch.context() as mp: ...``. This is the recommended way to use
  ``MonkeyPatch`` directly, since unlike the ``monkeypatch`` fixture, an instance created directly
  is not ``undo()``-ed automatically.



Improvements
------------

- `1265 &lt;https://github.com/pytest-dev/pytest/issues/1265&gt;`_: Added an ``__str__`` implementation to the :class:`~pytest.pytester.LineMatcher` class which is returned from ``pytester.run_pytest().stdout`` and similar. It returns the entire output, like the existing ``str()`` method.


- `2044 &lt;https://github.com/pytest-dev/pytest/issues/2044&gt;`_: Verbose mode now shows the reason that a test was skipped in the test&#39;s terminal line after the &quot;SKIPPED&quot;, &quot;XFAIL&quot; or &quot;XPASS&quot;.


- `7469 &lt;https://github.com/pytest-dev/pytest/issues/7469&gt;`_ The types of builtin pytest fixtures are now exported so they may be used in type annotations of test functions.
  The newly-exported types are:

  - ``pytest.FixtureRequest`` for the :fixture:`request` fixture.
  - ``pytest.Cache`` for the :fixture:`cache` fixture.
  - ``pytest.CaptureFixture[str]`` for the :fixture:`capfd` and :fixture:`capsys` fixtures.
  - ``pytest.CaptureFixture[bytes]`` for the :fixture:`capfdbinary` and :fixture:`capsysbinary` fixtures.
  - ``pytest.LogCaptureFixture`` for the :fixture:`caplog` fixture.
  - ``pytest.Pytester`` for the :fixture:`pytester` fixture.
  - ``pytest.Testdir`` for the :fixture:`testdir` fixture.
  - ``pytest.TempdirFactory`` for the :fixture:`tmpdir_factory` fixture.
  - ``pytest.TempPathFactory`` for the :fixture:`tmp_path_factory` fixture.
  - ``pytest.MonkeyPatch`` for the :fixture:`monkeypatch` fixture.
  - ``pytest.WarningsRecorder`` for the :fixture:`recwarn` fixture.

  Constructing them is not supported (except for `MonkeyPatch`); they are only meant for use in type annotations.
  Doing so will emit a deprecation warning, and may become a hard-error in pytest 7.0.

  Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.


- `7527 &lt;https://github.com/pytest-dev/pytest/issues/7527&gt;`_: When a comparison between :func:`namedtuple &lt;collections.namedtuple&gt;` instances of the same type fails, pytest now shows the differing field names (possibly nested) instead of their indexes.


- `7615 &lt;https://github.com/pytest-dev/pytest/issues/7615&gt;`_: :meth:`Node.warn &lt;_pytest.nodes.Node.warn&gt;` now permits any subclass of :class:`Warning`, not just :class:`PytestWarning &lt;pytest.PytestWarning&gt;`.


- `7701 &lt;https://github.com/pytest-dev/pytest/issues/7701&gt;`_: Improved reporting when using ``--collected-only``. It will now show the number of collected tests in the summary stats.


- `7710 &lt;https://github.com/pytest-dev/pytest/issues/7710&gt;`_: Use strict equality comparison for non-numeric types in :func:`pytest.approx` instead of
  raising :class:`TypeError`.

  This was the undocumented behavior before 3.7, but is now officially a supported feature.


- `7938 &lt;https://github.com/pytest-dev/pytest/issues/7938&gt;`_: New ``--sw-skip`` argument which is a shorthand for ``--stepwise-skip``.


- `8023 &lt;https://github.com/pytest-dev/pytest/issues/8023&gt;`_: Added ``&#39;node_modules&#39;`` to default value for :confval:`norecursedirs`.


- `8032 &lt;https://github.com/pytest-dev/pytest/issues/8032&gt;`_: :meth:`doClassCleanups &lt;unittest.TestCase.doClassCleanups&gt;` (introduced in :mod:`unittest` in Python and 3.8) is now called appropriately.



Bug Fixes
---------

- `4824 &lt;https://github.com/pytest-dev/pytest/issues/4824&gt;`_: Fixed quadratic behavior and improved performance of collection of items using autouse fixtures and xunit fixtures.


- `7758 &lt;https://github.com/pytest-dev/pytest/issues/7758&gt;`_: Fixed an issue where some files in packages are getting lost from ``--lf`` even though they contain tests that failed. Regressed in pytest 5.4.0.


- `7911 &lt;https://github.com/pytest-dev/pytest/issues/7911&gt;`_: Directories created by by :fixture:`tmp_path` and :fixture:`tmpdir` are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.


- `7913 &lt;https://github.com/pytest-dev/pytest/issues/7913&gt;`_: Fixed a crash or hang in :meth:`pytester.spawn &lt;_pytest.pytester.Pytester.spawn&gt;` when the :mod:`readline` module is involved.


- `7951 &lt;https://github.com/pytest-dev/pytest/issues/7951&gt;`_: Fixed handling of recursive symlinks when collecting tests.


- `7981 &lt;https://github.com/pytest-dev/pytest/issues/7981&gt;`_: Fixed symlinked directories not being followed during collection. Regressed in pytest 6.1.0.


- `8016 &lt;https://github.com/pytest-dev/pytest/issues/8016&gt;`_: Fixed only one doctest being collected when using ``pytest --doctest-modules path/to/an/__init__.py``.



Improved Documentation
----------------------

- `7429 &lt;https://github.com/pytest-dev/pytest/issues/7429&gt;`_: Add more information and use cases about skipping doctests.


- `7780 &lt;https://github.com/pytest-dev/pytest/issues/7780&gt;`_: Classes which should not be inherited from are now marked ``final class`` in the API reference.


- `7872 &lt;https://github.com/pytest-dev/pytest/issues/7872&gt;`_: ``_pytest.config.argparsing.Parser.addini()`` accepts explicit ``None`` and ``&quot;string&quot;``.


- `7878 &lt;https://github.com/pytest-dev/pytest/issues/7878&gt;`_: In pull request section, ask to commit after editing changelog and authors file.



Trivial/Internal Changes
------------------------

- `7802 &lt;https://github.com/pytest-dev/pytest/issues/7802&gt;`_: The ``attrs`` dependency requirement is now &gt;=19.2.0 instead of &gt;=17.4.0.


- `8014 &lt;https://github.com/pytest-dev/pytest/issues/8014&gt;`_: `.pyc` files created by pytest&#39;s assertion rewriting now conform to the newer PEP-552 format on Python&gt;=3.7.
  (These files are internal and only interpreted by pytest itself.)
   ```
   
  
  
   ### 6.1.2
   ```
   =========================

Bug Fixes
---------

- `7758 &lt;https://github.com/pytest-dev/pytest/issues/7758&gt;`_: Fixed an issue where some files in packages are getting lost from ``--lf`` even though they contain tests that failed. Regressed in pytest 5.4.0.


- `7911 &lt;https://github.com/pytest-dev/pytest/issues/7911&gt;`_: Directories created by `tmpdir` are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.



Improved Documentation
----------------------

- `7815 &lt;https://github.com/pytest-dev/pytest/issues/7815&gt;`_: Improve deprecation warning message for ``pytest._fillfuncargs()``.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

8 participants