Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve help for --runxfail flag #5188

Merged
merged 1 commit into from May 1, 2019

Conversation

Projects
None yet
2 participants
@allanlewis
Copy link
Contributor

commented Apr 30, 2019

As per 528eea7:

The help for the '--runxfail' flag is incorrect. The default behaviour is to run tests marked as 'xfail' but to ignore the results. This flag alters that behaviour in two ways:

  1. Only tests marked 'xfail' are run.
  2. The results of these tests are reported as if they weren't marked 'xfail'.

This commit updates the help to describe point 1 above; point 2 is implied.

  • Create a new changelog file in the changelog folder, with a name like <ISSUE NUMBER>.<TYPE>.rst. See changelog/README.rst for details.
  • Target the master branch for bug fixes, documentation updates and trivial changes.
  • Target the features branch for new features and removals/deprecations.
  • Include documentation when adding new features.
  • Include new tests or update existing tests when applicable.

@allanlewis allanlewis force-pushed the allanlewis:fix-runxfail-docs branch from 528eea7 to 14c9529 Apr 30, 2019

@allanlewis allanlewis marked this pull request as ready for review Apr 30, 2019

@codecov

This comment has been minimized.

Copy link

commented Apr 30, 2019

Codecov Report

Merging #5188 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5188      +/-   ##
==========================================
+ Coverage   96.06%   96.06%   +<.01%     
==========================================
  Files         114      114              
  Lines       25824    25824              
  Branches     2550     2550              
==========================================
+ Hits        24807    24809       +2     
+ Misses        706      705       -1     
+ Partials      311      310       -1
Impacted Files Coverage Δ
src/_pytest/skipping.py 95.55% <ø> (ø) ⬆️
testing/test_pdb.py 99.19% <0%> (+0.4%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 204004c...f050203. Read the comment docs.

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

Hi @allanlewis,

First of all thanks for the PR.

Only tests marked 'xfail' are run.

I don't think this is true, consider:

import pytest

@pytest.mark.xfail
def test_x():
    assert 0

def test_y():
    pass
λ pytest .tmp\test-foo.py -q
x.
1 passed, 1 xfailed in 0.06 seconds


λ pytest .tmp\test-foo.py -q --runxfail
F.
============================================ FAILURES =============================================
_____________________________________________ test_x ______________________________________________

    @pytest.mark.xfail
    def test_x():
>       assert 0
E       assert 0

.tmp\test-foo.py:6: AssertionError
1 failed, 1 passed in 0.06 seconds

As you can see, the run with --runxfail still executed test_y, even if it was not marked as xfail.

So I think the help information is correct, can you tell why did you think otherwise?

@allanlewis

This comment has been minimized.

Copy link
Contributor Author

commented May 1, 2019

Hi @nicoddemus, thanks - you're absolutely right, I wrote this in haste, bear with me...

Improve help for --runxfail flag
The help for the '--runxfail' flag is somewhat misleading. The default
behaviour is to run tests marked as 'xfail' but to ignore the results. This
flag alters that behaviour by running these tests as if they weren't marked
'xfail', i.e. their results are not ignored.

@allanlewis allanlewis force-pushed the allanlewis:fix-runxfail-docs branch from 14c9529 to f050203 May 1, 2019

@allanlewis allanlewis changed the title Correct help for --runxfail flag Improve help for --runxfail flag May 1, 2019

@allanlewis

This comment has been minimized.

Copy link
Contributor Author

commented May 1, 2019

I've rephrased the help message and the commit message.

@nicoddemus
Copy link
Member

left a comment

you're absolutely right, I wrote this in haste, bear with me...

Sure, no worries!

Thanks, the new message reads does better!

@nicoddemus nicoddemus merged commit f17b734 into pytest-dev:master May 1, 2019

5 checks passed

WIP Ready for review
Details
codecov/patch Coverage not affected when comparing 204004c...f050203
Details
codecov/project 96.06% (+<.01%) compared to 204004c
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
pytest-CI #20190501.1 succeeded
Details

bors bot added a commit to rehandalal/therapist that referenced this pull request May 10, 2019

Merge #69
69: Update pytest to 4.4.2 r=rehandalal a=pyup-bot


This PR updates [pytest](https://pypi.org/project/pytest) from **4.4.1** to **4.4.2**.



<details>
  <summary>Changelog</summary>
  
  
   ### 4.4.2
   ```
   =========================

Bug Fixes
---------

- `5089 &lt;https://github.com/pytest-dev/pytest/issues/5089&gt;`_: Fix crash caused by error in ``__repr__`` function with both ``showlocals`` and verbose output enabled.


- `5139 &lt;https://github.com/pytest-dev/pytest/issues/5139&gt;`_: Eliminate core dependency on &#39;terminal&#39; plugin.


- `5229 &lt;https://github.com/pytest-dev/pytest/issues/5229&gt;`_: Require ``pluggy&gt;=0.11.0`` which reverts a dependency to ``importlib-metadata`` added in ``0.10.0``.
  The ``importlib-metadata`` package cannot be imported when installed as an egg and causes issues when relying on ``setup.py`` to install test dependencies.



Improved Documentation
----------------------

- `5171 &lt;https://github.com/pytest-dev/pytest/issues/5171&gt;`_: Doc: ``pytest_ignore_collect``, ``pytest_collect_directory``, ``pytest_collect_file`` and ``pytest_pycollect_makemodule`` hooks&#39;s &#39;path&#39; parameter documented type is now ``py.path.local``


- `5188 &lt;https://github.com/pytest-dev/pytest/issues/5188&gt;`_: Improve help for ``--runxfail`` flag.



Trivial/Internal Changes
------------------------

- `5182 &lt;https://github.com/pytest-dev/pytest/issues/5182&gt;`_: Removed internal and unused ``_pytest.deprecated.MARK_INFO_ATTRIBUTE``.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>



Co-authored-by: pyup-bot <github-bot@pyup.io>

bors bot added a commit to mozilla/normandy that referenced this pull request May 17, 2019

Merge #1887
1887: Scheduled weekly dependency update for week 19 r=rehandalal a=pyup-bot






### Update [botocore](https://pypi.org/project/botocore) from **1.12.142** to **1.12.146**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.12.146
   ```
   ========

* api-change:``glue``: Update glue client to latest version
* api-change:``sts``: Update sts client to latest version
   ```
   
  
  
   ### 1.12.145
   ```
   ========

* api-change:``servicecatalog``: Update servicecatalog client to latest version
* api-change:``eks``: Update eks client to latest version
* api-change:``sagemaker``: Update sagemaker client to latest version
* api-change:``kinesisanalytics``: Update kinesisanalytics client to latest version
* api-change:``kinesisanalyticsv2``: Update kinesisanalyticsv2 client to latest version
   ```
   
  
  
   ### 1.12.144
   ```
   ========

* api-change:``appsync``: Update appsync client to latest version
* api-change:``storagegateway``: Update storagegateway client to latest version
* api-change:``ssm``: Update ssm client to latest version
* api-change:``alexaforbusiness``: Update alexaforbusiness client to latest version
   ```
   
  
  
   ### 1.12.143
   ```
   ========

* api-change:``config``: Update config client to latest version
* api-change:``iam``: Update iam client to latest version
* api-change:``sts``: Update sts client to latest version
* api-change:``codepipeline``: Update codepipeline client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/botocore
  - Changelog: https://pyup.io/changelogs/botocore/
  - Repo: https://github.com/boto/botocore
</details>





### Update [Faker](https://pypi.org/project/Faker) from **1.0.5** to **1.0.6**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/faker
  - Changelog: https://pyup.io/changelogs/faker/
  - Repo: https://github.com/joke2k/faker
</details>





### Update [pluggy](https://pypi.org/project/pluggy) from **0.9.0** to **0.11.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.11.0
   ```
   ==========================

Bug Fixes
---------

- `205 &lt;https://github.com/pytest-dev/pluggy/issues/205&gt;`_: Revert changes made in 0.10.0 release breaking ``.egg`` installs.
   ```
   
  
  
   ### 0.10.0
   ```
   ==========================

Features
--------

- `199 &lt;https://github.com/pytest-dev/pluggy/issues/199&gt;`_: Switch from ``pkg_resources`` to ``importlib-metadata`` for entrypoint detection for improved performance and import time.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pluggy
  - Changelog: https://pyup.io/changelogs/pluggy/
  - Repo: https://github.com/pytest-dev/pluggy
</details>





### Update [Pygments](https://pypi.org/project/Pygments) from **2.3.1** to **2.4.0**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pygments
  - Homepage: http://pygments.org/
</details>





### Update [pyrsistent](https://pypi.org/project/pyrsistent) from **0.15.1** to **0.15.2**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.15.2
   ```
   * Fix 166, Propagate &#39;ignore_extra&#39; param in hierarchy. Thanks ss18 for this!
 * Fix 167, thaw typing. Thanks nattofriends for this!
 * Fix 154, not possible to insert empty pmap as leaf node with transform.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pyrsistent
  - Changelog: https://pyup.io/changelogs/pyrsistent/
  - Repo: http://github.com/tobgu/pyrsistent/
</details>





### Update [boto3](https://pypi.org/project/boto3) from **1.9.142** to **1.9.146**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.9.146
   ```
   =======

* api-change:``glue``: [``botocore``] Update glue client to latest version
* api-change:``sts``: [``botocore``] Update sts client to latest version
   ```
   
  
  
   ### 1.9.145
   ```
   =======

* api-change:``servicecatalog``: [``botocore``] Update servicecatalog client to latest version
* api-change:``eks``: [``botocore``] Update eks client to latest version
* api-change:``sagemaker``: [``botocore``] Update sagemaker client to latest version
* api-change:``kinesisanalytics``: [``botocore``] Update kinesisanalytics client to latest version
* api-change:``kinesisanalyticsv2``: [``botocore``] Update kinesisanalyticsv2 client to latest version
   ```
   
  
  
   ### 1.9.144
   ```
   =======

* api-change:``appsync``: [``botocore``] Update appsync client to latest version
* api-change:``storagegateway``: [``botocore``] Update storagegateway client to latest version
* api-change:``ssm``: [``botocore``] Update ssm client to latest version
* api-change:``alexaforbusiness``: [``botocore``] Update alexaforbusiness client to latest version
   ```
   
  
  
   ### 1.9.143
   ```
   =======

* api-change:``config``: [``botocore``] Update config client to latest version
* api-change:``iam``: [``botocore``] Update iam client to latest version
* api-change:``sts``: [``botocore``] Update sts client to latest version
* api-change:``codepipeline``: [``botocore``] Update codepipeline client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/boto3
  - Changelog: https://pyup.io/changelogs/boto3/
  - Repo: https://github.com/boto/boto3
</details>





### Update [django-cors-headers](https://pypi.org/project/django-cors-headers) from **2.5.3** to **3.0.1**.


<details>
  <summary>Changelog</summary>
  
  
   ### 3.0.0
   ```
   ------------------

* ``CORS_ORIGIN_WHITELIST`` now requires URI schemes, and optionally ports.
  This is part of the CORS specification
  (`Section 3.2 &lt;https://tools.ietf.org/html/rfc6454section-3.2&gt;`_) that was
  not implemented in this library, except from with the
  ``CORS_ORIGIN_REGEX_WHITELIST`` setting. It fixes a security issue where the
  CORS middleware would allow requests between schemes, for example from
  insecure ``http://`` Origins to a secure ``https://`` site.

  You will need to update your whitelist to include schemes, for example from
  this:

  .. code-block:: python

      CORS_ORIGIN_WHITELIST = [&#39;example.com&#39;]

  ...to this:

  .. code-block:: python

      CORS_ORIGIN_WHITELIST = [&#39;https://example.com&#39;]

* Removed the ``CORS_MODEL`` setting, and associated class. It seems very few,
  or no users were using it, since there were no bug reports since its move to
  abstract in version 2.0.0 (2017-01-07). If you *are* using this
  functionality, you can continue by changing your model to not inherit from
  the abstract one, and add a signal handler for ``check_request_enabled`` that
  reads from your model. Note you&#39;ll need to handle the move to include schemes
  for Origins.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/django-cors-headers
  - Changelog: https://pyup.io/changelogs/django-cors-headers/
  - Repo: https://github.com/ottoyiu/django-cors-headers
</details>





### Update [djangorestframework](https://pypi.org/project/djangorestframework) from **3.9.3** to **3.9.4**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/djangorestframework
  - Changelog: https://pyup.io/changelogs/djangorestframework/
  - Homepage: https://www.django-rest-framework.org/
</details>





### Update [factory_boy](https://pypi.org/project/factory_boy) from **2.11.1** to **2.12.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 2.12.0
   ```
   -------------------

*New:*

    - Add support for Python 3.7
    - Add support for Django 2.1
    - Add :attr:`~factory.fuzzy.FuzzyChoice.getter` to :class:`~factory.fuzzy.FuzzyChoice` that mimics
      the behavior of ``getter`` in :class:`~factory.Iterator`
    - Make the ``extra_kwargs`` parameter of :meth:`~factory.faker.Faker.generate` optional
    - Add :class:`~factory.RelatedFactoryList` class for one-to-many support, thanks `Sean Harrington &lt;https://github.com/seanharr11&gt;`_.
    - Make the `locale` argument for :class:`~factory.faker.Faker` keyword-only

*Bugfix:*

    - Allow renamed arguments to be optional, thanks to `Justin Crown &lt;https://github.com/mrname&gt;`_.
    - Fix `django_get_or_create` behavior when using multiple fields with `unique=True`, thanks to `YPCrumble &lt;https://github.com/YPCrumble&gt;`
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/factory-boy
  - Changelog: https://pyup.io/changelogs/factory-boy/
  - Repo: https://github.com/FactoryBoy/factory_boy
</details>





### Update [kinto-http](https://pypi.org/project/kinto-http) from **10.3.0** to **10.4.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 10.4.0
   ```
   ===================

- Add support for Bearer tokens in the CLI utilities.
- Use black for code formatting.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/kinto-http
  - Changelog: https://pyup.io/changelogs/kinto-http/
  - Repo: https://github.com/Kinto/kinto-http.py/
</details>





### Update [pytest](https://pypi.org/project/pytest) from **4.4.1** to **4.5.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 4.4.2
   ```
   =========================

Bug Fixes
---------

- `5089 &lt;https://github.com/pytest-dev/pytest/issues/5089&gt;`_: Fix crash caused by error in ``__repr__`` function with both ``showlocals`` and verbose output enabled.


- `5139 &lt;https://github.com/pytest-dev/pytest/issues/5139&gt;`_: Eliminate core dependency on &#39;terminal&#39; plugin.


- `5229 &lt;https://github.com/pytest-dev/pytest/issues/5229&gt;`_: Require ``pluggy&gt;=0.11.0`` which reverts a dependency to ``importlib-metadata`` added in ``0.10.0``.
  The ``importlib-metadata`` package cannot be imported when installed as an egg and causes issues when relying on ``setup.py`` to install test dependencies.



Improved Documentation
----------------------

- `5171 &lt;https://github.com/pytest-dev/pytest/issues/5171&gt;`_: Doc: ``pytest_ignore_collect``, ``pytest_collect_directory``, ``pytest_collect_file`` and ``pytest_pycollect_makemodule`` hooks&#39;s &#39;path&#39; parameter documented type is now ``py.path.local``


- `5188 &lt;https://github.com/pytest-dev/pytest/issues/5188&gt;`_: Improve help for ``--runxfail`` flag.



Trivial/Internal Changes
------------------------

- `5182 &lt;https://github.com/pytest-dev/pytest/issues/5182&gt;`_: Removed internal and unused ``_pytest.deprecated.MARK_INFO_ATTRIBUTE``.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: https://docs.pytest.org/en/latest/
</details>







Co-authored-by: pyup-bot <github-bot@pyup.io>
Co-authored-by: Michael Cooper <mythmon@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.