Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_marker doesn't work for parametrized tests using custom markers #3605

Closed
petr-balogh opened this Issue Jun 20, 2018 · 11 comments

Comments

Projects
None yet
4 participants
@petr-balogh
Copy link

petr-balogh commented Jun 20, 2018

When we are using parametrize regarding to doc and in pytest hook pytest_collection_modifyitems we would like to get marker and it's value we are failing with error:
ValueError: MarkInfo expects Mark instances, got MarkDecorator(mark=Mark(name='custom_mark', args=('custom mark1',), kwargs={})) (<class '_pytest.mark.structures.MarkDecorator'>)

How to reproduce:
I attached full example,, you just need to run_test.sh script.

test_file.py:

import pytest

first_custom_mark = pytest.mark.custom_marker
custom_mark = pytest.mark.custom_mark

@custom_mark("custom mark non parametrized")
def test_custom_mark_non_parametrized():
    print "Hey from test"

@pytest.mark.parametrize(
    "obj_type",
    [
        first_custom_mark("first custom mark")("template"),
        pytest.param( # Think this should be recommended way?
            "disk",
            marks=custom_mark('custom mark1')
        ),
        custom_mark("custom mark2")("vm"),  # Tried also this
    ]
)
def test_custom_mark_parametrized(obj_type):
    print "obj_type is: %s" % obj_type

conftest.py

def pytest_collection_modifyitems(session, config, items):
    for item in items:
        # here we are failing with:
        # INTERNALERROR> ValueError: MarkInfo expects Mark instances, got
        # MarkDecorator(mark=Mark(name='custom_mark', args=('custom mark1',), kwargs={})) (<class '_pytest.mark.structures.MarkDecorator'>)
        custom_mark = item.get_marker('custom_mark')
        print("Custom mark %s" % custom_mark)
@pytestbot

This comment has been minimized.

Copy link

pytestbot commented Jun 20, 2018

GitMate.io thinks possibly related issues are #1697 (Fixture scope doesn't work when it is parametrized), #2133 (Pytest.mark doesn't work on a list of markers as suggested in the documentation), #379 (Can't not apply multiple markers to individual test when using parametrize), #396 (Class scope in parametrized fixtures does not work if first class to be collected has only one method using that fixture), and #1601 (Overriding autouse fixture with a parametrized fixture does not work).

@petr-balogh

This comment has been minimized.

Copy link
Author

petr-balogh commented Jun 20, 2018

This bug was introduced in version 3.6.2, I've just tested with 3.6.1 and here it's working well.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jun 20, 2018

i introduced it fixing another one, ill investigate tommorow

@petr-balogh

This comment has been minimized.

Copy link
Author

petr-balogh commented Jun 20, 2018

Thanks

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jun 20, 2018

ps - i believe i missed to add a unpacking while making the internals more strict, so the code is already broken from before, now it just actually shows instead of silently hiding it

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jun 20, 2018

i transformed your post into a fialing test i believe i will find a nice solution tommorow

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 20, 2018

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 21, 2018

@petr-balogh

This comment has been minimized.

Copy link
Author

petr-balogh commented Jun 21, 2018

Thanks Ronny for the effort. Please, just a question about usage of multiple markers cause we want also use it. Is there any known issue at the moment? Or will it also work with multiple marks? Like:

pytest.param( # Think this should be recommended way?
            "disk",
            marks=(custom_mark('custom mark1'), custo_mark2('custom mark2'))
        ),
# Or like this?:
first_custom_mark("first custom mark")(second_custom_mark('second custom mark'))("template"),
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jun 21, 2018

@petr-balogh the ´pytest.param´ version is supposed to be the correct one, however the mark shift towards correct internals has some fallout due to a incomplete test-suite

#3606 fixes the issue you see

@Midnighter

This comment has been minimized.

Copy link

Midnighter commented Jun 22, 2018

Hitting the same issue in combination with pytest-raises. Here's is an example of the parametrization:

@pytest.mark.parametrize("model", [
    pytest.param("missing_energy_partner",
                 marks=pytest.mark.raises(exception=ZeroDivisionError)),
    pytest.param("infeasible",
                 marks=pytest.mark.raises(exception=Infeasible))
], indirect=["model"])
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jun 22, 2018

@Midnighter the fix was not yet released

@Midnighter

This comment has been minimized.

Copy link

Midnighter commented Jun 22, 2018

Sorry, meant to write I look forward to the fix 😃

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jun 26, 2018

ashutoshb added a commit to zonnepanelendelen/pytest-raises that referenced this issue Jul 2, 2018

jezdez added a commit to mozilla/telemetry-analysis-service that referenced this issue Aug 14, 2018

Update pytest to 3.7.1 (#1247)
This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.2** to **3.7.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.1
   ```
   =========================

Bug Fixes
---------

- `3473 &lt;https://github.com/pytest-dev/pytest/issues/3473&gt;`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn&#39;t understand (e.g. strings, nested dicts, etc.).


- `3712 &lt;https://github.com/pytest-dev/pytest/issues/3712&gt;`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``.

- `3742 &lt;https://github.com/pytest-dev/pytest/issues/3742&gt;`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute &#39;_collectfile&#39;``.

- `3745 &lt;https://github.com/pytest-dev/pytest/issues/3745&gt;`_: Display the absolute path if ``cache_dir`` is not relative to the ``rootdir`` instead of failing.


- `3747 &lt;https://github.com/pytest-dev/pytest/issues/3747&gt;`_: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly.


- `3748 &lt;https://github.com/pytest-dev/pytest/issues/3748&gt;`_: Fix infinite recursion in ``pytest.approx`` with arrays in ``numpy&lt;1.13``.


- `3757 &lt;https://github.com/pytest-dev/pytest/issues/3757&gt;`_: Pin pathlib2 to ``&gt;=2.2.0`` as we require ``__fspath__`` support.


- `3763 &lt;https://github.com/pytest-dev/pytest/issues/3763&gt;`_: Fix ``TypeError`` when the assertion message is ``bytes`` in python 3.
   ```
   
  
  
   ### 3.7.0
   ```
   =========================

Deprecations and Removals
-------------------------

- `2639 &lt;https://github.com/pytest-dev/pytest/issues/2639&gt;`_: ``pytest_namespace`` has been deprecated.

  See the documentation for ``pytest_namespace`` hook for suggestions on how to deal
  with this in plugins which use this functionality.


- `3661 &lt;https://github.com/pytest-dev/pytest/issues/3661&gt;`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``.

  This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model.



Features
--------

- `2283 &lt;https://github.com/pytest-dev/pytest/issues/2283&gt;`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly.


- `3576 &lt;https://github.com/pytest-dev/pytest/issues/3576&gt;`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first.


- `3579 &lt;https://github.com/pytest-dev/pytest/issues/3579&gt;`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler.


- `3610 &lt;https://github.com/pytest-dev/pytest/issues/3610&gt;`_: New ``--trace`` option to enter the debugger at the start of a test.


- `3623 &lt;https://github.com/pytest-dev/pytest/issues/3623&gt;`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project.



Bug Fixes
---------

- `2220 &lt;https://github.com/pytest-dev/pytest/issues/2220&gt;`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test.


- `3695 &lt;https://github.com/pytest-dev/pytest/issues/3695&gt;`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results.
  Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()``


- `980 &lt;https://github.com/pytest-dev/pytest/issues/980&gt;`_: Fix truncated locals output in verbose mode.



Improved Documentation
----------------------

- `3295 &lt;https://github.com/pytest-dev/pytest/issues/3295&gt;`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed.



Trivial/Internal Changes
------------------------

- `3519 &lt;https://github.com/pytest-dev/pytest/issues/3519&gt;`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists.
   ```
   
  
  
   ### 3.6.4
   ```
   =========================

Bug Fixes
---------

- Invoke pytest using ``-mpytest`` so ``sys.path`` does not get polluted by packages installed in ``site-packages``. (`742 &lt;https://github.com/pytest-dev/pytest/issues/742&gt;`_)


Improved Documentation
----------------------

- Use ``smtp_connection`` instead of ``smtp`` in fixtures documentation to avoid possible confusion. (`3592 &lt;https://github.com/pytest-dev/pytest/issues/3592&gt;`_)


Trivial/Internal Changes
------------------------

- Remove obsolete ``__future__`` imports. (`2319 &lt;https://github.com/pytest-dev/pytest/issues/2319&gt;`_)

- Add CITATION to provide information on how to formally cite pytest. (`3402 &lt;https://github.com/pytest-dev/pytest/issues/3402&gt;`_)

- Replace broken type annotations with type comments. (`3635 &lt;https://github.com/pytest-dev/pytest/issues/3635&gt;`_)

- Pin ``pluggy`` to ``&lt;0.8``. (`3727 &lt;https://github.com/pytest-dev/pytest/issues/3727&gt;`_)
   ```
   
  
  
   ### 3.6.3
   ```
   =========================

Bug Fixes
---------

- Fix ``ImportWarning`` triggered by explicit relative imports in
  assertion-rewritten package modules. (`3061
  &lt;https://github.com/pytest-dev/pytest/issues/3061&gt;`_)

- Fix error in ``pytest.approx`` when dealing with 0-dimension numpy
  arrays. (`3593 &lt;https://github.com/pytest-dev/pytest/issues/3593&gt;`_)

- No longer raise ``ValueError`` when using the ``get_marker`` API. (`3605
  &lt;https://github.com/pytest-dev/pytest/issues/3605&gt;`_)

- Fix problem where log messages with non-ascii characters would not
  appear in the output log file.
  (`3630 &lt;https://github.com/pytest-dev/pytest/issues/3630&gt;`_)

- No longer raise ``AttributeError`` when legacy marks can&#39;t be stored in
  functions. (`3631 &lt;https://github.com/pytest-dev/pytest/issues/3631&gt;`_)


Improved Documentation
----------------------

- The description above the example for ``pytest.mark.skipif`` now better
  matches the code. (`3611
  &lt;https://github.com/pytest-dev/pytest/issues/3611&gt;`_)


Trivial/Internal Changes
------------------------

- Internal refactoring: removed unused ``CallSpec2tox ._globalid_args``
  attribute and ``metafunc`` parameter from ``CallSpec2.copy()``. (`3598
  &lt;https://github.com/pytest-dev/pytest/issues/3598&gt;`_)

- Silence usage of ``reduce`` warning in Python 2 (`3609
  &lt;https://github.com/pytest-dev/pytest/issues/3609&gt;`_)

- Fix usage of ``attr.ib`` deprecated ``convert`` parameter. (`3653
  &lt;https://github.com/pytest-dev/pytest/issues/3653&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Repo: https://github.com/pytest-dev/pytest/issues
  - Homepage: http://pytest.org
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.