Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow module level skip #2808

Merged

Conversation

Projects
None yet
5 participants
@georgeyk
Copy link
Contributor

commented Oct 1, 2017

Work-in-progress fix for #2805
I need to update the PR with all contributing guideline recommendations, but a initial feedback would be good.

--//--
Thanks for submitting a PR, your contribution is really appreciated!

Here's a quick checklist that should be present in PRs:

  • Add a new news fragment into the changelog folder
    • name it $issue_id.$type for example (588.bug)
    • if you don't have an issue_id change it to the pr id after creating the pr
    • ensure type is one of removal, feature, bugfix, vendor, doc or trivial
    • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."
  • Target: for bugfix, vendor, doc or trivial fixes, target master; for removals or features target features;
  • Make sure to include reasonable tests for your change if necessary

Unless your change is a trivial or a documentation fix (e.g., a typo or reword of a small section) please:

  • Add yourself to AUTHORS;
@coveralls

This comment has been minimized.

Copy link

commented Oct 1, 2017

Coverage Status

Coverage increased (+0.003%) to 92.235% when pulling 0fb31f8 on georgeyk:allow-module-level-skip into 6690b8a on pytest-dev:master.

@nicoddemus
Copy link
Member

left a comment

Great work!

Aside from my comments, we also need a changelog file.

""" skip an executing test with the given message. Note: it's usually
better to use the pytest.mark.skipif marker to declare a test to be
skipped under certain conditions like mismatching platforms or
dependencies. See the pytest_skipping plugin for details.
:kwarg bool allow_module_level: allows this function to be called at
module level, skipping the rest of the module. Default to False.

This comment has been minimized.

Copy link
@nicoddemus

nicoddemus Oct 1, 2017

Member

I think this needs to be indented otherwise tox -e docs fails

allow_module_level = kwargs.pop('allow_module_level', False)
if kwargs:
keys = [k for k in kwargs.keys()]
raise TypeError('unexpected keyworkd arguments: {}'.format(keys))

This comment has been minimized.

Copy link
@nicoddemus

nicoddemus Oct 1, 2017

Member

Needs to use {0} to work on Python 2.6:

raise TypeError('unexpected keyworkd arguments: {0}'.format(keys))

This comment has been minimized.

Copy link
@The-Compiler

The-Compiler Oct 2, 2017

Member

keyworkd -> keyword

@The-Compiler

This comment has been minimized.

Copy link
Member

commented Oct 2, 2017

This probably should go to the features branch.



.. code-block:: python

This comment has been minimized.

Copy link
@RonnyPfannschmidt

RonnyPfannschmidt Oct 2, 2017

Member

i do wonder if there is a better example for this (thats not looking like a skipif marker or a importorskip would already solve

This comment has been minimized.

Copy link
@georgeyk

georgeyk Oct 2, 2017

Author Contributor

Maybe using the use case from previous discussion ? Something like:

if not pytest.config.getoption('--custom-flag'):
    pytest.skip('--custom-flag is missing, skipping tests', allow_module_level=True)
@georgeyk

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2017

Should I close this and re-open targeting the features branch ?

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 2, 2017

You can rebase on the features branch, push force to this branch and then edit the target branch of the PR here on GH to features.

@georgeyk georgeyk force-pushed the georgeyk:allow-module-level-skip branch from 0fb31f8 to 9824499 Oct 3, 2017

@georgeyk georgeyk changed the base branch from master to features Oct 3, 2017

@georgeyk

This comment has been minimized.

Copy link
Contributor Author

commented Oct 3, 2017

Is this event.when supposed to exist in CollectReport instance ? https://github.com/pytest-dev/pytest/blob/features/_pytest/skipping.py#L352

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 3, 2017

Hmm folded_skips is relatively new I think. @RonnyPfannschmidt do you have a suggestion on how to proceed here?

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

nope

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

actually i do, whole module skips should be ignored, collectrports have no when

so at collect time we should never fold skips

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

so at collect time we should never fold skips

I see, thanks. Could you tell @georgeyk what should be changed then? I'm not sure what should be done with collectreports.

And the error is happening at the end of the session, when we are about to show the terminal summary, not during collection:

Traceback (most recent call last):
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/pytester.py", line 744, in runpytest_inprocess
    reprec = self.inline_run(*args, **kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/pytester.py", line 717, in inline_run
    ret = pytest.main(list(args), plugins=plugins)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/config.py", line 59, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 131, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 125, in wrap_session
    exitstatus=session.exitstatus)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 302, in __call__
    return outcome.get_result()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 279, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 265, in __init__
    self.result = func()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 300, in <lambda>
    outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 250, in _wrapped_call
    wrap_controller.send(call_outcome)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/terminal.py", line 427, in pytest_sessionfinish
    exitstatus=exitstatus)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 302, in __call__
    return outcome.get_result()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 279, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 265, in __init__
    self.result = func()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 300, in <lambda>
    outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 291, in pytest_terminal_summary
    show_skipped(terminalreporter, lines)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 370, in show_skipped
    fskips = folded_skips(skipped)
  File "/home/travis/build/pytest-dev/pytest/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 352, in folded_skips
    if event.when == 'setup' and 'skip' in keywords and 'pytestmark' not in keywords:
AttributeError: 'CollectReport' object has no attribute 'when'
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

correct - a skip in a collect-report is added to the skip stats, and such a stat shouldn't be folded in the display

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

one solution could be to add a when property to collect reports

another solution could be to just ignore them

its not clear what has the best effect

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

correct - a skip in a collect-report is added to the skip stats, and such a stat shouldn't be folded in the display

That was my first thought too, but after thinking some more I think it might make sense in some cases. For example, if (say) 3 whole modules are skipped for the same reason, it makes sense to fold it.

In light of that it might make sense to just make a special case for it in folded_skips.

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

Something like (untested):

when = getattr(event, 'when', None)  # might be a collection report
if when == 'setup' and 'skip' in keywords and 'pytestmark' not in keywords:
@coveralls

This comment has been minimized.

Copy link

commented Oct 4, 2017

Coverage Status

Coverage increased (+0.006%) to 92.506% when pulling 03ce0ad on georgeyk:allow-module-level-skip into d132c50 on pytest-dev:features.

@nicoddemus
Copy link
Member

left a comment

Thanks @georgeyk, well done!

@georgeyk

This comment has been minimized.

Copy link
Contributor Author

commented Oct 4, 2017

@nicoddemus Thank you all to support this great project. I hope to do more in the future!

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 4, 2017

I hope to do more in the future!

We really appreciate it, thanks!

@coveralls

This comment has been minimized.

Copy link

commented Oct 5, 2017

Coverage Status

Coverage increased (+0.006%) to 92.506% when pulling 0668a6c on georgeyk:allow-module-level-skip into d132c50 on pytest-dev:features.

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Oct 11, 2017

@RonnyPfannschmidt you think we can merge this?

@RonnyPfannschmidt RonnyPfannschmidt merged commit df37cdf into pytest-dev:features Oct 12, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.