New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indicate non-pytest warnings in the UI #1262

Closed
DRMacIver opened this Issue Dec 16, 2015 · 6 comments

Comments

Projects
None yet
6 participants
@DRMacIver

DRMacIver commented Dec 16, 2015

My library, Hypothesis potentially emits a number of warnings indicating that a test is in some way doing the wrong thing.

Currently these will only show up if the code is being run with --capture=no, or if a test actually fails. It would be useful if the user were in some way notified that their code is emitting warnings even within a passing test, because warnings might indicate that the test is passing erroneously.

Things I would ideally like to see:

  1. A visual indicator somewhere that there have been warnings raised during tests. e.g. in the report where pytest warnings are indicated an 'N other warnings' label
  2. Some easy way of running the tests in a way that indicates the presence of warnings. e.g. emitting warnings in verbose mode would be nice, but maybe too noisy.

I leave the exact UI up to your judgement, I just want the warnings my library is emitting to not get swallowed. :-)

@DRMacIver DRMacIver changed the title from Indicate warnings in the UI to Indicate non-pytest warnings in the UI Dec 16, 2015

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Dec 16, 2015

Member

I agree test frameworks should make python warnings apparent in some way.

I use this for my own testsuite:

@pytest.yield_fixture(autouse=True)
def fail_tests_on_warnings():
    warnings.simplefilter('error')
    # https://github.com/pytest-dev/pytest-bdd/issues/153
    warnings.filterwarnings('ignore', message=r'inspect.getargspec\(\) is '
                            r'deprecated, use inspect.signature\(\) instead',
                            category=DeprecationWarning)
    yield
    warnings.resetwarnings()

Of course that'd be a bit harsh as a default, but maybe something like showing the warnings in the summary (and coloring it yellow), and printing the warnings if -v was used would help.

Member

The-Compiler commented Dec 16, 2015

I agree test frameworks should make python warnings apparent in some way.

I use this for my own testsuite:

@pytest.yield_fixture(autouse=True)
def fail_tests_on_warnings():
    warnings.simplefilter('error')
    # https://github.com/pytest-dev/pytest-bdd/issues/153
    warnings.filterwarnings('ignore', message=r'inspect.getargspec\(\) is '
                            r'deprecated, use inspect.signature\(\) instead',
                            category=DeprecationWarning)
    yield
    warnings.resetwarnings()

Of course that'd be a bit harsh as a default, but maybe something like showing the warnings in the summary (and coloring it yellow), and printing the warnings if -v was used would help.

@nicoddemus

This comment has been minimized.

Show comment
Hide comment
@nicoddemus

nicoddemus Dec 16, 2015

Member

We would also probably need a way to silence some warnings, probably using some configuration variable.

Member

nicoddemus commented Dec 16, 2015

We would also probably need a way to silence some warnings, probably using some configuration variable.

@davehunt

This comment has been minimized.

Show comment
Hide comment
@davehunt

davehunt Dec 17, 2015

Contributor

@bobsilverberg I think this relates to your recent question about showing warnings during a test run.

Contributor

davehunt commented Dec 17, 2015

@bobsilverberg I think this relates to your recent question about showing warnings during a test run.

@bobsilverberg

This comment has been minimized.

Show comment
Hide comment
@bobsilverberg

bobsilverberg Dec 17, 2015

@davehunt Yes this seems to. @The-Compiler I tried your code but I am still not seeing warnings anywhere. Could this relate to the fact that I am issuing the warning from inside a fixture? It seems that no matter what I do I cannot get the warning to appear anywhere.

bobsilverberg commented Dec 17, 2015

@davehunt Yes this seems to. @The-Compiler I tried your code but I am still not seeing warnings anywhere. Could this relate to the fact that I am issuing the warning from inside a fixture? It seems that no matter what I do I cannot get the warning to appear anywhere.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Dec 17, 2015

Member

@bobsilverberg Hm, that's a good point, this can run after other fixtures. Can you try adding scope='module' to the yield_fixture decorator?

Probably it'd be cleaner to do this in a pytest_configure hook or so.

Member

The-Compiler commented Dec 17, 2015

@bobsilverberg Hm, that's a good point, this can run after other fixtures. Can you try adding scope='module' to the yield_fixture decorator?

Probably it'd be cleaner to do this in a pytest_configure hook or so.

@RonnyPfannschmidt

This comment has been minimized.

Show comment
Hide comment
@RonnyPfannschmidt

RonnyPfannschmidt Jul 19, 2016

Member

@fschulze created https://pypi.python.org/pypi/pytest-warnings

it will move to core afer some more incubation

closing this issue as solved by a plugin, please reopen in case of disagreement

Member

RonnyPfannschmidt commented Jul 19, 2016

@fschulze created https://pypi.python.org/pypi/pytest-warnings

it will move to core afer some more incubation

closing this issue as solved by a plugin, please reopen in case of disagreement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment