Indicate non-pytest warnings in the UI #1262

Closed
DRMacIver opened this Issue Dec 16, 2015 · 6 comments

Projects

None yet

6 participants

@DRMacIver

My library, Hypothesis potentially emits a number of warnings indicating that a test is in some way doing the wrong thing.

Currently these will only show up if the code is being run with --capture=no, or if a test actually fails. It would be useful if the user were in some way notified that their code is emitting warnings even within a passing test, because warnings might indicate that the test is passing erroneously.

Things I would ideally like to see:

  1. A visual indicator somewhere that there have been warnings raised during tests. e.g. in the report where pytest warnings are indicated an 'N other warnings' label
  2. Some easy way of running the tests in a way that indicates the presence of warnings. e.g. emitting warnings in verbose mode would be nice, but maybe too noisy.

I leave the exact UI up to your judgement, I just want the warnings my library is emitting to not get swallowed. :-)

@DRMacIver DRMacIver changed the title from Indicate warnings in the UI to Indicate non-pytest warnings in the UI Dec 16, 2015
@The-Compiler
Member

I agree test frameworks should make python warnings apparent in some way.

I use this for my own testsuite:

@pytest.yield_fixture(autouse=True)
def fail_tests_on_warnings():
    warnings.simplefilter('error')
    # https://github.com/pytest-dev/pytest-bdd/issues/153
    warnings.filterwarnings('ignore', message=r'inspect.getargspec\(\) is '
                            r'deprecated, use inspect.signature\(\) instead',
                            category=DeprecationWarning)
    yield
    warnings.resetwarnings()

Of course that'd be a bit harsh as a default, but maybe something like showing the warnings in the summary (and coloring it yellow), and printing the warnings if -v was used would help.

@nicoddemus
Member

We would also probably need a way to silence some warnings, probably using some configuration variable.

@davehunt
Contributor

@bobsilverberg I think this relates to your recent question about showing warnings during a test run.

@bobsilverberg

@davehunt Yes this seems to. @The-Compiler I tried your code but I am still not seeing warnings anywhere. Could this relate to the fact that I am issuing the warning from inside a fixture? It seems that no matter what I do I cannot get the warning to appear anywhere.

@The-Compiler
Member

@bobsilverberg Hm, that's a good point, this can run after other fixtures. Can you try adding scope='module' to the yield_fixture decorator?

Probably it'd be cleaner to do this in a pytest_configure hook or so.

@RonnyPfannschmidt
Contributor

@fschulze created https://pypi.python.org/pypi/pytest-warnings

it will move to core afer some more incubation

closing this issue as solved by a plugin, please reopen in case of disagreement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment