Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compat with pytest-capturelog for note()-style logging capture? #356

Closed
Daenyth opened this issue Aug 24, 2016 · 2 comments
Closed

Add compat with pytest-capturelog for note()-style logging capture? #356

Daenyth opened this issue Aug 24, 2016 · 2 comments
Labels
enhancement it's not broken, but we want it to be better

Comments

@Daenyth
Copy link

Daenyth commented Aug 24, 2016

Using hypothesis with code that has logging statements makes it very difficult to use those log calls to debug.

Having the ability to have logging output captured in the style of note() would be extremely useful. The pytest-capturelog plugin collects the logging output into the test failure message. It would be really nice to have some kind of cross-compatibility with them so that it can group captured logs by example rather than at the test-function level

@Zac-HD Zac-HD added the enhancement it's not broken, but we want it to be better label Jul 19, 2017
@Zac-HD
Copy link
Member

Zac-HD commented Jan 14, 2018

As of pytest 3.3, log capturing is built in - meaning that logs are probably being captured (for all examples, eek) more often, and there's a standard thing that we can be compatible with in the pytest plugin (in hypothesis.extra)

See https://docs.pytest.org/en/latest/logging.html

@Zac-HD
Copy link
Member

Zac-HD commented Aug 18, 2018

This may be as simple as disabling logging for all but the final example of a test, which can be done in hypothesis.control.BuildContext:

if not self.is_final:  # in __enter__
    logging.disable()
...
if not self.is_final:  # in __exit__
    logging.disable(logging.NOTSET)

and adding a test in tests/pytest/ that a test with a log statement only logs it once.

garyd203 added a commit to garyd203/hypothesis that referenced this issue Aug 28, 2018
…isWorks#356

Caveats:
* If the subject under test uses really high custom log levels (higher
  than CRITICAL), those log messages will be retained.
* If the user has defined multiple log managers whilst running tests,
  then log messages may be retained.
* Our implementation relies on internal details of the logging module in
  order to reset logging state, which may break in the future.
garyd203 added a commit to garyd203/hypothesis that referenced this issue Aug 28, 2018
…isWorks#356

Caveats:
* If the subject under test uses really high custom log levels (higher
  than CRITICAL), those log messages will be retained.
* If the user has defined multiple log managers whilst running tests,
  then log messages may be retained.
* Our implementation relies on internal details of the logging module in
  order to reset logging state, which may break in the future.
garyd203 added a commit to garyd203/hypothesis that referenced this issue Aug 28, 2018
…isWorks#356

Caveats:
* If the subject under test uses really high custom log levels (higher
  than CRITICAL), those log messages will be retained.
* If the user has defined multiple log managers whilst running tests,
  then log messages may be retained.
* Our implementation relies on internal details of the logging module in
  order to reset logging state, which may break in the future.
garyd203 added a commit to garyd203/hypothesis that referenced this issue Aug 28, 2018
…isWorks#356

Caveats:
* If the subject under test uses really high custom log levels (higher
  than CRITICAL), those log messages will be retained.
* If the user has defined multiple log managers whilst running tests,
  then log messages may be retained.
* Our implementation relies on internal details of the logging module in
  order to reset logging state, which may break in the future.
Zac-HD added a commit to Zac-HD/hypothesis that referenced this issue Aug 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement it's not broken, but we want it to be better
Projects
None yet
Development

No branches or pull requests

2 participants