Address #3821 by adding a flag to turn off coverage reporting for observability #3826
Conversation
- Add OBSERVABILITY_COLLECT_COVERAGE exported from internal/observability - Set that variable based on "HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER"
| @@ -19,6 +19,7 @@ | |||
| from hypothesis.internal.conjecture.data import ConjectureData, Status | |||
|
|
|||
| TESTCASE_CALLBACKS: List[Callable[[dict], None]] = [] | |||
| OBSERVABILITY_COLLECT_COVERAGE = True | |||
There was a problem hiding this comment.
Let's set this as OBSERVABILITY_COLLECT_COVERAGE = "HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER" not in os.environ, put it just above if "HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY" in os.environ:, and change that condition to if "HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY" in os.environ or OBSERVABILITY_COLLECT_COVERAGE is False:.
That way setting only HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER will enable observability in nocover mode.
The last thing I'd do is emit a warning in nocover mode on Python >= 3.12, because the sys.monitoring system is expected to have much lower overhead and users might not know that. Maybe worth a sentence in the docs too?
There was a problem hiding this comment.
OK, I think I did all of this correctly!
Not sure if this is correct yet, but I'll make a PR to start a discussion. I added a new command line flag that allows users to turn coverage collection off when doing observability logging.