PyTracer: RecursionError triggers "Trace function changed" warning

[blueyed]

Given t_coverage_recursion.py:

def f():
    f()


try:
    f()
except Exception as exc:
    print(exc)

print(1)

coverage run -m t_coverage_recursion.py:

maximum recursion depth exceeded in comparison
1
Coverage.py warning: Trace function changed, measurement is likely wrong: None (trace-changed)
__path__ attribute not found on 't_coverage_recursion' while trying to find 't_coverage_recursion.py'

This only happens when using "timid = 1" (PyTracer):

[run]
timid = 1

coverage report then shows "t_coverage_recursion.py 7 3 57%".

Coverage.py, version 4.5.3 with C extension, Python 3.7.2

[blueyed]

Looks more like a Python issue, i.e. it appears to unset any trace function in case of a RecursionError?!

[blueyed]

It is likely due to the RecursionError being raised while the trace function is executed (and it never returns non-None to keep tracing).

[blueyed]

Filed https://bugs.python.org/issue36474.

[blueyed]

Closing as a Python issue for now.

[tolomea]

They don't seem much interested in fixing it
https://bugs.python.org/issue10933

This just cost me an entire morning.
There's a test deep in my test suite that does this 'with self.assertRaises(RecursionError):'
Currently you just get the warning at the end and incomplete coverage info.
And nothing about that warning leads to looking for RecursionErrors
Is there anything coverage could do to make this more obvious?

[nedbat]

@tolomea I'm not sure what coverage could do. We could add a note in the documentation somewhere that the warning about trace functions could be caused by a RecursionError, but I'm not sure what you could do to fix the situation. Did you find a way to fix the problem with your coverage measurement?

I don't see how coverage could defend against the trace function being unset, which is the actual problem.

BTW: why are you using the Python tracer? If this isn't an issue with the C tracer, it would be better all around if you could use it.

[tolomea]

Did you find a way to fix the problem with your coverage measurement?

Once I'd found the actual problem I used a pytest conditional skip to not run that test when there's a tracer installed.
The hard part was getting from a warning at the end of a half hour test run to the root cause
(tests with coverage be slow and I have some other issue with cov and xdist so have to run single threaded as well).

I don't see how coverage could defend against the trace function being unset

If defending is not possible then informing would help.

Detecting it closer to the source would be handy, that might be a thing for pytest-cov an end of test check that makes sure the trace function is still set

Maybe another thing for pytest-cov but the current warning is not the most visible, maybe a hard fail would be better.

Also it would've helped me a lot if the warning mentioned that RecursionError's were known to cause this.

BTW: why are you using the Python tracer?

Because it's the default and I wasn't aware that changing it was a thing.

[nedbat]

Also it would've helped me a lot if the warning mentioned that RecursionError's were known to cause this.

I'm hesitant to put RecursionError into the warning because I'm not sure it is a common reason for the warning. Also, the warning is rare, so I'm not even sure what the likely cause it.

BTW: why are you using the Python tracer?

Because it's the default and I wasn't aware that changing it was a thing.

Actually, the CTracer is the default if it's possible, so something might be going on with your installation. Can you run coverage debug sys and show the output?

tests with coverage be slow

If you can use the C tracer, they will still be slow, but not as slow as with the Python tracer.

[nedbat]

I just did a quick experiment to see if handling event="exception" in the Python tracer would help. Looks like the trace function is unset before the exception is raised, so we wouldn't even hear about a RecursionError in order to fix the problem.

[tolomea]

poking around, turns out we have an old version of coverage, not for any good reason, just because no one has moved it up for ages
I bumped it to the most recent and it uses the CTracer and is noticeably faster than before and resolves the RecursionError thing
so I guess that makes me yet another entry on the long long list of people who should pay more attention to package updates
sorry if I've wasted your time

p.s. I don't even know why we would have it tied to a specific version and not just latest, I'd talk to the person who did that but they are long gone