PyEval_SetTrace used in way that breaks sys.gettrace()

[nedbat]

Originally reported by Devin Jeanpierre (Bitbucket: devin.jeanpierre, GitHub: Unknown)

---

tl;dr: //sys.settrace(sys.gettrace())// breaks coverage by causing the trace function to raise an exception, and this is avoidable.

The (C extension) Tracer.c installs a trace function using PyEval_SetTrace, as opposed to the usual sys.settrace. This is problematic because unless PyEval_SetTrace is used in a very particular way (in particular, the way sys.settrace uses it), it will break sys.gettrace.

PyEval_SetTrace(func, self) is a C function that takes "func" and "self". "self" is passed as the first argument to func, which generally does something with it. With sys.settrace for example, self is a callable PyObject* that gets called by func. sys.gettrace is designed to work with sys.settrace, so sys.gettrace returns self.

coverage, however, takes "self" literally and passes the instance of Tracer, so that the tracer method involved with actually tracing execution can maintain state. So when sys.gettrace is called, it returns the instance of Tracer, which is not safe to pass back in to sys.settrace.

An executable example is attached, which fails when run under coverage, but succeeds when run normally.

There are two ways to fix this: either PyEval_SetTrace can be eschewed completely, in favor of sys.settrace, or else Tracer can be made callable appropriately. I'm not really familiar with the codebase of coverage (nor am I familiar with writing C extensions in general), but I think that making Tracer callable would be the simplest approach.

I'll probably work on a patch later, but first I want to investigate a bit more why doctest doesn't appear (at least with some minimal tests that I think I did a few days ago) to be bitten by this behavior: doctest gets the trace function with sys.gettrace(), and restores it later using sys.settrace(). Also, as I said before, I'm not incredibly familiar with the codebase or with writing C extensions.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/123
• This issue had attachments: coverage_tracer.py, tracer.c.diff. See the original issue for details.

[nedbat]

Original comment by Devin Jeanpierre (Bitbucket: devin.jeanpierre, GitHub: Unknown)

---

Alright, so I just looked into doctest. It turns out that only the most recent version of doctest does this, not the version included in 3.2. I'm not sure if coverage //ever// worked with doctest, though, so it's not like I can say "doctest will no longer work with coverage if you don't act soon!"

[nedbat]

Fixed in <<changeset ebca9c8e46a6 (bb)>>. Thanks!