Accessing `cython_runtime` object when its reference count is 0

[cshtarkov]

The following piece of code ends up accessing the cython_runtime module when its reference count is 0 and its destructor has already been called. This is undefined behaviour and can cause the interpreter to crash.

import sys

def generator():
    yield
    yield

g = generator()
next(g)

del sys.modules["cython_runtime"]
del g

Even if sys.modules["cython_runtime"] is not deleted explicitly, the bug can still occur during shutdown depending on whether the reference count for g or cython_runtime goes down to 0 first.

The problem is caused by __Pyx_CLineForTraceback being called when stopping a started generator and trying to look up the cython_runtime.cline_in_traceback variable introduced in 0.26. See Exceptions.c#L665 and Exceptions.c#L673.

It only has a borrowed reference to cython_runtime from ModuleNode.py#L2839 obtained during initialisation which is not reference counted and thus there is no guarantee that the object has not already been destroyed.

I believe adding a call to code.put_incref() after obtaining the reference would be sufficient to resolve the issue, but there may be better ways of going about this.

Note that this particular example is not guaranteed to crash. However I confirmed the illegal access by
breaking at the lines mentioned above in a debugger, and seeing that __pyx_cython_runtime->ob_refcnt is 0 at the point of access.

[scoder]

Right, PyImport_AddModule() returns a borrowed reference that we should better not keep un-owned.

[scoder]

Thanks for the excellent analysis.