def square(double x): return x*x import scipy.integrate integrator = scipy.integrate.ode(square) ... integrator.integrate(10)
Now, this is efficient compiled code in Cython, and the (Fortran) code in SciPy calling the callback is also compiled, but because of the Python in the middle each callback pays a major penalty.
- Extend the def syntax so that it takes a return value
- Define a protocol for getting access to the underlying cdef function on a Cython def function.
- Allow using this from Cython by allowing casting of (callable) objects to C function pointers. 1. For objects exposing the protocol of 1., this is simple 2. For other objects we need to use libffi and create a thunk
After this, one can also add another optimizing branch for all ``PyObject_Call``s.
def double square(double x): return x*x cdef double (*fncptr)(double) fncptr = <double (*)(double)>square # (A) Directly access underlying cdef function funcptr = <double (*)(double)>somepythonmodule.somefunc # (B) use libffi and create thunk somemodule.otherfunc(2) # (C) Has two branches: If it has the native interface in (2) above, use that, # otherwise do PyObject_Call
print square._native_signature # e.g., "d>d" print square._native_callptr # A Python capsule
Let's leave the string format unspecified until a time comes such that somebody is ready to implement this.
Cython's own C function sub-type should contain this information, and PyType_Check can then be used for a fast check, useful for case (C) in the example. A goal here is to still achieve somewhat high speed when not using a pxd file for early binding. For this to work well it is therefore necesarry that Cython's own C function sub-type is canonicalized across Cython modules (one can coordinate, e.g., through some _cythoninternal entry in sys.modules).
The case of casting a Cython def function, supporting the above protocol, is obvious.
In the other case we need to create a thunk using libffi. Since libffi ships with Python on most platforms this can probably be made pretty portable. At any rate this must be made an optional feature as we can't support it everywhere.
Thunking has a lot of potential for optimizing other parts of Cython, so efforts spent here could be used for more things.
Like with bytes->``char*``, we are going to assume it is the users responsibility to hold the Python object for the life of the pointer. We must use a WeakKeyDictionary in order to reuse existing thunks (think repeatedly casting a function in a loaded module), and also hold a weak reference that can destruct the thunk when, e.g., a closure object is freed.
cdef float x = someobj(1.0, 3)
We may, e.g., have three cases:
- someobj is a Cython function (PyType_Check)
- ...and the format string matches "di>f". Then just dispatch directly (do a C cast and call).
- ...otherwise, build the stack using libffi? (Very optional, and potentially a lot faster than boxing/unboxing)
- Fall back to PyObject_Call