New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add custom code to tp_traverse and tp_clear? #2737

Open
mdavidsaver opened this Issue Nov 28, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@mdavidsaver

mdavidsaver commented Nov 28, 2018

I've been looking at Cython with an eye toward replacing some hand written
cpython extension code binding a C++ library. This library is for a network
client and server, and includes a number of callbacks from C++ into python code.
It also makes heavy use of std::shared_ptr.

I find myself wanting the ability to add custom code to tp_traverse and tp_clear
to find additional python objects referenced indirectly through c++ objects.

I don't think this is presently possible?

A strawman example of the situation is as follows.
The reference cycles to be broken would include.

Holder.iface -> PyImpl.held -> Held -> ... -> Holder

cdef extern from "foo.h":
    cdef cppclass Interface:
        ...
        void callback() # pure virtual
    cdef cppclass PyImpl(Interface):
        ...
        void callback() # calls held.callback()
        object held

    ctypedef Interface* pInterface

cdef class Held:
    def callback(self, arg):
        pass

cdef class Holder:
    cdef shared_ptr[Interface] iface

    # I would like to be able to do something like this:
    def tp_traverse(self, visit):
        cdef PyImpl* impl
        if self.iface:
            impl = dynamic_cast[pInterface](self.iface.get())
            if impl:
                visit(impl.held)
@scoder

This comment has been minimized.

Contributor

scoder commented Nov 28, 2018

Could probably be added as two new special methods that get called by tp_traverse and tp_clear, so I'll leave this open as a feature request.

You can already do this by manually casting the type to a PyTypeObject and setting its slot methods. That way, you can store away the old slot value and insert your own one that calls the original one.

@mdavidsaver

This comment has been minimized.

mdavidsaver commented Dec 6, 2018

As advertised, it is possible to replace tp_traverse. A working example: https://github.com/mdavidsaver/cython-c--demo

I found that the definition of cpython.traverseproc changed at some point to include "except -1". In exploring ways to handle with this sort of API change I was surprised to find that eg. IF CYTHON_HEX_VERSION>=0x001D01F0 isn't possible.

@scoder

This comment has been minimized.

Contributor

scoder commented Dec 7, 2018

The example looks good.

IF CYTHON_HEX_VERSION>=0x001D01F0 isn't possible

Ah, yes, that makes sense. There aren't many real use cases for translation time environment tests, but this one is reasonable (although your specific use case is clearly better served by a cast). PR welcome that sets CYTHON_HEX_VERSION to what we now write into the C file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment