[QUESTION] How do I stop pybind11 from automatically deallocating an object created on the python side?

[dograt]

Copying my question from here.

So, I know that pybind lets you set a return value policy for methods that you wrap up. However, that doesn't seem to be working for me when I try to use this policy on a constructor. I have a class to wrap my C++ type that looks like this:

class PyComponent{
public:


    static Component* Create(ComponentType type) {
        Component* c = new Component(type);
        // Irrelevant stuff removed here
        return c;
    }

    /// @brief Wrap a behavior for Python
    static void PyInitialize(py::module_& m);

};

void PyComponent::PyInitialize(py::module_ & m)
{
    py::class_<Component>(m, "Component")
        .def(py::init<>(&PyComponent::Create), py::return_value_policy::reference)
        ;
}

However, this does NOT stop my Component type from getting deallocated from the Python side if I call Component() and the created object goes out of scope. Any suggestions?

[YannickJadoul]

You're probably right, yes. First of all, you're creating an implementation for __init__ and since __init__/a constructor in Python doesn't have a return value, return_value_policy doesn't really apply to anything. (The thing you're returning from that lambda is not returned to Python; it's just used by the init auxiliary construction, such that you don't have to call an in-place C++ constructor.)

Moreover, as far as I know, pybind11 also does this since this is a constructor, so you're creating a new object, so it's very reasonable to assume you're creating a new object. But we should probably add some kind of a compile-time error, at least.

Finally, I don't see the problem with your code, though. PyComponent::Create should work perfectly, since it acts as factor constructor.

[dograt]

Hi Yannick, thanks for the reply. That makes sense, and I figured that setting the return policy wouldn't do anything for exactly those reasons you mentioned.

My problem is that I am managing the lifetime of my Component class from the C++ side, so if it is deallocated from Python, I end up with a crash since my C++ code is trying to access the now-deleted Component. Can you think of anything I could do to give me the desired behavior? I could always use a factory function like addComponent or something instead of using a Component constructor, but it would be pretty messy and won't scale well for my application.

[bstaletic]

Does using std::unique_ptr<Component, py::nodelete> as the holder type work for you? The destructor of the python wrapper object will still run, but the destructor of Component won't.

[YannickJadoul]

Hi @dantamont, I reckoned so, or you wouldn't be complaining about pybind11 deleting your object, but I wasn't sure from your example :-)

I kind of agree, that making this possible doesn't seem all too crazy (as pybind11 often tries to create a nicer Python interface), so feel free to keep this issue open.

I would suggest to now have Component.create (a static method), as some kind of "named constructor". It's not exactly perfect, but it's the best/simplest I can think of (without going and trying out some hack).

[YannickJadoul]

Does using std::unique_ptr<Component, py::nodelete> as the holder type work for you? The destructor of the python wrapper object will still run, but the destructor of Component won't.

Oh, good point!

[dograt]

@bstaletic Wonderful! And sure enough, there is a section on this in the documentation. How wonderfully thorough. I'll close this out.