segmentation fault with std::shared_ptr

[zhqrbitee]

I have a example like below:

#include <memory>
#include <vector>

#include "nanobind/nanobind.h"
#include "nanobind/stl/shared_ptr.h"

namespace nb = nanobind;
using namespace nb::literals;

struct Time {
  Time() noexcept = default;
  double value{};
  virtual ~Time() = default;
};

struct MyTime: public Time {
  MyTime() noexcept {
    value = 1.0 * rand() / RAND_MAX;
    a = 1.0 * rand() / RAND_MAX;
  }

  double a{};
};

static std::vector<std::shared_ptr<Time>> times{};

NB_MODULE(test_nanobind_shared_ptr_in_cpp, m) {
  nb::class_<Time>(m, "Time", nb::dynamic_attr())
      .def(nb::init<>())
      .def_prop_rw(
          "value", [](Time &self) { return self.value; },
          [](Time &self, double value) { self.value = value; });

  nb::class_<MyTime, Time>(m, "MyTime")
      .def(nb::init<>())
      .def_prop_rw("a", [](MyTime &self) { return self.a; },
                   [](MyTime &self, double value) { self.a = value; });

  m.def("push", []() {
    // create MyTime and push to queue.
    times.emplace_back(std::static_pointer_cast<Time>(std::make_shared<MyTime>()));
  });

  m.def("pop", []() {
    // pop from queue
    std::shared_ptr<Time> last = std::move(times.back());
    times.pop_back();
    return last;
  });

}

When I call push/pop in Python side, e.g.

from test_nanobind_shared_ptr_in_cpp import MyTime, Time, push, pop

push()
popped = pop()
print(popped)

I get below error msgs:

nanobind: leaked 1 instances!
nanobind: leaked 1 keep_alive records!
nanobind: leaked 2 types!
 - leaked type "test_nanobind_shared_ptr_in_cpp.Time"
 - leaked type "test_nanobind_shared_ptr_in_cpp.MyTime"
nanobind: leaked 10 functions!
 - leaked function ""
 - leaked function "push"
 - leaked function "__init__"
 - leaked function ""
 - leaked function ""
 - leaked function ""
 - leaked function "pop"
 - leaked function ""
 - leaked function ""
 - leaked function "__init__"
nanobind: this is likely caused by a reference counting issue in the binding code.

If I push a few more times and then pop, then I can see a segmentation fault:

for i in range(10):
    push()

for i in range(10):
    popped = pop()
    print(f"Popped: {popped}")

I checked the document about std::shared_ptr: https://nanobind.readthedocs.io/en/latest/ownership.html, but didn't found any thing obvious wrong. I also tried to casted from Time to MyTime in the pop method, but it doesn't seems to help:

  m.def("pop", []() {
    std::shared_ptr<Time> last = std::move(times.back());
    times.pop_back();
    return std::static_pointer_cast<MyTime>(last);
  });

Any suggestion would be appreciated!

My environment is: macOS Ventura, Python 3.9.17, with nanobind 1.4.0.

[zhqrbitee]

If I put the nb::dynamic_attr() from the base class to the derived class (from the Time class to MyTime class), then the segmentation fault and leaking warnings seems gone:

  nb::class_<Time>(m, "Time")

  nb::class_<MyTime, Time>(m, "MyTime", nb::dynamic_attr())

So the issue seems comes from the nb::dynamic_attr()?

[wjakob]

I didn't have time to investigate yet, but some general observations:

• some of the older versions of Python (e.g. 3.8, 3.9) don't do a great job at cleaning up objects needing GC at the end, which can call leak warnings. Try del popped followed by gc.collect() twice to see if that removes them. Generally, putting such code in a function (with local symbol scope) will also fix the issue, it's just globals in the top-level scope that are sometimes not cleaned up. You can disable the leak warnings manually if they bother you (see nanobind API reference).

• There might be a problem in your example when the static std::vector<std::shared_ptr<Time>> vector still contains entries when Python shuts down. That's because they install a custom shared_ptr destructor, which calls Py_DECREF, but Python is already shut down at that point, which likely causes a segfault. That might be the issue you're experiencing.

[zhqrbitee]

Thanks for your suggestion, I tried to simplify the test case which still reproduce the segmentation fault with nanobind 1.4.0:

#include <memory>
#include <vector>
#include <iostream>

#include "nanobind/nanobind.h"
#include "nanobind/stl/shared_ptr.h"

namespace nb = nanobind;
using namespace nb::literals;

struct Time {
  Time() noexcept = default;
  double value{};
  virtual void dummy() {};
  virtual ~Time() = default;
};

struct MyTime: public Time {
  MyTime(): Time() {
    value = 1.0 * rand() / RAND_MAX;
    a = 1.0 * rand() / RAND_MAX;
  }

  double a{};
};

NB_MODULE(test_nanobind_shared_ptr_in_cpp, m) {
  nb::class_<Time>(m, "Time", nb::dynamic_attr())
      .def(nb::init<>());

  nb::class_<MyTime, Time>(m, "MyTime")
      .def(nb::init<>());

  m.def("gen_mytime_from_cpp", []() {
    return std::make_shared<MyTime>();
  });
}

from test_nanobind_shared_ptr_in_cpp import gen_mytime_from_cpp

print(gen_mytime_from_cpp())
print(gen_mytime_from_cpp())

By debugging into the code, it seems the segmentation faults comes from below line:

However, I tried a wheel built from latest master, it seems the segmentation issue is gone. Maybe it is already fixed by some commits after 1.4.0?
So @wjakob do you have any estimated date for the next release?

[wjakob]

I just released version 1.5.0. Can you confirm that it fixes your issue?

[zhqrbitee]

Confirmed fixed. Thanks!

[wjakob]

More fixes in version 1.5.2 :-)

[zhqrbitee]

Thanks for your quick fix!