leak report

[tbrekalo]

How do I interpret the following?

nanobind: leaked 1 types!
 - leaked type "Pile"
nanobind: leaked 5 functions!
 - leaked function "<anonymous>"
 - leaked function "<anonymous>"
 - leaked function "<anonymous>"
 - leaked function "<anonymous>"
 - leaked function "__init__"
nanobind: this is likely caused by a reference counting issue in the binding code.

Is it a bug on my part for getting something wrong in binding code or is it an issue with nanobind?
Side note: code causing this issue deals with shared pointers for another type that is not brought up by the error message.
Type 'Pile' is regular value type, I have a std::vector of them. So I am not sure how reference counting is being mentioned here.

Thanks for your help in advance.

[wjakob]

Yes, this most likely means that there is a bug in your program. I recommend minimizing the bindings as much as possible to produce a tiny reproducer that could be posted here. Probably you will realize what the problem is along the way.

[tbrekalo]

Managed to construct a minimum program which triggers the error with slight modifications to nanobind_example.

Here's the git diff:

index 95ecfec..7d3562a 100644
--- a/src/nanobind_example/__init__.py
+++ b/src/nanobind_example/__init__.py
@@ -1 +1 @@
-from .nanobind_example_ext import add
+from .nanobind_example_ext import add, Blob
diff --git a/src/nanobind_example_ext.cpp b/src/nanobind_example_ext.cpp
index 602ee79..c3d65f7 100644
--- a/src/nanobind_example_ext.cpp
+++ b/src/nanobind_example_ext.cpp
@@ -4,6 +4,13 @@ namespace nb = nanobind;

 using namespace nb::literals;

+struct Blob {
+  Blob(int a) : a(a) {}
+  int a;
+};
+
 NB_MODULE(nanobind_example_ext, m) {
-    m.def("add", [](int a, int b) { return a + b; }, "a"_a, "b"_a);
+  nb::class_<Blob>(m, "Blob").def(nb::init<int>());
+  m.def(
+      "add", [](int a, int b) { return a + b; }, "a"_a, "b"_a);
 }
diff --git a/tests/test_basic.py b/tests/test_basic.py
index 9c670fc..285c927 100644
--- a/tests/test_basic.py
+++ b/tests/test_basic.py
@@ -1,4 +1,9 @@
 import nanobind_example as m
+import seaborn as sns

-def test_add():
+from typing import List
+
+def test_add():
     assert m.add(1, 2) == 3
+
+blobs: List[m.Blob] = []

If I exclude the: import seaborn as sns from the test file, nanobind doesn't raise the issue for the Blob type.

[jbms]

In general, Python does not guarantee that all references are cleaned up when the interpreter is shut down. For example, in the case of multiple threads, I believe leaks are practically guaranteed.

For details you can review the CPython source code:
https://github.com/python/cpython/blob/75280944e5ca957eec7f814b9d0608fc84fc5811/Python/pylifecycle.c#L1749

[tbrekalo]

Then should an issue be created from this discussion?

[wjakob]

Yes, that would be great. I would generally say that these warning messages are far too useful to simply turn them off. (I already found a ton of bugs with them, both in my own projects, and in nanobind itself). However, perhaps they can be "restrained" somehow so that they are aware of potential leaks involving type annotations.

[tbrekalo]

Opened an issues from this discussion. Would love to help resolving this issue but I am afraid my knoweledge of python inner workings is too shallow. Maybe having an option to disable warnings for objects whoes ownership is not shared between the binding code and python interpreter?

[wjakob]

For anyone running into this in the future: the problem arises due to a reference leak in typing (not exactly this issue, but this is what pointed me into the right direction: https://bugs.python.org/issue28649).

To avoid this issue, a set of LRU caches related to type information must be cleared before interpreter shutdown, which can be accomplished by running the following anywhere (just once is enough).

# Uncomment to stop leak warnings
import atexit

def cleanup():
    import typing
    for cleanup in typing._cleanups:
        cleanup()

atexit.register(cleanup)

In any case, this is not a nanobind issue. It's just that nanobind complains very loudly about leaks (a good thing!) which revealed this issue.