pybind11 stuff that is missing in nanobind

[steven-johnson]

I experimentally tried to port Halide's python bindings to use nanobind instead of pybind11, and it seems there are a few things still missing; this isn't meant to be a criticism of what I presume is still a work in progress, but rather, feedback for future work (or maybe a request for correction if things are present in a way I haven't found):

• Documentation on nb::tensor, which presumably replaces py::buffer. Overall the concept looks like a big improvement, but this should definitely be called out as an API change in the README, as the changes are nontrivial enough that it's not just a drop-in change.
• py::exception and py::register_exception_translator are missing. It looks like there is internal support for the latter, but I can't find where custom exceptions can be defined, as with the former.
• Can I make a custom constructor that is also an implicit type conversion? (e.g. for types that don't exist in the underlying code, like py::tuple)
• How can I create (and populate) a tuple of a given size? There doesn't appear to be a ctor that takes a size argument.

[eirrgang]

How can I create (and populate) a tuple of a given size? There doesn't appear to be a ctor that takes a size argument.

Do you mean a py::tuple or a std::tuple? I don't see how a size argument could make sense in either case.

[steven-johnson]

A py::tuple... in pybind11 you can create a tuple of size N via the ctor, then set each value with operator[].

[wjakob]

Are you sure? Python tuple objects are immutable, which is a Python limitation. Both pybind11 and nanobind have py::make_tuple / nb::make_tuple for this reason.

[froody]

@wjakob In python you can create a tuple from a dynamically-sized collection, e.g. tuple(range(5)), but there's no facility in nanobind to do this. In pybind11 you could assign via operator[]. I get the desire to conform to python semantics for tuple immutability, but some convenience methods to create a tuple from an iterator pair would be much more ergonomic.
pybind11:

py::tuple result(num_items);
for (int i = 0; i < num_items; i++) {
    result[i] = make_thing(i);
}

nanobind:

nb::tuple result = nb::steal<nb::tuple>(PyTuple_New(num_items));
for (int i = 0; i < num_items; i++) {
    nb::object obj = nb::cast(make_thing(i));
    NB_TUPLE_SET_ITEM(result.ptr(), i, obj.inc_ref().ptr());
}

[wjakob]

Generally, it's easy to create your own wrappers or extensions (writeable_tuple) in your own part of the codebase. What you describe checks several boxes that make me lean against including it in nanobind:

1. It's an advanced use case (creating a tuple using a size that is only known at runtime)
2. The API is potentially dangerous. Tuple mutability is only legal in a narrow set of conditions.
3. This is a "nano" library

[wjakob]

Documentation on nb::tensor, which presumably replaces py::buffer. Overall the concept looks like a big improvement, but this should definitely be called out as an API change in the README, as the changes are nontrivial enough that it's not just a drop-in change.

There was never a guarantee of API compatibility. I am trying to stay mostly close when the existing API was a good choice, but in other cases I am making drastic changes. The tensors are one such example.

They are quite extensively documented through a Markdown page (https://github.com/wjakob/nanobind/blob/master/docs/tensor.md) and a testcase (https://github.com/wjakob/nanobind/blob/master/tests/test_tensor.py, https://github.com/wjakob/nanobind/blob/master/tests/test_tensor.cpp). Is that not enough?

py::exception and py::register_exception_translator are missing. It looks like there is internal support for the latter, but I can't find where custom exceptions can be defined, as with the former.

This feature was just added to master.

Can I make a custom constructor that is also an implicit type conversion? (e.g. for types that don't exist in the underlying code, like py::tuple)

nanobind supports implicit conversions (see README.md for an explanation for the API differences), but I am not sure we are talking about the same thing. Probably it's best if you give an example of what you did in pybind11 that now no longer works.

How can I create (and populate) a tuple of a given size? There doesn't appear to be a ctor that takes a size argument.

See the comment above.

[steven-johnson]

There was never a guarantee of API compatibility.

Oh, I get that, and I agree, right choice

They are quite extensively documented through a Markdown page

Embarassingly, I didn't see that page on my first read-thru, somehow. I'll take a look. Sorry for the noise.