How to bind numpy arrays in-place when given as STL vectors?

[tienviitta]

I have C++ header to bind to Python:

#include <cstdint>
#include <vector>

void bit2byte(const std::vector<uint8_t> &bits, std::vector<uint8_t> &bytes, bool lsb_order);
void byte2bit(const std::vector<uint8_t> &bytes, std::vector<uint8_t> &bits, bool lsb_order);

And I have done following wrapper:

    m.def("bit2byte",
          [](nb::ndarray<uint8_t, nb::shape<nb::any>, nb::c_contig, nb::device::cpu> &bits,
             nb::ndarray<uint8_t, nb::shape<nb::any>, nb::c_contig, nb::device::cpu>
                 &bytes) {
              bit2byte((std::vector<uint8_t> &)bits, (std::vector<uint8_t> &)bytes, true);
          });

Which compiles but gives segmentation fault. There must be some other means to use the nb:ndarray for std::vector but I couldn't find this on documentation? I have previously used Cython for similar with memoryviews but I'm new to pybind or nanobind.

[wjakob]

You cannot just reference-cast the nb::ndarray to a std::vector<uint8_t>. They are different types and have an incompatible layout. Use a static_cast if you feel unsure. This would have resulted in an error message in this case.

std::vector can never be used as a view of some other pre-existing data, since it wants to own the underlying storage.

I suggest that you review the nanobind documentation, including how to iterate over the contents of an ndarray. You will likely have to update the interface of your bit2byte functions. Another option is to not use an ndarray but use bind_vector.h to expose this type. Again, please refer to the documentation for detail.

[tienviitta]

OK, thank you, basically I can then make this bit2byte using nb::ndarray as inputs instead of std::vector. Another option I think would be return the bytes using that bind_vector.h. I understood the problem with std:.vector, using uint8_t* could be an option too here with size_t given.

I'll review the documentation and decide what way to go.

[tienviitta]

Thank you for your quick reply. This seems to be the easiest solution to still have in-place computation using uint8_t*:

void bit2byte(const uint8_t *bits, uint8_t *bytes, size_t n_bytes, bool lsb_order);

And then:

    m.def(
        "bit2byte",
        [](nb::ndarray<uint8_t, nb::shape<nb::any>, nb::c_contig, nb::device::cpu> &bits,
           nb::ndarray<uint8_t, nb::shape<nb::any>, nb::c_contig, nb::device::cpu>
               &bytes, bool lsb_order) { bit2byte(bits.data(), bytes.data(), bytes.shape(0), lsb_order); });

I'll work with the other solutions later...