Convert vector of string to numpy array

[damondd]

If the max length of string is known in advance, I can write code like:

std::vector<std::string> v = {"1", "23", "456"};
using S8 = char[8];
py::array_t<S8> arr(v.size());
auto pdest = static_cast<S8*>(arr.request().ptr);
for (std::size_t i = 0; i < v.size(); ++i) {
  std::strncpy(pdest[i], v[i].data(), 8);
}

But what if the vector is created dynamically and the longest string is unknown? Please advise.

[damondd]

Figure out a way, not sure if there is any other solution:

py::array array_char() {
  std::vector<std::string> v = {"01", "234", "5678"};
  int n = v.size();
  int len = std::max_element(v.begin(), v.end(),
      [](const auto& a, const auto& b) { return a.size() < b.size(); })->size();
  int arrlen = n * len + 1;
  auto sp = std::shared_ptr<char[]>(new char[arrlen]);
  auto p = sp.get();
  for (int i = 0; i < n; ++i) {
    std::strncpy(p + len * i, v[i].data(), len);
  }

  py::array arr(py::dtype("S" + std::to_string(len)), {n}, {len}, p);
  return arr;
}

// python: array([b'01', b'234', b'5678'], dtype='|S4')

[eacousineau]

AFAIK, numpy itself shouldn't really care about the string being dynamically sized at construction of the string array:

>>> import numpy as np
>>> np.array(["hello", "world", "you", "cool", "asdfasdfasdfasdf"])
array(['hello', 'world', 'you', 'cool', 'asdfasdfasdfasdf'],
      dtype='<U16')

Hacking a quick example, seems to work just fine to use py::array(py::cast(my_strings)) if you're OK with additional overhead of conversion and the type-erased py::array in lieu of concrete py::array_t<> (which seems to be the case):
https://github.com/eacousineau/repro/blob/ff4718e185051d173695f423890c029920d4e582/python/pybind11/custom_tests/test_numpy_issue1773.cc#L17

Output:

[ Eval ]
array(['hello', 'world', 'you', 'cool', 'asdfasdfasdfasdf'],
      dtype='<U16')
[ Done ]

[damondd]

Thanks @eacousineau , py::array(py::cast(my_strings)) is really a concise way. Hope there could be more official documentation.

[eacousineau]

Agreed! If you're up for it, might you be able to take a crack at adding a short update to this doc? :D
https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html#arrays