Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structured arrays support #1

Closed
wants to merge 70 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a2aec09
Change npy_format_descriptor typenum to static fn
aldanor Jun 17, 2016
0f4a7ef
Add buffer_info::as_pybuffer() method
aldanor Jun 19, 2016
f06e8c8
Change format_descriptor::value to a static func
aldanor Jun 19, 2016
4719de4
Dump test output if the test runner fails
aldanor Jun 19, 2016
1afd6f6
Add memoryview type
aldanor Jun 19, 2016
7c77a29
Update gitignore to ignore debug test builds
aldanor Jun 19, 2016
73291d3
Use a macro for numpy API definitions
aldanor Jun 19, 2016
dff03a4
Use memoryview for constructing array from buffer
aldanor Jun 19, 2016
88cab92
Switch away from typenums for numpy descriptors
aldanor Jun 19, 2016
fb32995
Add PYBIND11_DTYPE macro for registering dtypes
aldanor Jun 19, 2016
f3c0dba
Add a basic test for recarrays and complex dtypes
aldanor Jun 19, 2016
98307ab
Release format descriptor args before converting
aldanor Jun 19, 2016
2479cd6
Add empty recarray test, check for calloc fail
aldanor Jun 19, 2016
a22c3cf
Add packed recarray tests
aldanor Jun 19, 2016
3c2ffd5
Borrow field descriptors for recarray dtype
aldanor Jun 19, 2016
aa276b2
Add tests for nested recarrays
aldanor Jun 19, 2016
b97ac4c
Add explicit test for recarray format descriptors
aldanor Jun 19, 2016
efedfc2
Simplify npy_format_descriptor slightly
aldanor Jun 21, 2016
15acfbd
Add test for a function accepting recarray (WIP)
aldanor Jun 21, 2016
512b1c4
Incref descriptors properly when creating arrays
aldanor Jun 21, 2016
20ce202
Use object instead of ptrs in numpy descriptors
aldanor Jun 21, 2016
17d93f5
Prefix all macros in numpy.h to avoid name clashes
aldanor Jun 21, 2016
1c54e08
Remove erroneous py:: prefix in numpy.h
aldanor Jun 21, 2016
8fcecf9
Add dtype_of<T>() function, update the tests
aldanor Jun 22, 2016
cd48be6
Make changes to format_descriptor backwards-compat
aldanor Jun 26, 2016
bcc65a6
Prefix the FIELD_DESCRIPTOR macro
aldanor Jun 26, 2016
60d8780
Make npy_format_descriptor backwards-compat
aldanor Jun 26, 2016
9cfa4ae
Revert accidental whitespace change
aldanor Jun 26, 2016
3a644d6
npy_format_descriptor::format() - fail if unbound
aldanor Jun 26, 2016
cd23f16
Add a test for buffer format of unbound struct
aldanor Jun 26, 2016
beab8a8
Add detail::is_pod_struct<T> helper
aldanor Jun 26, 2016
81b7fe0
Cosmetic: fix indentation
aldanor Jun 27, 2016
82f4f07
Fix PYBIND11_DTYPE to work with MSVC compiler
aldanor Jun 27, 2016
bcb69e1
Change PB11_IMPL prefix to PYBIND11, add comment
aldanor Jun 27, 2016
c5c9a15
Add a few braces for clarity
aldanor Jun 29, 2016
5b6c75e
Always allocate at least one element
aldanor Jun 29, 2016
4f97566
Add a few comments throughout numpy.h
aldanor Jun 29, 2016
bc6cdc7
Exclude double type from is_pod_struct
aldanor Jun 29, 2016
25767cd
Add documentation re: PYBIND11_DTYPE macro
aldanor Jul 2, 2016
bdbe3c1
Use proper type for an int literal
aldanor Jul 2, 2016
49e592c
Minor fix in the docs
aldanor Jul 2, 2016
62ae351
Rename PYBIND11_DTYPE to PYBIND11_NUMPY_DTYPE
aldanor Jul 2, 2016
1c0a394
Use malloc insterad of calloc for numpy arrays
aldanor Jul 2, 2016
03b0032
Change field descriptor offset type to size_t
aldanor Jul 3, 2016
065a1c6
Bugfix: pass struct size as itemsize to descriptor
aldanor Jul 3, 2016
6c4dd0e
Strip padding fields in dtypes, update the tests
aldanor Jul 5, 2016
3d7e87b
Various cleanup
aldanor Jul 17, 2016
e73aba5
Restore dtype equivalence sanity check
aldanor Jul 17, 2016
1c7e11f
Update npy_format_descriptor::name()
aldanor Jul 18, 2016
4ec176b
Make buffer_info::as_pybuffer a memoryview ctor
aldanor Jul 18, 2016
21183e8
Store array requests in local variables in tests
aldanor Jul 18, 2016
1f7747e
Fix a comment and wrong indentation
aldanor Jul 18, 2016
dbeb5c2
Make struct packing in example20 MSVC-compliant
aldanor Jul 18, 2016
72acf83
Replace 4096B format buffer with std::string
aldanor Jul 18, 2016
55f3604
Remove redundant definition
aldanor Jul 18, 2016
826a046
Add numpy wrappers for char[] and std::array<char>
aldanor Jul 19, 2016
8637851
Rename example20 -> example-numpy-dtypes
aldanor Jul 19, 2016
3e874fe
Code reordering / cleanup only
aldanor Jul 19, 2016
98e00e3
Cleanup: move numpy API bindings out of py::array
aldanor Jul 19, 2016
87e7438
Initial implementation of py::dtype
aldanor Jul 23, 2016
88d383a
Fix a segfault where func object wasn't released
aldanor Jul 24, 2016
6e75c44
Add dtype(names, offsets, formats, itemsize) ctor
aldanor Jul 24, 2016
c1d2f12
Add all possible ctors for py::array
aldanor Jul 24, 2016
b27a3ba
Also add the new ctors to py::array_t
aldanor Jul 24, 2016
a09d05e
Add a simplified buffer_info ctor for 1-D case
aldanor Jul 24, 2016
0f576c6
Add tests for new array/array_t ctors
aldanor Jul 24, 2016
9459e99
Add tests for py::dtype ctors
aldanor Jul 24, 2016
55623e2
Add a few more dtype tests
aldanor Jul 24, 2016
d96d564
Allow nullptr in array ctors wherever possible
aldanor Jul 24, 2016
01dfc40
Added dtype from const char pointer ctor
aldanor Jul 24, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ CMakeFiles
Makefile
cmake_install.cmake
.DS_Store
/example/example.so
/example/example*.so
/example/example.cpython*.so
/example/example.pyd
/example/example*.dll
Expand Down Expand Up @@ -31,3 +31,4 @@ MANIFEST
.DS_Store
/dist
/build
/cmake/
43 changes: 33 additions & 10 deletions docs/advanced.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1159,12 +1159,12 @@ completely avoid copy operations with Python expressions like
py::class_<Matrix>(m, "Matrix")
.def_buffer([](Matrix &m) -> py::buffer_info {
return py::buffer_info(
m.data(), /* Pointer to buffer */
sizeof(float), /* Size of one scalar */
py::format_descriptor<float>::value, /* Python struct-style format descriptor */
2, /* Number of dimensions */
{ m.rows(), m.cols() }, /* Buffer dimensions */
{ sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
m.data(), /* Pointer to buffer */
sizeof(float), /* Size of one scalar */
py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
2, /* Number of dimensions */
{ m.rows(), m.cols() }, /* Buffer dimensions */
{ sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
sizeof(float) }
);
});
Expand Down Expand Up @@ -1208,7 +1208,7 @@ buffer objects (e.g. a NumPy matrix).
py::buffer_info info = b.request();

/* Some sanity checks ... */
if (info.format != py::format_descriptor<Scalar>::value)
if (info.format != py::format_descriptor<Scalar>::format())
throw std::runtime_error("Incompatible format: expected a double array!");

if (info.ndim != 2)
Expand All @@ -1234,7 +1234,7 @@ as follows:
m.data(), /* Pointer to buffer */
sizeof(Scalar), /* Size of one scalar */
/* Python struct-style format descriptor */
py::format_descriptor<Scalar>::value,
py::format_descriptor<Scalar>::format(),
/* Number of dimensions */
2,
/* Buffer dimensions */
Expand Down Expand Up @@ -1293,6 +1293,30 @@ template paramenter, and it ensures that non-conforming arguments are converted
into an array satisfying the specified requirements instead of trying the next
function overload.

NumPy structured types
======================

In order for ``py::array_t`` to work with structured (record) types, we first need
to register the memory layout of the type. This can be done via ``PYBIND11_NUMPY_DTYPE``
macro which expects the type followed by field names:

.. code-block:: cpp

struct A {
int x;
double y;
};

struct B {
int z;
A a;
};

PYBIND11_NUMPY_DTYPE(A, x, y);
PYBIND11_NUMPY_DTYPE(B, z, a);

/* now both A and B can be used as template arguments to py::array_t */

Vectorizing functions
=====================

Expand Down Expand Up @@ -1374,7 +1398,7 @@ simply using ``vectorize``).
auto result = py::array(py::buffer_info(
nullptr, /* Pointer to data (nullptr -> ask NumPy to allocate!) */
sizeof(double), /* Size of one item */
py::format_descriptor<double>::value, /* Buffer format */
py::format_descriptor<double>::format(), /* Buffer format */
buf1.ndim, /* How many dimensions? */
{ buf1.shape[0] }, /* Number of elements for each dimension */
{ sizeof(double) } /* Strides for each dimension */
Expand Down Expand Up @@ -1760,4 +1784,3 @@ is always ``none``).

// Evaluate the statements in an separate Python file on disk
py::eval_file("script.py", scope);

2 changes: 1 addition & 1 deletion example/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ set(PYBIND11_EXAMPLES
example-stl-binder-vector.cpp
example-eval.cpp
example-custom-exceptions.cpp
example-numpy-dtypes.cpp
issues.cpp
)

Expand Down Expand Up @@ -68,4 +69,3 @@ foreach(VALUE ${PYBIND11_EXAMPLES})
string(REGEX REPLACE "^(.+).cpp$" "\\1" EXAMPLE_NAME "${VALUE}")
add_test(NAME ${EXAMPLE_NAME} COMMAND ${RUN_TEST} ${EXAMPLE_NAME})
endforeach()

14 changes: 7 additions & 7 deletions example/example-buffers.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ void init_ex_buffers(py::module &m) {
/// Construct from a buffer
.def("__init__", [](Matrix &v, py::buffer b) {
py::buffer_info info = b.request();
if (info.format != py::format_descriptor<float>::value || info.ndim != 2)
if (info.format != py::format_descriptor<float>::format() || info.ndim != 2)
throw std::runtime_error("Incompatible buffer format!");
new (&v) Matrix(info.shape[0], info.shape[1]);
memcpy(v.data(), info.ptr, sizeof(float) * v.rows() * v.cols());
Expand All @@ -103,12 +103,12 @@ void init_ex_buffers(py::module &m) {
/// Provide buffer access
.def_buffer([](Matrix &m) -> py::buffer_info {
return py::buffer_info(
m.data(), /* Pointer to buffer */
sizeof(float), /* Size of one scalar */
py::format_descriptor<float>::value, /* Python struct-style format descriptor */
2, /* Number of dimensions */
{ m.rows(), m.cols() }, /* Buffer dimensions */
{ sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
m.data(), /* Pointer to buffer */
sizeof(float), /* Size of one scalar */
py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
2, /* Number of dimensions */
{ m.rows(), m.cols() }, /* Buffer dimensions */
{ sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
sizeof(float) }
);
});
Expand Down
Loading