Skip to content

Add an example for sharing the memory buffer of the xarray between C++ and Python (numpy)#175

Merged
JohanMabille merged 3 commits intoxtensor-stack:masterfrom
yungyuc:array-ref
Nov 6, 2018
Merged

Add an example for sharing the memory buffer of the xarray between C++ and Python (numpy)#175
JohanMabille merged 3 commits intoxtensor-stack:masterfrom
yungyuc:array-ref

Conversation

@yungyuc
Copy link
Contributor

@yungyuc yungyuc commented Nov 2, 2018

#113 added the ability to take the memory buffer of an xarray and share it with ndarray in Python, but there wasn't an example for how to do it. This PR adds an example in the pytest directory.

I also fixed some formatting and compatibilty issue with pybind11 2.3.

Demonstrate how to return an array and share the buffer to Python (numpy).
@yungyuc
Copy link
Contributor Author

yungyuc commented Nov 2, 2018

I originally thought numpy view binding (#130) is necessary, but it turned out the feature I needed is already in the code base.

@yungyuc
Copy link
Contributor Author

yungyuc commented Nov 5, 2018

Would someone be willing to take a look at this? If the fixes for pybind11 2.3 should be in a separate PR, I can close this and file two new ones.

@SylvainCorlay
Copy link
Member

@yungyuc sorry for the late reply! I am looking at it now.

test/main.cpp Outdated

#include "pybind11/numpy.h"

#include "xtensor/xarray.hpp"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. On OSX if commenting it out, I got:

In file included from /Users/yungyuc/hack/code/xtensor-python/test/main.cpp:17:
In file included from /Users/yungyuc/hack/code/xtensor-python/include/xtensor-python/pyarray.hpp:17:
/Users/yungyuc/hack/usr/opt36_180914/include/xtensor/xiterator.hpp:406:25: error: no member named 'mpl' in namespace 'xtl'
            return xtl::mpl::static_if<has_storage_iterator<C>::value>([&](auto self)
                   ~~~~~^

Copy link
Member

@SylvainCorlay SylvainCorlay Nov 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, this seems to be something that was fixed about 2 months ago in xtensor-stack/xtensor@8a61692.

Copy link
Contributor Author

@yungyuc yungyuc Nov 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I was testing against xtensor-stack/xtensor@18a26c8 (Fri Sep 14 15:55:58 2018 +0200). After updating xtensor to the latest master (xtensor-stack/xtensor@d61b71e) I don't need this line anymore. I'll remove it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yungyuc excellent!

(By the way, is there a specific reason why you are running on specific commits and not version tags?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SylvainCorlay It's only because of my careless practice. My time on my side project is limited so I am developing against master of primary dependencies (pybind11, xtensor, xtensor-python, etc.). It's fine but sometimes (like this time) I was distracted for weeks and forgot to update the dependencies.

Sorry for my silly mistake. I've add a new change reverting this line and other two spots of unnecessary change.


#include "gtest/gtest.h"

#include "xtensor-python/pyarray.hpp"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is reverted in a newer change.


#include "gtest/gtest.h"

#include "xtensor-python/pytensor.hpp"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is reverted in a newer change.

@JohanMabille JohanMabille merged commit 7f96473 into xtensor-stack:master Nov 6, 2018
@yungyuc yungyuc deleted the array-ref branch November 7, 2018 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants