Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifications to enable sparse xarrays #261

Merged
merged 2 commits into from
Jul 13, 2019

Conversation

nvictus
Copy link
Contributor

@nvictus nvictus commented Jul 13, 2019

Hello!

I'm working on a SciPy sprint to get sparse arrays into xarray.

With a couple tweaks to xarray anticipating objects implementing __array_function__, I can get it to work with the following:

  • Make COO.astype() provide the copy argument

  • Expose result_type() though sparse to bypass numpy's dispatcher -- I'm importing the lower level function.

(I'm using numpy 1.16 with 'NUMPY_EXPERIMENTAL_ARRAY_FUNCTION' = '1'
Numpy 1.17.0rc1 currently fails due to an issue with numba numba/numba#4319)

@codecov
Copy link

codecov bot commented Jul 13, 2019

Codecov Report

Merging #261 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #261      +/-   ##
==========================================
+ Coverage   97.28%   97.28%   +<.01%     
==========================================
  Files          10       10              
  Lines        1620     1621       +1     
==========================================
+ Hits         1576     1577       +1     
  Misses         44       44

Copy link
Collaborator

@hameerabbasi hameerabbasi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hameerabbasi hameerabbasi merged commit 939f3ff into pydata:master Jul 13, 2019
@hameerabbasi
Copy link
Collaborator

Thanks, @nvictus!

nvictus added a commit to nvictus/xarray that referenced this pull request Aug 2, 2019
shoyer pushed a commit to pydata/xarray that referenced this pull request Aug 5, 2019
)

* Support for __array_function__ implementers

* Pep8

* Consistent naming

* Check for NEP18 enabled and nep18 non-numpy arrays

* Replace .values with .data

* Add initial test for nep18

* Fix linting issues

* Add parameterized tests

* Internal clean-up of isnull() to avoid relying on pandas

This version should be much more compatible out of the box with duck typing.

* Add sparse to ci requirements

* Moar tests

* Two more patches for __array_function__ duck-arrays

* Don't use coords attribute from duck-arrays that aren't derived from DataWithCoords

* Improve checking for coords, and autopep8

* Skip tests if NEP-18 envvar is not set

* flake8

* Update xarray/core/dataarray.py

Co-Authored-By: Stephan Hoyer <shoyer@gmail.com>

* Fix coords parsing

* More tests

* Add align tests

* Replace nep18 tests with more extensive tests on pydata/sparse

* Add xfails for missing np.result_type (fixed by pydata/sparse/pull/261)

* Fix xpasses

* Revert isnull/notnull

* Fix as_like_arrays by coercing dense arrays to COO if any sparse

* Make Variable.load a no-op for non-dask duck arrays

* Add additional method tests

* Fix utils.as_scalar to handle duck arrays with ndim>0
@@ -5,6 +5,8 @@
eye, full, full_like, zeros, zeros_like, ones, ones_like,
kron, argwhere, isposinf, isneginf)

from numpy.core._multiarray_umath import result_type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beware, this is likely to break in a future NumPy release — this isn’t a public API

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants