Skip to content

dask.array.asarray should handle case where xarray class is in top-level namespace#7335

Merged
jrbourbeau merged 4 commits intodask:mainfrom
tomwhite:flexible-xarray-module-name
Mar 9, 2021
Merged

dask.array.asarray should handle case where xarray class is in top-level namespace#7335
jrbourbeau merged 4 commits intodask:mainfrom
tomwhite:flexible-xarray-module-name

Conversation

@tomwhite
Copy link
Copy Markdown
Contributor

@tomwhite tomwhite commented Mar 8, 2021

In some cases (e.g. pydata/xarray#4279), xarray's DataArray class is in the xarray module, rather than the usual xarray.core.dataarray module. This causes dask.array.asarray to fail to wrap the array correctly, since it checks for a module name starting with xarray. (note the dot). This PR handles both cases.

  • Closes #xxxx
  • Tests added / passed
  • Passes black dask / flake8 dask

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tomwhite! What does type(a).__module__ look like in one of these problematic cases? Is this something we could easily test in dask/array/tests/test_xarray.py?

@tomwhite
Copy link
Copy Markdown
Contributor Author

tomwhite commented Mar 8, 2021

Thanks @tomwhite! What does type(a).__module__ look like in one of these problematic cases? Is this something we could easily test in dask/array/tests/test_xarray.py?

type(a).__module__ is simply xarray in the case I saw (due to the workaround in pydata/xarray#4279).

Do you think it would be safe to set the __module__ property on the DataArray class for the duration of the test?

Base automatically changed from master to main March 8, 2021 20:20
Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, so this issue is specifically related to using that intersphinx workaround, thanks for clarifying @tomwhite. This LGTM, I left a few small comments below mostly about adding some additional context around the added test

xr.DataArray.__module__ = "xarray"
y = da.asarray(xr.DataArray([1, 2, 3.0]))
assert isinstance(y, da.Array)
assert type(y._meta).__name__ == "ndarray"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious why this check is needed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the assert that fails without the fix.

assert_eq(y, y)


def test_asarray_recognises_xarray_data():
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment here which points out that this test ensures we support users who are using the intersphinx workaround in pydata/xarray#4279

@jrbourbeau
Copy link
Copy Markdown
Member

Also the test failures look to be unrelated to the changes here (I suspect it's due to a new pyarrow build on conda-forge). I'm investigating further over in #7344

@jrbourbeau
Copy link
Copy Markdown
Member

FWIW I'm not sure if you're running into it, but there is similar looking module checking code over in Xarray itself

tomwhite and others added 2 commits March 9, 2021 17:49
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tomwhite!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants