Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Opening via xarray backendentrypoint #35

Open
TomNicholas opened this issue Mar 15, 2024 · 3 comments
Open

Opening via xarray backendentrypoint #35

TomNicholas opened this issue Mar 15, 2024 · 3 comments
Labels
enhancement New feature or request xarray Requires changes to xarray upstream

Comments

@TomNicholas
Copy link
Collaborator

Some changes are needed in xarray to support using the xarray backend entrypoint system to open datasets from disk as ManifestArray-backed Variables just by passing a keyword arg to open_dataset/open_mfdataset. This requires dodging some internal array wrapping that occurs in the depths of xarray's backend machinery.

Originally posted by @TomNicholas in #14 (comment)

@TomNicholas
Copy link
Collaborator Author

This would be cool, but it's actually not needed for an MVP, because we have open_dataset_via_kerchunk.

@TomNicholas
Copy link
Collaborator Author

Trying to get this to work leads down rabbit holes like the one in pydata/xarray#8712

@TomNicholas TomNicholas added the xarray Requires changes to xarray upstream label Mar 15, 2024
@TomNicholas TomNicholas added the enhancement New feature or request label Mar 26, 2024
@TomNicholas
Copy link
Collaborator Author

TomNicholas commented Mar 26, 2024

Note that to get the syntax open_dataset(file, engine='virtualizarr', indexes={}) to work as a way of avoiding creating indexes does not actually require that indexes be added to xr.open_dataset upstream (xref pydata/xarray#6633). That's because if the kwarg is not recognized by xarray it should be passed on to our backend engine, which already does support an indexes kwarg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request xarray Requires changes to xarray upstream
Projects
None yet
Development

No branches or pull requests

1 participant