open_datatree performance improvement on NetCDF and Zarr files #9014
Conversation
|
Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient. |
TomNicholas
added
the
topic-DataTree
aladinor
changed the title
| @@ -416,6 +415,104 @@ class ZarrStore(AbstractWritableDataStore): | |||
| "_close_store_on_close", | |||
| ) | |||
|
|
|||
| @classmethod | |||
| def open_store( | |||
There was a problem hiding this comment.
Could you rewrite open_group to call open_store() internally? That would reduce the amount of duplicated code and make this easier to maintain going forward.
…to datatree-zarr merging branches
| @@ -16,7 +16,6 @@ | |||
| BackendEntrypoint, | |||
| WritableCFDataStore, | |||
| _normalize_path, | |||
| _open_datatree_netcdf, | |||
There was a problem hiding this comment.
Before this PR _open_datatree_netcdf was used by both the netCDF4_.py and h5netcdf_.py backends. Would it be possible to move these changes back into the backends/common location and remove completely rewrite the _open_datatree_netcdf function?
There was a problem hiding this comment.
I think if you move these changes back to backends/common and leave them in _open_datatree_netcdf, You might be able to import the store for both the legacyhdf5 and the netcdf4 libraries. and then call _open_datatree_netcdf for both legacyhdf and netcdf.
currently _open_datatree_netcdftakes ncDataset: ncDataset | ncDatasetLegacyH5,
you might be able to include a new param with type cdfDataStore: NetCDF4DataStore | H5NetCDFStore and pass the appropritate one from both the n5netcdf_.py and netCDF4_.py backends.
open_datatree performance improvement on NetCDF files
whats-new.rst