-
Notifications
You must be signed in to change notification settings - Fork 7
Datatree support #89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Datatree support #89
Conversation
|
Well, it's unfortunate to see that this doesn't work with Python I think we should drop support for
|
|
When it comes to supporting Python versions, there are two places I always reference:
Python 3.10 itself will still be supported until the end of this year, but SPEC 0 recommends keeping to the cutting-edge. If you want to keep pace with the likes of NumPy and Xarray, the recommendation is actually to drop 3.10 and 3.11 at this stage. With the rise of tools like It's your call if you want to drop 3.10. I'm probably overly conservative about this because I recall being stuck on 3.8 for ages in the days before |
|
Thanks for the advice Liam! I think to stay in-line with SPEC 0 but still allow for people who may have slightly older versions of Python installed I shall follow your advice and drop 3.10 but maintain 3.11 |
…h `open_mfdataset`
2477804 to
40e5943
Compare
|
I've rebased on main, apologies that GitHub is now giving me credit for all of your hard work! The failing readthedocs build seems to be because my commit was immediately followed by an auto-commit that cancelled the previous workflow. I tried to resolve a |
6795ba5 to
880a996
Compare
|
Rebased again to remove the little dance with the conflicting formatting tools. |
|
Thanks for fixing the conflict! Regarding the |
No worries! I'll raise an issue/PR for the workflow change later. I've read through the code and all looks good to me, but I haven't yet had a chance to play around with the |
LiamPattinson
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All looks good! I've tested it myself and the DataTrees are a lovely addition to the library. I can think of a few previous projects I've worked on where they would have been really helpful.
|
|
||
| ds = ds.rename_vars(final_renames) | ||
|
|
||
| return xr.DataTree.from_dict(ds) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I must admit, I'm quite impressed that this works. I would have expected much more manual intervention to be necessary!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was just as shocked that it worked when I first tried it, such an elegant system!
xarray.DataTree. Thankfully this is quite a simple addition that takes an existing dataset, converts the names back to the original ones in the raw SDF files and then builds a datatree using thefrom_dictmethod.xarray.DataTreean additional attribute calledflat_structure_nameis added that points to the name used to reference the variable when opening as a regularxarray.Dataset.Resolves #23
The motivation behind this addition is so that we are able to interface with a package like xr-tui. This makes interfacing with and playing around with datasets on HPCs significantly easier.
Requires samueljackson92/xr-tui#4 to work with
open_mfdatatree.