This repository was archived by the owner on Sep 11, 2023. It is now read-only.
Update NetCDF loading to use the cleaner code #623
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Pull Request ## Description Before this, the NetCDF loading code used a small hack (read file as bytes, then pass bytes into `xarray.open_dataset`, then delete bytes variable) because it was faster. Now the library fixed this and we can use the cleaner code. Fixes Issue openclimatefix-archives#602 (I hope) ## How Has This Been Tested? - Currently setting up the env on my machine to test locally. _If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?_ - [] Not yet ## Checklist: - [x] My code follows [OCF's coding style guidelines](https://github.com/openclimatefix/nowcasting/blob/main/coding_style.md) - [x] I have performed a self-review of my own code - [x] I have made corresponding changes to the documentation (noop) - [-] I have added tests that prove my fix is effective or that my feature works - [x] I have checked my code and corrected any misspellings
JanEbbing
changed the title
Contributor
|
Thanks for doing this @JanEbbing . Ill merge this in. What were the errors you found when running tests from master? |
Contributor
|
Ill keep this open |
Contributor
|
hi @all-contributors, could you please add @JanEbbing for code |
Contributor
|
@all-contributors please add @JanEbbing for code |
Contributor
|
I've put up a pull request to add @JanEbbing! 🎉 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request
Description
Before this, the NetCDF loading code used a small hack (read file as bytes, then pass bytes into
xarray.open_dataset, then delete bytes variable) because it was faster. Now the library fixed this and we can use the cleaner code.Fixes Issue #602 (I hope)
How Has This Been Tested?
py.test -s, and wanted to compare runtime ofpy.test -s --use_cloud_databefore and after this change (possibly limited to tests that touch this area of the code, e.g.tests/data_sources/test_pv_data_source.pybased on method names). Unfortunately I had 11 failed on master, possibly due to me not having set up external accounts (for data sources I presume?) correctly. Would love some feedback here how to proceed.If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?
Checklist: