Streamline input data loading and interpolation code#38
Merged
Conversation
Suggest by CoPilot to speed up this calculation.
Replace the `apply` call with a function and reduce number of df transformations.
Also introduces class constants to easier identify which algorithm was requested.
This renames classes, variables, and method args to conform with Python naming conventions.
The `variables` and `data_variables` were unused when determining if wind data needs to be loaded or not based on the config values.
Reduce the complexity of accessioning the loaded data from the loading class that was previously copied to the proxy class to determine the data type and have the model framework logic directly access the loader class.
Expand the base class for all loaded gridded data to hold attributes for metadata and data.
Split out the call to load metadata versus variable data for HRRR. Now HRRR metadata gets only loaded once per class initialization and the variable data per time step. Stations are now only filtered when there are any set with the config corresponding section. This should be a small performance boost by reducing data load that is not needed/used.
Ran into this while running the "baseline" iSnobal. Not particularly related to this branch but will hopefully be merged soon.
jomey
commented
Oct 30, 2025
arobledano
approved these changes
Oct 31, 2025
arobledano
left a comment
arobledano
left a comment
Contributor
There was a problem hiding this comment.
Wow, big effort here. Nice job!!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Couple of changes to this PR:
.inifileNote regards the change to use KDTree - I have been running that successfully for multiple water years and domains at the CBRFC.