Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bnb/dual dh patch #164

Merged
merged 9 commits into from
Sep 14, 2023
Merged

Bnb/dual dh patch #164

merged 9 commits into from
Sep 14, 2023

Conversation

bnb32
Copy link
Collaborator

@bnb32 bnb32 commented Aug 31, 2023

No description provided.

@bnb32 bnb32 requested a review from grantbuster August 31, 2023 14:52
@bnb32 bnb32 force-pushed the bnb/dual_dh_patch branch 2 times, most recently from 852cb65 to 4f4e76a Compare September 6, 2023 17:42
{feature} is one of the features contained by this DataHandler and
the data is a 3D array of shape (lat, lon, time) where time is
length 1 for annual correction or 12 for monthly correction. Bias
correction is only run if bc_files is not None.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This handler has to be init on independent DataHandler objects, each of which has a bias correction method. Why should we add more logic in this dual handler instead of just bias correcting the LR data handler before being passed in here? Bias correction code should ideally exist in only as many places as is necessary, i would advocate for removing here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this because the lr_handler is expected to load the full ERA domain before regridding (to in many cases a much smaller domain) so bias correcting the regridded data can in theory require touching far fewer points. But maybe it's still not worth duplicating code?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code that makes the bias correction factors will operate on the original ERA domain though, won't it? Bias correction is really fast. Strong preference for keeping bc functions only in the base handler.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it will but the original ERA domain is not the same as the final domain after regridding to the wtk grid. I load the entire ERA domain in the lr_handler before regridding because its hard to select an ERA subdomain that still covers the wtk domain (due to the curvilinear grid). But if theres no concern about speed I'm fine with removing this.

sup3r/preprocessing/batch_handling.py Outdated Show resolved Hide resolved
sup3r/preprocessing/data_handling/base.py Show resolved Hide resolved
sup3r/preprocessing/data_handling/base.py Outdated Show resolved Hide resolved
sup3r/preprocessing/data_handling/base.py Show resolved Hide resolved
…lat_lon to use min(dist) instead of kdtree. kept as static method since it is called by class methods.
…dler before sending through dual_data_handler.
@bnb32 bnb32 merged commit 63ae740 into main Sep 14, 2023
5 checks passed
@bnb32 bnb32 deleted the bnb/dual_dh_patch branch September 14, 2023 16:55
github-actions bot pushed a commit that referenced this pull request Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants