Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lazy regridding for wflow diagnostic #1630

Merged
merged 2 commits into from
Apr 25, 2020
Merged

Conversation

bouweandela
Copy link
Member

@bouweandela bouweandela commented Apr 23, 2020

The wflow diagnostic was using too much memory since #1618, because regridding in iris is not lazy. This pull request implements a lazy regridder for that diagnostic.

The second commit reduces memory consumption by consistently using float32 everywhere, this causes small differences in the resulting pet (order 1e-6). On my computer, this reduced peak memory usage from 1.6 GB to 1.2 GB, so not strictly needed.


Tasks

  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.

@SarahAlidoost SarahAlidoost self-requested a review April 24, 2020 09:16
Copy link
Contributor

@SarahAlidoost SarahAlidoost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bouweandela Thanks. To review this, I run the recipe for Rhine catchment (the largest one) using both ERA5 and ERA-Interim during 2002-2016 (the longest period).

  • The recipe run was successful. Therefore, the memory issue is now solved.
  • To compare the results of the diagnostic with those generated without lazy-regrid, I plotted the results with and without lazy-regrid for one point in time. They look the same.
  • I run the wflow model using forcing data with and without lazy-regrid in 2002-2003. The results look the same.

Therefore, this PR can be merged.

@bouweandela
Copy link
Member Author

Thank you for the quick review @SarahAlidoost!

@bouweandela bouweandela merged commit 44d7a72 into master Apr 25, 2020
@bouweandela bouweandela deleted the wflow-lazy-regrid branch April 25, 2020 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants