Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the climwip weighting scheme in a recipe and diagnostic #1648

Merged
merged 94 commits into from
Oct 7, 2020

Conversation

JaroCamphuijsen
Copy link
Contributor

@JaroCamphuijsen JaroCamphuijsen commented May 6, 2020

Before you start, read CONTRIBUTING.md and the guide for diagnostic developers.

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub. If you are planning to modify an existing functionality, please discuss it with the original author(s) by tagging them in the issue.


Tasks

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • Please use yamllint to check that your YAML files do not contain mistakes
  • (Only if really necessary) Add any additional dependencies needed for the diagnostic script to setup.py, esmvaltool/install/R/r_requirements.txt or esmvaltool/install/Julia/julia_requirements.txt (depending on the language of your script) and also to package/meta.yaml for conda dependencies (includes Python and others, but not R/Julia)
  • If new dependencies are introduced, check that the license is compatible with Apache2.0

New recipe/diagnostic

  • Add documentation for the recipe to the doc/sphinx/source/recipes folder and add a new entry to index.rst
  • Add provenance information

Closes #1640

@bouweandela bouweandela added diagnostic EUCP www.eucp-project.eu labels Jun 8, 2020
@lukasbrunner
Copy link
Contributor

I do no longer think that the sea masking is the issue. It seems to be the detrended_std operation.

detrended_std:
<<: *general
detrend:
dimension: time
method: linear
climate_statistics:
operator: std_dev

If I change this line also to climatological_mean everything works fine:

preprocessor: detrended_std

For this case (including my suggestions above, I get very similar results when comparing the output from the ESMValTool recipe and our implementation:
climwip_esmvaltool

It would still be interesting to understand where the difference comes from, but this is a detail I'm okay with postponing.

@ruthlorenz
Copy link
Contributor

ruthlorenz commented Oct 5, 2020

I do no longer think that the sea masking is the issue. It seems to be the detrended_std operation.

I agree that the masking works fine for the climatology, so detrended_std is causing the issue. Somehow the detrend preprocessor messes with the masking (as already reported here ESMValGroup/ESMValCore#768).

But these are problems with the preprocessor functions, not the weighting code as such.
I suggest to sort out a more realistic example recipe at a later stage and change the recipe as suggested by Lukas. It is minimal (and maybe scientifically not great) but works.

Updates to the recipe file so that it makes more sense scientifically.

Co-authored-by: Lukas Brunner <lukas.brunner@live.at>
@Peter9192
Copy link
Contributor

@lukasbrunner In the last commit I used YAML anchors to condense the recipe a bit. Can you check if you prefer this or not? Otherwise I will revert it.

@lukasbrunner
Copy link
Contributor

@lukasbrunner In the last commit I used YAML anchors to condense the recipe a bit. Can you check if you prefer this or not? Otherwise I will revert it.

I think this is quite neat feel free to leave it like that from my side!

@ruthlorenz
Copy link
Contributor

looks good to me too, I like to see the variable groups as examples.

if we really wanted to include detrend_std we could do like this:

  detrended_std:
    custom_order: true
    <<: *general
    annual_statistics:
      operator: mean
    detrend:
      dimension: time
      method: linear
    climate_statistics:
      operator: std_dev
    mask_landsea:
      mask_out: sea

reapplying the ocean mask at the end. but its a bit of a hack and the masking is different because of the different order of operations...

Copy link
Contributor

@ruthlorenz ruthlorenz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, works, and will be even better when we figured out the issues with the preprocessors ;-)

@Peter9192
Copy link
Contributor

@ESMValGroup/esmvaltool-coreteam this recipe has a technical approval from @bouweandela and a scientific approval from @ruthlorenz . Can this be merged now? 1 small note is that 'natsort' was added as a dependency after Bouwe's review, but it this looks like a decent package, so I think it should not be a problem.

Copy link
Member

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The natsort package would also need to be added to package/meta.yml, or the conda package will be broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
diagnostic EUCP www.eucp-project.eu
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add diagnostic for model weighting with ClimWIP
6 participants