Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recipe with new diagnostics for ESA-CMUG H2O #1834

Merged
merged 38 commits into from
Jan 13, 2022
Merged

Recipe with new diagnostics for ESA-CMUG H2O #1834

merged 38 commits into from
Jan 13, 2022

Conversation

katjaweigel
Copy link
Contributor

@katjaweigel katjaweigel commented Sep 30, 2020

Before you start, please read our contribution guidelines.

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub. If you are planning to modify an existing functionality, please discuss it with the original author(s) by tagging them in the issue.


Tasks

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • Please use yamllint to check that your YAML files do not contain mistakes

New recipe/diagnostic

  • Add documentation for the recipe to the doc/sphinx/source/recipes folder and add a new entry to index.rst
  • Add provenance information

Modified recipe/diagnostic

New data reformatting script

Modified data reformatting script

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.


Closes #1833

@katjaweigel
Copy link
Contributor Author

katjaweigel commented Sep 30, 2020

I still need to add provenance and complete the documentation, sorry. Cmorizers for the ESACCI water vapor data are not part of the pull request, because the data are not available, yet. Instead I used HOAPS (nadir) and ERA-Interim (for limb) data as placeholders. There is already a function to do the necessary conversion of the unit to cmorize the limb data when available.

@katjaweigel katjaweigel self-assigned this Sep 30, 2020
“katjaweigel” added 2 commits April 7, 2021 10:22
@valeriupredoi valeriupredoi self-requested a review December 6, 2021 13:50
Copy link
Contributor

@valeriupredoi valeriupredoi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work! A couple minor comments from me 👍

The envisaged coverage is 2010-2014. The calculation of hus from water vapour in ppmv will be part of the cmorizer.
Here, ERA-Interim data are used.

The diagnostic diag_tropopause_zonalmean.py plots zonal mean for given variable at for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The diagnostic diag_tropopause_zonalmean.py plots zonal mean for given variable at for
The diagnostic diag_tropopause_zonalmean.py plots zonal mean for given variable for

For each model, two experiments must be given:
a pre industrial control run, and a scenario with 4 times CO\ :sub:`2`\.
Possibly, 150 years should be given, but shorter time series work as well.
Currently, HOAPS data are incuded as place holder for expected ESACCI-WV data, type CDR-2:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently, HOAPS data are incuded as place holder for expected ESACCI-WV data, type CDR-2:
Currently, HOAPS data are included as place holder for expected ESACCI-WV data, type CDR-2:

:align: center
:width: 50%

Scatter plot and regression line the between the ratio of the change of net short wave radiation (rsnst) and the change of the Water Vapor Path (prw) against the ratio of the change of netshort wave radiation for clear skye (rsnstcs) and the the change of surface temperature (tas). The width of horizontal shading for models and the vertical dashed lines for observations (Obs.) represent statistical uncertainties of the ratio, as the 95% confidence interval (CI) of the regression slope to the rsnst versus prw curve. For the prw observations ESACCI CDR-2 data from 2003 to 2014 are used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Scatter plot and regression line the between the ratio of the change of net short wave radiation (rsnst) and the change of the Water Vapor Path (prw) against the ratio of the change of netshort wave radiation for clear skye (rsnstcs) and the the change of surface temperature (tas). The width of horizontal shading for models and the vertical dashed lines for observations (Obs.) represent statistical uncertainties of the ratio, as the 95% confidence interval (CI) of the regression slope to the rsnst versus prw curve. For the prw observations ESACCI CDR-2 data from 2003 to 2014 are used.
Scatter plot and regression line computed between the ratio of the change of net short wave radiation (rsnst) and the change of the Water Vapor Path (prw) against the ratio of the change of netshort wave radiation for clear skye (rsnstcs) and the the change of surface temperature (tas). The width of horizontal shading for models and the vertical dashed lines for observations (Obs.) represent statistical uncertainties of the ratio, as the 95% confidence interval (CI) of the regression slope to the rsnst versus prw curve. For the prw observations ESACCI CDR-2 data from 2003 to 2014 are used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @valeriupredoi , I corrected the documentation (here and for the points above).


def h2o_to_hus(h2o):
"""Calculate H2O in specific humidity instead of vmr."""
mda = 28.966 # melocular mass of dry air
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mda = 28.966 # melocular mass of dry air
mda = 28.966 # molecular mass of dry air

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, done


def hus_to_h2o(hus):
"""Calculate H2O in vmr instead of specific humidity."""
mda = 28.966 # melocular mass of dry air
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mda = 28.966 # melocular mass of dry air
mda = 28.966 # molecular mass of dry air

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, done

mda = 28.966 # melocular mass of dry air
mwv = 18.016 # molecular mass of water vapour
helpval = h2o * (mwv / mda)
hus = helpval / (1.0 + (helpval))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
hus = helpval / (1.0 + (helpval))
hus = helpval / (1.0 + helpval)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, done


from esmvaltool.diag_scripts.shared import (group_metadata, run_diagnostic,
select_metadata, sorted_metadata)
from esmvaltool.diag_scripts.shared._base import (get_plot_filename)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please run an isort on this script to fix the import order

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, done

# pformat(provenance_record))
# with ProvenanceLogger(cfg) as provenance_logger:
# provenance_logger.log(diagnostic_file, provenance_record)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please remove commented out code or reinstate it if needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, seems like I never completed the provenance here, trying to do that now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provenance added for esmvaltool/diag_scripts/cmug_h2o/diag_tropopause.py, but it is also missing for a part of the figures in esmvaltool/diag_scripts/cmug_h2o/diag_tropopausezonal.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katjaweigel is this now ready for a final look before merging this into main?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, on more figure type in esmvaltool/diag_scripts/cmug_h2o/diag_tropopausezonal.py missing, I hope I'll manage now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@axel-lauer it should be complete now. I'm running an other tests for all parts on Mistral now (so far I only ran the diagnostics I changed, not deangelis).

@katjaweigel
Copy link
Contributor Author

Provenance for one more figure type is missing, I'll add it later today or tomorrow.

@valeriupredoi
Copy link
Contributor

cheers @katjaweigel - solid progress here 🍺 Could you please add a title to recipe_cmug_h2o_deangelis.yml - that's why the tests are failing; I'll have a one final tech look after 👍

@katjaweigel
Copy link
Contributor Author

@valeriupredoi thanks for checking that! I did not realize, that recipe_cmug_h2o_deangelis.yml is actuall part of the branch. I created it mainly for testing, it calls only one of the 3 diagnostics from recipe_cmug_h2o.yml. This is also the only diagnostic, which really uses CMUG data so far (the other ones using reanalysis data, because we still wait for the CMUG data with vertical coordinate). @axel-lauer I guess I better remove recipe_cmug_h2o_deangelis.yml because it is kind of redundant?

@katjaweigel
Copy link
Contributor Author

recipe_cmug_h2o.yml ran successfully on Mistral now (but I needed the development version of the core, else it had an issue with the capital letters in obs4MIPs).
@axel-lauer I removed recipe_cmug_h2o_deangelis.yml now (but kept a copy in case you want it back).

@valeriupredoi
Copy link
Contributor

nice work @katjaweigel 🍺 Could you please take care of the PR description section (check all boxes that are relevant, strike-through or delete the ones that are not)? Then all will be snug like a bug in a rug 🐛

@katjaweigel
Copy link
Contributor Author

@valeriupredoi done, thanks! (I forgot to rerun the yamllint checks before, spyder tends to introduce new trailing white spaces, so I removed them now.)

@axel-lauer
Copy link
Contributor

@katjaweigel Is your PR now ready to be merged? If you think it helps, I can try to test the recipe one last time once the fx files on Mistral have been restored.

@katjaweigel
Copy link
Contributor Author

@axel-lauer I think it is ready (you can test it, after the first round of CMIP5 removal it still worked, so I guess fx is no issue. i had however to use the development version of the Core, somehow related to the way "obs4MIPs" should be written.
I'm actually already working on new figures, e.g. to visualize the "tape recorder", but this should go in an own PR later.)

@axel-lauer
Copy link
Contributor

axel-lauer commented Jan 13, 2022

I successfully tested this recipe with the following models:

  • CMIP6: BCC-CSM2-MR, IPSL-CM6A-LR, MIROC6, MRI-ESM2-0
  • CMIP5: GISS-E2-R, inmcm4, MIROC-ESM, MPI-ESM-LR

The results look fine to me. For all other CMIP5 models, fx files for abrupt4xCO2 were missing on Mistral (I couldn't get the automatic download to work).

@katjaweigel if you are fine with this, I propose to merge this PR now.

@katjaweigel
Copy link
Contributor Author

@axel-lauer thanks for testing! I think you can merge it (I'll make an update with additional figures later).

@axel-lauer axel-lauer merged commit 9cbe4f1 into main Jan 13, 2022
@axel-lauer axel-lauer deleted the CMUG_ESACCI_H2O branch January 13, 2022 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Recipe for ESA CMUG H2O including new diagnostics for cold point tropopause
4 participants