Extended ESRL cmorizer

[bettina-gier]

Required for #1915

---

Checklist for technical review

• Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
• The pull request has a descriptive title that can be used as a one line summary in the changelog
• The code is composed of functions of no more than 50 lines and uses meaningful names for variables and follows the style guide
• Documentation is available
• Please use yamllint to check that your YAML files do not contain mistakes
• (Only if really necessary) Add any additional dependencies needed for the diagnostic script to setup.py, esmvaltool/install/R/r_requirements.txt or esmvaltool/install/Julia/Project.toml (depending on the language of your script) and also to package/meta.yaml for conda dependencies (includes Python and others, but not R/Julia). Also check that the license of the dependency you want to add and any of its dependencies are compatible with Apache2.0.

New or updated data reformatting script:

• Add a new dataset to the table in the documentation
• Add a test for the CMORized data to recipes/example/recipe_check_obs.yml and run the recipe, to make sure the CMOR checks pass without errors
• There are clear instructions on how to obtain the data
• Tag @remi-kazeroni in this pull request, so that the new dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

Automated checks pass, status can be seen below the pull request:

• Circle/CI tests pass. If the tests are failing, click the Details link to find out why.
• Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
• The documentation is building successfully on readthedocs and looks well formatted, click the Details link to see it.

Checklist for scientific review

New or updated data reformatting script:

• The numbers and units of the data look physically meaningful

---

If you need help with any of the items on the checklists above, please do not hesitate to ask by commenting in the issue or pull request.

Old NCL ESRL cmorizer only worked for baseline observatories. New cmorizer rewritten in python, includes capability to process surface flask monthly data, as well as global mean data from ESRL website. Additionally writes altitude to the netcdf files, gets this information from the ESRL website table, no external file required. Kept the "OBS" datatable so recipes using data from the old cmorizer will not need to be changed. Will download data if not present in RAW folder.
Changed recipe_check_obs.yml to include one of each case. No change to documentation necessary as ESRL cmorizer already noted there.
Required for #1915

[bettina-gier]

@axel-lauer Wanna do me the honor of a review, since you looked at Sabrinas original implementation for just the main station data? Remaining codacy error is the use of unsecure ftp for data download, copied this from the CT2019 cmorizer script. Feel free to change if you have a better idea.

[axel-lauer]

Is there a reason why the new CMORizer only produces only ASK, GLOBAL and MLO? What happened to BRW, SMO, SPO?

[bettina-gier]

I just put them there as example, the cmorizer can handle all base stations (BRW, SMO, SPO, MLO), the global average from their website, as well as all the single flask sites in the ESRL network (https://www.esrl.noaa.gov/gmd/dv/site/?program=ccgg). So I used one station for each example. I wasn't sure if it should take all the possible stations as default input.

[axel-lauer]

The old CMORizer would output BRW, MLO, SMO, SPO. I would therefore suggest to additionally output BRW, SMO, SPO with the new version.

[bettina-gier]

I've added support for an "all" keyword put in as default, which will process all available datasets - so this also includes brw, spo, mlo and smo -, and added a flag for downloading additional files, in case someone only wants to format all datasets locally available.
Also extended comment section in the yml file to better explain what files are supported.

I hope that addresses your concern sufficiently!

[axel-lauer]

This looks good. I tested the CMORizer and it does exactly what it is supposed to do. The only thing left to do would be to try to fix (some of) the Codacy issues. @bettina-gier could you please take a look? Other than that, this is ready for merging.

[bettina-gier]

Thanks for testing Axel!
Fixed all I could, only remaining one is the "insecure" use of the ftp package, but that is already merged with the CT2019 cmorizer so doesn't introduce anything new.

[axel-lauer]

Thanks for addressing the Codacy issues.