Added monthly mean vars (ta, va, zg) to era5 cmorizer via recipe

[egalytska]

Tasks

• Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
• Give this pull request a descriptive title that can be used as a one line summary in a changelog
• Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
• Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
• Please use yamllint to check that your YAML files do not contain mistakes

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.

---

Closes #issue_number

I would like to comment on a few issues:

1. In order to cmorize geopotential height (zg) I had to edit ESMValCore/esmvalcore/cmor/_fixes/native6/era5.py to convert "geopotential" to "geopotential height". Should I create a new request in ESMValCore to add those couple lines?
2. After additional ERA5 data was moved to native6, seems that 2 variables "temperature" (ta) and "temperature_of_snow_layer" (tsn) are now found simultaneously by recipe_era5.yml when given era5_name: temperature. Maybe, it worth renaming "temperature" to "t" or so?

[bouweandela]

I have created an issue with era5cli to see if the filenames it generates can be changed to look more like the filenames used with other climate data: eWaterCycle/era5cli#53

[bouweandela]

Should I create a new request in ESMValCore to add those couple lines?

Yes

[bouweandela]

@valeriupredoi Would you be able to test this?

[mattiarighi]

The recipes fails with several variables, not only with the new ones.
The error looks always the same:

2020-05-18 18:12:56,169 UTC [48027] INFO    Starting task monthly/zg in process [48027]
2020-05-18 18:12:56,213 UTC [48028] ERROR   Failed to run fix_metadata([<iris 'Cube' of air_temperature / (K) (time: 12; pressure_level: 37; latitude: 721; longitude: 1440)>, <iris 'Cube' of temperature_in_surface_snow / (K) (time: 12; latitude: 721; longitude: 1440)>, <iris 'Cube' of air_temperature / (K) (time: 12; pressure_level: 37; latitude: 721; longitude: 1440)>, <iris 'Cube' of temperature_in_surface_snow / (K) (time: 12; latitude: 721; longitude: 1440)>], {'project': 'native6', 'dataset': 'ERA5', 'short_name': 'ta', 'mip': 'Amon', 'frequency': 'mon', 'check_level': <CheckLevels.DEFAULT: 3>})
2020-05-18 18:12:56,268 UTC [47969] INFO    Progress: 8 tasks running, 1 tasks waiting for ancestors, 8/17 done
2020-05-18 18:12:56,315 UTC [47969] INFO    Maximum memory used (estimate): 4.5 GB
2020-05-18 18:12:56,316 UTC [47969] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2020-05-18 18:12:56,318 UTC [47969] ERROR   Program terminated abnormally, see stack trace below for more information
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 693, in _run_task
    output_files = task.run()
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 240, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 426, in _run
    product.apply(step, self.debug)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 294, in apply
    self.cubes = preprocess(self.cubes, step, **self.settings[step])
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 236, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 222, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/fix.py", line 107, in fix_metadata
    cube_list = fix.fix_metadata(cube_list)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/_fixes/native6/era5.py", line 325, in fix_metadata
    cube = self._fix_coordinates(cube)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/_fixes/native6/era5.py", line 277, in _fix_coordinates
    coord = cube.coord(axis=axis)
  File "/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/cube.py", line 1497, in coord
    raise iris.exceptions.CoordinateNotFoundError(msg)
iris.exceptions.CoordinateNotFoundError: 'Expected to find exactly 1  coordinate, but found none.'

[bouweandela]

The recipes fails with several variables, not only with the new ones.

This is exactly the error reported under point 2 at the top of this pull request. Maybe you can try with the new default DRS proposed here? ESMValGroup/ESMValCore#645

[mattiarighi]

That does not help. I think the problem is the use of _ as a separator in the variable name, it should be changed to - in era5cli.

However, I managed to test thir PR by running each variable separately and moving out the non required files from the input directory.

Since the problem is not related to this PR, we can merge it. I also tested ESMValGroup/ESMValCore#640 in this branch.

@egalytska can you please open an issue about the Point 2. above (if not already done)?

[bouweandela]

That does not help. I think the problem is the use of _ as a separator in the variable name, it should be changed to - in era5cli.

That is exactly the kind of problem ESMValGroup/ESMValCore#645 tries to solve. Why did it not work? Maybe we can discuss further in that pull request?

[egalytska]

Dear all. Just an update on this issue:
(i) By the time I downloaded ERA5 data I did not know I could do it via era5cli. So I used CDS API.
(ii) Raw ERA5 files I named as era5_temperature_monthly.nc . After data was moved to ~work/bd0854/DATA/ESMValTool2/RAWOBS/ my recipe started giving an error. That's when I realized that for e.g. ERA-Interim raw data is named as ERA-Interim_t_monthly.nc. So if I would name ERA5 raw data as era5_t_*_monthly.nc, I assume there would be no problem at all. (I consider it my mistake).
(iii) Sorry, it takes me longer than it should. I am still somewhat learning.