Wrong values in vertices_longitude of NEMO/LIM variables

[uwefladrich]

The cmorised NEMO/LIM variables have an auxiliary variable vertices_longitude in the files and there seems to be a problem with it. Below is a plot of corner 1 longitudes for an arbitrary NEMO variable, taken from the cmorised file:

There is a strange line of low(~0)/high(~360) values where the lon values pass 180 degrees. This feature is present for all four vertices and (not proven but very likely) for all NEMO/LIM variables, at least for ORCA1.
For comparison, this is what the vertices longitudes look like in the original NEMO output:

So it appears that the line coincides with the transition from -180 to 180 degrees in the original values.

[goord]

Hmm yes the line consists of wrong values of nearly 360 degrees, I'm looking into it. This will mean another round with the cmor-fixer I guess...

[goord]

ece2cmor3 calculates the vertex longitudes as midpoints of gridpoint longitudes since there used to be no nav_lon_bounds yet. The line originates from the crossing from 180 to -180 degrees where the naive midpoint calculation fails.

I guess the best way is to use nav_lat/lon_bounds in ece2cmor3 from now on and modify published data with the correct values

[etiennesky]

Not sure if this is relevant but I noticed that the lons and lats variable in the cmorized files are not exactly equal to the nav_lon and nav_lat variables in the nemo files...

This might in turn have an impact on the vertices?

lon_lat_cmor.txt.gz
lon_lat_orig.txt.gz

[klauswyser]

ece2cmor3 calculates the vertex longitudes as midpoints of gridpoint longitudes since there used to be no nav_lon_bounds yet.

Not sure I understand this, each NEMO output file contains variables bounds_nav_lon/lat in addition to nav_lon/lat, so there would be no need to compute anything in ece2cmor3, or?

[goord]

Yes but I believe historically this was not the case, so I agree we should use those now that they're available

[treerink]

Not sure I understand this, each NEMO output file contains variables bounds_nav_lon/lat in addition to nav_lon/lat, so there would be no need to compute anything in ece2cmor3, or?

I have been looking in recent test data, i.e. raw XIOS-NEMO output and I can't find any bounds_nav_lon or something similar when looking with ncdump? This is what I find (in for instance t004_1d_19910101_19911231_lim_grid_U_2D) :

	float nav_lon(y, x) ;
		nav_lon:standard_name = "longitude" ;
		nav_lon:long_name = "Longitude" ;
		nav_lon:units = "degrees_east" ;

No more attributes.

[treerink]

An approach could be that, if someone has the correct files for bounds_nav_lon that we upload them as small netcdf files to the ece2cmor and cmor-fixer repositories. But we expect that we need different ones for the different staggered grids: For temperature, u, v. Maybe w is for the bounds_nav_lon equal to the temperature one?

Then ec2cmor3 and cmor-fixer can read in these small netcdf and use them to provide or correct the vertices_longitue and vertices_latitude. Note that at cmorisation ece2cmor3 knows the grid because the NEMO file name contains this info, for cmor-fixer this info is absent, and needs another measure.

[treerink]

@klauswyser provided me a NEMO output file which has the bounds_nav_lon included for the T-grid. In all our KNMI raw NEMO output even back to December 2018 we don't have this, it is simply not there as an attribute of nav_lon. Also in recent SMHI files it isn't there anymore.

I will try whether I can easily reproduce the NEMO bounds_nav_lon in ece2cmor3 and also check whether I can see a difference in the XIOS controling files.

[treerink]

See cmor-fixer issue 27 and ec-earth portal issue 808-3.

[treerink]

The NEMO vertices can be fixed with the cmor-fixer v3.0 release.

[treerink]

The fix-vertices branch is created.

Reading the vertices files and shifting the longitude vertices to the 0-360 degree range, as in cmor-fixer is added in this branch. Note place of code and sys.exit might need a review.

[treerink]

Though the regular cases seem to work now in the fix-vertices branch, ece2cmor3will exit for such cases:

The file has horizonatal grid dimensions: (292, 1) which are not supported because they differ from ORCA1 or ORCA025.

where vertices are tried to add for a file which has a non 2D horizontal grid.

Another point, these are covered:

'T_2D', 'T_3D', 'W_2D', 'W_3D'
'U_2D', 'U_3D'
'V_2D', 'V_3D'

Testing has to prove that this coveres all 2D/3D situations. Obviously the 1D case above is also not covered at this point yet.

[treerink]

With the latest 4309a80 commit, the 1D & 0D cases are now covered with the earlier method.

However, in the test-all (#542) test, I have another problematic case: _sum :

CRITICAL:ece2cmor3.nemo2cmor: The grid type _sum has not been implemented yet.

Another problem I encountered on the HPC with the batch sysytem is that the relative path to the nemo-vertices fails. So as long you cmorice locally from your ece2cmor root directory it goes fine. But when submitting the job on the HPC I had to give a hack like this in my case in the __load_nemo_vertices__.py file:

def load_vertices(vertices_file_name):
    # Loading once at the start the NEMO longitude and latitude vertices from a netcdf file:
   #nemo_vertices_file_name=os.path.join("ece2cmor3/resources/nemo-vertices/", vertices_file_name)
    nemo_vertices_file_name=os.path.join("/nfs/home/users/reerink/cmorize/ece2cmor3/ece2cmor3/resources/nemo-vertices/", vertices_file_name)

[treerink]

The above mentioned _sum case and another tice case are catched, thus solved now.

[treerink]

Another problem I encountered on the HPC with the batch sysytem is that the relative path to the nemo-vertices fails. So as long you cmorice locally from your ece2cmor root directory it goes fine. But when submitting the job on the HPC I had to give a hack like this in my case in the __load_nemo_vertices__.py file:

def load_vertices(vertices_file_name):
    # Loading once at the start the NEMO longitude and latitude vertices from a netcdf file:
   #nemo_vertices_file_name=os.path.join("ece2cmor3/resources/nemo-vertices/", vertices_file_name)
    nemo_vertices_file_name=os.path.join("/nfs/home/users/reerink/cmorize/ece2cmor3/ece2cmor3/resources/nemo-vertices/", vertices_file_name)

This path need to be organised in a neat way. @goord can I leave that to you? And please review this branch. But I think for the rest it is ready for merge into master now.

[goord]

Yes I will fix the path. Usually I take the path of the current source file to get to the resources directory.

[goord]

Paths have been fixed, files are automatically downloaded from b2share if necessary. Closing this issue.