Modify ERA5 fixes to support data produced by the new GRIB to netcdf converter version of ECMWF

[jlenh]

Description

ECMWF is now using a different netCDF converter for the ERA5 GRIB files available on the CDS and ADS systems. These changes notably involve modifications in the coordinates (notably time and latitude) that can lead to errors when using newly downloaded ERA5 data. The detailed description of the modifications can be found on the ECMWF Confluence page here.

This PR aims to tackle two identified issues:

• The time coordinate unit and calendar have been changed from "hours since 1900-01-01 00:00:00.0, gregorian" to "seconds since 1970-01-01, proleptic_gregorian" which leads to an error when converting the time unit in the ERA5 _fix_coordinates method.
  • This is solved by including the calendar in the target unit for the time conversion.
• The latitude coordinate metadata has been changed to include the stored_direction which leads to an error when trying to concatenate data files from the legacy and the new version because of differing metadata in the iris cubes.
  • This is solved by removing this attribute in cubes downloaded with the legacy version of the converter. If the direction of the coordinate is erroneous, it will be fixed by esmvalcore.cmor._fixes.fix._fix_coord_direction

Closes #3026

Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.70%. Comparing base (9e24b51) to head (36cc684).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3030   +/-   ##
=======================================
  Coverage   95.70%   95.70%           
=======================================
  Files         267      267           
  Lines       15768    15771    +3     
=======================================
+ Hits        15090    15093    +3     
  Misses        678      678           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[schlunma]

Are the differing calendars a problem when concatenating legacy data and new data?

[jlenh]

Are the differing calendars a problem when concatenating legacy data and new data?

It does not seem like it. The calendars are changed again when concatenating the cubes by _fix_calendars , and in the case of ERA5 data we are out of the date range where this could be a problem (if I understood these calendars correctly).

I have managed to run some diagnostics with ERA5 spanning 1980-2025 (2024 and 2025 being downloaded with the new version), and the preprocessed data looks fine e.g. for JJA temperature data:

DimCoord :  time / (days since 1850-1-1 00:00:00, standard calendar)
    points: [
        1980-07-17 00:00:00, 1981-07-17 00:00:00, ...,
        2024-07-17 00:00:00, 2025-07-17 00:00:00]
    bounds: [
        [1980-06-01 00:00:00, 1980-09-01 00:00:00],
        [1981-06-01 00:00:00, 1981-09-01 00:00:00],
        ...,
        [2024-06-01 00:00:00, 2024-09-01 00:00:00],
        [2025-06-01 00:00:00, 2025-09-01 00:00:00]]
    shape: (46,)  bounds(46, 2)
    dtype: float64
    standard_name: 'time'
    long_name: 'time'
    var_name: 'time'

[schlunma]

Ah, that makes sense! I wasn't aware that our concantenation can change calendars!

Looks good to me, just one tiny comment.

[schlunma]

Thanks Julien 🚀