Skip to content

cloud-radiative-effects diagnostic fails to find reference data when used with CMIP7 simulations #653

@bouweandela

Description

@bouweandela

Running

ref test-cases run --provider esmvaltool --diagnostic cloud-radiative-effects --test-case cmip7 --fetch --force-regen --clean

fails with the following error message:

...
using data sources:
- data source: LocalDataSource(name='local', project='OBS', priority=1, rootpath=PosixPath('/home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS'), dirname_template='Tier{tier}/{dataset}', filename_template='{project}_{dataset}_{type}_{version}_{mip}_{short_name}[_.]*nc', ignore_warnings=[])
  message: No files found matching glob pattern /home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS/Tier2/ISCCP-FH/OBS_ISCCP-FH_sat_v0_atmos_rsutcs[_.]*nc
2026-05-04 09:12:47,813 UTC [37859] ERROR   Set 'log_level' to 'debug' to get more information
2026-05-04 09:12:47,813 UTC [37859] ERROR   Could not create all tasks
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_maps/lwcre:
- Missing data for Dataset: rlut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlutcs, atmos, OBS, CERES-EBAF, , Ed4.2
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_maps/swcre:
- Missing data for Dataset: rsut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsutcs, atmos, OBS, CERES-EBAF, , Ed4.2
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_profiles/lwcre:
- Missing data for Dataset: rlut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlut, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rlut, atmos, OBS, ISCCP-FH, , v0
- Missing data for Dataset: rlutcs, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlutcs, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rlutcs, atmos, OBS, ISCCP-FH, , v0
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_profiles/swcre:
- Missing data for Dataset: rsut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsut, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rsut, atmos, OBS, ISCCP-FH, , v0
- Missing data for Dataset: rsutcs, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsutcs, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rsutcs, atmos, OBS, ISCCP-FH, , v0
...

because mip is set to atmos for the CMIP5 style OBS data, while it should be Amon to match the filenames, e.g.:

/home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS/Tier2/ISCCP-FH/OBS_ISCCP-FH_sat_v0_Amon_rsutcs_198401-201612.nc

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions