New MEPS Archive 2023

IMPORTANT - NEW ARCHIVE REGIME FROM 30. SEPTEMBER

The updated MEPS archive structure is as follows:
Each day we produce a set of NetCDF files per member, level type and time step. The easiest way to access these is through ncml files, e.g. for https://thredds.met.no/thredds/catalog/meps25epsarchive/2023/11/01/catalog.html:
For each control cycle (hh=00,03,..,21):

• meps_det_sfc_20231101ThhZ.ncml -> Surface parameters
• meps_det_pl_20231101ThhZ.ncml -> Pressure level parameters
• meps_det_ml_20231101ThhZ.ncml -> Model level parameters
• meps_det_hl_20231101ThhZ.ncml -> Height level parameters
  
  These point to files under subfolders, e.g.
• meps_det_sfc_20231101T00Z.ncml -> 00/member_00/meps_sfc_*_20231101T00Z.nc
  Thus, you have available files for each time step as NetCDF files, if you prefer to use those directly instead of the ncml files.
  
  For each perturbed member (hh=00,01,02,..,23, ensemble_member=0,1,2,..14) you will find ncml files under each respective folder, i.e. for member 13 which is produced on cycle 01:
• 01/meps_mbr013_sfc_20231101T01Z.ncml -> 01/member_13/meps_sfc_*_20231101T00Z.nc
  Note that we archive much fewer parameters on perturbed members than on the control member. For instance there are no model level parameters on perturbed members.

Cleaning strategy

The cleaning strategy will be revised in 2024.
The following old archive files will still be produced, but will be removed after 30 days:

• meps_det_2_5km_yyyymmddThhZ.nc
  
  This file will NO LONGER BE PRODUCED after September 2024:
• meps_lagged_6_h_subset_2_5km_yyyymmddThhZ.nc
  
  

We currently plan to keep the following after 16 months:

• det_sfc: Keep everything
• det_pl, det_hl: Keep 00,06,12,18z cycles
• det_ml: Keep only 00z cycle
• mbr_sfc, mbr_pl: There will be reduction of members archived, details not yet decided.

Background for change

Due to limitations of scalability in the current archive strategy of MEPS, we need to change how archiving is done.
We want to enforce the new regime for files produced after September 30th 2023. We strongly encourage users to adapt to this regime as soon as possible.
Please note that this is only affects the MEPS Archive/. The production folder MEPS Latest/ stays unchanged.

• No longer archiving meps_det_*.nc and meps_subset_*.nc NetCDF-files
• Instead archive one NetCDF file per ensemble member, time step and level type
• Ncml-files available per cycle, member and level type
• No production of lagged-ensemble files, users must construct these themselves
• Transition period: meps_det_*.nc and meps_subset_*.nc NetCDF files will be available for 30 days before deletion. This will be revised in 2024
• It is not yet decided whether data before 30. Sept 2023 will be converted to the new archive regime, we might keep it as is.

Users of meps_det_*.nc should use:

• meps_det_sfc_*.ncml for surface parameters
• meps_det_hl_*.ncml for height level parameters
• meps_det_pl_*.ncml for pressure level parameters
• meps_det_ml_*.ncml for model level parameters

If you want ensemble data you must construct this yourselves.