Skip to content
Guoqing Ge edited this page May 26, 2026 · 4 revisions

User can set DO_ARCHIVE=true the exp file to activate the archive task.
It can archive files under either com or stmp to HPSS, etc.

There are the following archive settings:

  1. ARCHIVE_INTERVAL: how frequently to run the archive task, with options of every 1, 2, 3, 4, 6, 8, 12, or 24 hours
  2. ARCHIVE_HPSSDIR: the top directory in HPSS to save archived tarballs.
    For example, ARCHIVE_HPSSDIR=5year/BMC/wrfruc/Guoqing.Ge/${EXP_NAME} and then the archive tarballs will save as YYYY/YYYYMM/YYYYMMDD/com1.YYYMMDDH1-H2.tar under it. Make sure the workflow runtime account has the write access to this directory.
  3. ARCHIVE_COM1_SPEC: specify what directories/files to include/exclude the com1 tarball.
    For example, ARCHIVE_COM1_SPEC='{"include":"ioda*, jedivar, nonvar*, fcst, upp, pyDAmonitor, graphics", "exclude":""}'. It accepts a Python dictionary value. Use " inside and ' outside. For the include key, one may list the task names (supporting the * wildcard) to be archived and the archive script will append the corresponding WGF (det or enkf) automatically (so det and enkf will archive in separate workflows). One may list files as well, for example, jedivar/det/jedivar.yaml, fcst/det/mpasout.*, etc. For the exclude key, one has to list down to under WGF, for example, upp/det/done.*
  4. ARCHIVE_STMP: specify what task's umbrella directory under STMP to include in the stmp tarball, example, ARCHIVE_STMP="fcst,mpassit"
  5. ARCHIVE_STMP_INTERVAL: how frequently to archive the STMP files. ARCHIVE_STMP_INTERVAL must divide ARCHIVE_INTERVAL evenly. STMP fcst files or mpassit files have larger disk footprint, so we may archive them more frequently.

Clone this wiki locally