-
Notifications
You must be signed in to change notification settings - Fork 68
Do Archive
Guoqing Ge edited this page May 26, 2026
·
4 revisions
User can set DO_ARCHIVE=true the exp file to activate the archive task.
It can archive files under either com or stmp to HPSS, etc.
There are the following archive settings:
-
ARCHIVE_INTERVAL: how frequently to run the archive task, with options of every 1, 2, 3, 4, 6, 8, 12, or 24 hours -
ARCHIVE_HPSSDIR: the top directory in HPSS to save archived tarballs.
For example,ARCHIVE_HPSSDIR=5year/BMC/wrfruc/Guoqing.Ge/${EXP_NAME}and then the archive tarballs will save asYYYY/YYYYMM/YYYYMMDD/com1.YYYMMDDH1-H2.tarunder it. Make sure the workflow runtime account has the write access to this directory. -
ARCHIVE_COM1_SPEC: specify what directories/files to include/exclude the com1 tarball.
For example,ARCHIVE_COM1_SPEC='{"include":"ioda*, jedivar, nonvar*, fcst, upp, pyDAmonitor, graphics", "exclude":""}'. It accepts a Python dictionary value. Use"inside and'outside. For theincludekey, one may list the task names (supporting the*wildcard) to be archived and the archive script will append the correspondingWGF(det or enkf) automatically (so det and enkf will archive in separate workflows). One may list files as well, for example,jedivar/det/jedivar.yaml,fcst/det/mpasout.*, etc. For theexcludekey, one has to list down to underWGF, for example,upp/det/done.* -
ARCHIVE_STMP: specify what task's umbrella directory under STMP to include in the stmp tarball, example,ARCHIVE_STMP="fcst,mpassit" -
ARCHIVE_STMP_INTERVAL: how frequently to archive the STMP files.ARCHIVE_STMP_INTERVALmust divideARCHIVE_INTERVALevenly. STMP fcst files or mpassit files have larger disk footprint, so we may archive them more frequently.