Make atmos_prep job work using GFSv17 and new COM structure #119
Conversation
On branch feature/v13_atmos_prep modified: rocoto/parm/setbase modified: sorc/link_gefs.sh new file: versions/fix.ver Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: rocoto/bin/wcoss2/atmos_prep.sh modified: rocoto/bin/wcoss2/common.sh modified: rocoto/py/GEFS_XML_For_Tasks.py modified: scripts/exgefs_atmos_prep.sh modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/parm/setbase Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: fix/.gitignore modified: scripts/.gitignore modified: sorc/.gitignore modified: ush/.gitignore Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: sorc/link_gefs.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep renamed: parm/gefs.parm -> parm/parm_gefs/gefs.parm renamed: parm/gefs_atmos_prep.parm -> parm/parm_gefs/gefs_atmos_prep.parm Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: parm/parm_gefs/gefs.parm modified: parm/parm_gefs/gefs_atmos_prep.parm Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: rocoto/user_full.conf modified: ush/gefs_atmos_prep.sh modified: versions/run.ver Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP new file: parm/config/config.com modified: parm/parm_gefs/gefs.parm modified: parm/parm_gefs/gefs_atmos_prep.parm modified: scripts/exgefs_atmos_prep.sh modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/bin/hera/atmos_prep.sh modified: rocoto/bin/hera/common.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/user_full.conf Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/dev/versions/run_hera.ver modified: rocoto/parm/gefs_config Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/py/GEFS_Crontab.py modified: rocoto/py/GEFS_Parm.py modified: rocoto/py/GEFS_UserConfig.py modified: rocoto/py/GEFS_XML.py modified: rocoto/py/GEFS_XML_For_Tasks.py modified: rocoto/py/add_crontab.py modified: rocoto/py/run_pyGEFS.py Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: sorc/link_gefs.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/py/GEFS_XML.py modified: rocoto/py/GEFS_XML_For_Tasks.py modified: rocoto/py/user_hera.conf Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/bin/hera/atmos_prep.sh modified: rocoto/bin/hera/common.sh modified: rocoto/compile_install_all.sh modified: rocoto/dev/versions/run_hera.ver modified: rocoto/py/GEFS_XML.py Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: parm/parm_gefs/gefs.parm modified: rocoto/bin/hera/atmos_prep.sh modified: rocoto/parm/gefs_config modified: rocoto/parm/setbase modified: rocoto/py/GEFS_XML_For_Tasks.py Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/parm/gefs_config Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep deleted: rocoto/bin/gefs_load_modules.inc deleted: rocoto/bin/gefs_pre_job.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/compile_install_all.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/compile_install_all.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/compile_install_all.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/bin/hera/common.sh modified: rocoto/bin/wcoss2/common.sh Refs: NOAA-EMC#87
There was a problem hiding this comment.
This probably should've been a few different PRs. (at the very least, changes to use os.path.join() should've been separate). A bit hypocritical of me given the COM refactor PR, but I do usually try to keep PRs to one thing at a time.
Outside the scope of this PR, but we definitely have too much machine-dependent code all over the place.
parm/parm_gefs/gefs_atmos_prep.parm
Outdated
| #export pdycycp=$($NDATE -$fhrp $PDY$cyc) | ||
| #export pdyp=$(echo $pdycycp|cut -c1-8) | ||
| #export cycp=$(echo $pdycycp|cut -c9-10) |
There was a problem hiding this comment.
Go ahead and delete these
rocoto/bin/hera/atmos_prep.sh
Outdated
| . $GEFS_ROCOTO/dev/versions/run_hera.ver | ||
|
|
||
| # Load modules | ||
| module list |
There was a problem hiding this comment.
Not much point in listing right before you purge/reset
There was a problem hiding this comment.
The original design is to list the modules before the purge to double check whether the job can load module properly. Yes, it can be deleted.
rocoto/bin/hera/atmos_prep.sh
Outdated
|
|
||
| # Load modules | ||
| module list | ||
| module purge |
There was a problem hiding this comment.
You should use module reset rather than module purge.
There was a problem hiding this comment.
No problem. I will check what is the difference between reset and purge since I never use reset before
There was a problem hiding this comment.
NCO wants us to use reset since the move to WCOSS2. I'm surprised they didn't ask you to switch for the port (they did for GFS). It restores the base lmod environment instead of unloading all modules, but also drops any additional module use you may have added locally. On dev machines, you need to LMOD_SYSTEM_DEFAULT_MODULES='contrib' first because the admins haven't set it yet. You can look and see how workflow does it: https://github.com/NOAA-EMC/gfs-utils/blob/develop/ush/module-setup.sh
Really, you should probably be using that script, just like GW does. You'll already be getting it as part of the GW checkout.
There was a problem hiding this comment.
No problem. I will use it in another PR.
ush/gefs_atmos_prep.sh
Outdated
| export ATM_FILES_INPUT="enkfgfs.t${cyc}z.ratmanl.nc" | ||
| ATMFILE="${COMINenkfgfs}/$memchar/atmos/${ATM_FILES_INPUT}" #gfs.t${cycp}z.atmanl.nc" |
There was a problem hiding this comment.
See above. Also, ratmanl.nc does not appear to be an output of the GFS system. Only see enkfgfs.t00z.ratmi003.nc, enkfgfs.t00z.ratmi009.nc, and enkfgfs.t00z.ratminc.nc in my trees. Am I missing a file and need to hunt down an error?
There was a problem hiding this comment.
When DOIAU=no. Since GEFS will not use IAU
There was a problem hiding this comment.
But what about enkfgfs? That's where the problem lies.
There was a problem hiding this comment.
When set DOIAU=NO, then enkfgfs will generate ratmanl.nc. This function was added by Travis.
ush/gefs_atmos_prep.sh
Outdated
| export SFC_FILES_INPUT="gfs.t${cyc}z.sfcanl.nc" | ||
| SFCFILE="${COMINgfs}/atmos/${SFC_FILES_INPUT}" |
ush/gefs_atmos_prep.sh
Outdated
| YMD=${PDY} HH=${cyc} MEMDIR=${mem2} generate_com -x COM_ATMOS_INPUT | ||
| COMDIR2=${COM_ATMOS_INPUT} #${echo "${!COM_ATMOS_INPUT_TMPL}"} # $(echo "${!COM_ATMOS_INPUT_TMPL}" | envsubst) #${COM_ATMOS_INPUT} #$COMOUT/${mem2}/atmos/INPUT #init/$mem2 |
There was a problem hiding this comment.
| YMD=${PDY} HH=${cyc} MEMDIR=${mem2} generate_com -x COM_ATMOS_INPUT | |
| COMDIR2=${COM_ATMOS_INPUT} #${echo "${!COM_ATMOS_INPUT_TMPL}"} # $(echo "${!COM_ATMOS_INPUT_TMPL}" | envsubst) #${COM_ATMOS_INPUT} #$COMOUT/${mem2}/atmos/INPUT #init/$mem2 | |
| YMD=${PDY} HH=${cyc} MEMDIR=${mem2} generate_com COMDIR2:COM_ATMOS_INPUT_TMPL |
(It is only used locally, no need to export.)
Thanks Walter. Yes, I agree, at least |
On branch feature/v13_atmos_prep modified: parm/parm_gefs/gefs_atmos_prep.parm modified: rocoto/bin/wcoss2/atmos_prep.sh modified: rocoto/bin/wcoss2/common.sh modified: rocoto/py/GEFS_Crontab.py modified: rocoto/py/GEFS_Parm.py modified: rocoto/py/GEFS_UserConfig.py modified: rocoto/py/run_pyGEFS.py modified: scripts/exgefs_atmos_prep.sh Refs: NOAA-EMC#87
…FS into feature/v13_atmos_prep
On branch feature/v13_atmos_prep modified: sorc/link_gefs.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep deleted: parm/config/config.com modified: sorc/link_gefs.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: parm/.gitignore modified: sorc/link_gefs.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/py/GEFS_Crontab.py modified: rocoto/py/GEFS_Parm.py modified: rocoto/py/GEFS_UserConfig.py modified: rocoto/py/GEFS_XML.py modified: rocoto/py/GEFS_XML_For_Tasks.py modified: rocoto/py/add_crontab.py modified: rocoto/py/run_pyGEFS.py Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/py/GEFS_XML.py Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/compile_install_all.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: parm/parm_gefs/gefs.parm RefsP NOAA-EMC#87
On branch feature/v13_atmos_prep modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: rocoto/bin/hera/atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: jobs/JGEFS_ATMOS_PREP modified: rocoto/bin/hera/atmos_prep.sh modified: ush/gefs_atmos_prep.sh Refs: NOAA-EMC#87
On branch feature/v13_atmos_prep modified: sorc/link_gefs.sh deleted: versions/fix.ver Refs: NOAA-EMC#87
|
@WalterKolczynski-NOAA I have modified and could you please review my changes? Thanks. |
On branch feature/v13_atmos_prep modified: rocoto/bin/hera/atmos_prep.sh Refs: NOAA-EMC#87
Since the rsfcanl.nc is not variable in early cycle enkf, so I copy the c00 data to all ensemble members, I will do the change after rsfcanl.nc is available.
For ensemble member path, I still use c00, p01, ..., p30 for this time, I will open another issue to change the member path from c00, p01, .. to mem000, mem001, ... after all the atm workflow is ready.
The job works on both WCOSS2 and HARA.