From a329e103c4f9bb0048da46e1cd0cd56a1aa2e3ee Mon Sep 17 00:00:00 2001 From: JianKuang-NOAA <51758200+JianKuang-NOAA@users.noreply.github.com> Date: Mon, 20 Jul 2020 15:40:50 -0400 Subject: [PATCH] wave configuration update (#81) * Pointing to v3.1 tag for ufs-s2s-model making readme consistant with user.yaml.default * bug fix, POSTGRB2TBL undefined * Squashing commits to wave2global * Fix to JWAVE_PREP to look back a day for rtofs * Moving standalone fv3 model_config exglobal_fcst block into if/else/fi cplwav model_config block. Reinstating config.wave block in JGLOBAL_FORECAST. Pointing EXECwave to HOMEgfs/exec directory for WW3 util executables (changed link_fv3gfs.sh accordingly). Removing debug options from compile.sh line in build_fv3.sh. * Clean up exwave_post_sbs * clean up unused schema move default value into defaults/ * move all default values into /defaults from /schema It's better if we have one place to manage default values for configurable variables other than two Default values only for reference. Please throughly test to make sure it could reproduce. * model_configure_DATM * merge in the latest revision from IPD work * add stoch variables to config.fcst scripts update: CCPP and IPD works * capital case CDUMP values in config.fv3 * stochy variable namelist update, add hard-coded variables into config.fcst (fcst.yaml) * OCNTIM and ICETIM are included in config.fv3 ice variable istep0 bug fix * stochastic variables update * delete falseful copy after namelist and model_configure parsing add FNMSKH conditional clause, coupled vs non-coupled * update on sandbox platform, for FIX_SCRUB settings * turn off stochastic features in case file * configuration update for CCPP scripts * generalize fix file directory for imp_physics 8 * reiflag default value changed to 1 * port to Orion * remove HERA-like paths in platform.yaml of Orion * checkout and External.cfg update * update checkout and External.cfg * case file adjustment for Orion IC * expand gitignore * automate fix file linking, to this location: /work/noaa/marine/jmeixner/tempFixICdir/fix/fix_prep_benchmark3 * updates to checkout, build and link to run prototype 4 * update UPP version to v1.0.8 * making Externals_coupled.cfg consistent with checkout.sh * removing fix from .gitignore * fixed error when linking on hera * replace MOM6IC with BASE_CPLIC for base directory location of IC for s2s model and made it platform dependent. (Orion location is temporary and needs to be replaced, can also be changed in cases/coupled_free_forecast.yaml to users prefered location) * updates for copying ICs * more updates for ICs * updates for new fix (which includes prep for multiple resolutions) and some clean-up * renaming exe to reflect updated build * update diag_table for ocean * updates for ice_in to match prototype 4 * updates for CCPP suite used in p4 * making a fix dir of all available ccpp suite files copying suite xml file to rundirectory * updating for reorganized tar files, pulling in changes from wcoss-p2 workflow * last of archive updates for p4 * updating module files to match s2s model and ncep post modules * we don't want to cycle/have 4 runs per day so setting the cases to be just 1 IC * updating module files on hera * updates to readme for orion * mediator cold start succeeds on hera and forecast job starts post jobs for atmosphere are no longer triggered on cold starts * adding ncks, updating ocean post, some fixes for tar files * updates for tar files and do not keep data * fix typo * updating module_base.orion that allows cold start to run. Not sure if other jobs will now fail though or if what is in your .bashrc file will effect the success/failure of the run. * both forecast and atm post seem to be running okay on orion * wave configuration files * wave configuration update * fix module-setup.sh.inc script, which causes module purge unsuccessful * 1, Moved some of the variables into configuration system from JWAVE scripts 2, Wave configuration system built out 3, Modularized scripts updated to accomodate wave variables * Fix suite name of CCPP Move cplwav and cplwav2atm variable * Revise CCPP_SUITE variable name, default option, to "IPD" * wave post-process job update * reg2grb2 for test on feature coupled-crow * orion hera build test * reg2grb2 Build on Hera and Orion * Merged and ready for test * add ocn-ice build to readme * adding updates to wave scripts from gfsv16b branch by @ajhenrique * cleaning up double cplwav * changes to wave case * bugfix in layout file * bug fix in layout file * move OCNPETS, ICEPETS and WAVEPETS into dedicated model-specific sessions instead of fv3_gfs_settings * first try for adding wave ICs * fix bugs in config.wave and coupled_ic.sh * updates for wave defaults for s2s p4 * updates for running wave jobs * point WW3-related executables through HOMEgfs/fv3_coupled.fd/exec * ocean post hours at boundary are processed twice, now fixed. * wave init job now works on hera * adding missing file from last commit * add ocean, ice and wave configurable variables * add ocean, ice and wave configurable variables * put ic type variable into config.fv3ic * wave init now works on orion * updates to run wave prep for s2s case * updating forecast job for wave jobs * updates for forecast with waves * at this point the ocean post is now running * fix nems.configure for cold start * updated wave-realted scripts to match gfsv16 branch updated environment files env/ORION.env updated configuration system * S2s prototype4 (#2) * add waves to jglobal forecast config * updates - everything but wave post seems to run now, part of wave post does updating model to latest tag * change for lmod -> lmod/lmod on orion updated README * fixing syntax error in JWAVE_PREP job for variables that are not actively being used * adding cycle definition to wave post job * cleanup from duplications Co-authored-by: Jessica.Meixner Co-authored-by: Henrique Alves Co-authored-by: kate.friedman Co-authored-by: Jose-Henrique Alves <47567389+ajhenrique@users.noreply.github.com> Co-authored-by: Lin.Gan --- .gitignore | 2 + README.md | 10 +- env/HERA.env | 5 + env/ORION.env | 5 + jobs/JGLOBAL_FORECAST | 4 +- jobs/JGLOBAL_FORECAST_MEDCOLD | 4 +- jobs/JWAVE_INIT | 41 +- jobs/JWAVE_POST_SBS | 46 +- jobs/JWAVE_PREP | 50 +- jobs/rocoto/coupled_ic.sh | 27 +- jobs/rocoto/ocnpost.sh | 40 +- jobs/rocoto/waveinit.sh | 4 - modulefiles/module-setup.sh.inc | 6 +- modulefiles/module_base.orion | 7 + modulefiles/modulefile.reg2grb2.hera | 14 + modulefiles/modulefile.reg2grb2.orion | 14 + parm/config/config.base | 2 - parm/config/config.base.emc.dyn_coupled | 2 - scripts/exwave_init.sh | 19 +- scripts/exwave_post_sbs.sh | 197 +-- scripts/exwave_prep.sh | 268 +-- scripts/run_reg2grb2.sh | 71 + scripts/run_regrid.sh | 24 + sorc/build_all.sh | 12 +- sorc/build_fv3_coupled.sh | 1 - sorc/build_reg2grb2.sh | 41 + sorc/checkout.sh | 2 +- sorc/fv3gfs_build.cfg | 1 + sorc/link_fv3gfs.sh | 17 +- sorc/partial_build.sh | 3 +- sorc/reg2grb2.fd/Makefile | 26 + sorc/reg2grb2.fd/reg2grb2.f | 185 +++ sorc/reg2grb2.fd/regdiag.f | 1431 +++++++++++++++++ ush/cplvalidate.sh | 2 + ush/forecast_postdet.sh | 72 +- ush/icepost.ncl | 382 +++++ ush/nems.configure.med_atm_ocn_ice_wav.IN | 4 +- ush/nems.configure.medcold_atm_ocn_ice_wav.IN | 4 +- ush/nems_configure.sh | 4 + ush/ocnpost.ncl | 587 +++++++ ush/parsing_namelists_FV3.sh | 10 +- ush/wave_grib2_sbs.sh | 18 +- ush/wave_grid_interp.sh | 8 +- ush/wave_grid_interp_sbs.sh | 8 +- ush/wave_outp_spec.sh | 9 +- ush/wave_prnc_cur.sh | 29 +- ush/wave_prnc_ice.sh | 4 +- ush/wave_tar.sh | 9 +- workflow/cases/coupled_free_forecast.yaml | 8 +- .../cases/coupled_free_forecast_wave.yaml | 60 + workflow/config/base.yaml | 14 +- workflow/config/fcst.yaml | 5 +- workflow/config/fv3ic.yaml | 4 + workflow/config/ocnpost.yaml | 1 + workflow/config/wave.yaml | 143 ++ workflow/config/waveprep.yaml | 33 + workflow/defaults/case.yaml | 8 + workflow/defaults/default_resources.yaml | 4 +- workflow/defaults/fv3_enkf.yaml | 2 +- workflow/defaults/fv3_gdas.yaml | 2 +- workflow/defaults/fv3_gfs.yaml | 5 +- workflow/defaults/ice.yaml | 2 + workflow/defaults/ocn.yaml | 2 + workflow/defaults/output_settings.yaml | 14 + workflow/defaults/places.yaml | 2 + workflow/defaults/settings.yaml | 2 + workflow/defaults/wave.yaml | 19 + workflow/layout/free_forecast_gfs.yaml | 44 +- workflow/schema/ice.yaml | 6 + workflow/schema/ocn.yaml | 6 + workflow/schema/wave.yaml | 12 + 71 files changed, 3671 insertions(+), 458 deletions(-) create mode 100644 modulefiles/modulefile.reg2grb2.hera create mode 100644 modulefiles/modulefile.reg2grb2.orion create mode 100755 scripts/run_reg2grb2.sh create mode 100755 scripts/run_regrid.sh create mode 100755 sorc/build_reg2grb2.sh create mode 100755 sorc/reg2grb2.fd/Makefile create mode 100755 sorc/reg2grb2.fd/reg2grb2.f create mode 100755 sorc/reg2grb2.fd/regdiag.f create mode 100644 ush/icepost.ncl create mode 100644 ush/ocnpost.ncl create mode 100644 workflow/cases/coupled_free_forecast_wave.yaml create mode 100644 workflow/config/wave.yaml create mode 100644 workflow/config/waveprep.yaml create mode 100644 workflow/defaults/wave.yaml create mode 100644 workflow/schema/wave.yaml diff --git a/.gitignore b/.gitignore index 7bf237f397..b56d24eb21 100644 --- a/.gitignore +++ b/.gitignore @@ -165,3 +165,5 @@ util/sub_wcoss_d workflow/user.yaml sorc/checkout-gldas.fd.log sorc/gldas.fd/ +sorc/reg2grb2.fd/*.o +sorc/reg2grb2.fd/*.mod diff --git a/README.md b/README.md index cb7589882b..e3a33c3086 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,9 @@ sh checkout.sh coupled # Check out the couple ``` ## Compile code used in ufs-s2s-model and EMC_post and link fixed files and executable programs: ``` -sh build_ncep_post.sh #This command will build ncep_post. +sh build_ncep_post.sh #This command will build ncep_post sh build_fv3_coupled.sh #This command will build ufs-s2s-model +sh build_reg2grb2.sh #This command will build exes for ocean-ice post To link fixed files and executable programs for the coupled application: On Hera: @@ -85,6 +86,13 @@ module load rocoto rocotorun -w workflow.xml -d workflow.db ``` +## Using crons on Orion + +On orion, you will need to update the path to rocotorun in the autogenerated workflow.crontab to be +/apps/contrib/rocoto/1.3.1/bin/rocotorun + +Also note, on orion you need to be on login node 1 for cronjobs. + ## Resource handling There are two ways of changing resource settings (cpu count, time limits, threads) for a job that has already been defined in the workflow. diff --git a/env/HERA.env b/env/HERA.env index 642fc33c4b..31a1e43fcc 100755 --- a/env/HERA.env +++ b/env/HERA.env @@ -154,6 +154,11 @@ elif [ $step = "fv3ic" ]; then [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max export APRUN_CHGRES="time" +elif [ $step = "waveinit" -o $step = "waveprep" -o $step = "wavepostsbs" ]; then + + export mpmd="--multi-prog" + export CFP_MP="YES" + elif [ $step = "postsnd" ]; then nth_max=$(($npe_node_max / $npe_node_postsnd)) diff --git a/env/ORION.env b/env/ORION.env index 6a196fe2a1..b2daccdcf3 100755 --- a/env/ORION.env +++ b/env/ORION.env @@ -189,6 +189,11 @@ elif [ $step = "fv3ic" ]; then [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max export APRUN_CHGRES="time" +elif [ $step = "waveinit" -o $step = "waveprep" -o $step = "wavepostsbs" ]; then + + export mpmd="--multi-prog" + export CFP_MP="YES" + elif [ $step = "postsnd" ]; then nth_max=$(($npe_node_max / $npe_node_postsnd)) diff --git a/jobs/JGLOBAL_FORECAST b/jobs/JGLOBAL_FORECAST index 824638dcf0..e9ad054024 100755 --- a/jobs/JGLOBAL_FORECAST +++ b/jobs/JGLOBAL_FORECAST @@ -9,7 +9,7 @@ date ############################# # Source relevant config files ############################# -configs="base fcst" +configs="base fcst wave" export EXPDIR=${EXPDIR:-$HOMEgfs/parm/config} config_path=${EXPDIR:-$NWROOT/gfs.${gfs_ver}/parm/config} for config in $configs; do @@ -62,6 +62,8 @@ if [ $RUN_ENVIR = "nco" ]; then export RSTDIR=${GESROOT:?}/$envir fi +export COMPONENTwave=${COMPONENTwave:-${RUN}wave} +export WAV_MOD_TAG=${COMPONENTwave}${waveMEMB} ############################################## # Begin JOB SPECIFIC work diff --git a/jobs/JGLOBAL_FORECAST_MEDCOLD b/jobs/JGLOBAL_FORECAST_MEDCOLD index a0826d9c6f..f7b7737d8b 100755 --- a/jobs/JGLOBAL_FORECAST_MEDCOLD +++ b/jobs/JGLOBAL_FORECAST_MEDCOLD @@ -9,7 +9,7 @@ date ############################# # Source relevant config files ############################# -configs="base fcst" +configs="base fcst wave" export EXPDIR=${EXPDIR:-$HOMEgfs/parm/config} config_path=${EXPDIR:-$NWROOT/gfs.${gfs_ver}/parm/config} for config in $configs; do @@ -62,6 +62,8 @@ if [ $RUN_ENVIR = "nco" ]; then export RSTDIR=${GESROOT:?}/$envir fi +export COMPONENTwave=${COMPONENTwave:-${RUN}wave} +export WAV_MOD_TAG=${COMPONENTwave}${waveMEMB} ############################################## # Begin JOB SPECIFIC work diff --git a/jobs/JWAVE_INIT b/jobs/JWAVE_INIT index 781c69c33e..57300adad5 100755 --- a/jobs/JWAVE_INIT +++ b/jobs/JWAVE_INIT @@ -7,7 +7,7 @@ set -x -e ############################# # Source relevant config files ############################# -configs="base wave waveinit" +configs="base wave" export EXPDIR=${EXPDIR:-$HOMEgfs/parm/config} config_path=${EXPDIR:-$NWROOT/gfs.${gfs_ver}/parm/config} for config in $configs; do @@ -26,18 +26,13 @@ status=$? # PATH for working directory export NET=${NET:-gfs} export RUN=${RUN:-gfs} -export COMPONENTwave=${COMPONENTwave:-${RUN}wave} -export HOMEgfs=${HOMEgfs:-$NWROOT/${NET}.${gfs_ver}} +export COMPONENTwave=${COMPONENTwave:-${RUN}wave} # Add default errchk = err_chk export errchk=${errchk:-err_chk} -# Set HOMEwave to HOMEgfs -HOMEwave=${HOMEwave:-${HOMEgfs}} - -# Create and go to DATA directory -export DATA=${DATA:-${DATAROOT:?}/${jobid}} +export DATA=${DATA:-${DATAROOT}/${jobid:?}} mkdir -p $DATA cd $DATA @@ -50,21 +45,10 @@ sh ./PDY export pgmout=OUTPUT.$$ -export MP_PULSE=0 - -# Set resources to propagate NTASKS across cfp call -NTASKS=${NTASKS:-${npe_node_waveinit}} -export NTASKS=${NTASKS:?NTASKS required to be set} - -# Path to HOME Directory -export CODEwave=${CODEwave:-${HOMEfv3gfs}/WW3} -export EXECwave=${EXECwave:-$HOMEwave/exec} -export FIXwave=${FIXwave:-$HOMEwave/fix/fix_wave_${NET}} -export PARMwave=${PARMwave:-$HOMEwave/parm/wave} -export USHwave=${USHwave:-$HOMEwave/ush} -export EXECcode=${EXECcode:-$CODEwave/exec} +export COMINice=${COMINice:-$COMROOTp2/omb/prod} +export COMINwnd=${COMINwnd:-$COMROOT/gfs/prod} +export COMIN_WAV_CUR=${COMIN_WAV_CUR:-$COMROOTp2/rtofs/prod} -# Set COM Paths and GETGES environment export COMINwave=${COMINwave:-${ROTDIR:?}} export COMOUTwave=${COMOUTwave:-${ROTDIR:?}} export COMIN=${COMIN:-${COMINwave}/${COMPONENTwave}.${PDY}/${cyc}} @@ -75,19 +59,14 @@ if [ $SENDCOM = YES ]; then mkdir -p $COMOUT/rundata fi -export wavelog=${COMOUTwave}/wave.log - # Set mpi serial command -export wavempexec=${wavempexec:-"mpirun -n"} -export wave_mpmd=${wave_mpmd:-"cfp"} +export wavempexec=${launcher:-"mpirun -n"} +export wave_mpmd=${mpmd:-"cfp"} # Execute the Script $HOMEwave/scripts/exwave_init.sh # Remove temp directories -if [ "$KEEPDATA" != "YES" ]; then - cd $DATAROOT - rm -rf $DATA -fi -date +[[ $KEEPDATA = "NO" ]] && rm -rf $DATA +date diff --git a/jobs/JWAVE_POST_SBS b/jobs/JWAVE_POST_SBS index 6e60567d3f..66676b0eae 100755 --- a/jobs/JWAVE_POST_SBS +++ b/jobs/JWAVE_POST_SBS @@ -7,7 +7,7 @@ set -x -e ############################# # Source relevant config files ############################# -configs="base wave wavepostsbs" +configs="base wave" export EXPDIR=${EXPDIR:-$HOMEgfs/parm/config} config_path=${EXPDIR:-$NWROOT/gfs.${gfs_ver}/parm/config} for config in $configs; do @@ -26,72 +26,45 @@ status=$? # PATH for working directory export NET=${NET:-gfs} export RUN=${RUN:-gfs} -export COMPONENTwave=${COMPONENTwave:-${RUN}wave} -export HOMEgefs=${HOMEgefs:-$NWROOT/$NET.${gefs_ver}} -export HOMEgfs=${HOMEgfs:-$NWROOT/$NET.${gfs_ver}} +export COMPONENTwave=${COMPONENTwave:-${RUN}wave} # Add default errchk = err_chk export errchk=${errchk:-err_chk} -# Set HOMEwave to HOMEgefs -HOMEwave=${HOMEwave:-${HOMEgfs}} - -# Set resources to propagate NTASKS across cfp call -NTASKS=${NTASKS:-${npe_node_waveprep}} -export NTASKS=${NTASKS:?NTASKS required to be set} - -# Create and go to DATA directory -export DATA=${DATA:-${DATAROOT:?}/${jobid}} +export DATA=${DATA:-${DATAROOT}/${jobid:?}} mkdir -p $DATA cd $DATA -export cyc=${cyc:-00} +cyc=${cyc:-00} export cycle=${cycle:-t${cyc}z} - + # Set PDY setpdy.sh sh ./PDY export pgmout=OUTPUT.$$ -export MP_PULSE=0 +export COMINice=$((COMINice:-$COMROOTp2/omb/prod)) +export COMINwnd=$((COMINwnd:-$COMROOT/gfs/prod)) +export COMIN_WAV_CUR=$((COMIN_WAV_CUR:-$COMROOTp2/rtofs/prod)) -# Path to HOME Directory -export CODEwave=${CODEwave:-${HOMEfv3gfs}/WW3} -export EXECwave=${EXECwave:-$HOMEwave/exec} -export FIXwave=${FIXwave:-$HOMEwave/fix/fix_wave_${NET}} -export PARMwave=${PARMwave:-$HOMEwave/parm/wave} -export USHwave=${USHwave:-$HOMEwave/ush} -export EXECcode=${EXECcode:-$CODEwave/exec} - -# Set COM Paths and GETGES environment export COMINwave=${COMINwave:-${ROTDIR:?}} export COMOUTwave=${COMOUTwave:-${ROTDIR:?}} export COMIN=${COMIN:-${COMINwave}/${COMPONENTwave}.${PDY}/${cyc}} export COMOUT=${COMOUT:-${COMOUTwave}/${COMPONENTwave}.${PDY}/${cyc}} -export COMINice=${COMINice:-${COMROOTp2}/omb/prod} -export COMINwnd=${COMINwnd:-${COMROOT}/gfs/prod} -export COMIN_WAV_CUR=${COMIN_WAV_CUR:-${COMROOTp2}/rtofs/prod} - mkdir -p $COMOUT/gridded mkdir -p $COMOUT/station mkdir -p $COMOUT/stats -export wavelog=${COMOUTwave}/wave.log - -# Set mpi serial command -export wavempexec=${wavempexec:-"mpirun -n"} -export wave_mpmd=${wave_mpmd:-"cfp"} - env | sort # Set wave model ID tag to include member number # if ensemble; waveMEMB var empty in deterministic # Set wave model ID tag to include member number # if ensemble; waveMEMB var empty in deterministic -membTAG='p' +export membTAG='p' if [ "${waveMEMB}" == "00" ]; then membTAG='c'; fi export membTAG export WAV_MOD_TAG=${COMPONENTwave}${waveMEMB} @@ -114,4 +87,3 @@ if [ "$KEEPDATA" != "YES" ]; then rm -rf $DATA fi date - diff --git a/jobs/JWAVE_PREP b/jobs/JWAVE_PREP index dcb7b0df06..b1a657bcc2 100755 --- a/jobs/JWAVE_PREP +++ b/jobs/JWAVE_PREP @@ -1,5 +1,7 @@ #!/bin/bash +set -x + date export PS4=' $SECONDS + ' set -x -e @@ -26,22 +28,14 @@ status=$? # PATH for working directory export NET=${NET:-gfs} export RUN=${RUN:-gfs} -export COMPONENTwave=${COMPONENTwave:-${RUN}wave} -export HOMEgfs=${HOMEgfs:-$NWROOT/gfs.${gfs_ver}} +export COMPONENTwave=${COMPONENTwave:-${RUN}wave} +export WAV_MOD_TAG=${COMPONENTwave}${waveMEMB} # Add default errchk = err_chk export errchk=${errchk:-err_chk} -# Set HOMEwave to HOMEgfs -HOMEwave=${HOMEwave:-${HOMEgfs}} - -# Set resources to propagate NTASKS across cfp call -NTASKS=${NTASKS:-${npe_node_waveprep}} -export NTASKS=${NTASKS:?NTASKS required to be set} - -# Create and go to DATA directory -export DATA=${DATA:-${DATAROOT:?}/${jobid}} +export DATA=${DATA:-${DATAROOT}/${jobid:?}} mkdir -p $DATA cd $DATA @@ -56,24 +50,10 @@ export RPDY=$PDY export pgmout=OUTPUT.$$ -export MP_PULSE=0 +export COMINice=${COMINice:-${COMROOTp2}/omb/prod} +export COMINwnd=${COMINwnd:-${COMROOT}/gfs/prod} +export COMIN_WAV_CUR=${COMIN_WAV_CUR:-${COMROOTp2}/rtofs/prod} -# CDO required for processing RTOFS currents -# export CDO=${COMROOTp2}/nwprod/rtofs_glo.v1.2.0/bin/cdo -export CDO=/gpfs/dell2/emc/verification/noscrub/Todd.Spindler/CDO/bin/cdo - -# Path to HOME Directory -export CODEwave=${CODEwave:-${HOMEfv3gfs}/WW3} -export EXECwave=${EXECwave:-$HOMEwave/exec} -export FIXwave=${FIXwave:-$HOMEwave/fix/fix_wave_${NET}} -export PARMwave=${PARMwave:-$HOMEwave/parm/wave} -export USHwave=${USHwave:-$HOMEwave/ush} -export EXECcode=${EXECcode:-$CODEwave/exec} - -################################### -# Set COM Paths and GETGES environment -################################### -# Set COM Paths and GETGES environment export COMINwave=${COMINwave:-${ROTDIR:?}} export COMOUTwave=${COMOUTwave:-${ROTDIR:?}} export COMIN=${COMIN:-${COMINwave}/${COMPONENTwave}.${PDY}/${cyc}} @@ -88,7 +68,7 @@ if [ $RUN_ENVIR = "nco" ]; then export RPDY=`$NDATE -24 ${PDY}00 | cut -c1-8` export COMIN_WAV_CUR=$(compath.py ${WAVECUR_DID}/prod)/${WAVECUR_DID}.${RPDY} fi -else +else if [ ! -d $DMPDIR/${WAVECUR_DID}.${RPDY} ]; then export RPDY=`$NDATE -24 ${PDY}00 | cut -c1-8`; fi if [ ! -L $ROTDIR/${WAVECUR_DID}.${RPDY} ]; then # Check if symlink already exists in ROTDIR $NLN $DMPDIR/${WAVECUR_DID}.${RPDY} $ROTDIR/${WAVECUR_DID}.${RPDY} @@ -104,15 +84,9 @@ if [ $SENDCOM = YES ]; then mkdir -p $COMOUT fi -export wavelog=${COMOUTwave}/wave.log - # Set mpi serial command -export wavempexec=${wavempexec:-"mpirun -n"} -export wave_mpmd=${wave_mpmd:-"cfp"} - -# Set wave model ID tag to include member number -# if ensemble; waveMEMB var empty in deterministic -export WAV_MOD_TAG=${COMPONENTwave}${waveMEMB} +export wavempexec=${launcher:-"mpirun -n"} +export wave_mpmd=${mpmd:-"cfp"} # Execute the Script $HOMEwave/scripts/exwave_prep.sh @@ -123,4 +97,4 @@ if [ "$KEEPDATA" != "YES" ]; then rm -rf $DATA fi date - +exit 0 diff --git a/jobs/rocoto/coupled_ic.sh b/jobs/rocoto/coupled_ic.sh index 045171f807..8e954db818 100755 --- a/jobs/rocoto/coupled_ic.sh +++ b/jobs/rocoto/coupled_ic.sh @@ -1,5 +1,7 @@ #!/bin/bash +set -x + ############################################################### ## Abstract: ## Create FV3 initial conditions from GFS intitial conditions @@ -20,7 +22,7 @@ status=$? ############################################################### # Source relevant configs -configs="base fv3ic" +configs="base fv3ic wave" for config in $configs; do . $EXPDIR/config.${config} status=$? @@ -33,15 +35,6 @@ done status=$? [[ $status -ne 0 ]] && exit $status - -#TODO: make the following configurable variables instead of hardcoded -CPL_ATMIC=CFSR -CPL_OCNIC=CPC3Dvar -CPL_ICEIC=CPC -OCNRES=025 -ICERES=025 -ICERES=${ICERES:-"025"} - # Create ICSDIR if needed [[ ! -d $ICSDIR/$CDATE ]] && mkdir -p $ICSDIR/$CDATE [[ ! -d $ICSDIR/$CDATE/ocn ]] && mkdir -p $ICSDIR/$CDATE/ocn @@ -63,11 +56,13 @@ cp -r $ORIGIN_ROOT/$CPL_OCNIC/$CDATE/ocn/$OCNRES/MOM*.nc $ICSDIR/$CDATE/ocn/ #Setup Ice IC files cp $ORIGIN_ROOT/$CPL_ICEIC/$CDATE/ice/$ICERES/cice5_model_${ICERESdec}.res_$CDATE.nc $ICSDIR/$CDATE/ice/ -#TODO for wave coupling -#if cplwav=true - #Setup Wave IC files -#fi - +if [ $cplwav = ".true." ]; then + [[ ! -d $ICSDIR/$CDATE/wav ]] && mkdir -p $ICSDIR/$CDATE/wav + for grdID in $waveGRD + do + cp $ORIGIN_ROOT/$CPL_WAVIC/$CDATE/wav/$grdID/*restart.$grdID $ICSDIR/$CDATE/wav/ + done +fi export OUTDIR="$ICSDIR/$CDATE/$CDUMP/$CASE/INPUT" @@ -80,4 +75,6 @@ $NLN $OUTDIR . ############################################################## # Exit cleanly + +set +x exit 0 diff --git a/jobs/rocoto/ocnpost.sh b/jobs/rocoto/ocnpost.sh index d4a75e00af..183fd25281 100755 --- a/jobs/rocoto/ocnpost.sh +++ b/jobs/rocoto/ocnpost.sh @@ -108,27 +108,43 @@ for fhr in $fhrlst; do export CDATE=$VDATE cd $DATA if [ $fhr -gt 0 ]; then - export MOM6REGRID=$UGCSsrc/mom6_regrid_025 - $MOM6REGRID/run_regrid.sh + export MOM6REGRID=${MOM6REGRID:-$HOMEgfs} + $MOM6REGRID/scripts/run_regrid.sh status=$? [[ $status -ne 0 ]] && exit $status # Convert the netcdf files to grib2 export executable=$MOM6REGRID/exec/reg2grb2.x - $MOM6REGRID/run_reg2grb2.sh + $MOM6REGRID/scripts/run_reg2grb2.sh + status=$? + [[ $status -ne 0 ]] && exit $status + + + #break up ocn netcdf into multiple files: + if [ -f $COMOUT/ocn_2D_$VDATE.$ENSMEM.$IDATE.nc ]; then + echo "File $COMOUT/ocn_2D_$VDATE.$ENSMEM.$IDATE.nc already exists" + else + $ncks -x -v vo,uo,so,temp $COMOUT/ocn$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn_2D_$VDATE.$ENSMEM.$IDATE.nc + fi + if [ -f $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc ]; then + echo "File $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc already exists" + else + $ncks -x -v Heat_PmE,LW,LwLatSens,MLD_003,MLD_0125,SSH,SSS,SST,SSU,SSV,SW,cos_rot,ePBL,evap,fprec,frazil,latent,lprec,lrunoff,sensible,sin_rot,speed,taux,tauy,wet_c,wet_u,wet_v $COMOUT/ocn$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc + fi + if [ -f $COMOUT/ocn-temp-EQ_$VDATE.$ENSMEM.$IDATE.nc ]; then + echo "File $COMOUT/ocn-temp-EQ_$VDATE.$ENSMEM.$IDATE.nc already exists" + else + $ncks -v temp -d yh,503 -d xh,-299.92,60.03 $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn-temp-EQ_$VDATE.$ENSMEM.$IDATE.nc + fi + if [ -f $COMOUT/ocn-uo-EQ_$VDATE.$ENSMEM.$IDATE.nc ]; then + echo "File $COMOUT/ocn-uo-EQ_$VDATE.$ENSMEM.$IDATE.nc already exists" + else + $ncks -v uo -d yh,503 -d xh,-299.92,60.03 $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn-uo-EQ_$VDATE.$ENSMEM.$IDATE.nc + fi status=$? [[ $status -ne 0 ]] && exit $status fi - #break up ocn netcdf into multiple files: - $ncks -x -v vo,uo,so,temp $COMOUT/ocn_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn_2D_$VDATE.$ENSMEM.$IDATE.nc - $ncks -x -v Heat_PmE,LW,LwLatSens,MLD_003,MLD_0125,SSH,SSS,SST,SSU,SSV,SW,cos_rot,ePBL,evap,fprec,frazil,latent,lprec,lrunoff,sensible,sin_rot,speed,taux,tauy,wet_c,wet_u,wet_v $COMOUT/ocn_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc - $ncks -v temp -d yh,503 -d xh,-299.92,60.03 $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn-temp-EQ_$VDATE.$ENSMEM.$IDATE.nc - $ncks -v uo -d yh,503 -d xh,-299.92,60.03 $COMOUT/ocn_3D_$VDATE.$ENSMEM.$IDATE.nc $COMOUT/ocn-uo-EQ_$VDATE.$ENSMEM.$IDATE.nc - status=$? - [[ $status -ne 0 ]] && exit $status - - done # Restore CDATE to what is expected export CDATE=$IDATE diff --git a/jobs/rocoto/waveinit.sh b/jobs/rocoto/waveinit.sh index ce7397c3e7..2111727d6e 100755 --- a/jobs/rocoto/waveinit.sh +++ b/jobs/rocoto/waveinit.sh @@ -15,7 +15,3 @@ $HOMEgfs/jobs/JWAVE_INIT status=$? exit $status -############################################################### -# Force Exit out cleanly -if [ ${KEEPDATA:-"NO"} = "NO" ] ; then rm -rf $DATAROOT ; fi -exit 0 diff --git a/modulefiles/module-setup.sh.inc b/modulefiles/module-setup.sh.inc index f4aad8b9f1..75c7249ea4 100644 --- a/modulefiles/module-setup.sh.inc +++ b/modulefiles/module-setup.sh.inc @@ -28,8 +28,8 @@ elif [[ -d /scratch1 ]] ; then source /apps/lmod/lmod/init/$__ms_shell fi module purge -elif [[ -d /scratch3 ]] ; then - # We are on NOAA Theia +elif [[ -d /work ]] ; then + # We are on MSU Orion if ( ! eval module help > /dev/null 2>&1 ) ; then source /apps/lmod/lmod/init/$__ms_shell fi @@ -83,7 +83,7 @@ elif [[ -d /lustre && -d /ncrc ]] ; then __ms_source_etc_profile=no fi module purge -# clean up after purge + # clean up after purge unset _LMFILES_ unset _LMFILES_000 unset _LMFILES_001 diff --git a/modulefiles/module_base.orion b/modulefiles/module_base.orion index c053ffa49c..b1d0ef4aa0 100644 --- a/modulefiles/module_base.orion +++ b/modulefiles/module_base.orion @@ -15,6 +15,10 @@ module load contrib noaatools module load intel/2018 module load impi/2018 +#For ocean post: +module load ncl/6.6.2 +module load nco/4.8.1 + ## ## NCEP libraries (temporary version to match the CCPP requirements) ## @@ -47,6 +51,9 @@ module load netcdf_parallel/4.7.4 module load hdf5_parallel/1.10.6 module load wgrib/2.0.8 module load grib_util/1.2.0 + +module load slurm/19.05.3-2 + ## ### load cmake ### diff --git a/modulefiles/modulefile.reg2grb2.hera b/modulefiles/modulefile.reg2grb2.hera new file mode 100644 index 0000000000..5a1d7dc537 --- /dev/null +++ b/modulefiles/modulefile.reg2grb2.hera @@ -0,0 +1,14 @@ +#%Module##################################################### +# reg2grb2 - hera +############################################################# + +module load intel/18.0.5.274 +module load netcdf/4.7.0 +module load ip/3.0.1 +module load sp/2.0.2 +module load w3nco/2.0.7 +module load bacio/2.0.2 +module load landsfcutil/2.1.0 +module use /apps/modules/modulefamilies/intel +module load wgrib2/2.0.8 +set FCMP ifort diff --git a/modulefiles/modulefile.reg2grb2.orion b/modulefiles/modulefile.reg2grb2.orion new file mode 100644 index 0000000000..17f16a3448 --- /dev/null +++ b/modulefiles/modulefile.reg2grb2.orion @@ -0,0 +1,14 @@ +#%Module##################################################### +# reg2grb2 - orion +############################################################# +export FCOMP=ifort +module load intel/2020 +module load netcdf/4.7.2 +module load ip/3.0.2 +module load sp/2.0.3 +module load w3nco/2.0.7 +module load bacio/2.0.3 +module load landsfcutil/2.2.0 +module use -a /apps/contrib/NCEPLIBS/orion/modulefiles +module load wgrib/2.0.8 +export WGRIB2_ROOT=/apps/contrib/NCEPLIBS/orion/NCEPLIBS/NCEPLIBS-external/build5/wgrib2/grib2 diff --git a/parm/config/config.base b/parm/config/config.base index 1fd8ce741c..fd3092c94a 100755 --- a/parm/config/config.base +++ b/parm/config/config.base @@ -208,8 +208,6 @@ export ARCDIR="$NOSCRUB/archive/$PSLOT" export ICSDIR="@ICSDIR@" export ATARDIR="/NCEPDEV/$HPSS_PROJECT/1year/$USER/$machine/scratch/$PSLOT" -# For coupled -export UGCSsrc="/scratch2/NCEPDEV/climate/Bin.Li/S2S/fix/ocean_ice_post" # Commonly defined parameters in JJOBS export envir=${envir:-"prod"} diff --git a/parm/config/config.base.emc.dyn_coupled b/parm/config/config.base.emc.dyn_coupled index 1fd8ce741c..fd3092c94a 100755 --- a/parm/config/config.base.emc.dyn_coupled +++ b/parm/config/config.base.emc.dyn_coupled @@ -208,8 +208,6 @@ export ARCDIR="$NOSCRUB/archive/$PSLOT" export ICSDIR="@ICSDIR@" export ATARDIR="/NCEPDEV/$HPSS_PROJECT/1year/$USER/$machine/scratch/$PSLOT" -# For coupled -export UGCSsrc="/scratch2/NCEPDEV/climate/Bin.Li/S2S/fix/ocean_ice_post" # Commonly defined parameters in JJOBS export envir=${envir:-"prod"} diff --git a/scripts/exwave_init.sh b/scripts/exwave_init.sh index 5a5ed2d669..4b49d5b32e 100755 --- a/scripts/exwave_init.sh +++ b/scripts/exwave_init.sh @@ -14,6 +14,7 @@ # Script history log: # 2019-05-06 J-Henrique Alves First Version. # 2019-11-02 J-Henrique Alves Ported to global-workflow. +# 2020-06-10 J-Henrique Alves Ported to R&D machine Hera # # $Id$ # @@ -125,12 +126,15 @@ echo ' ' echo $msg [[ "$LOUD" = YES ]] && set -x - echo "$COMPONENTwave init config $date $cycle : ww3_grid.inp.$grdID missing." >> $wavelog err=2;export err;${errchk} fi [[ ! -d $COMOUT/rundata ]] && mkdir -m 775 -p $COMOUT/rundata - echo "$USHwave/wave_grid_moddef.sh $grdID > $grdID.out 2>&1" >> cmdfile + if [ ${CFP_MP:-"NO"} = "YES" ]; then + echo "$nmoddef $USHwave/wave_grid_moddef.sh $grdID > $grdID.out 2>&1" >> cmdfile + else + echo "$USHwave/wave_grid_moddef.sh $grdID > $grdID.out 2>&1" >> cmdfile + fi nmoddef=`expr $nmoddef + 1` @@ -163,7 +167,11 @@ if [ "$NTASKS" -gt '1' ] then - ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + if [ ${CFP_MP:-"NO"} = "YES" ]; then + ${wavempexec} -n ${wavenproc} ${wave_mpmd} cmdfile + else + ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + fi exit=$? else ./cmdfile @@ -208,7 +216,6 @@ echo $msg sed "s/^/$grdID.out : /g" $grdID.out [[ "$LOUD" = YES ]] && set -x - echo "$COMPONENTwave prep $date $cycle : mod_def.$grdID missing." >> $wavelog err=3;export err;${errchk} fi done @@ -224,7 +231,5 @@ echo ' ' [[ "$LOUD" = YES ]] && set -x - msg="$job completed normally" - postmsg "$jlogfile" "$msg" - + exit $err # End of MWW3 init config script ------------------------------------------- # diff --git a/scripts/exwave_post_sbs.sh b/scripts/exwave_post_sbs.sh index 6f1a1dec38..8d637f0b2b 100755 --- a/scripts/exwave_post_sbs.sh +++ b/scripts/exwave_post_sbs.sh @@ -108,12 +108,13 @@ # --------------------------------------------------------------------------- # # 1. Get files that are used by most child scripts - fieldOK='yes' - pointOK='yes' - gribOK='yes' - grintOK='yes' - specOK='yes' - bullOK='yes' + export DOIBP_WAV='NO' + export DOFLD_WAV='YES' + export DOPNT_WAV='YES' + export DOGRB_WAV='YES' + export DOGRI_WAV='YES' + export DOSPC_WAV='YES' + export DOBLL_WAV='YES' exit_code=0 @@ -144,9 +145,7 @@ cp -f $COMIN/rundata/${COMPONENTwave}.mod_def.${grdID} mod_def.$grdID iloop=`expr $iloop + 1` - fi - done for grdID in $waveGRD $wavesbsGRD $wavepostGRD $waveinterpGRD $waveuoutpGRD @@ -160,17 +159,15 @@ echo '*************************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $grdID $date $cycle : mod_def file missing." >> $wavelog postmsg "$jlogfile" "FATAL ERROR : NO MOD_DEF file mod_def.$grdID" - fieldOK='no' + DOFLD_WAV='NO' err=2; export err;${errchk} exit $err - gribOK='no' + DOGRB_WAV='NO' else set +x echo "File mod_def.$grdID found. Syncing to all nodes ..." [[ "$LOUD" = YES ]] && set -x - $FSYNC mod_def.$grdID fi done @@ -183,12 +180,9 @@ cp -f $FIXwave/wave_${NET}.buoys buoy.loc.temp # Reverse grep to exclude IBP points sed -n '/^\$.*/!p' buoy.loc.temp | grep -v IBP > buoy.loc -# Grep to include IBP points - sed -n '/^\$.*/!p' buoy.loc.temp | grep IBP > buoy.ibp - rm -f buoy.loc.temp fi - if [ -s buoy.loc ] && [ -s buoy.ibp ] + if [ -s buoy.loc ] then set +x echo " buoy.loc and buoy.ibp copied and processed ($FIXwave/wave_${NET}.buoys)." @@ -201,18 +195,41 @@ echo '************************************* ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$AV_MOD_ID post $date $cycle : buoy location file missing." >> $wavelog postmsg "$jlogfile" "FATAL ERROR : NO BUOY LOCATION FILE" err=3; export err;${errchk} exit $err - pointOK='no' - specOK='no' - bullOK='no' + DOPNT_WAV='NO' + DOSPC_WAV='NO' + DOBLL_WAV='NO' + fi + + if [ "$DOIBP_WAV" = 'YES' ] + then + sed -n '/^\$.*/!p' buoy.loc.temp | grep IBP > buoy.ibp + if [ -s buoy.ibp ]; then + set +x + echo " buoy.loc and buoy.ibp copied and processed ($FIXwave/wave_${NET}.buoys)." + [[ "$LOUD" = YES ]] && set -x + else + set +x + echo ' ' + echo '************************************* ' + echo ' FATAL ERROR : NO BUOY LOCATION FILE ' + echo '************************************* ' + echo ' ' + [[ "$LOUD" = YES ]] && set -x + postmsg "$jlogfile" "FATAL ERROR : NO BUOY LOCATION FILE" + err=3; export err;${errchk} + exit $err + DOPNT_WAV='NO' + DOSPC_WAV='NO' + DOBLL_WAV='NO' + fi fi # 1.d Input template files - if [ "$grintOK" = 'yes' ] + if [ "$DOGRI_WAV" = 'YES' ] then for intGRD in $waveinterpGRD do @@ -226,7 +243,6 @@ set +x echo " ${intGRD}_interp.inp.tmpl copied. Syncing to all nodes ..." [[ "$LOUD" = YES ]] && set -x - $FSYNC ${intGRD}_interp.inp.tmpl else set +x echo ' ' @@ -235,15 +251,14 @@ echo '*********************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : GRINT template file missing." >> $wavelog postmsg "$jlogfile" "NON-FATAL ERROR : NO TEMPLATE FOR GRINT INPUT FILE" exit_code=1 - grintOK='no' + DOGRI_WAV='NO' fi done fi - if [ "$gribOK" = 'yes' ] + if [ "$DOGRB_WAV" = 'YES' ] then for grbGRD in $waveinterpGRD $wavepostGRD do @@ -257,7 +272,6 @@ set +x echo " ww3_grib2.${grbGRD}.inp.tmpl copied. Syncing to all nodes ..." [[ "$LOUD" = YES ]] && set -x - $FSYNC ww3_grib2.inp.tmpl else set +x echo ' ' @@ -266,10 +280,9 @@ echo '*********************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : GRIB2 template file missing." >> $wavelog postmsg "$jlogfile" "NON-FATAL ERROR : NO TEMPLATE FOR GRIB2 INPUT FILE" exit_code=2 - gribOK='no' + DOGRB_WAV='NO' fi done fi @@ -284,7 +297,6 @@ set +x echo " ww3_outp_spec.inp.tmpl copied. Syncing to all grids ..." [[ "$LOUD" = YES ]] && set -x - $FSYNC ww3_outp_spec.inp.tmpl else set +x echo ' ' @@ -293,11 +305,10 @@ echo '*********************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : ww3_outp_spec.inp.tmpl file missing." >> $wavelog postmsg "$jlogfile" "NON-FATAL ERROR : NO TEMPLATE FOR SPEC INPUT FILE" exit_code=3 - specOK='no' - bullOK='no' + DOSPC_WAV='NO' + DOBLL_WAV='NO' fi if [ -f $FIXwave/ww3_outp_bull.inp.tmpl ] @@ -310,7 +321,6 @@ set +x echo " ww3_outp_bull.inp.tmpl copied. Syncing to all nodes ..." [[ "$LOUD" = YES ]] && set -x - $FSYNC ww3_outp_bull.inp.tmpl else set +x echo ' ' @@ -319,15 +329,14 @@ echo '*************************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : bulletin template file missing." >> $wavelog postmsg "$jlogfile" "NON-FATAL ERROR : NO TEMPLATE FOR BULLETIN INPUT FILE" exit_code=4 - bullOK='no' + DOBLL_WAV='NO' fi # 1.e Getting buoy information for points - if [ "$specOK" = 'yes' ] || [ "$bullOK" = 'yes' ] + if [ "$DOSPC_WAV" = 'YES' ] || [ "$DOBLL_WAV" = 'YES' ] then ymdh=`$NDATE -${WAVHINDH} $CDATE` tstart="`echo $ymdh | cut -c1-8` `echo $ymdh | cut -c9-10`0000" @@ -361,7 +370,7 @@ rm -f buoy_tmp.loc buoy_log.ww3 ww3_oup.inp ln -fs ./out_pnt.${waveuoutpGRD} ./out_pnt.ww3 ln -fs ./mod_def.${waveuoutpGRD} ./mod_def.ww3 - $EXECcode/ww3_outp > buoy_lst.loc 2>&1 + $EXECwave/ww3_outp > buoy_lst.loc 2>&1 err=$? if [ "$err" != '0' ] && [ ! -f buoy_log.ww3 ] @@ -376,12 +385,11 @@ echo '******************************************** ' echo ' ' cat buoy_tmp.loc - echo "$WAV_MOD_TAG post $date $cycle : buoy log file failed to be created." >> $wavelog echo $msg [[ "$LOUD" = YES ]] && set -x err=4;export err;${errchk} - specOK='no' - bullOK='no' + DOSPC_WAV='NO' + DOBLL_WAV='NO' exit $err fi @@ -401,7 +409,6 @@ then set +x echo 'Buoy log file created. Syncing to all nodes ...' - $FSYNC buoy_log.dat [[ "$LOUD" = YES ]] && set -x else set +x @@ -411,49 +418,43 @@ echo '**************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : buoy log file missing." >> $wavelog postmsg "$jlogfile" "FATAL ERROR : NO BUOY LOG FILE GENERATED FOR SPEC AND BULLETIN FILES" err=5;export err;${errchk} - specOK='no' - bullOK='no' - OspecOK='no' - ObullOK='no' + DOSPC_WAV='NO' + DOBLL_WAV='NO' fi # Create new buoy_log.ww3 including all IBP files - ibspecOK='yes' - cat buoy.ibp | awk '{print $3}' | sed 's/'\''//g' > ibp_tags - grep -F -f ibp_tags buoy_log.ww3 > buoy_log.tmp - rm -f buoy_log.ibp - mv buoy_log.tmp buoy_log.ibp - - grep -F -f ibp_tags buoy_lst.loc > buoy_tmp1.loc - sed '$d' buoy_tmp1.loc > buoy_tmp2.loc - ibpoints=`awk '{ print $1 }' buoy_tmp2.loc` - Nibp=`wc buoy_tmp2.loc | awk '{ print $1 }'` - rm -f buoy_tmp1.loc buoy_tmp2.loc - - if [ -s buoy_log.ibp ] - then - set +x - echo 'IBP log file created. Syncing to all nodes ...' - $FSYNC buoy_log.ibp - [[ "$LOUD" = YES ]] && set -x - else - set +x - echo ' ' - echo '**************************************** ' - echo '*** ERROR : NO IBP LOG FILE CREATED *** ' - echo '**************************************** ' - echo ' ' - [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $date $cycle : ibp log file missing." >> $wavelog - postmsg "$jlogfile" "FATAL ERROR : NO IBP LOG FILE GENERATED FOR SPEC AND BULLETIN FILES" - err=6;export err;${errchk} - ibspecOK='no' + if [ "$DOIBP_WAV" = 'YES' ]; then + cat buoy.ibp | awk '{print $3}' | sed 's/'\''//g' > ibp_tags + grep -F -f ibp_tags buoy_log.ww3 > buoy_log.tmp + rm -f buoy_log.ibp + mv buoy_log.tmp buoy_log.ibp + + grep -F -f ibp_tags buoy_lst.loc > buoy_tmp1.loc + sed '$d' buoy_tmp1.loc > buoy_tmp2.loc + ibpoints=`awk '{ print $1 }' buoy_tmp2.loc` + Nibp=`wc buoy_tmp2.loc | awk '{ print $1 }'` + rm -f buoy_tmp1.loc buoy_tmp2.loc + if [ -s buoy_log.ibp ] + then + set +x + echo 'IBP log file created. Syncing to all nodes ...' + [[ "$LOUD" = YES ]] && set -x + else + set +x + echo ' ' + echo '**************************************** ' + echo '*** ERROR : NO IBP LOG FILE CREATED *** ' + echo '**************************************** ' + echo ' ' + [[ "$LOUD" = YES ]] && set -x + postmsg "$jlogfile" "FATAL ERROR : NO IBP LOG FILE GENERATED FOR SPEC AND BULLETIN FILES" + err=6;export err;${errchk} + DOIBP_WAV='NO' + fi fi - - fi + fi # 1.f Data summary @@ -463,11 +464,11 @@ echo ' ' echo ' Data summary : ' echo ' ---------------------------------------------' - echo " Sufficient data for GRID interpolation : $grintOK" - echo " Sufficient data for GRIB files : $gribOK" - echo " Sufficient data for spectral files : $specOK ($Nb points)" - echo " Sufficient data for bulletins : $bullOK ($Nb points)" - echo " Sufficient data for Input Boundary Points : $ibspecOK ($Nibp points)" + echo " Sufficient data for GRID interpolation : $DOGRI_WAV" + echo " Sufficient data for GRIB files : $DOGRB_WAV" + echo " Sufficient data for spectral files : $DOSPC_WAV ($Nb points)" + echo " Sufficient data for bulletins : $DOBLL_WAV ($Nb points)" + echo " Sufficient data for Input Boundary Points : $DOIBP_WAV ($Nibp points)" echo ' ' [[ "$LOUD" = YES ]] && set -x @@ -505,10 +506,9 @@ fcmdibp=ibpcmdfile.${FH3} rm -f ${fcmdnow} ${fcmdigrd} ${fcmdpnt} ${fcmdibp} touch ${fcmdnow} ${fcmdigrd} ${fcmdpnt} ${fcmdibp} -# echo "mkdir output_$YMDHMS" >> ${fcmdnow} mkdir output_$YMDHMS -# echo "cd output_$YMDHMS" >> ${fcmdnow} cd output_$YMDHMS + # Create instances of directories for spec and gridded output export SPECDATA=${DATA}/output_$YMDHMS export BULLDATA=${DATA}/output_$YMDHMS @@ -525,14 +525,13 @@ echo " FATAL ERROR : NO RAW POINT OUTPUT FILE out_pnt.$waveuoutpGRD echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $waveuoutpGRD $date $cycle : point output missing." >> $wavelog postmsg "$jlogfile" "FATAL ERROR : NO RAW POINT OUTPUT FILE out_pnt.$waveuoutpGRD err=6; export err;${errchk} exit $err fi ln -fs ${pfile} ./out_pnt.${waveuoutpGRD} - if [ "$specOK" = 'yes' ] + if [ "$DOSPC_WAV" = 'YES' ] then export dtspec=3600. for buoy in $buoys @@ -541,7 +540,7 @@ done fi - if [ "$ibspecOK" = 'yes' ] && [ "$DOIBP_WAV" = "YES" ] + if [ "$DOIBP_WAV" = 'YES' ] then export dtspec=3600. for buoy in $ibpoints @@ -550,7 +549,7 @@ done fi - if [ "$bullOK" = 'yes' ] + if [ "$DOBLL_WAV" = 'YES' ] then export dtspec=3600. for buoy in $buoys @@ -572,16 +571,15 @@ echo '*************************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG post $grdID $date $cycle : field output missing." >> $wavelog postmsg "$jlogfile" "NON-FATAL ERROR : NO RAW FIELD OUTPUT FILE out_grd.$grdID" - fieldOK='no' + DOFLD_WAVE='NO' err=7; export err;${errchk} exit $err fi ln -s ${gfile} ./out_grd.${wavGRD} done - if [ "$grintOK" = 'yes' ] + if [ "$DOGRI_WAV" = 'YES' ] then nigrd=1 for grdID in $waveinterpGRD @@ -591,7 +589,7 @@ glo_30mxt) ymdh_int=`$NDATE -${WAVHINDH} $ymdh`; dt_int=3600.; n_int=9999 ;; esac echo "$USHwave/wave_grid_interp_sbs.sh $grdID $ymdh_int $dt_int $n_int > grint_$grdID.out 2>&1" >> ${fcmdigrd}.${nigrd} - if [ "$gribOK" = 'yes' ] + if [ "$DOGRB_WAV" = 'YES' ] then gribFL=\'`echo ${OUTPARS_WAV}`\' case $grdID in @@ -606,7 +604,7 @@ done fi - if [ "$gribOK" = 'yes' ] + if [ "$DOGRB_WAV" = 'YES' ] then for grdID in ${wavepostGRD} # First concatenate grib files for sbs grids do @@ -615,6 +613,8 @@ aoc_9km) GRDNAME='arctic' ; GRDRES=9km ; GRIDNR=255 ; MODNR=11 ;; ant_9km) GRDNAME='antarc' ; GRDRES=9km ; GRIDNR=255 ; MODNR=11 ;; glo_10m) GRDNAME='global' ; GRDRES=0p16 ; GRIDNR=255 ; MODNR=11 ;; + gnh_10m) GRDNAME='global' ; GRDRES=0p16 ; GRIDNR=255 ; MODNR=11 ;; + gsh_15m) GRDNAME='gsouth' ; GRDRES=0p25 ; GRIDNR=255 ; MODNR=11 ;; glo_15m) GRDNAME='global' ; GRDRES=0p25 ; GRIDNR=255 ; MODNR=11 ;; ao_20m) GRDNAME='arctic' ; GRDRES=0p33 ; GRIDNR=255 ; MODNR=11 ;; so_20m) GRDNAME='antarc' ; GRDRES=0p33 ; GRIDNR=255 ; MODNR=11 ;; @@ -631,7 +631,7 @@ set +x echo ' ' - echo " Executing the copy command file at : `date`" + echo " Executing the grib2_sbs scripts at : `date`" echo ' ------------------------------------' echo ' ' [[ "$LOUD" = YES ]] && set -x @@ -700,9 +700,12 @@ # 6.b Spectral data files - if [ "$specOK" = 'yes' ] + if [ "$DOIBP_WAV" = 'YES' ] then echo "$USHwave/wave_tar.sh $WAV_MOD_TAG ibp $Nibp > ${WAV_MOD_TAG}_ibp_tar.out 2>&1 " >> cmdtarfile + fi + if [ "$DOSPC_WAV" = 'YES' ] + then echo "$USHwave/wave_tar.sh $WAV_MOD_TAG spec $Nb > ${WAV_MOD_TAG}_spec_tar.out 2>&1 " >> cmdtarfile echo "$USHwave/wave_tar.sh $WAV_MOD_TAG bull $Nb > ${WAV_MOD_TAG}_spec_tar.out 2>&1 " >> cmdtarfile echo "$USHwave/wave_tar.sh $WAV_MOD_TAG cbull $Nb > ${WAV_MOD_TAG}_spec_tar.out 2>&1 " >> cmdtarfile @@ -713,7 +716,7 @@ set +x echo ' ' - echo " Executing the copy command file at : `date`" + echo " Executing the wave_tar scripts at : `date`" echo ' ------------------------------------' echo ' ' [[ "$LOUD" = YES ]] && set -x diff --git a/scripts/exwave_prep.sh b/scripts/exwave_prep.sh index 1cd88fd41a..fcd443f2db 100755 --- a/scripts/exwave_prep.sh +++ b/scripts/exwave_prep.sh @@ -65,15 +65,7 @@ echo ' ' [[ "$LOUD" = YES ]] && set -x -# export MP_PGMMODEL=mpmd -# export MP_CMDFILE=./cmdfile - - if [ "$INDRUN" = 'no' ] - then - FHMAX_WAV=${FHMAX_WAV:-3} - else - FHMAX_WAV=${FHMAX_WAV:-384} - fi + FHMAX_WAV=${FHMAX_WAV:-384} # 0.b Date and time stuff @@ -105,7 +97,7 @@ RSTOFFSET=$(( ${RSTOFFSET} + ${RSTIOFF_WAV} )) ymdh_rst_ini=`$NDATE ${RSTOFFSET} $YMDH` RST2OFFSET=$(( DT_2_RST_WAV / 3600 )) - ymdh_rst2_ini=`$NDATE ${RST2OFFSET} $ymdh_rst_ini` # DT2 relative to first-first-cycle restart file + ymdh_rst2_ini=`$NDATE ${RST2OFFSET} $YMDH` # DT2 relative to first-first-cycle restart file # First restart file for cycling time_rst_ini="`echo $ymdh_rst_ini | cut -c1-8` `echo $ymdh_rst_ini | cut -c9-10`0000" if [ ${DT_1_RST_WAV} = 1 ]; then @@ -116,15 +108,20 @@ time_rst1_end="`echo $ymdh_rst1_end | cut -c1-8` `echo $ymdh_rst1_end | cut -c9-10`0000" fi # Second restart file for checkpointing - time_rst2_ini="`echo $ymdh_rst2_ini | cut -c1-8` `echo $ymdh_rst2_ini | cut -c9-10`0000" - time_rst2_end=$time_end -# Condition for gdas run or any other run when checkpoint stamp is > ymdh_end - if [ $ymdh_rst2_ini -ge $ymdh_end ]; then - ymdh_rst2_ini=`$NDATE 3 $ymdh_end` + if [ "${RSTTYPE_WAV}" = "T" ]; then time_rst2_ini="`echo $ymdh_rst2_ini | cut -c1-8` `echo $ymdh_rst2_ini | cut -c9-10`0000" - time_rst2_end=$time_rst2_ini + time_rst2_end=$time_end +# Condition for gdas run or any other run when checkpoint stamp is > ymdh_end + if [ $ymdh_rst2_ini -ge $ymdh_end ]; then + ymdh_rst2_ini=`$NDATE 3 $ymdh_end` + time_rst2_ini="`echo $ymdh_rst2_ini | cut -c1-8` `echo $ymdh_rst2_ini | cut -c9-10`0000" + time_rst2_end=$time_rst2_ini + fi + else + time_rst2_ini="$" + time_rst2_end= + DT_2_RST_WAV= fi - set +x echo ' ' echo 'Times in wave model format :' @@ -185,25 +182,27 @@ echo ' ' echo $msg [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : ${COMPONENTwave}.mod_def.${grdID} missing." >> $wavelog err=2;export err;${errchk} fi done # 1.b Netcdf Preprocessor template files + if [ "$WW3ATMINP" = 'YES' ]; then itype="$itype wind" ; fi + if [ "$WW3ICEINP" = 'YES' ]; then itype="$itype ice" ; fi + if [ "$WW3CURINP" = 'YES' ]; then itype="$itype cur" ; fi - for grdID in $grdINP + for type in $itype do - case $grdID in - $WAVECUR_FID ) - type='cur' + case $type in + wind ) + grdID=$WAVEWND_FID ;; - $WAVEWND_FID ) - type='wind' + ice ) + grdID=$WAVEICE_FID ;; - $WAVEICE_FID ) - type='ice' + cur ) + grdID=$WAVECUR_FID ;; * ) echo 'Input type not yet implemented' @@ -236,7 +235,6 @@ echo $msg echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : ww3_prnc.${type}.$grdID.tmpl missing." >> $wavelog err=4;export err;${errchk} fi done @@ -270,7 +268,6 @@ err=5;export err;${errchk} else mv -f ice.out $DATA/outtmp - rm -f ww3_prep.$WAVEICE_FID.tmpl mod_def.$WAVEICE_FID set +x echo ' ' echo ' Ice field unpacking successful.' @@ -306,9 +303,15 @@ # 3.a Gather and pre-process grib2 files ymdh=$ymdh_beg + if [ ${CFP_MP:-"NO"} = "YES" ]; then nm=0 ; fi # Counter for MP CFP while [ "$ymdh" -le "$ymdh_end" ] do - echo "$USHwave/wave_g2ges.sh $ymdh > grb_$ymdh.out 2>&1" >> cmdfile + if [ ${CFP_MP:-"NO"} = "YES" ]; then + echo "$nm $USHwave/wave_g2ges.sh $ymdh > grb_$ymdh.out 2>&1" >> cmdfile + nm=`expr $nm + 1` + else + echo "$USHwave/wave_g2ges.sh $ymdh > grb_$ymdh.out 2>&1" >> cmdfile + fi ymdh=`$NDATE $WAV_WND_HOUR_INC $ymdh` done @@ -322,14 +325,18 @@ set +x echo ' ' - echo " Executing the copy command file at : `date`" + echo " Executing the wnd grib cmd file at : `date`" echo ' ------------------------------------' echo ' ' [[ "$LOUD" = YES ]] && set -x if [ "$wavenproc" -gt '1' ] then - ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + if [ ${CFP_MP:-"NO"} = "YES" ]; then + ${wavempexec} -n ${wavenproc} ${wave_mpmd} cmdfile + else + ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + fi exit=$? else ./cmdfile @@ -423,7 +430,6 @@ echo '**********************************' echo ' Possibly in multiple calls' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : error in wind grib2 files." >> $wavelog set +x for file in grb_*.out do @@ -448,7 +454,6 @@ echo ' ' echo $msg [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : fatal error in grib2 wind files." >> $wavelog err=6;export err;${errchk} fi @@ -475,7 +480,6 @@ echo '******************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : no wind files found." >> $wavelog err=7;export err;${errchk} fi @@ -505,10 +509,10 @@ ln -sf mod_def.$grdID mod_def.ww3 set +x - echo "Executing $EXECcode/ww3_prnc" + echo "Executing $EXECwave/ww3_prnc" [[ "$LOUD" = YES ]] && set -x - $EXECcode/ww3_prnc > prnc.out + $EXECwave/ww3_prnc > prnc.out err=$? if [ "$err" != '0' ] @@ -522,7 +526,6 @@ echo '*************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $grdID $date $cycle : error in waveprnc." >> $wavelog err=8;export err;${errchk} fi @@ -539,7 +542,6 @@ echo '****************************************' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $grdID $date $cycle : wind.ww3 missing." >> $wavelog err=9;export err;${errchk} fi @@ -582,7 +584,7 @@ echo '************************************************' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $grdID $date $cycle : error in wind increment." >> $wavelog + postmsg "$jlogfile" "$WAV_MOD_TAG prep $grdID $date $cycle : error in wind increment." err=10;export err;${errchk} fi @@ -626,57 +628,57 @@ # Prepare files for cfp process rm -f cmdfile touch cmdfile - chmod 744 cmfile + chmod 744 cmdfile - ymdh_rtofs=${PDY}00 # RTOFS runs once daily - ymdh_end=`$NDATE ${FHMAX_WAV_CUR} ${ymdh_rtofs}` - NDATE_DT=${WAV_CUR_HF_DT} - FLGHF='T' + ymdh_rtofs=${PDY}00 # RTOFS runs once daily use ${PDY}00 + ymdh_end=`$NDATE ${FHMAX_WAV_CUR} ${PDY}00` + NDATE_DT=${WAV_CUR_HF_DT} + FLGHF='T' - while [ "$ymdh_rtofs" -le "$ymdh_end" ] - do + if [ ${CFP_MP:-"NO"} = "YES" ]; then nm=0 ; fi # Counter for MP CFP + while [ "$ymdh_rtofs" -le "$ymdh_end" ] + do # Timing has to be made relative to the single 00z RTOFS cycle for that PDY - fhr_rtofs=`${NHOUR} ${ymdh_rtofs} ${PDY}00` - fext='f' + fhr_rtofs=`${NHOUR} ${ymdh_rtofs} ${PDY}00` + fext='f' - if [ ${fhr_rtofs} -lt 0 ] - then -# Data from nowcast phase - fhr_rtofs=`expr 48 + ${fhr_rtofs}` - fext='n' - fi + fh3_rtofs=`printf "%03d" "${fhr_rtofs#0}"` - fhr_rtofs=`printf "%03d\n" ${fhr_rtofs}` + curfile1h=${COMIN_WAV_CUR}/rtofs_glo_2ds_${fext}${fh3_rtofs}_1hrly_prog.nc + curfile3h=${COMIN_WAV_CUR}/rtofs_glo_2ds_${fext}${fh3_rtofs}_3hrly_prog.nc - curfile1h=${COMIN_WAV_CUR}/rtofs_glo_2ds_${fext}${fhr_rtofs}_1hrly_prog.nc - curfile3h=${COMIN_WAV_CUR}/rtofs_glo_2ds_${fext}${fhr_rtofs}_3hrly_prog.nc + if [ -s ${curfile1h} ] && [ "${FLGHF}" = "T" ] ; then + curfile=${curfile1h} + elif [ -s ${curfile3h} ]; then + curfile=${curfile3h} + FLGHF='F' + else + echo ' ' + set $setoff + echo ' ' + echo '************************************** ' + echo "*** FATAL ERROR: NO CUR FILE $curfile *** " + echo '************************************** ' + echo ' ' + set $seton + postmsg "$jlogfile" "FATAL ERROR - NO CURRENT FILE (RTOFS)" + err=11;export err;${errchk} + exit 0 + echo ' ' + fi - if [ -s ${curfile1h} ] && [ "${FLGHF}" = "T" ] ; then - curfile=${curfile1h} - elif [ -s ${curfile3h} ]; then - curfile=${curfile3h} - FLGHF='F' - else - echo ' ' - set $setoff - echo ' ' - echo '************************************** ' - echo "*** FATAL ERROR: NO CUR FILE $curfile *** " - echo '************************************** ' - echo ' ' - set $seton - postmsg "$jlogfile" "FATAL ERROR - NO CURRENT FILE (RTOFS)" - err=11;export err;${errchk} - exit 0 - echo ' ' - fi + if [ ${CFP_MP:-"NO"} = "YES" ]; then + echo "$nm $USHwave/wave_prnc_cur.sh $ymdh_rtofs $curfile $fhr_rtofs > cur_$ymdh_rtofs.out 2>&1" >> cmdfile + nm=`expr $nm + 1` + else + echo "$USHwave/wave_prnc_cur.sh $ymdh_rtofs $curfile $fhr_rtofs > cur_$ymdh_rtofs.out 2>&1" >> cmdfile + fi - echo "$USHwave/wave_prnc_cur.sh $ymdh_rtofs $curfile > cur_$ymdh_rtofs.out 2>&1" >> cmdfile - if [ $fhr_rtofs -ge ${WAV_CUR_HF_FH} ] ; then - NDATE_DT=${WAV_CUR_DT} - fi - ymdh_rtofs=`$NDATE $NDATE_DT $ymdh_rtofs` - done + if [ $fhr_rtofs -ge ${WAV_CUR_HF_FH} ] ; then + NDATE_DT=${WAV_CUR_DT} + fi + ymdh_rtofs=`$NDATE $NDATE_DT $ymdh_rtofs` + done # Set number of processes for mpmd wavenproc=`wc -l cmdfile | awk '{print $1}'` @@ -684,14 +686,18 @@ set +x echo ' ' - echo " Executing the copy command file at : `date`" + echo " Executing the curr prnc cmdfile at : `date`" echo ' ------------------------------------' echo ' ' [[ "$LOUD" = YES ]] && set -x if [ $wavenproc -gt '1' ] then - ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + if [ ${CFP_MP:-"NO"} = "YES" ]; then + ${wavempexec} -n ${wavenproc} ${wave_mpmd} cmdfile + else + ${wavempexec} ${wavenproc} ${wave_mpmd} cmdfile + fi exit=$? else chmod 744 ./cmdfile @@ -711,7 +717,7 @@ [[ "$LOUD" = YES ]] && set -x fi - files=`ls ${WAVECUR_FID}.* 2> /dev/null` + files=`ls ${WAVECUR_DID}.* 2> /dev/null` if [ -z "$files" ] then @@ -724,7 +730,6 @@ echo '******************************************** ' echo ' ' [[ "$LOUD" = YES ]] && set -x - echo "$WAV_MOD_TAG prep $date $cycle : no current files found." >> $wavelog err=11;export err;${errchk} fi @@ -732,8 +737,8 @@ for file in $files do + echo $file cat $file >> cur.${WAVECUR_FID} - rm -f $file done cp -f cur.${WAVECUR_FID} ${COMOUT}/rundata/${COMPONENTwave}.${WAVECUR_FID}.$cycle.cur @@ -772,7 +777,6 @@ echo '*** FATAL ERROR : NO TEMPLATE FOR INPUT FILE *** ' echo '************************************************ ' echo ' ' - echo "${WAV_MOD_TAG} fcst $date $cycle : ww3_multi file missing." >> $wavelog echo $msg [[ "$LOUD" = YES ]] && set -x err=12;export err;${errchk} @@ -796,7 +800,6 @@ [[ "$LOUD" = YES ]] && set -x postmsg "$jlogfile" " FATAL ERROR : buoy.loc ($FIXwave/wave_${NET}.buoys) NOT FOUND" touch buoy.loc - echo "$WAV_MOD_TAG fcst $date $cycle : no buoy locations file ($FIXwave/wave_${NET}.buoys)." >> $wavelog err=13;export err;${errchk} fi @@ -831,35 +834,56 @@ WINDFLAG="$WAVEWND_FID" ;; 'CPL' ) - WINDFLAG="CPL:${waveesmfGRD}" WNDIFLAG='T' - CPLILINE=" '${waveesmfGRD}' F F T F F F F" + if [ ${waveesmfGRD} ] + then + WINDFLAG="CPL:${waveesmfGRD}" + CPLILINE=" '${waveesmfGRD}' F F T F F F F" + else + WINDFLAG="CPL:native" + fi ;; esac case ${WW3ICEINP} in 'YES' ) NFGRIDS=`expr $NFGRIDS + 1` + ICEIFLAG='T' ICELINE=" '$WAVEICE_FID' F F F T F F F" ICEFLAG="$WAVEICE_FID" ;; 'CPL' ) - ICEFLAG="CPL:${waveesmfGRD}" ICEIFLAG='T' - CPLILINE=" '${waveesmfGRD}' F F ${WNDIFLAG} T F F F" + if [ ${waveesmfGRD} ] + then + ICEFLAG="CPL:${waveesmfGRD}" + CPLILINE=" '${waveesmfGRD}' F F ${WNDIFLAG} T F F F" + else + ICEFLAG="CPL:native" + fi ;; esac case ${WW3CURINP} in 'YES' ) - NFGRIDS=`expr $NFGRIDS + 1` - CURRLINE=" '$WAVECUR_FID' F T F F F F F" - CURRFLAG="$WAVECUR_FID" + if [ "$WAVECUR_FID" != "$WAVEICE_FID" ]; then + NFGRIDS=`expr $NFGRIDS + 1` + CURRLINE=" '$WAVECUR_FID' F T F F F F F" + CURRFLAG="$WAVECUR_FID" + else # cur fields share the same grid as ice grid + ICELINE=" '$WAVEICE_FID' F T F ${ICEIFLAG} F F F" + CURRFLAG="$WAVEICE_FID" + fi ;; 'CPL' ) - CURRFLAG="CPL:${waveesmfGRD}" CURIFLAG='T' - CPLILINE=" '${waveesmfGRD}' F T ${WNDIFLAG} ${ICEIFLAG} F F F" + if [ ${waveesmfGRD} ] + then + CURRFLAG="CPL:${waveesmfGRD}" + CPLILINE=" '${waveesmfGRD}' F T ${WNDIFLAG} ${ICEFLAG} F F F" + else + CURRFLAG="CPL:native" + fi ;; esac @@ -883,7 +907,7 @@ sed -e "s/NFGRIDS/$NFGRIDS/g" \ -e "s/NMGRIDS/${NMGRIDS}/g" \ -e "s/FUNIPNT/${FUNIPNT}/g" \ - -e "s/PNTSRV/${PNTSRV}/g" \ + -e "s/IOSRV/${IOSRV}/g" \ -e "s/FPNTPROC/${FPNTPROC}/g" \ -e "s/FGRDPROC/${FGRDPROC}/g" \ -e "s/OUTPARS/${OUTPARS_WAV}/g" \ @@ -901,6 +925,8 @@ -e "s/OUT_BEG/$time_beg_out/g" \ -e "s/OUT_END/$time_end/g" \ -e "s/DTFLD/ $DTFLD_WAV/g" \ + -e "s/FLAGMASKCOMP/ $FLAGMASKCOMP/g" \ + -e "s/FLAGMASKOUT/ $FLAGMASKOUT/g" \ -e "s/GOFILETYPE/ $GOFILETYPE/g" \ -e "s/POFILETYPE/ $POFILETYPE/g" \ -e "s/FIELDS/$FIELDS/g" \ @@ -922,32 +948,33 @@ if [ -f ww3_multi.inp ] then echo " Copying file ww3_multi.${WAV_MOD_TAG}.inp to $COMOUT " - cp ww3_multi.inp ${COMOUT}/rundata/ww3_multi.${WAV_MOD_TAG}.$cycle.inp + cp ww3_multi.inp ${COMOUT}/rundata/ww3_multi.${WAV_MOD_TAG}.${cycle}.inp else - echo "FATAL ERROR: file ww3_multi.${WAV_MOD_TAG}.$cycle.inp NOT CREATED, ABORTING" + echo "FATAL ERROR: file ww3_multi.${WAV_MOD_TAG}.${cycle}.inp NOT CREATED, ABORTING" err=13;export err;${errchk} fi # 6. Copy rmp grid remapping pre-processed coefficients - if ls $FIXwave/rmp_src_to_dst_conserv_* 2> /dev/null - then - for file in $(ls $FIXwave/rmp_src_to_dst_conserv_*) ; do - cp -f $file ${COMOUT}/rundata - done - else - msg="NO rmp precomputed nc files found, is this OK???" - postmsg "$jlogfile" "$msg" - set +x - echo ' ' - echo '************************************************ ' - echo '*** FATAL ERROR : NO PRECOMPUTED RMP FILES FOUND *** ' - echo '************************************************ ' - echo ' ' - echo "${WAV_MOD_TAG} prep $date $cycle : rmp*.nc not found." >> $wavelog - echo $msg - [[ "$LOUD" = YES ]] && set -x - err=13;export err;${errchk} + if [ "${USE_WAV_RMP:-YES}" = "YES" ]; then + if ls $FIXwave/rmp_src_to_dst_conserv_* 2> /dev/null + then + for file in $(ls $FIXwave/rmp_src_to_dst_conserv_*) ; do + cp -f $file ${COMOUT}/rundata + done + else + msg="NO rmp precomputed nc files found, is this OK???" + postmsg "$jlogfile" "$msg" + set +x + echo ' ' + echo '************************************************ ' + echo '*** FATAL ERROR : NO PRECOMPUTED RMP FILES FOUND *** ' + echo '************************************************ ' + echo ' ' + echo $msg + [[ "$LOUD" = YES ]] && set -x + err=13;export err;${errchk} + fi fi @@ -1001,7 +1028,6 @@ echo ' ' [[ "$LOUD" = YES ]] && set -x - msg="$job completed normally" - postmsg "$jlogfile" "$msg" + exit $err # End of MWW3 preprocessor script ------------------------------------------- # diff --git a/scripts/run_reg2grb2.sh b/scripts/run_reg2grb2.sh new file mode 100755 index 0000000000..ffe0a0a57d --- /dev/null +++ b/scripts/run_reg2grb2.sh @@ -0,0 +1,71 @@ +#!/bin/bash +set -x + +#requires grib_util module + +MOM6REGRID=${MOM6REGRID:-$HOMEgfs} +export mask_file=$MOM6REGRID/fix/fix_reg2grb2/mask.0p25x0p25.grb2 + +# offline testing: +#export DATA= +#export icefile=$DATA/DATA0p5/icer2012010106.01.2012010100_0p5x0p5.nc +#export ocnfile=$DATA/DATA0p5/ocnr2012010106.01.2012010100_0p5x0p5.nc +#export outfile=$DATA/DATA0p5/out/ocnh2012010106.01.2012010100.grb2 +# +# workflow testing: +export icefile=icer${CDATE}.${ENSMEM}.${IDATE}_0p25x0p25_CICE.nc +export ocnfile=ocnr${CDATE}.${ENSMEM}.${IDATE}_0p25x0p25_MOM6.nc +export outfile=ocn_ice${CDATE}.${ENSMEM}.${IDATE}_0p25x0p25.grb2 +export outfile0p5=ocn_ice${CDATE}.${ENSMEM}.${IDATE}_0p5x0p5.grb2 + +export mfcstcpl=${mfcstcpl:-1} +export IGEN_OCNP=${IGEN_OCNP:-197} + +# PT This is the forecast date +export year=`echo $CDATE | cut -c1-4` +export month=`echo $CDATE | cut -c5-6` +export day=`echo $CDATE | cut -c7-8` +export hour=`echo $CDATE | cut -c9-10` + +# PT This is the initialization date +export syear=`echo $IDATE | cut -c1-4` +export smonth=`echo $IDATE | cut -c5-6` +export sday=`echo $IDATE | cut -c7-8` +export shour=`echo $IDATE | cut -c9-10` + +# PT Need to get this from above - could be 6 or 1 hour +export hh_inc_ocn=6 +# +# set for 1p0 lat-lon +#export im=360 +#export jm=181 +# export km=40 +#export imo=360 +#export jmo=181 +# +# set for 0p5 lat-lon +#export im=720 +#export jm=361 +#export km=40 +#export imo=720 +#export jmo=361 +# +# set for 0p25 lat-lon +export im=1440 +export jm=721 +export imo=1440 +export jmo=721 +export km=40 + +export flats=-90. +export flatn=90. +export flonw=0.0 +export flone=359.75 + +ln -sf $mask_file ./iceocnpost.g2 +$executable > reg2grb2.$CDATE.$IDATE.out + +# interpolated from 0p25 to 0p5 grid +grid2p05="0 6 0 0 0 0 0 0 720 361 0 0 90000000 0 48 -90000000 359500000 500000 500000 0" +#### $NWPROD/util/exec/copygb2 -g "${grid2p05}" -i0 -x $outfile $outfile0p5 +$COPYGB2 -g "${grid2p05}" -i0 -x $outfile $outfile0p5 diff --git a/scripts/run_regrid.sh b/scripts/run_regrid.sh new file mode 100755 index 0000000000..e99913d835 --- /dev/null +++ b/scripts/run_regrid.sh @@ -0,0 +1,24 @@ +#!/bin/bash +set -x + +echo "Entered $0" +MOM6REGRID=${MOM6REGRID:-$HOMEgfs} +export EXEC_DIR=$MOM6REGRID/exec +export USH_DIR=$MOM6REGRID/ush +export COMOUT=$COMOUT +export IDATE=$IDATE +export ENSMEM=$ENSMEM +export FHR=$fhr +export DATA=$DATA +export OCNFIXDIR=$OCNFIXDIR + +###### DO NOT MODIFY BELOW UNLESS YOU KNOW WHAT YOU ARE DOING ####### +#Need NCL module to be loaded: +echo $NCARG_ROOT +export NCL=$NCARG_ROOT/bin/ncl + +ls -alrt + +$NCL $USH_DIR/icepost.ncl > regrid_CICE.log 2>&1 +$NCL $USH_DIR/ocnpost.ncl > regrid_MOM6.log 2>&1 +##################################################################### diff --git a/sorc/build_all.sh b/sorc/build_all.sh index ff4eca3079..944d8b72fe 100755 --- a/sorc/build_all.sh +++ b/sorc/build_all.sh @@ -1,5 +1,5 @@ #!/bin/sh -set -eu +set -x #------------------------------------ # USER DEFINED STUFF: # @@ -207,6 +207,16 @@ echo " .... grib_util build not currently supported .... " #./build_grib_util.sh > $logs_dir/build_grib_util.log 2>&1 } +#------------------------------------ +# build reg2grb2 +#------------------------------------ +if [ $target = hera -o $target = orion ]; then + $Build_reg2grb2 && { + echo " .... Building reg2grb2 .... " + ./build_reg2grb2.sh > $logs_dir/build_reg2grb2.log 2>&1 +} +fi + echo;echo " .... Build system finished .... " exit 0 diff --git a/sorc/build_fv3_coupled.sh b/sorc/build_fv3_coupled.sh index b7ea3cc038..88cf75fe65 100755 --- a/sorc/build_fv3_coupled.sh +++ b/sorc/build_fv3_coupled.sh @@ -16,7 +16,6 @@ if [ ! -d "../exec" ]; then mkdir ../exec fi -if [ $target = theia ]; then target=theia.intel ; fi cd fv3_coupled.fd/NEMS make -j 8 app=coupledFV3_CCPP_MOM6_CICE_WW3 build diff --git a/sorc/build_reg2grb2.sh b/sorc/build_reg2grb2.sh new file mode 100755 index 0000000000..16d627d6fd --- /dev/null +++ b/sorc/build_reg2grb2.sh @@ -0,0 +1,41 @@ +#! /usr/bin/env bash +set -x + +# Check final exec folder exists +if [ ! -d "../exec" ]; then + mkdir ../exec +fi + +source ./machine-setup.sh > /dev/null 2>&1 + +source ../modulefiles/modulefile.reg2grb2.$target + +export FCMP=${FCMP:-ifort} +export FCMP95=$FCMP +export CPP=fpp + +export INCS="-I${LANDSFCUTIL_INCd} \ + -I${IP_INCd} \ + -I${NETCDF}/include" + +export FFLAGSM="-O3 -free -convert big_endian -traceback -qopenmp -fp-model precise -assume byterecl ${INCS}" +export FFLAGSM2="-O3 -free -r8" + +export LIBSM="${LANDSFCUTIL_LIBd} \ + ${IP_LIBd} \ + ${SP_LIBd} \ + ${W3NCO_LIB4} \ + ${BACIO_LIB4} \ + -L${NETCDF}/lib -lnetcdff -lnetcdf" + +WGRIB2DIR=$WGRIB2_ROOT +export LIB2="-L$WGRIB2DIR/lib -lwgrib2 -lwgrib2_api" +export MOD2="-I$WGRIB2DIR/lib" + +cd reg2grb2.fd + +pwd + +make -f Makefile clean +make -f Makefile +make -f Makefile install diff --git a/sorc/checkout.sh b/sorc/checkout.sh index 70ac21bdd7..3a6707ba33 100755 --- a/sorc/checkout.sh +++ b/sorc/checkout.sh @@ -16,7 +16,7 @@ rm -f ${topdir}/checkout-fv3_coupled.log if [[ ! -d fv3_coupled.fd ]] ; then git clone https://github.com/ufs-community/ufs-s2s-model fv3_coupled.fd >> ${topdir}/checkout-fv3_coupled.log 2>&1 cd fv3_coupled.fd - git checkout s2s_prototype4.0 + git checkout ufss2s_cmeps_v0.7 git submodule update --init --recursive cd ${topdir} else diff --git a/sorc/fv3gfs_build.cfg b/sorc/fv3gfs_build.cfg index 09c095e215..05f65b0052 100644 --- a/sorc/fv3gfs_build.cfg +++ b/sorc/fv3gfs_build.cfg @@ -21,5 +21,6 @@ Building gfs_util (gfs_util) .......................... yes Building prod_util (prod_util) ........................ no Building grib_util (grib_util) ........................ no + Building reg2grb2 (reg2grb2) .......................... yes # -- END -- diff --git a/sorc/link_fv3gfs.sh b/sorc/link_fv3gfs.sh index 1cf2eb4878..643a5092a0 100755 --- a/sorc/link_fv3gfs.sh +++ b/sorc/link_fv3gfs.sh @@ -336,19 +336,18 @@ fi #--choose dynamic config.base for EMC installation #--choose static config.base for NCO installation cd $pwd/../parm/config -[[ -s config.base ]] && rm -f config.base -if [ $RUN_ENVIR = nco ] ; then - cp -p config.base.nco.static config.base -else - cp -p config.base.emc.dyn config.base -fi +#[[ -s config.base ]] && rm -f config.base +#if [ $RUN_ENVIR = nco ] ; then +# cp -p config.base.nco.static config.base +#else +# cp -p config.base.emc.dyn config.base +#fi #------------------------------ if [ $model = "coupled" ] ; then - rm -f config.base - cp -p config.base.emc.dyn_coupled config.base + #rm -f config.base + #cp -p config.base.emc.dyn_coupled config.base cd $pwd/../fix # Add fixed files needed for coupled ufs-s2s-model - $LINK $FIX_DIR/fix_ocnice . $LINK $FIX_DIR/fix_cice5 . $LINK $FIX_DIR/fix_mom6 . $LINK $FIX_DIR/fix_fv3grid . diff --git a/sorc/partial_build.sh b/sorc/partial_build.sh index 885548f052..6b6351e061 100755 --- a/sorc/partial_build.sh +++ b/sorc/partial_build.sh @@ -16,7 +16,8 @@ "Build_regrid_nemsio" \ "Build_gfs_util" \ "Build_prod_util" \ - "Build_grib_util") + "Build_grib_util" \ + "Build_reg2grb2") # # function parse_cfg: read config file and retrieve the values diff --git a/sorc/reg2grb2.fd/Makefile b/sorc/reg2grb2.fd/Makefile new file mode 100755 index 0000000000..edd523f061 --- /dev/null +++ b/sorc/reg2grb2.fd/Makefile @@ -0,0 +1,26 @@ +SHELL =/bin/sh +INCMOD =${INCS} + +SRCM =reg2grb2.f +OBJS =regdiag.o +FC =ifort +FC90 =ifort + +FFLAGS = ${FFLAGSM} +FFLAGS2 = ${FFLAGSM2} + +NCDFLIB = ${NCDF} +LIBS = ${LIBSM} ${NCDFLIB} + +CMD = reg2grb2.x +$(CMD): $(SRCM) $(OBJS) + $(FCMP) $(FFLAGS) -o $(CMD) $(LDFLAGS) $(SRCM) $(OBJS) $(LIBS) ${MOD2} ${LIB2} + +regdiag.o:regdiag.f + $(FCMP) $(FFLAGS) -I $(INCMOD) -c regdiag.f ${MOD2} ${LIB2} + +clean: + -rm -f $(OBJS) *.mod + +install: + mv $(CMD) ../../exec/$(CMD) diff --git a/sorc/reg2grb2.fd/reg2grb2.f b/sorc/reg2grb2.fd/reg2grb2.f new file mode 100755 index 0000000000..6fd98cd35a --- /dev/null +++ b/sorc/reg2grb2.fd/reg2grb2.f @@ -0,0 +1,185 @@ + program reg2grb2 + +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! +! ! +! This program reads lat-lon ocean and ice data, calculates diagnostics, ! +! and saves them in grib2 format. ! +! ! +! Xingren Wu (Xingren.Wu@noaa.gov) ! +! April 20, 2007 ! +! ! +! Xingren Wu - Update for MOM5/CICE with grib2 output ! +! February 16, 2017 ! +! ! +! Christopher Melhauesr - Update for MOM6/CICE5 with grib2 output ! +! September 8th, 2017 ! +! ! +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! + + use regdiag_mod, only: diag + use wgrib2api + + implicit none + + integer iyr,imth,iday,ihr,im,jm,km,imo,jmo + integer isyr,ismth,isday,ishr + integer mfh,mfhout,mfcstcpl,igenocnp +! integer idbug,mkmoc,jmtpo,jmtp,imos,nreg + + real*4 dlat,dlon,flats,flatn,flonw,flone +! real*4 tripolat + CHARACTER*1 kmfcstcpl +! CHARACTER*1 kdbug,kmkmoc + CHARACTER*300 icefile,ocnfile,outfile +! CHARACTER*120 mocfile,cicefile + CHARACTER*10 syr,smth,sday,shr + CHARACTER*10 kyr,kmth,kday,khr + CHARACTER*10 kfh,kfhout + CHARACTER*10 kim,kjm,kkm +! CHARACTER*10 knreg,kimos,kjmtp,kjmtpo + CHARACTER*10 kimo,kjmo + CHARACTER*10 klats,klatn,klonw,klone,kigenocnp +! CHARACTER*10 ktripolat + + call start() +! call getenv("idbug",kdbug) +! read(kdbug,'(i1)') idbug +! write(*,*) "idbug= ",idbug +!! +! call getenv("mkmoc",kmkmoc) +! read(kmkmoc,'(i1)') mkmoc +! write(*,*) "mkmoc= ",mkmoc +! + call getenv("mfcstcpl",kmfcstcpl) + read(kmfcstcpl,'(i1)') mfcstcpl + write(*,*) "mfcstcpl= ",mfcstcpl +! + call getenv("IGEN_OCNP",kigenocnp) + read(kigenocnp,'(i3)') igenocnp + write(*,*) "IGEN_OCNP= ",igenocnp +! + call getenv("icefile",icefile) + write(*,*) "icefile= ",icefile +! + call getenv("ocnfile",ocnfile) + write(*,*) "ocnfile= ",ocnfile +!! +! call getenv("cicefile",cicefile) +! write(*,*) "cicefile= ",cicefile +!! + call getenv("outfile",outfile) + write(*,*) "outfile= ",outfile +!! +! call getenv("mocfile",mocfile) +! write(*,*) "mocfile= ",mocfile +!! + call getenv("syear",syr) + read(syr,'(I4)') isyr + write(*,*) "syear= ",isyr +! + call getenv("smonth",smth) + read(smth,'(I2)') ismth + write(*,*) "smonth= ",ismth +! + call getenv("sday",sday) + read(sday,'(I2)') isday + write(*,*) "sday= ",isday +! + call getenv("shour",shr) + read(shr,'(I2)') ishr + write(*,*) "shour= ",ishr +! + call getenv("year",kyr) + read(kyr,'(I4)') iyr + write(*,*) "year= ",iyr +! + call getenv("month",kmth) + read(kmth,'(I2)') imth + write(*,*) "month= ",imth +! + call getenv("day",kday) + read(kday,'(I2)') iday + write(*,*) "day= ",iday +! + call getenv("hour",khr) + read(khr,'(I2)') ihr + write(*,*) "hour= ",ihr +! + call getenv("fh",kfh) + read(kfh,'(I10)') mfh + write(*,*) "fh= ",mfh +! + call getenv("hh_inc_ocn",kfhout) + read(kfhout,'(I10)') mfhout + write(*,*) "hh_inc_ocn= ",mfhout +! + call getenv("im",kim) + read(kim,'(I10)') im + write(*,*) "im= ",im +! + call getenv("jm",kjm) + read(kjm,'(I10)') jm + write(*,*) "jm= ",jm +! + call getenv("imo",kimo) + read(kimo,'(I10)') imo + write(*,*) "imo= ",imo +! + call getenv("jmo",kjmo) + read(kjmo,'(I10)') jmo + write(*,*) "jmo= ",jmo +!! +! call getenv("jmtp",kjmtp) +! read(kjmtp,'(I10)') jmtp +! write(*,*) "jmtp= ",jmtp +!! +! call getenv("jmtpo",kjmtpo) +! read(kjmtpo,'(I10)') jmtpo +! write(*,*) "jmtpo= ",jmtpo +!! +! call getenv("imos",kimos) +! read(kimos,'(I10)') imos +! write(*,*) "imos= ",imos +!! + call getenv("km",kkm) + read(kkm,'(I10)') km + write(*,*) "km= ",km +! + call getenv("flats",klats) + read(klats,'(f8.1)') flats + write(*,*) "flats= ",flats +! + call getenv("flatn",klatn) + read(klatn,'(f8.1)') flatn + write(*,*) "flatn= ",flatn +! + call getenv("flonw",klonw) + read(klonw,'(f8.1)') flonw + write(*,*) "flonw= ",flonw +! + call getenv("flone",klone) + read(klone,'(f8.1)') flone + write(*,*) "flone= ",flone +! +! call getenv("nreg",knreg) +! read(knreg,'(i10)') nreg +! write(*,*) "nreg= ",nreg +!! +! call getenv("tripolat",ktripolat) +! read(ktripolat,'(f8.1)') tripolat +! write(*,*) "tripolat= ",tripolat +!! + dlat = (flatn-flats) / (jmo-1) +! + dlon = (flone-flonw) / (imo-1) +! + call diag(im,jm,km,icefile,ocnfile, & + isyr,ismth,isday,ishr,iyr,imth,iday,ihr,mfh,mfhout, & + flonw,flone,dlon,flatn,flats,dlat,imo,jmo, & + outfile,mfcstcpl,igenocnp) +! + call summary() + call errexit(0) + stop + end + diff --git a/sorc/reg2grb2.fd/regdiag.f b/sorc/reg2grb2.fd/regdiag.f new file mode 100755 index 0000000000..1130ce9476 --- /dev/null +++ b/sorc/reg2grb2.fd/regdiag.f @@ -0,0 +1,1431 @@ +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! +! ! +! This program reads ocean and ice data on lat/lon grid ! +! It contains 4 subroutines: diag, read_ice, read_ocn ! +! ! +! Xingren Wu (Xingren.Wu@noaa.gov) ! +! May 18, 2007 ! +! Xingren Wu ! +! Nov 16, 2007 - modified ! +! fix kpds(13) to kpds(15) for handle different forecast time unit ! +! Xingren Wu ! +! Dec 4, 2007 - modified ! +! add extra fields to match production output ! +! Xingren Wu ! +! Sep 7, 2016 - modified ! +! add cice ! +! Xingren Wu ! +! Feb 10, 2017 - modified ! +! writing grib2 ! +! Christopher Melhauser ! +! Sep 8, 2017 - overhauled code for latlon --> grb2 only ! +! Xingren Wu ! +! Oct 23, 2017 - Bug/fix ! +! Suranjana Saha ! +! Nov 8, 2017 - Added new variables in the MOM6 NetCDF file to grib2 ! +! ! +! Xingren Wu ! +! Nov 22, 2017 - Fixed the following ! +! 1. Ice concentration ! +! 2. Ice thickness ! +! 3. Snow depth ! +! 4. Surface Temperature over Water and Ice ! +! 5. u-component of ice drift ! +! 6. v-component of ice drift ! +! Mar 30, 2018 - Bug fix ! +! ! +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! +! Notes: ! +! 1. uflx, vflx, lhtfl and shtfl are opposite in sign to data in the flx file ! +! 2. nlwrs=net lw radiation at sfc i.e dlwrfavesfc-ulwrfavesfc flx file ! +! 3. nswrs=net sw radiation at sfc i.e. dswrfavesfc-uswrfavesfc in flx file ! +! 4. sfc_hflux (THFLX) = Total net heat, i.e. ! +! dswrfavesfc-uswrfavesfc+dlwrfavesfc-ulwrfavesfc-lhtflsfc-shtflsfc in flx ! +! 5. evp=evaporation i.e -lhtflsfc*0.03456 in flx file ! +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! +! Note that the input NetCDF and output grib2 files have the following: ! +! z=1 is topmost level; z=40 is bottom most level ! +! even though grads control file puts z=1 at the bottom and z=40 at the top ! +!~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~! + +module regdiag_mod +use wgrib2api + + implicit none + +contains + + subroutine read_ocn(im,jm,km,lon,lat, & + t,s,u,v,eta,sfcflx,pme,mld,taux,tauy,ocean_file, & + sss,ssu,ssv,speed,sensible,latent,sw,lw,lprec,evap) + include "netcdf.inc" + integer im,jm,km,i,j,k + integer status, ncid + integer :: id_lat,id_lon, & + id_temp,id_salt,id_u,id_v,id_wt, & + id_eta_t,id_sfc_hflux,id_pme,id_mld,id_tau_x,id_tau_y, & + id_sss,id_ssu,id_ssv,id_speed, & + id_sensible,id_latent,id_sw,id_lw,id_lprec,id_evap + + real, dimension(im,jm,km) :: t,s,u,v + real, dimension(im,jm) :: eta,sfcflx,pme,mld,taux,tauy,LwLatSens + real, dimension(im,jm) :: sss,ssu,ssv,speed,sensible,latent,sw,lw + real, dimension(im,jm) :: lprec,evap + real, dimension(im) :: lon + real, dimension(jm) :: lat + real, dimension(km,jm,im) :: tmp3D + real, dimension(jm,im) :: tmp2D + character(len=300) :: ocean_file + + real, parameter :: c2k=273.15 + real :: undef + + undef=-1.0E+34 + status = nf_open(ocean_file, NF_NOWRITE, ncid) + if(status.ne.NF_NOERR) stop 'cannot open ocean_file' + + status = nf_inq_varid(ncid, 'temp' , id_temp) + status = nf_inq_varid(ncid, 'so' , id_salt) + status = nf_inq_varid(ncid, 'uo' , id_u ) + status = nf_inq_varid(ncid, 'vo' , id_v ) + status = nf_inq_varid(ncid, 'SSH' , id_eta_t) + status = nf_inq_varid(ncid, 'LwLatSens',id_sfc_hflux) + status = nf_inq_varid(ncid, 'Heat_PmE', id_pme) + status = nf_inq_varid(ncid, 'mld' , id_mld) + status = nf_inq_varid(ncid, 'taux' , id_tau_x) + status = nf_inq_varid(ncid, 'tauy' , id_tau_y) + status = nf_inq_varid(ncid, 'lon' , id_lon) + status = nf_inq_varid(ncid, 'lat' , id_lat) + status = nf_inq_varid(ncid, 'SSS' , id_sss) + status = nf_inq_varid(ncid, 'SSU' , id_ssu) + status = nf_inq_varid(ncid, 'SSV' , id_ssv) + status = nf_inq_varid(ncid, 'speed' , id_speed) + status = nf_inq_varid(ncid, 'sensible', id_sensible) + status = nf_inq_varid(ncid, 'latent' , id_latent) + status = nf_inq_varid(ncid, 'SW' , id_sw) + status = nf_inq_varid(ncid, 'LW' , id_lw) + status = nf_inq_varid(ncid, 'lprec' , id_lprec) + status = nf_inq_varid(ncid, 'evap' , id_evap) + +! read variable + status = nf_get_var_real(ncid, id_lon, lon) + status = nf_get_var_real(ncid, id_lat, lat) + status = nf_get_var_real(ncid, id_temp, t) !temp + status = nf_get_var_real(ncid, id_salt, s) !salt + status = nf_get_var_real(ncid, id_u, u) !u + status = nf_get_var_real(ncid, id_v, v) !v + status = nf_get_var_real(ncid, id_eta_t, eta) !eta_t + status = nf_get_var_real(ncid, id_sfc_hflux, LwLatSens) !sfc_hflux + status = nf_get_var_real(ncid, id_pme, pme) !pme + status = nf_get_var_real(ncid, id_mld, mld) !mld + status = nf_get_var_real(ncid, id_tau_x, taux) !tau_x + status = nf_get_var_real(ncid, id_tau_y, tauy) !tau_y + status = nf_get_var_real(ncid, id_sss, sss) !sss + status = nf_get_var_real(ncid, id_ssu, ssu) !ssu + status = nf_get_var_real(ncid, id_ssv, ssv) !ssv + status = nf_get_var_real(ncid, id_speed, speed) !speed + status = nf_get_var_real(ncid, id_sensible, sensible) !sensible + status = nf_get_var_real(ncid, id_latent, latent) !latent + status = nf_get_var_real(ncid, id_sw, sw) !sw + status = nf_get_var_real(ncid, id_lw, lw) !lw + status = nf_get_var_real(ncid, id_lprec, lprec) !lprec + status = nf_get_var_real(ncid, id_evap, evap) !evap + +! status = nf_get_var_real(ncid, id_fprec, fprec) !fprec +! status = nf_get_var_real(ncid, id_lrunoff, lrunoff) !lrunoff +! status = nf_get_var_real(ncid, id_frunoff, frunoff) !frunoff + + sfcflx=SW+LwLatSens + +! convert all temps in Celcius that are read in to Kelvin below.. + + t=t+c2k + + return + + end subroutine read_ocn + + subroutine flip_ijk(im,jm,km,datain,dataout) + implicit none + integer :: im,jm,km,i,j,k + real, intent(out), dimension(im,jm,km) :: dataout + real, intent(in), dimension(km,jm,im) :: datain + do k=1,km + do j=1,jm + do i=1,im + dataout(i,j,k)=datain(k,j,i) + enddo + enddo + enddo + + return + end subroutine flip_ijk + + subroutine flip_ij(im,jm,datain,dataout) + implicit none + integer :: im,jm,i,j + real, intent(out), dimension(im,jm) :: dataout + real, intent(in), dimension(jm,im) :: datain + + do j=1,jm + do i=1,im + dataout(i,j)=datain(j,i) + enddo + enddo + + return + end subroutine flip_ij + + subroutine set_undef(im,jm,spv,undef,data) + implicit none + integer :: im,jm,i,j + real :: spv,undef + real, intent(inout), dimension(im,jm) :: data + + do j=1,jm + do i=1,im + if (data(i,j) .GE. spv) data(i,j)=undef + enddo + enddo + + return + end subroutine set_undef + + subroutine read_ice(im,jm,hi_h,hs_h,aice_h,ts_h,uvel_h,vvel_h,icefile) + + include "netcdf.inc" + + character(len=300) :: icefile + integer :: status,ncid + integer :: i,j + integer :: im,jm + integer :: id_hi_h,id_hs_h,id_tsfc_h,id_aice_h,id_sst_h,id_uvel_h,id_vvel_h + + real :: spv_ci,undef + real, dimension(im,jm) :: hi_h,hs_h,Tsfc_h,aice_h,sst_h,ts_h,uvel_h,vvel_h,fi,fw + + spv_ci=1.0e+30 + undef=-1.0E+34 + print *, 'icefile:', icefile + status = nf_open(icefile, NF_NOWRITE, ncid) + print *, 'status:', status + if(status.ne.NF_NOERR) stop 'cannot open ice_file' + + status = nf_inq_varid(ncid, 'hi_h ', id_hi_h) + status = nf_inq_varid(ncid, 'hs_h ', id_hs_h) + status = nf_inq_varid(ncid, 'Tsfc_h ', id_tsfc_h) + status = nf_inq_varid(ncid, 'aice_h ', id_aice_h) + status = nf_inq_varid(ncid, 'sst_h ', id_sst_h) + status = nf_inq_varid(ncid, 'uvel_h ', id_uvel_h) + status = nf_inq_varid(ncid, 'vvel_h ', id_vvel_h) + + status = nf_get_var_real(ncid, id_hi_h ,hi_h) + status = nf_get_var_real(ncid, id_hs_h ,hs_h) + status = nf_get_var_real(ncid, id_tsfc_h ,Tsfc_h) + status = nf_get_var_real(ncid, id_aice_h ,aice_h) + status = nf_get_var_real(ncid, id_sst_h ,sst_h) + status = nf_get_var_real(ncid, id_uvel_h ,uvel_h) + status = nf_get_var_real(ncid, id_vvel_h ,vvel_h) + + do j=1,jm + do i=1,im + if (aice_h(i,j) < spv_ci) then + if (aice_h(i,j) > 1.0) then + print *,'Warning: aice_h>1:',aice_h(i,j) + aice_h(i,j)=1.0 + endif + fi(i,j)=aice_h(i,j) + else + fi(i,j)=0.0 + endif + fw=1.0-fi(i,j) + ts_h(i,j)=fw(i,j)*sst_h(i,j)+fi(i,j)*Tsfc_h(i,j) + enddo + enddo + + + return + end subroutine read_ice + + subroutine diag(im,jm,km,icefile,ocnfile, & + isyr,ismth,isday,ishr,iyr,imth,iday,ihr,mfh,mfhout, & + flonw,flone,dlon,flatn,flats,dlat,imo,jmo, & + outfile,mfcstcpl,igenocnp) + + integer nvar,ko,ki,kk,ndtc,mfcstcpl,igenocnp +! integer mkmoc,nreg + real hfc + + parameter(nvar=37,ko=40,ki=5) + parameter(ndtc=7) +! + character*300 cicefile,icefile,ocnfile,outfile +! character*120 mocfile + character*120 template_file,template_inv,metadata,template_var + character*10 datecode + character*4 level4 + character*8 hr8 + character*80 levelm(nvar) + character*80 levelcode,ftime + character*5 varcode(nvar) + + integer im,jm,km + integer imo,jmo + integer iyr,imth,iday,ihr,mfh,mfhout + integer isyr,ismth,isday,ishr + integer hrdif + integer isdate(8),iedate(8) + real dlat,dlon,flats,flatn,flonw,flone +! real tripolat,dtripolat +! integer jtripolat + real factor,undef,spv_ci,spv_pme,spv_tau,val + integer i,j,k,ierr,ilev,ind,iret,nv,ndata,nr,nundef,kreg + integer nx,ny + real datedif(5) +! + real*8, dimension(im,jm) :: hi,hs,ts,t1,t2,fi,alb,ui,vi,sst,saltf +! + real, dimension(im,jm) :: hi_ci,hs_ci,ts_ci,fi_ci,ui_ci,vi_ci +! + real, dimension(im,jm,km) :: t,s,u,v,w,vv +! real, dimension(im,jm,km) :: dckt,dcks,vfc + real, dimension(im,jm) :: eta,sfcflx,pme,mld,taux,tauy,uice,vice + real, dimension(im,jm) :: sss,ssu,ssv,speed,sensible,latent,sw,lw + real, dimension(im,jm) :: lprec,evap + real, dimension(im,jm) :: varice + real, dimension(im) :: lon + real, dimension(jm) :: lat +! + real, dimension(imo,jmo) :: grid,grdtmp,varsfc + real, dimension(imo,jmo,km) :: varocn,tocn,socn + + real zt(ko) + real zw(ko) + real dtc(ndtc) + real grid2(imo*jmo) +! + real, dimension(nvar) :: fac + integer, dimension(nvar) :: kpds5,kpds6,kpds7,kpds22 + integer, dimension(ko) :: levs + + integer, parameter :: kpds_dim=200 + integer, dimension(kpds_dim) :: KPDS,KGDS,JPDS,JGDS + logical*1 lbms(imo,jmo) + logical :: climate = .false. +! new wgrib2api requires this + integer, parameter :: regex=1 +! + data dtc/2.5,5.,10.,15.,20.,25.,28./ +! +! NV kpds5=Variable (Parameter Table) +! http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html#TABLE128 +! +! 1. 13=Potential temperature (2) +! 2. 88=Salinity (2) +! 3. 49=u-component of current (2) +! 4. 50=v-component of current (2) +! 5. 40=Geometric Vertical velocity (2) +! 6. 124=Momentum flux, u component (2) +! 7. 125=Momentum flux, v component (2) +! 8. 198=Sea Surface Height Relative to Geoid (129) +! 9. 91=Ice concentration (2) +! 10. 92=Ice thickness (2) +! 11. 66=Snow depth (2) +! 12. 11=Surface Temperature over Water and Ice(2) +! 13. 95=u-component of ice drift (2) +! 14. 96=v-component of ice drift (2) +! 15. 188=Evaporation - Precipitation (2) +! 16. 202=Total downward heat flux at surface (downward is positive) (129) +! 17. 195=Geometric Depth Below Sea Surface (129) +! 18. 195=Geometric Depth Below Sea Surface (129) +! 19. 197=Ocean Heat Content (129) +! 20. 194=Tropical Cyclone Heat Potential (129) +! 21. 195=Geometric Depth Below Sea Surface for the 2.5C isotherm (129) +! 22. 195=Geometric Depth Below Sea Surface for the 5C isotherm (129) +! 23. 195=Geometric Depth Below Sea Surface for the 10C isotherm (129) +! 24. 195=Geometric Depth Below Sea Surface for the 15C isotherm (129) +! 25. 195=Geometric Depth Below Sea Surface for the 20C isotherm (129) +! 26. 195=Geometric Depth Below Sea Surface for the 25C isotherm (129) +! 27. 195=Geometric Depth Below Sea Surface for the 28C isotherm (129) +! 28. 88=Sea Surface Salinity (2) +! 29. 49=Sea Surface u-current (2) +! 30. 50=Sea Surface v-current (2) +! 31. 32=Sea Surface speed (2) +! 32. 122=Sensible Heat (2) +! 33. 121=Latent Heat (2) +! 34. 111=Net surface Downward Short Wave flux (2) +! 35. 112=Net surface Downward Long Wave flux (2) +! 36. 59=Precipitation (2) +! 37. 57=Evaporation (2) +! + data kpds5/ 13, 88, 49, 50, 40,124,125,198, 91, 92, & + 66, 11, 95, 96,188,202,195,195,197,194, & + 195,195,195,195,195,195,195, 88, 49, 50, & + 32,122,121,111,112, 59, 57/ + + data kpds6/ 160,160,160,160,160, 1, 1, 1, 1, 1, & + 1, 1, 1, 1, 1, 1,237,238,236,239, & + 235,235,235,235,235,235,235, 1, 1, 1, & + 1, 1, 1, 1, 1, 1, 1/ + + data kpds7/ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, & + 0, 0, 0, 0, 0, 0, 0, 0, 30,260, & + 25, 50,100,150,200,250,280, 0, 0, 0, & + 0, 0, 0, 0, 0, 0, 0/ +!------------------------------------------------------------------- +! Xingren: check why fac=273.15 for Ice temperature +! check why fac=-8.64e6 for evap minus precip +!------------------------------------------------------------------- + data fac/ 0.0, 0.001,1.0,1.0, 1.0, 1.0, 1.0,1.0,1.0,1.0, & + 1.0,273.15,1.0,1.0,-8.64e6, 1.0, 1.0,1.0,1.0,1.0, & + 1.0, 1.0,1.0,1.0, 1.0, 1.0, 1.0,1.0,1.0,1.0, & + 1.0, 1.0,1.0,1.0, 1.0, 1.0, 1.0/ + + data kpds22/ 2, 5, 3, 3, 9, 3, 3, 3, 3, 2, & + 3, 2, 3, 3, 3, 3, 0, 0, -5, -4, & + 0, 0, 0, 0, 0, 0, 0, 5, 3, 3, & + 0, 0, 0, 0, 0, 3, 3/ +! + data levs/ 5, 15, 25, 35, 45, 55, 65, 75, & + 85, 95, 105, 115, 125, 135, 145, 155, & + 165, 175, 185, 195, 205, 215, 225, 238, & + 262, 303, 366, 459, 584, 747, 949,1193, & + 1479,1807,2174,2579,3016,3483,3972,4478/ +! + data spv_tau/-1.0E+5/ + data spv_ci/1.0E+29/ + data spv_pme/-1.0E+10/ + data undef/-1.0E+34/ +! + data zt/ 5., 15., 25., 35., 45., 55., 65., 75., & + 85., 95., 105., 115., 125., 135., 145., 155., & + 165., 175., 185., 195., 205., 215., 225., 238., & + 262., 303., 366., 459., 584., 747., 949.,1193., & + 1479.,1807.,2174.,2579.,3016.,3483.,3972.,4478./ +! + data zw/ 10., 20., 30., 40., 50., 60., 70., 80., & + 90., 100., 110., 120., 130., 140., 150., 160., & + 170., 180., 190., 200., 210., 220., 232., 250., & + 283., 335., 413., 522., 666., 848.,1072.,1337., & + 1643.,1991.,2377.,2798.,3250.,3728.,4225.,4737./ +! + data levelm/' m below sea level', ' m below sea level', & + ' m below sea level', ' m below sea level', & + ' m below sea level', & + 'surface', 'surface', 'surface', 'surface', & + 'surface', 'surface', 'surface', 'surface', & + 'surface', 'surface', 'surface', & + 'bottom of ocean mixed layer', & + 'bottom of ocean isothermal layer', & + '0-300 m ocean layer', & + 'layer ocean surface and 26C ocean isothermal level', & + '2.5C ocean isotherm', & + '5C ocean isotherm', & + '10C ocean isotherm', & + '15C ocean isotherm', & + '20C ocean isotherm', & + '25C ocean isotherm', & + '28C ocean isotherm', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface', & + 'surface'/ + + data varcode/'POT', 'SALTY', 'UOGRD', 'VOGRD', 'DZDT', & + 'UFLX', 'VFLX', 'SSHG', 'ICEC', 'ICETK', & + 'SNOD', 'TMP', 'UICE', 'VICE', 'EMNP', & + 'THFLX', 'DBSS', 'DBSS', 'OHC', 'TCHP', & + 'DBSS', 'DBSS', 'DBSS', 'DBSS', 'DBSS', & + 'DBSS', 'DBSS', & + 'SALTY', 'UOGRD', 'VOGRD','SPEED','SHTFL', & + 'LHTFL', 'NSWRS', 'NLWRS','PRATE','EVP'/ +! + template_file = 'iceocnpost.g2' + template_inv = '@mem:0' + +! find grid size to make sure + iret = grb2_mk_inv(template_file, template_inv) + if (iret.ne.0) stop 1 + +! search using variable and regular date YYYYMMDDHH + template_var = '^1:' +! Based on Wesley's suggestion +! iret = grb2_inq(template_file,template_inv,template_var,nx=nx,ny=ny) + iret = grb2_inq(template_file,template_inv,template_var,nx=nx,ny=ny,regex=regex) + if (iret.ne.1) then + if (iret.eq.0) write(*,*) 'could not find message' + if (iret.gt.1) write(*,*) 'found multiple messages ', iret + stop 2 + endif + +! + if (mfcstcpl.eq.1) climate = .true. +! + print *,'im = ',im + print *,'jm = ',jm + print *,'km = ',km + + print *,'imo = ',imo + print *,'jmo = ',jmo + + print *,'IGEN_OCNP = ',igenocnp + print *,'mfcstcpl = ',mfcstcpl + print *,'climate = ',climate + + isdate=0 + isdate(1)=isyr + isdate(2)=ismth + isdate(3)=isday + isdate(5)=ishr + print *,'isdate ',isdate + + iedate=0 + iedate(1)=iyr + iedate(2)=imth + iedate(3)=iday + iedate(5)=ihr + + call w3difdat(iedate,isdate,2,datedif) + hrdif=datedif(2) + write(hr8,'(i8)') hrdif +! print *,'datedif',datedif + print *,'hrdif',hrdif + + write(datecode,'(i4.4,i2.2,i2.2,i2.2)') isyr, ismth, isday, ishr + print*, 'datecode=',datecode + + call read_ocn(im,jm,km,lon,lat, & + t,s,u,v,eta,sfcflx,pme,mld,taux,tauy,ocnfile, & + sss,ssu,ssv,speed,sensible,latent,sw,lw,lprec,evap) + print *,'after call read_ocn' + call read_ice(im,jm,hi_ci,hs_ci,fi_ci,ts_ci,ui_ci,vi_ci,icefile) + print *,'after call read_ice' + uice(:,:)=ui_ci(:,:) + vice(:,:)=vi_ci(:,:) + + do k=1,80 + ftime(k:k)=' ' + enddo +! ftime='6 hour fcst' + + kpds(1)=7 + if (igenocnp.GT.0) then + kpds(2)=igenocnp + else + kpds(2)=98 + endif + kpds(3)=10 + kpds(4)=192 + kpds(8)=mod(isyr-1,100)+1 + kpds(9)=ismth + kpds(10)=isday + kpds(11)=ishr + kpds(12)=0 + if (mfhout == 1) then + kpds(13)=1 + else if (mfhout == 3) then + kpds(13)=10 + else if (mfhout == 6) then + kpds(13)=11 + else if (mfhout == 12) then + kpds(13)=12 + else if (mfhout == 24) then + kpds(13)=2 + else + print *,'invalid mhout, must be one of (1 3 6 12 24).' + stop + endif + if (climate) then + kpds(13)=1 + kpds(14)=mfh + print *,'kpds(14)=',kpds(14) + print *,'mfhout=',mfhout + kpds(15)=0 + kpds(16)=10 + ftime=hr8 // ' hour fcst' + print *, 'ftime:', ftime + else + if (mfh > 1530) then + kpds(13)=1 + kpds(14)=mfh + kpds(15)=0 + kpds(16)=10 + else + kpds(14)=mfh/mfhout-1 + kpds(15)=kpds(14)+1 + kpds(16)=3 + endif + endif + kpds(17)=0 + kpds(18)=1 + kpds(19)=2 + kpds(20)=0 + kpds(21)=((iyr-1)/100)+1 + kpds(23)=4 + kpds(24)=0 + kpds(25)=32 + + print*,'kpds:',kpds(1:25) +! + kgds(1)=0 + kgds(2)=imo + kgds(3)=jmo + kgds(4)=nint(flatn*1000.) + kgds(5)=nint(flonw*1000.) + kgds(6)=128 + kgds(7)=nint(flats*1000.) + kgds(8)=nint(flone*1000.) + kgds(9)=nint(dlon*1000.) + kgds(10)=nint(dlat*1000.) + kgds(11)=0 + kgds(12)=0 + kgds(13)=0 + kgds(14)=0 + kgds(15)=0 + kgds(16)=0 + kgds(17)=0 + kgds(18)=0 + kgds(19)=0 + kgds(20)=255 + kgds(21)=0 + kgds(22)=0 +! + print*,'kgds:',kgds(1:22) +! + ndata=imo*jmo +! + ind=0 + do nv=1,5 + factor=fac(nv) + kpds(22)=kpds22(nv) + print *,nv,' factor ',factor,' kpds22 ',kpds(22) + +!temp/potdsl... + if (nv.eq.1) then + varocn=t + do k=1,km + do j=1,jmo + do i=1,imo + if (varocn(i,j,k).LE.undef) then + tocn(i,j,k)=undef + else + tocn(i,j,k)=varocn(i,j,k) + endif + enddo + enddo + enddo + endif + +!salinity + if (nv.eq.2) then + varocn=s + do k=1,km + do j=1,jmo + do i=1,imo + socn(i,j,k)=varocn(i,j,k) + enddo + enddo + enddo + endif +!u-current + if (nv.eq.3) then + varocn=u + endif +!v-current + if (nv.eq.4) then + varocn=v + endif +!------------------------------------------------------------------- +! Xingren: +! w vertical velocity (not present in raw NetCDF file) +! so how can you compute the vertical velocity with the program below? +!------------------------------------------------------------------- + if (nv.eq.5) then + varocn=w + do k=1,km-1 + kk=km-k+1 + do j=1,jmo + do i=1,imo + if (varocn(i,j,k).LE.undef) then + varocn(i,j,k)=varocn(i,j,k+1) + else + varocn(i,j,k)=(varocn(i,j,k+1)*(zw(kk)-zt(kk)) & + +varocn(i,j,k)*(zt(kk)-zw(kk-1)))/(zw(kk)-zw(kk-1)) + endif + enddo + enddo + enddo + continue + endif + + do k=1,km + ind=ind+1 + ilev=levs(k) + +! flip N-S + do j=1,jmo + grdtmp(:,j)=varocn(:,jmo-j+1,k) + enddo + +! make bit-map.... + do j=1,jmo + do i=1,imo + val=grdtmp(i,j) + if (val.eq.undef) then + lbms(i,j) = .false. + grid2(i+(j-1)*nx)=9.999E+20 + else + if (nv.eq.1) then + grid(i,j)=val+factor + else + grid(i,j)=val*factor + endif + lbms(i,j) = .true. + grid2(i+(j-1)*nx)=grid(i,j) + endif + enddo + enddo + + print *,' record written ',ind,nv,k,grid(92,125),lbms(92,125) + kpds(5)=kpds5(nv) + kpds(6)=kpds6(nv) + kpds(7)=ilev + + write(level4,'(i4)') ilev + levelcode=level4 // levelm(nv) + print *, 'levelcode= ', levelcode + print *, 'trim(ftime):', trim(ftime) + metadata='d=' // datecode // ':' // trim(varcode(nv)) // ':' // trim(levelcode) // ':' // trim(ftime) // ':' + iret = grb2_wrt(outfile,template_file,1,data1=grid2,meta=metadata) + write(*,*) iret +! +!.. end level-loop + enddo +!.. end variable-loop + enddo + +!... now read and grib 5 surface records... +! + do nv=6,nvar + levelcode=levelm(nv) + print *, 'levelcode= ', levelcode + print *, 'process data for nv=',nv + factor=fac(nv) + kpds(22)=kpds22(nv) + print *,nv,' factor ',factor,' kpds22 ',kpds(22) + ind=ind+1 + kpds(5)=kpds5(nv) + kpds(6)=kpds6(nv) + kpds(7)=kpds7(nv) + if (nv .EQ. 8 .or. (nv.GE.16 .AND. nv.LE.27)) then + kpds(19)=129 + else if (nv .EQ. 15) then + kpds(19)=128 + else + kpds(19)=2 + endif +! taux + if (nv.eq.6) then + varsfc=taux + do j=1,jmo + do i=1,imo + if (varsfc(i,j) .LE. spv_tau) varsfc(i,j)=undef + enddo + enddo + endif +! tauy + if (nv.eq.7) then + varsfc=tauy + do j=1,jmo + do i=1,imo + if (varsfc(i,j) .LE. spv_tau) varsfc(i,j)=undef + enddo + enddo + endif +! eta + if (nv.eq.8) then + varsfc=eta + endif +! fi + if (nv.eq.9) then + varsfc=fi_ci + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! hi + if (nv.eq.10) then + varsfc=hi_ci + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! hs + if (nv.eq.11) then + varsfc=hs_ci + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! ts + if (nv.eq.12) then + varsfc=ts_ci + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! uice + if (nv.eq.13) then + varsfc=uice + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! vice + if (nv.eq.14) then + varsfc=vice + call set_undef(im,jm,spv_ci,undef,varsfc) + endif +! pme + if (nv.eq.15) then + varsfc=pme + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. spv_pme) varsfc(i,j)=undef + enddo + enddo + endif +! sfc_flx + if (nv.eq.16) then + varsfc=sfcflx + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! mld + if (nv.eq.17) then +! call mixed_layer(imo,jmo,km,tocn,socn,zt,varsfc,undef) + varsfc=mld + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! sfc isothm layer depth + if (nv.eq.18) then + call sfc_isothm_layer(imo,jmo,km,tocn,zt,varsfc,undef) + endif +! ocean heat content + if (nv.eq.19) then + call ocean_heat(imo,jmo,km,tocn,socn,zw,zt,varsfc,undef) + endif +! tropical cyc heat potential + if (nv.eq.20) then + call tchp26(imo,jmo,km,tocn,socn,zw,zt,varsfc,undef) + endif +! depth of 7 different isotherms... + if (nv.ge.21 .AND. nv.le.27) then + i=nv-20 + call isothm_layer(imo,jmo,km,dtc(i),tocn,zt,varsfc,undef) + endif +! sea surface salinity + if (nv.eq.28) then + varsfc=sss + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! sea surface u-current + if (nv.eq.29) then + varsfc=ssu + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! sea surface v-current + if (nv.eq.30) then + varsfc=ssv + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! sea surface speed + if (nv.eq.31) then + varsfc=speed + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! sensible heat + if (nv.eq.32) then + varsfc=sensible + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! latent heat + if (nv.eq.33) then + varsfc=latent + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! net downward shortwave radiation at the surface + if (nv.eq.34) then + varsfc=sw + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! net downward longwave radiation at the surface + if (nv.eq.35) then + varsfc=lw + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! precipitation rate + if (nv.eq.36) then + varsfc=lprec + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif +! evaporation + if (nv.eq.37) then + varsfc=evap + do j=1,jm + do i=1,im + if (varsfc(i,j) .LE. undef) varsfc(i,j)=undef + enddo + enddo + endif + + nundef=0 + do j=1,jmo + grdtmp(:,j)=varsfc(:,jmo-j+1) + enddo + do j=1,jmo + do i=1,imo + val=grdtmp(i,j) + if (val.eq.undef) then + lbms(i,j) = .false. + nundef=nundef+1 + grid2(i+(j-1)*nx)=9.999E+20 + else + if (nv.EQ.12) then + grid(i,j)=val+factor + else + grid(i,j)=val*factor + endif + lbms(i,j) = .true. + grid2(i+(j-1)*nx)=grid(i,j) + endif + enddo + enddo + if(nundef.eq.imo*jmo) then + print *,' record not written because of all undef ', & + ind,kpds(5),kpds(6),kpds(7) + else + print *,' record written ',ind,kpds(5),kpds(6),kpds(7), & + grid(92,125),lbms(92,125),factor + endif + print *,'nv= ', nv + metadata='d=' // datecode // ':' // trim(varcode(nv)) // ':' // trim(levelcode) // ':' // trim(ftime) // ':' + iret = grb2_wrt(outfile,template_file,1,data1=grid2,meta=metadata) + write(*,*) iret + + enddo + + return + end subroutine diag + + subroutine isothm_layer(im,jm,km,dtc,temp,zlev,zisothm,undef) + + real, parameter :: c2k=273.15 + integer inumc,im,jm,km + integer i,j,k + real dtc + real, dimension(km) :: tz,zlev + real, dimension(im,jm) :: zisothm + real, dimension(im,jm,km) :: temp + real a,b,tc,undef + + tc=dtc+c2k + + do j=1,jm + do i=1,im + + zisothm(i,j)=undef + if (temp(i,j,1) .GE. tc) then + do k=1,km + tz(k)=temp(i,j,k) + enddo + do k=2,km + if (tz(k) .LT. -3.0) go to 111 + if (tz(k) .LT. tc) then + a = (tz(k)-tc) / (tz(k)-tz(k-1)) + b = (tc-tz(k-1)) / (tz(k)-tz(k-1)) + zisothm(i,j)=a*zlev(k-1)+b*zlev(k) + go to 111 + endif + enddo + endif + 111 continue + enddo + enddo + + return + end subroutine isothm_layer + + subroutine mixed_layer(im,jm,km,temp,salt,zlev,mld,undef) + + real, parameter :: disot=0.8, c2k=273.15 + integer im,jm,km + integer i,j,k,kmsk,kbm,kbp,krf + real, dimension(km) :: zlev,plev,sa,ta,th,rho + real, dimension(im,jm) :: mld + real, dimension(im,jm,km) :: temp,salt + real a,b,deltarho,dr,rb,undef + + do k=1,km + plev(k) = press(zlev(k),980.0) + enddo + + do j=1,jm + do i=1,im + kmsk = 0 + do k=1,km + if (temp(i,j,k).GT.0.0 .AND. salt(i,j,k).GT.0.0) then + ta(k) = temp(i,j,k)-c2k + sa(k) = salt(i,j,k) + kmsk = k + endif + enddo + + if (kmsk.EQ.0 .OR. ta(1).LT.-3.0) then + mld(i,j)=undef + else + deltarho = (density(0.0,ta(1)-disot,sa(1)) & + - density(0.0,ta(1),sa(1))) + do k=1,kmsk + th(k) = theta(plev(k),ta(k),sa(k),0.0) + rho(k) = density(0.0,th(k),sa(k)) - 1000.0 + enddo + krf = 1 + kbm = 0 + kbp = 0 + do k = krf,kmsk + if ((rho(k)-rho(krf)) .GE. deltarho) then + kbp = k + exit + endif + enddo + if (kbp .LE. 1) then + mld(i,j) = undef + else + kbm = kbp - 1 + rb = rho(krf) + deltarho + dr = rho(kbp) - rho(kbm) + a = (rho(kbp) - rb) / dr + b = (rb - rho(kbm)) / dr + mld(i,j) = zlev(kbm)*a + zlev(kbp)*b + endif + endif + enddo + enddo + + end subroutine mixed_layer + + subroutine sfc_isothm_layer(im,jm,km,temp,zlev,sitd,undef) + + real, parameter :: disot=0.8 + integer im,jm,km + integer i,j,k + real, dimension(im,jm) :: sitd + real, dimension(im,jm,km) :: temp + real, dimension(km) :: zlev,tz + real a,b,tc,undef + + do j=1,jm + do i=1,im + + sitd(i,j)=undef + + if (temp(i,j,1).GE.0.0) then + tc=temp(i,j,1)-disot + do k=1,km + tz(k)=temp(i,j,k) + enddo + do k=2,km + if (tz(k).LT.0.0) go to 112 + if (tz(k).LT.tc) then + a = (tz(k)-tc) / (tz(k)-tz(k-1)) + b = (tc-tz(k-1)) / (tz(k)-tz(k-1)) + sitd(i,j) = a*zlev(k-1) + b*zlev(k) + go to 112 + endif + enddo + endif + + 112 continue + + enddo + enddo + + return + end subroutine sfc_isothm_layer + + subroutine ocean_heat(im,jm,km,temp,salt,zblev,zlev,ocnhc,undef) + + integer, parameter :: kmh=26 + real, parameter :: c2k=273.15 + integer im,jm,km + integer i,j,k + real, dimension(km) :: zblev,zlev,plev + real, dimension(im,jm) :: ocnhc + real, dimension(im,jm,km) :: salt,temp + real dptlyr,rk,sk,tk,undef + real pk,rhm,rhp,tempk,zk + + k=kmh + zk=0.5*(300.0+zblev(k-1)) + pk=press(zk,980.0) + rhm = (zk-zlev(k-1))/(zlev(k)-zlev(k-1)) + rhp = (zlev(k)-zk)/(zlev(k)-zlev(k-1)) + + do k=1,km + plev(k) = press(zlev(k),980.0) + enddo + + do j=1,jm + do i=1,im + + ocnhc(i,j)=undef + + if (temp(i,j,kmh).GT.0.0 .AND. salt(i,j,kmh).GT.0.0) then + ocnhc(i,j)=0. + do k=1,kmh-1 + tk=temp(i,j,k)-c2k +! print *,'tk= ',tk + sk=salt(i,j,k) + rk=density(plev(k),tk,sk) + if (k .eq. 1) then + dptlyr=zblev(k) + else + dptlyr=zblev(k)-zblev(k-1) + endif + ocnhc(i,j)=ocnhc(i,j) + rk*temp(i,j,k)*dptlyr + enddo + k=kmh + tempk=rhp*temp(i,j,k-1) + rhm*temp(i,j,k) + tk=tempk - c2k + sk=rhp*salt(i,j,k-1) + rhm*salt(i,j,k) + rk=density(pk,tk,sk) + dptlyr=300.0-zblev(k-1) + ocnhc(i,j)=ocnhc(i,j)+rk*tempk*dptlyr + ocnhc(i,j)=ocnhc(i,j)*3996. + endif + enddo + enddo + + return + end subroutine ocean_heat + + subroutine tchp26(im,jm,km,temp,salt,zblev,zlev,ocnhcp,undef) + + real, parameter :: c2k=273.15, t26=26.0 + integer im,jm,km + integer i,j,k,k26 + real, dimension(km) :: zblev,zlev,plev,tz + real, dimension(im,jm) :: ocnhcp,z26isothm + real, dimension(im,jm,km) :: salt,temp + real dptlyr,rk,sk,tk,undef + real rhm,rhp,pk,zk + real a,b,skk,skm,tc,z26 + logical*1 lbms(im,jm) + + tc=c2k+t26 + + do k=1,km + plev(k) = press(zlev(k),980.0) + enddo + + do j=1,jm + do i=1,im + + z26isothm(i,j)=undef + lbms(i,j)=.false. + if (temp(i,j,1) .GE. tc) then + do k=1,km + tz(k)=temp(i,j,k) + enddo + k = 1 + do while (tz(k).GE.tc) + k26 = k + if (k.EQ.km) exit + k = k + 1 + enddo + k = k26 + if (tz(k) .GT. tc) then + if (k.LT.km .AND. tz(k+1).GT.0.0) then + k = k + 1 + a = (tz(k)-tc) / (tz(k)-tz(k-1)) + b = (tc-tz(k-1)) / (tz(k)-tz(k-1)) + z26isothm(i,j) = a*zlev(k-1) + b*zlev(k) + lbms(i,j)=.true. + else if (k.GE.2 .AND. tz(k).LT.tz(k-1)) then + a = (tz(k)-tc) / (tz(k)-tz(k-1)) + b = (tc-tz(k-1)) / (tz(k)-tz(k-1)) + z26 = a*zlev(k-1) + b*zlev(k) + if (z26.LE.zblev(k)) then + z26isothm(i,j) = z26 + lbms(i,j)=.true. + endif + endif + else if (tz(k).EQ.tc) then + z26isothm(i,j) = zlev(k) + lbms(i,j)=.true. + endif + endif + + enddo + enddo + +! +!---------- get ocean heat potential relative to 26C (TCHP) ------------ +! + do j=1,jm + do i=1,im + + if (temp(i,j,1) .GT. 0.0) then + ocnhcp(i,j)=0.0 + else + ocnhcp(i,j)=undef + cycle + endif + + if (lbms(i,j)) then ! we have water above 26c + + z26 = z26isothm(i,j) +! +! case where Z26 is within the topmost layer +! + if (z26 .LE. zblev(1)) then + tk=temp(i,j,1)-c2k + if (salt(i,j,1) .GT. 0.0) then + sk=salt(i,j,1) + else + sk=35. ! fake salinity + endif + rk=density(plev(1),tk,sk) + dptlyr=z26 + ocnhcp(i,j) = rk*(tk-t26)*dptlyr*3996. +! +! case where z26 is below the top layer and above the bottom +! + else + k26 = 1 + do k=2,km + if (z26.GT.zblev(k-1) .AND. z26.LE.zblev(k)) k26=k + enddo + + ocnhcp(i,j)=0.0 + do k=1,k26-1 + tk=temp(i,j,k)-c2k + if (salt(i,j,K) .GT. 0.0) then + sk=salt(i,j,k) + else + sk=35. ! fake salinity + endif + rk=density(plev(k),tk,sk) + if (k .EQ. 1) then + dptlyr=zblev(1) + else + dptlyr=zblev(k)-zblev(k-1) + endif + ocnhcp(i,j)=ocnhcp(i,j)+rk*(tk-26.0)*dptlyr + enddo + k=k26 + zk=0.5*(z26+zblev(k-1)) + pk=press(zk,980.0) + rhm = (zk-zlev(k-1))/(zlev(k)-zlev(k-1)) + rhp = (zlev(k)-zk)/(zlev(k)-zlev(k-1)) + tk=rhp*temp(i,j,k-1) + rhm*temp(i,j,k) - c2k + if (salt(i,j,k-1) .GT. 0.0) then + skm=salt(i,j,k-1) + else + skm=35. ! fake salinity + endif + if (salt(i,j,k) .GT. 0.0) then + skk=salt(i,j,k) + else + skk=35. ! fake salinity + endif + sk=(rhp*skm + rhm*skk) + rk=density(pk,tk,sk) + dptlyr=z26-zblev(k-1) + ocnhcp(i,j)=ocnhcp(i,j)+rk*(tk-26.0)*dptlyr + ocnhcp(i,j)=ocnhcp(i,j)*3996. + endif +! +! case where temperature is above 26C down to the bottom +! + else if ((temp(i,j,1)-c2k) .GT. t26) then + ocnhcp(i,j)=0.0 + do k=1,km + if (temp(i,j,k) .GT. undef) then + tk=temp(i,j,k)-c2k + if (salt(i,j,k) .GT. 0.0) then + sk=salt(i,j,k) + else + sk=35. ! fake salinity + endif + rk=density(plev(k),tk,sk) + if (k .EQ. 1) then + dptlyr=zblev(1) + else + dptlyr=zblev(k)-zblev(k-1) + endif + ocnhcp(i,j)=ocnhcp(i,j)+rk*(tk-26.0)*dptlyr + endif + enddo + ocnhcp(i,j)=ocnhcp(i,j)*3996. + endif + + enddo + enddo + + return + end subroutine tchp26 + + function press(z, g) + +! copy from cfs_ocean_time.f and modified +! depth (z) in meters and grav acc'l (g) in cm/sec**2 + + integer, parameter :: itr=20 + integer i + real p, a0, z, g, press + real(kind=8) :: e, ae, es +! + p = z*(1.0076+z*(2.3487e-6-z*1.2887e-11)) + e = zeta(p,g)-z + ae = abs(e) + es = ae*2. + do i = 1,itr + a0 = 0.972643+p*(1.32696e-5-p*(6.228e-12+p*1.885e-16)) + a0 = a0/(1.0+1.83e-5*p) + p = p-((g+1.113e-4*p)/a0)*e*0.001 + es = ae + e = zeta(p,g)-z + ae = abs(e) + if (ae .le. 0.01) exit + enddo +! + press = p +! + end function press + + function zeta(p, glat) +! +! copy from cfs_ocean_time.f and modified + + real p, glat, z, zeta + + z = ((-3.434e-12*p+1.113e-7)*p+0.712953)*p+14190.7*log(1.0+1.83e-5*p) + z = (z/(glat+1.113e-4*p))*1000. + + zeta = z +! + end function zeta + + function density(prs, tmp, sal) +! +! copy from cfs_ocean_time.f and modified +! Density is in units of kg/m**3 (1 g/cm**3 = 1000 kg/m**3) + + real density, prs, tmp, sal + real p, t, s, kstp, k0, kw, d0, dw +! + s = sal + t = tmp + p = prs/10.00 +! + kw = 19652.21+(148.4206-(2.327105-(1.360477e-2-5.155288e-5*t)*t)*t)*t +! + k0 = kw+s*(54.6746-(0.603459-(1.09987e-2-6.1670e-5*t)*t)*t) & + +sqrt(s*s*s)*(7.944e-2+(1.6483e-2-5.3009e-4*t)*t) +! + kstp = k0+p*((3.239908+(1.43713e-3+(1.16092e-4-5.77905e-7*t)*t)*t) & + +s*(2.2838e-3-(1.0981e-5+1.6078e-6*t)*t) & + +sqrt(s*s*s)*1.91075e-4 & + +p*((8.50935e-5-(6.12293e-6-5.2787e-8*t)*t) & + -s*(9.9348e-7-(2.0816e-8+9.1697e-10*t)*t))) +! + dw = 999.842594+(6.793952e-2-(9.095290e-3-(1.001685e-4 & + -(1.120083e-6-6.536332e-9*t)*t)*t)*t)*t +! + d0 = dw+s*(0.824493-(4.0899e-3-(7.6438e-5-(8.2467e-7 & + -5.3875e-9*t)*t)*t)*t) & + -sqrt(s*s*s)*(5.72466e-3-(1.0227e-4-1.6546e-6*t)*t) & + +s*s*4.8314e-4 +! + density = d0/(1.0-p/kstp) + + end function density + + function theta(p, t, s, pref) + + real(kind=8), parameter :: sqrt2 = 0.7071067811865475 + real theta, p,t, s, pref + real del_p, del_t1, del_t2, del_t3, del_t4, tp, th + + del_p = pref-p + del_t1 = del_p*atg(p,t,s) + tp = t+0.5*del_t1 + del_t2 = del_p*atg((p+0.5*del_p),tp,s) + tp = t+(-0.5+sqrt2)*del_t1+(1.0-sqrt2)*del_t2 + del_t3 = del_p*atg((p+0.5*del_p),tp,s) + tp = t-sqrt2*del_t2+(1.0+sqrt2)*del_t3 + del_t4 = del_p*atg(pref,tp,s) + th = (t+(del_t1+(1.0-sqrt2)*del_t2*2.0 & + + (1.0+sqrt2)*del_t3*2.0+del_t4)/6.0) + theta = th + + end function theta + + function atg(p, t, s) + + real atg, p, t, s, ds, a + + ds = s-35.0 + a = (((-2.1687e-16*t+1.8676e-14)*t-4.6206e-13)*p & + +((2.7759e-12*t-1.1351e-10)*ds+((-5.4481e-14*t & + +8.733e-12)*t-6.7795e-10)*t+1.8741e-8))*p & + +(-4.2393e-8*t+1.8932e-6)*ds & + +((6.6228e-10*t-6.836e-8)*t+8.5258e-6)*t+3.5803e-5 + + atg = a + + end function atg + + + end module regdiag_mod diff --git a/ush/cplvalidate.sh b/ush/cplvalidate.sh index f6ffa42ec0..ba5ec7b88a 100755 --- a/ush/cplvalidate.sh +++ b/ush/cplvalidate.sh @@ -17,8 +17,10 @@ case $confignamevarfornems in 'blocked_atm_wav') combination=.true..false..false..true..false.;; 'leapfrog_atm_wav')combination=.true..false..false..true..false.;; 'med_atm_ocn_ice_wav') combination=.true..true..true..true..false.;; + 'medcold_atm_ocn_ice_wav') combination=.true..true..true..true..false.;; 'med_atm_ocn_ice_wav1way') combination=.true..true..true..true..false.;; 'med_atm_ocn_ice_wav1waywcurr') combination=.true..true..true..true..false.;; + 'medcold_atm_ocn_ice_wav') combination=.true..true..true..true..false.;; 'medcold_atm_ocn_ice') combination=.true..true..true..false..false.;; *) echo "SUB cplvalidate: Combination not supported" exit ;; diff --git a/ush/forecast_postdet.sh b/ush/forecast_postdet.sh index 8bf2ff4c24..48ff14a7f7 100755 --- a/ush/forecast_postdet.sh +++ b/ush/forecast_postdet.sh @@ -502,17 +502,69 @@ $NCP -p $DATA/input.nml $COMOUT/ echo "SUB ${FUNCNAME[0]}: Output data for FV3 copied" } -WW3_in() +WW3_postdet() { - echo "SUB ${FUNCNAME[0]}: Linking input data for WW3" - # soft link commands insert here + echo "SUB ${FUNCNAME[0]}: Linking input data for WW3" + COMPONENTwave=${COMPONENTwave:-${RUN}wave} + + #Link mod_def files for wave grids + array=($WAVECUR_FID $WAVEICE_FID $WAVEWND_FID $waveuoutpGRD $waveGRD $waveesmfGRD $wavesbsGRD $wavepostGRD $waveinterpGRD) + echo "Wave Grids: $WAVECUR_FID $WAVEICE_FID $WAVEWND_FID $waveuoutpGRD $waveGRD $waveesmfGRD $wavesbsGRD $wavepostGRD $waveinterpGRD" + grdALL=`printf "%s\n" "${array[@]}" | sort -u | tr '\n' ' '` + for wavGRD in ${grdALL}; do + $NCP $ROTDIR/${COMPONENTwave}.${PDY}/${cyc}/rundata/${COMPONENTwave}.mod_def.$wavGRD $DATA/mod_def.$wavGRD + done + + #Copy initial condition files: + for wavGRD in $waveGRD ; do + # Link wave IC for current cycle + # $NLN ${WRDIR}/${sPDY}.${scyc}0000.restart.${wavGRD} $DATA/restart.${wavGRD} + # Link IC for S2S benchmarks: + $NCP -pf $ICSDIR/$CDATE/wav/${PDY}.${cyc}0000.restart.$wavGRD $DATA/restart.$wavGRD + # Link log files for computational grids: + eval $NLN $ROTDIR/${COMPONENTwave}.${PDY}/${cyc}/rundata/${COMPONENTwave}${WAV_MEMBER}.log.${wavGRD}.${PDY}${cyc} $DATA/log.${wavGRD} + done + + #link more log files: + eval $NLN $ROTDIR/${COMPONENTwave}.${PDY}/${cyc}/rundata/${COMPONENTwave}${WAV_MEMBER}.log.mww3.${PDY}${cyc} $DATA/log.mww3 + + datwave=$ROTDIR/${COMPONENTwave}.${PDY}/${cyc}/rundata + wavprfx=${COMPONENTwave}${WAV_MEMBER} + # Loop for gridded output (uses FHINC) + fhr=$FHMIN_WAV + while [ $fhr -le $FHMAX_WAV ]; do + YMDH=`$NDATE $fhr $CDATE` + YMD=$(echo $YMDH | cut -c1-8) + HMS="$(echo $YMDH | cut -c9-10)0000" + for wavGRD in ${waveGRD} ; do + eval $NLN $datwave/${wavprfx}.out_grd.${wavGRD}.${YMD}.${HMS} $DATA/${YMD}.${HMS}.out_grd.${wavGRD} + done + FHINC=$FHOUT_WAV + #if [ $FHMAX_HF_WAV -gt 0 -a $FHOUT_HF_WAV -gt 0 -a $fhr -lt $FHMAX_HF_WAV ]; then + # FHINC=$FHOUT_HF_WAV + #fi + fhr=$((fhr+FHINC)) + done + # Loop for point output (uses DTPNT) + fhr=$FHMIN_WAV + while [ $fhr -le $FHMAX_WAV ]; do + YMDH=`$NDATE $fhr $CDATE` + YMD=$(echo $YMDH | cut -c1-8) + HMS="$(echo $YMDH | cut -c9-10)0000" + eval $NLN $datwave/${wavprfx}.out_pnt.${waveuoutpGRD}.${YMD}.${HMS} $DATA/${YMD}.${HMS}.out_pnt.${waveuoutpGRD} + FHINC=$FHINCP_WAV + fhr=$((fhr+FHINC)) + done + + + } WW3_nml() { - echo "SUB ${FUNCNAME[0]}: Creating name list for WW3" - source $SCRIPTDIR/parsing_namelists_WW3.sh - WW3_namelists + echo "SUB ${FUNCNAME[0]}: Copying input files for WW3" + COMPONENTwave=${COMPONENTwave:-${RUN}wave} + $NCP $ROTDIR/${COMPONENTwave}.${PDY}/${cyc}/rundata/ww3_multi.${WAV_MOD_TAG}.${cycle}.inp $DATA/ww3_multi.inp } WW3_out() @@ -534,9 +586,11 @@ MOM6_postdet() $NCP -pf $FIXmom/$OCNRES/* $DATA/INPUT/ # Copy MOM6 input file - $NCP -pf $HOMEgfs/parm/mom6/MOM_input_$OCNRES $DATA/INPUT/MOM_input - - #TODO: if cplwav, copy MOM_input_$OCNRES_wav + if [ $cplwav = ".true." ] ; then + $NCP -pf $HOMEgfs/parm/mom6/MOM_input_${OCNRES}_wav $DATA/INPUT/MOM_input + else + $NCP -pf $HOMEgfs/parm/mom6/MOM_input_$OCNRES $DATA/INPUT/MOM_input + fi #TODO: update to make MOM_input configurable # Copy coupled grid_spec diff --git a/ush/icepost.ncl b/ush/icepost.ncl new file mode 100644 index 0000000000..9f831d1bff --- /dev/null +++ b/ush/icepost.ncl @@ -0,0 +1,382 @@ +;------------------------------------------------------------------ +; Denise.Worthen@noaa.gov (Feb 2019) +; +; This script will remap CICE5 output on the tripole grid to +; a set of rectilinear grids using pre-computed ESMF weights to remap +; the listed fields to the destination grid and write the results +; to a new netCDF file +; +; See ocnpost.ncl for a complete description +; +; Bin.Li@noaa.gov (May 2019) +; This script is revised to be used in the coupled workflow. +; Revised parts are marked by + + load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl" + +;---------------------------------------------------------------------- +begin + +;************************************************ +; specify parameters +;************************************************ +; + + output_masks = False + ; destination grid sizes and name + dsttype = (/"rect."/) + ;dstgrds = (/"1p0", "0p5", "0p25"/) +; + + ; specify a location to use + ; nemsrc = "/scratch4/NCEPDEV/ocean/save/Denise.Worthen/NEMS_INPUT0.1/ocnicepost/" + ; interpolation methods + methods = (/"bilinear" ,"conserve"/) + ; ocean model output location + ;dirsrc = "/scratch3/NCEPDEV/stmp2/Denise.Worthen/BM1_ice/" + + + ; variables to be regridded with the native tripole stagger location + + varlist = (/ (/ "hi_h", "Ct", "bilinear"/) \ + ,(/ "hs_h", "Ct", "bilinear"/) \ + ,(/ "Tsfc_h", "Ct", "bilinear"/) \ + ,(/ "aice_h", "Ct", "bilinear"/) \ + ,(/ "sst_h", "Ct", "bilinear"/) \ + /) + dims = dimsizes(varlist) + nvars = dims(0) + delete(dims) + ;print(varlist) + + ; vectors to be regridded with the native tripole stagger location + ; and dimensionality + ; note: vectors are always unstaggered using bilinear weights, but can + ; be remapped using conservative + nvpairs = 1 + veclist = new( (/nvpairs,3,2/),"string") + veclist = (/ (/ (/"uvel_h", "vvel_h"/), (/"Bu", "Bu"/), (/"bilinear", "bilinear"/) /) \ + /) + ;print(veclist) + + begTime = get_cpu_time() +;---------------------------------------------------------------------- +; make a list of the directories and files from the run +;---------------------------------------------------------------------- +; idate = "20120101" +; icefilelist = systemfunc("ls "+dirsrc+"gfs."+idate+"/00/"+"ice*.nc") +; icef = addfiles(icefilelist,"r") +; nfiles = dimsizes(icefilelist) +; + + ; get the rotation angle + angleT = icef[0]->ANGLET + + ; get a 2 dimensional fields for creating the interpolation mask + ; the mask2d contain 1's on land and 0's at valid points. + mask2d = where(ismissing(icef[0]->sst_h), 1.0, 0.0) + ;printVarSummary(mask2d) + + ; create conformed rotation arrays to make vector rotations cleaner + angleT2d=conform_dims(dimsizes(mask2d),angleT,(/1,2/)) + +;---------------------------------------------------------------------- +; loop over the output resolutions +;---------------------------------------------------------------------- + + jj = 1 + ii = 0 + + do jj = 0,dimsizes(dstgrds)-1 + ;outres = "_"+dstgrds(jj)+"x"+dstgrds(jj) + outres = dstgrds(jj)+"x"+dstgrds(jj) + outgrid = dstgrds(jj) + + ; regrid a field to obtain the output xy dimensions + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+".bilinear.nc" + tt = ESMF_regrid_with_weights(angleT,wgtsfile,False) + tt!0 = "lat" + tt!1 = "lon" + lat = tt&lat + lon = tt&lon + dims = dimsizes(tt) + nlat = dims(0) + nlon = dims(1) + print("fields will be remapped to destination grid size "\ + +nlon+" "+nlat) + + delete(tt) + delete(dims) + + ; regrid the masks to obtain the interpolation masks. + ; the mask2d contain 1's on land and 0's at valid points. + ; when remapped, any mask value > 0 identifies land values that + ; have crept into the field. remapped model fields are then + ; masked with this interpolation mask + + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+".bilinear.nc" + rgmask2d = ESMF_regrid_with_weights(mask2d, wgtsfile,False) + + if(output_masks)then + testfile = "masks_"+dstgrds(jj)+".nc" + system("/bin/rm -f "+testfile) + ; create + testcdf = addfile(testfile,"c") + testcdf->rgmask2d = rgmask2d + ; close + delete(testcdf) + end if + + ; create the interpolation mask + rgmask2d = where(rgmask2d .gt. 0.0, rgmask2d@_FillValue, 1.0) + +;---------------------------------------------------------------------- +; loop over each file in the icefilelist +;---------------------------------------------------------------------- +; + ; retrieve the time stamp + time = icef[0]->time + delete(time@bounds) + +;---------------------------------------------------------------------- +; set up the output netcdf file +;---------------------------------------------------------------------- +; system("/bin/rm -f " + outfile) ; remove if exists +; outcdf = addfile (outfile, "c") ; open output file +; +; + + ; explicitly declare file definition mode. Improve efficiency. + setfileoption(outcdf,"DefineMode",True) + + ; create global attributes of the file + fAtt = True ; assign file attributes + fAtt@creation_date = systemfunc ("date") + fAtt@source_file = infile + fileattdef( outcdf, fAtt ) ; copy file attributes + + ; predefine the coordinate variables and their dimensionality + dimNames = (/"time", "lat", "lon"/) + dimSizes = (/ -1 , nlat, nlon/) + dimUnlim = (/ True , False, False/) + filedimdef(outcdf,dimNames,dimSizes,dimUnlim) + + ; predefine the the dimensionality of the variables to be written out + filevardef(outcdf, "time", typeof(time), getvardims(time)) + filevardef(outcdf, "lat", typeof(lat), getvardims(lat)) + filevardef(outcdf, "lon", typeof(lon), getvardims(lon)) + + ; Copy attributes associated with each variable to the file + filevarattdef(outcdf, "time", time) + filevarattdef(outcdf, "lat", lat) + filevarattdef(outcdf, "lon", lon) + + ; predefine variables + do nv = 0,nvars-1 + varname = varlist(nv,0) + odims = (/"time", "lat", "lon"/) + ;print("creating variable "+varname+" in file") + filevardef(outcdf, varname, "float", odims) + delete(odims) + end do + + do nv = 0,nvpairs-1 + do nn = 0,1 + vecname = veclist(nv,0,nn) + odims = (/"time", "lat", "lon"/) + ;print("creating variable "+vecname+" in file") + filevardef(outcdf, vecname, "float", odims) + delete(odims) + end do + end do + + ; explicitly exit file definition mode. + setfileoption(outcdf,"DefineMode",False) + + lat=lat(::-1) + ; write the dimensions to the file + outcdf->time = (/time/) + outcdf->lat = (/lat/) + outcdf->lon = (/lon/) + +;---------------------------------------------------------------------- +; loop over nvars variables +;---------------------------------------------------------------------- + + ;nv = 1 + do nv = 0,nvars-1 + varname = varlist(nv,0) + vargrid = varlist(nv,1) + varmeth = varlist(nv,2) + + ;print(nv+" "+varname+" "+vargrid+" "+varmeth) + icevar = icef[ii]->$varname$ + ndims = dimsizes(dimsizes(icevar)) + ;print(ndims+" "+dimsizes(icevar)) + + if(vargrid .ne. "Ct")then + ; print error if the variable is not on the Ct grid + print("Variable is not on Ct grid") + exit + end if + + ; regrid to dsttype+dstgrd with method + ;print("remapping "+varname+" to grid "+dsttype+dstgrds(jj)) + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+"."+varmeth+".nc" + + rgtt = ESMF_regrid_with_weights(icevar,wgtsfile,False) + rgtt = where(ismissing(rgmask2d),icevar@_FillValue,rgtt) + rgtt=rgtt(:,::-1,:) + + ; enter file definition mode to add variable attributes + setfileoption(outcdf,"DefineMode",True) + filevarattdef(outcdf, varname, rgtt) + setfileoption(outcdf,"DefineMode",False) + + + outcdf->$varname$ = (/rgtt/) + + delete(icevar) + delete(rgtt) + + ; nv, loop over number of variables + end do + +;---------------------------------------------------------------------- +; +;---------------------------------------------------------------------- + + ;nv = 0 + do nv = 0,nvpairs-1 + vecnames = veclist(nv,0,:) + vecgrids = veclist(nv,1,:) + vecmeth = veclist(nv,2,:) + ;print(nv+" "+vecnames+" "+vecgrids+" "+vecmeth) + + ; create a vector pair list + vecpairs = NewList("fifo") + n = 0 + uvel = icef[ii]->$vecnames(n)$ + vecfld = where(ismissing(uvel),0.0,uvel) + copy_VarAtts(uvel,vecfld) + ;print("unstagger "+vecnames(n)+" from "+vecgrids(n)+" to Ct") + wgtsfile = nemsrc+"/"+"tripole.mx025."+vecgrids(n)+".to.Ct.bilinear.nc" + ut = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + delete(ut@remap) + + n = 1 + vvel = icef[ii]->$vecnames(n)$ + vecfld = where(ismissing(vvel),0.0,vvel) + copy_VarAtts(vvel,vecfld) + ;print("unstagger "+vecnames(n)+" from "+vecgrids(n)+" to Ct") + wgtsfile = nemsrc+"/"+"tripole.mx025."+vecgrids(n)+".to.Ct.bilinear.nc" + vt = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + delete(vt@remap) + + ListAppend(vecpairs,ut) + ListAppend(vecpairs,vt) + ;print(vecpairs) + + ; rotate + ; first copy Metadata + urot = vecpairs[0] + vrot = vecpairs[1] + urot = cos(angleT2d)*ut - sin(angleT2d)*vt + vrot = sin(angleT2d)*ut + cos(angleT2d)*vt + + ; change attribute to indicate these are now rotated velocities + urot@long_name=str_sub_str(urot@long_name,"(x)","zonal") + vrot@long_name=str_sub_str(vrot@long_name,"(y)","meridional") + ; copy back + vecpairs[0] = urot + vecpairs[1] = vrot + delete([/urot, vrot/]) + + ; remap + do n = 0,1 + vecfld = vecpairs[n] + ; regrid to dsttype+dstgrd with method + ;print("remapping "+vecnames(n)+" to grid "+dsttype+dstgrds(jj)) + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+"."+vecmeth(n)+".nc" + + rgtt = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + rgtt = where(ismissing(rgmask2d),vecfld@_FillValue,rgtt) + rgtt=rgtt(:,::-1,:) + + ; enter file definition mode to add variable attributes + setfileoption(outcdf,"DefineMode",True) + filevarattdef(outcdf, vecnames(n), rgtt) + setfileoption(outcdf,"DefineMode",False) + + outcdf->$vecnames(n)$ = (/rgtt/) + delete(rgtt) + end do + delete([/uvel,vvel,ut,vt,vecfld,vecpairs/]) + delete([/vecnames,vecgrids,vecmeth/]) + ; nv, loop over number of vector pairs + end do + +;---------------------------------------------------------------------- +; close the outcdf and continue through filelist +;---------------------------------------------------------------------- + + delete(outcdf) + + ; ii, loop over files + ;end do + ;jj, loop over destination grids + delete([/lat,lon,nlon,nlat/]) + delete([/rgmask2d/]) + end do + print("One complete ice file in " + (get_cpu_time() - begTime) + " seconds") +exit +end diff --git a/ush/nems.configure.med_atm_ocn_ice_wav.IN b/ush/nems.configure.med_atm_ocn_ice_wav.IN index 5acce5cd73..3a4ffe040d 100755 --- a/ush/nems.configure.med_atm_ocn_ice_wav.IN +++ b/ush/nems.configure.med_atm_ocn_ice_wav.IN @@ -61,13 +61,13 @@ WAV_attributes:: # Run Sequence # runSeq:: - @@[coupling_interval_slow_sec] + @@[CPL_SLOW] MED MedPhase_prep_ocn MED -> OCN :remapMethod=redist OCN -> WAV WAV -> OCN :srcMaskValues=1 OCN - @@[coupling_interval_fast_sec] + @@[CPL_FAST] MED MedPhase_prep_ice MED MedPhase_prep_atm MED -> ATM :remapMethod=redist diff --git a/ush/nems.configure.medcold_atm_ocn_ice_wav.IN b/ush/nems.configure.medcold_atm_ocn_ice_wav.IN index bd426d6eb5..12eaa9cc23 100644 --- a/ush/nems.configure.medcold_atm_ocn_ice_wav.IN +++ b/ush/nems.configure.medcold_atm_ocn_ice_wav.IN @@ -61,10 +61,10 @@ WAV_attributes:: # Run Sequence # runSeq:: - @@[coupling_interval_slow_sec] + @@[CPL_SLOW] OCN -> WAV WAV -> OCN :srcMaskValues=1 - @@[coupling_interval_fast_sec] + @@[CPL_FAST] MED MedPhase_prep_atm MED -> ATM :remapMethod=redist WAV -> ATM :srcMaskValues=1 diff --git a/ush/nems_configure.sh b/ush/nems_configure.sh index a362d12259..4573a8fa50 100755 --- a/ush/nems_configure.sh +++ b/ush/nems_configure.sh @@ -29,10 +29,12 @@ fi ATM_model=${ATM_model:-'fv3'} OCN_model=${OCN_model:-'mom6'} ICE_model=${ICE_model:-'cice'} +WAV_model=${WAV_model:-'ww3'} ATMPETS=${ATMPETS:-8} OCNPETS=${OCNPETS:-8} ICEPETS=${ICEPETS:-8} +WAVPETS=${WAVPETS:-8} rm -f $DATA/nems.configure @@ -40,6 +42,7 @@ med_petlist_bounds=${med_petlist_bounds:-"0 $(( $ATMPETS-1 ))"} atm_petlist_bounds=${atm_petlist_bounds:-"0 $(( $ATMPETS-1 ))"} #6*8*6+wrtgrps(24) ocn_petlist_bounds=${ocn_petlist_bounds:-"$ATMPETS $(( $ATMPETS+$OCNPETS-1 ))"} #120 ice_petlist_bounds=${ice_petlist_bounds:-"$(( $ATMPETS+$OCNPETS )) $(( $ATMPETS+$OCNPETS+$ICEPETS-1 ))"} #48 +wav_petlist_bounds=${wav_petlist_bounds:-"$(( $ATMPETS+$OCNPETS+$ICEPETS )) $(( $ATMPETS+$OCNPETS+$ICEPETS+$WAVPETS-1 ))"} #48 #if [ $CASE = "C96" ] ; then # med_petlist_bounds=${med_petlist_bounds:-'0 149'} @@ -86,6 +89,7 @@ if [ $cplflx = .true. ]; then fi if [ $cplwav = .true. ]; then sed -i -e "s;@\[wav_model\];ww3;g" tmp1 + sed -i -e "s;@\[wav_petlist_bounds\];$wav_petlist_bounds;g" tmp1 fi if [ $cplice = .true. ]; then sed -i -e "s;@\[ice_model\];$ICE_model;g" tmp1 diff --git a/ush/ocnpost.ncl b/ush/ocnpost.ncl new file mode 100644 index 0000000000..9bd82ef856 --- /dev/null +++ b/ush/ocnpost.ncl @@ -0,0 +1,587 @@ +;------------------------------------------------------------------ +; Denise.Worthen@noaa.gov (Feb 2019) +; +; This script will remap MOM6 ocean output on the tripole grid to +; a set of rectilinear grids using pre-computed ESMF weights to remap +; the listed fields to the destination grid and write the results +; to a new netCDF file +; +; Prior to running this script, files containing the conservative +; and bilinear regridding weights must be generated. These weights +; are created using the generate_iceocnpost_weights.ncl script. +; +; Note: the descriptive text below assumes fortran type indexing +; where the variables are indexed as (i,j) and indices start at 1 +; NCL indices are (j,i) and start at 0 +; +; The post involves these steps +; +; a) unstaggering velocity points +; MOM6 is on an Arakawa C grid. MOM6 refers to these +; locations as "Ct" for the centers and "Cu", "Cv" +; "Bu" for the left-right, north-south and corner +; points, respectively. +; +; The indexing scheme in MOM6 is as follows: +; +; Cv@i,j +; ----X------X Bu@i,j +; | +; | +; Ct@i,j | +; X X Cu@i,j +; | +; | +; | +; +; CICE5 is on an Arakawa B grid. CICE5 refers to these +; locations as TLAT,TLON for the centers and ULAT,ULON +; for the corners +; +; In UFS, the CICE5 grid has been created using the MOM6 +; supergrid file. Therefore, all grid points are consistent +; between the two models. +; +; In the following, MOM6's nomenclature will be followed, +; so that CICE5's U-grid will be referred to as "Bu". +; +; b) rotation of tripole vectors to East-West +; MOM6 and CICE6 both output velocties on their native +; velocity points. For MOM6, that is u-velocities on the +; Cu grid and v-velocites on the Cv grid. For CICE5, it is +; both u and v-velocities on the Bu grid. +; +; The rotation angle for both models are defined at center +; grid points; therefore the velocities need to be first +; unstaggered before rotation. MOM6 and CICE5 also define +; opposite directions for the rotations. Finally, while the +; grid points are identical between the two models, CICE5 +; calculates the rotation angle at center grid points by +; averaging the four surrounding B grid points. MOM6 derives +; the rotation angle at the center directly from the latitude +; and longitude of the center grid points. The angles are therefor +; not identical between the two grids. +; +; c) conservative regridding of some fields +; Fields such as ice concentration or fluxes which inherently +; area area-weighted require conservative regridding. Most other +; variables are state variables and can be regridded using +; bilinear weighting. +; +; An efficient way to accomplish the unstaggering of velocities +; is to use the bilinear interpolation weights between grid +; points of the Arakawa C grid and the center grid points (for example +; Cu->Ct). These weights are generated by the weight generation script +; +; Remapping from the tripole to rectilinear uses either the bilinear +; or conservative weights from the weight generation script. Bilinear weights +; generated for the first vertical level can be used on other levels +; (where the masking changes) by utilizing the correct masking procedure. +; Set output_masks to true to examine the interpolation masks. +; +; Intermediate file output can easily be generated for debugging by +; follwing the example in the output_masks logical +; +; Bin.Li@noaa.gov (May 2019) +; The scripts is revised for use in the coupled workflow. +; + load "$NCARG_ROOT/lib/ncarg/nclscripts/esmf/ESMF_regridding.ncl" + +;---------------------------------------------------------------------- +begin +; + + ; warnings (generated by int2p_n_Wrap) can be supressed by + ; the following (comment out to get the warnings) + err = NhlGetErrorObjectId() + setvalues err +; "errLevel" : "Fatal" ; only report Fatal errors + "errLevel" : "Verbose" + end setvalues + + output_masks = False + + ; specify a location to use + ; nemsrc = "/scratch4/NCEPDEV/ocean/save/Denise.Worthen/NEMS_INPUT0.1/ocnicepost/" + ; interpolation methods + methods = (/"bilinear" ,"conserve"/) + ; ocean model output location + ;dirsrc = "/scratch3/NCEPDEV/stmp2/Denise.Worthen/BM1_ocn/" + + ; destination grid sizes and name + dsttype = (/"rect."/) + ;dstgrds = (/"1p0", "0p5", "0p25"/) + ;dstgrds = (/"0p5"/) + dstgrds = (/"0p25"/) + + ; variables to be regridded with the native tripole stagger location + ; and dimensionality + ; first BM contained only field "mld", which was actually ePBL + ; the remaining BMs contain ePBL, MLD_003 and MLD_0125 + ; the following NCO command will be issued at the end + ; to rename the variable mld to ePBL if the variable mld is found + ; ncocmd = "ncrename -O -v mld,ePBL " + ncocmd = "ncrename -O -v MLD_003,mld" + + varlist = (/ (/ "SSH", "Ct", "bilinear", "2"/) \ + ,(/ "SST", "Ct", "bilinear", "2"/) \ + ,(/ "SSS", "Ct", "bilinear", "2"/) \ + ,(/ "speed", "Ct", "bilinear", "2"/) \ + ,(/ "temp", "Ct", "bilinear", "3"/) \ + ,(/ "so", "Ct", "bilinear", "3"/) \ + ,(/ "latent", "Ct", "conserve", "2"/) \ + ,(/ "sensible", "Ct", "conserve", "2"/) \ + ,(/ "SW", "Ct", "conserve", "2"/) \ + ,(/ "LW", "Ct", "conserve", "2"/) \ + ,(/ "evap", "Ct", "conserve", "2"/) \ + ,(/ "lprec", "Ct", "conserve", "2"/) \ + ,(/ "fprec", "Ct", "conserve", "2"/) \ + ,(/"LwLatSens", "Ct", "conserve", "2"/) \ + ,(/ "Heat_PmE", "Ct", "conserve", "2"/) \ +; ,(/ "mld", "Ct", "bilinear", "2"/) \ + ,(/ "ePBL", "Ct", "bilinear", "2"/) \ + ,(/ "MLD_003", "Ct", "bilinear", "2"/) \ + ,(/ "MLD_0125", "Ct", "bilinear", "2"/) \ + /) + dims = dimsizes(varlist) + nvars = dims(0) + delete(dims) + ;print(varlist) + + ; vectors to be regridded with the native tripole stagger location + ; and dimensionality + ; note: vectors are always unstaggered using bilinear weights, but can + ; be remapped using conservative + nvpairs = 3 + veclist = new( (/nvpairs,4,2/),"string") + veclist = (/ (/ (/ "SSU", "SSV"/), (/"Cu", "Cv"/), (/"bilinear", "bilinear"/), (/"2", "2"/) /) \ + , (/ (/ "uo", "vo"/), (/"Cu", "Cv"/), (/"bilinear", "bilinear"/), (/"3", "3"/) /) \ + , (/ (/ "taux", "tauy"/), (/"Cu", "Cv"/), (/"conserve", "conserve"/), (/"2", "2"/) /) \ + /) + ;print(veclist) + + begTime = get_cpu_time() +;---------------------------------------------------------------------- +; make a list of the directories and files from the run +;---------------------------------------------------------------------- + +; idate = "20120101" + +; ocnfilelist = systemfunc("ls "+dirsrc+"gfs."+idate+"/00/"+"ocn*.nc") +; ocnf = addfiles(ocnfilelist,"r") +; nfiles = dimsizes(ocnfilelist) +; + + ; get the rotation angles and vertical grid from the first file + ; two different name were used for the angles, either sinrot,cosrot + ; or sin_rot,cos_rot + if(isfilevar(ocnf[0],"sin_rot"))then + sinrot = ocnf[0]->sin_rot + else + sinrot = ocnf[0]->sinrot + end if + if(isfilevar(ocnf[0],"cos_rot"))then + cosrot = ocnf[0]->cos_rot + else + cosrot = ocnf[0]->cosrot + end if + z_l = ocnf[0]->z_l + z_i = ocnf[0]->z_i + nlevs = dimsizes(z_l) + + ; get a 2 and 3 dimensional fields for creating the interpolation masks + ; the mask2d,mask3d contain 1's on land and 0's at valid points. + mask2d = where(ismissing(ocnf[0]->SST), 1.0, 0.0) + mask3d = where(ismissing(ocnf[0]->temp), 1.0, 0.0) + ;printVarSummary(mask2d) + ;printVarSummary(mask3d) + + ; create conformed rotation arrays to make vector rotations cleaner + sinrot2d=conform_dims(dimsizes(mask2d),sinrot,(/1,2/)) + cosrot2d=conform_dims(dimsizes(mask2d),cosrot,(/1,2/)) + + sinrot3d=conform_dims(dimsizes(mask3d),sinrot,(/2,3/)) + cosrot3d=conform_dims(dimsizes(mask3d),cosrot,(/2,3/)) + + ; check for variables in file. this is only required because + ; of the missing/misnamed MLD variables in the first BM + ; only the varlist is checked, since it is assumed there are + ; no other variables missing after the first benchmark + valid = new((/nvars/),"logical") + valid = False + do nv = 0,nvars-1 + varname = varlist(nv,0) + if(isfilevar(ocnf[0],varname))then + valid(nv) = True + end if + print(varlist(nv,0)+" "+valid(nv)) + end do + +;---------------------------------------------------------------------- +; loop over the output resolutions +;---------------------------------------------------------------------- + + jj = 1 + ii = 0 + + do jj = 0,dimsizes(dstgrds)-1 + ;outres = "_"+dstgrds(jj)+"x"+dstgrds(jj) + outres = dstgrds(jj)+"x"+dstgrds(jj) + outgrid = dstgrds(jj) + + ; regrid a field to obtain the output xy dimensions + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+".bilinear.nc" + tt = ESMF_regrid_with_weights(sinrot,wgtsfile,False) + tt!0 = "lat" + tt!1 = "lon" + lat = tt&lat + lon = tt&lon + dims = dimsizes(tt) + nlat = dims(0) + nlon = dims(1) + + print("fields will be remapped to destination grid size "\ + +nlon+" "+nlat) + + delete(tt) + delete(dims) + + ; regrid the masks to obtain the interpolation masks. + ; the mask2d,mask3d contain 1's on land and 0's at valid points. + ; when remapped, any mask value > 0 identifies land values that + ; have crept into the field. remapped model fields are then + ; masked with this interpolation mask + + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+".bilinear.nc" + rgmask2d = ESMF_regrid_with_weights(mask2d, wgtsfile,False) + rgmask3d = ESMF_regrid_with_weights(mask3d, wgtsfile,False) + + if(output_masks)then + testfile = "masks_"+dstgrds(jj)+".nc" + system("/bin/rm -f "+testfile) + ; create + testcdf = addfile(testfile,"c") + testcdf->rgmask2d = rgmask2d + testcdf->rgmask3d = rgmask3d + ; close + delete(testcdf) + end if + + ; create the interpolation mask + rgmask2d = where(rgmask2d .gt. 0.0, rgmask2d@_FillValue, 1.0) + rgmask3d = where(rgmask3d .gt. 0.0, rgmask3d@_FillValue, 1.0) + + ; conformed depth array + depth = conform_dims(dimsizes(mask3d), z_l, (/1/)) + ;print(dimsizes(depth)) + +;---------------------------------------------------------------------- +; loop over each file in the ocnfilelist +;---------------------------------------------------------------------- +; + + ; retrieve the time stamp + time = ocnf[0]->time + delete(time@bounds) + +;---------------------------------------------------------------------- +; set up the output netcdf file +;---------------------------------------------------------------------- +; system("/bin/rm -f " + outfile) ; remove if exists +; outcdf = addfile (outfile, "c") ; open output file +; specify output file information and open file for output + FILENAME_REGRID = DATA_TMP+"/ocnr"+VDATE+"."+ENSMEM+"."+IDATE+"_"+outres+"_MOM6.nc" + if (isfilepresent(FILENAME_REGRID)) then + system("rm -f "+FILENAME_REGRID) + end if + outcdf = addfile(FILENAME_REGRID,"c") + outfile=FILENAME_REGRID + + ; explicitly declare file definition mode. Improve efficiency. + setfileoption(outcdf,"DefineMode",True) + + ; create global attributes of the file + fAtt = True ; assign file attributes + fAtt@creation_date = systemfunc ("date") + fAtt@source_file = infile + fileattdef( outcdf, fAtt ) ; copy file attributes + + ; predefine the coordinate variables and their dimensionality + ; dimNames = (/"time", "z_l", "z_i", "z_T", "lat", "lon"/) + dimNames = (/"time", "z_l", "z_i", "lat", "lon"/) + ;dimSizes = (/ -1 , nlevs, nlevs+1, nTd, nlat, nlon/) + dimSizes = (/ -1 , nlevs, nlevs+1, nlat, nlon/) + ;dimUnlim = (/ True , False, False, False, False, False/) + dimUnlim = (/ True , False, False, False, False/) + filedimdef(outcdf,dimNames,dimSizes,dimUnlim) + + ; predefine the the dimensionality of the variables to be written out + filevardef(outcdf, "time", typeof(time), getvardims(time)) + filevardef(outcdf, "z_l", typeof(z_l), getvardims(z_l)) + filevardef(outcdf, "z_i", typeof(z_i), getvardims(z_i)) + ;filevardef(outcdf, "z_T", typeof(z_T), getvardims(z_T)) + filevardef(outcdf, "lat", typeof(lat), getvardims(lat)) + filevardef(outcdf, "lon", typeof(lon), getvardims(lon)) + + ; Copy attributes associated with each variable to the file + filevarattdef(outcdf, "time", time) + filevarattdef(outcdf, "z_l", z_l) + filevarattdef(outcdf, "z_i", z_i) + ;filevarattdef(outcdf, "z_T", z_T) + filevarattdef(outcdf, "lat", lat) + filevarattdef(outcdf, "lon", lon) + + ; predefine variables + do nv = 0,nvars-1 + varname = varlist(nv,0) + vardims = varlist(nv,3) + if(valid(nv))then + if(vardims .eq. "2")then + odims = (/"time", "lat", "lon"/) + else + odims = (/"time", "z_l", "lat", "lon"/) + end if + ;print("creating variable "+varname+" in file") + filevardef(outcdf, varname, "float", odims) + delete(odims) + end if + end do + + do nv = 0,nvpairs-1 + do nn = 0,1 + vecname = veclist(nv,0,nn) + vecdims = veclist(nv,3,nn) + if(vecdims .eq. "2")then + odims = (/"time", "lat", "lon"/) + else + odims = (/"time", "z_l", "lat", "lon"/) + end if + ;print("creating variable "+vecname+" in file") + filevardef(outcdf, vecname, "float", odims) + delete(odims) + delete(vecdims) + end do + end do + + ; explicitly exit file definition mode. + setfileoption(outcdf,"DefineMode",False) + + ; write the dimensions to the file + outcdf->time = (/time/) + outcdf->z_l = (/z_l/) + outcdf->z_i = (/z_i/) +; outcdf->z_T = (/z_T/) +; + outcdf->lat = (/lat/) + outcdf->lon = (/lon/) + +;---------------------------------------------------------------------- +; loop over nvars variables +;---------------------------------------------------------------------- + + do nv = 0,nvars-1 + varname = varlist(nv,0) + vargrid = varlist(nv,1) + varmeth = varlist(nv,2) + vardims = varlist(nv,3) + + if(valid(nv))then + ;print(nv+" "+varname+" "+vargrid+" "+varmeth) + ocnvar = ocnf[ii]->$varname$ + ndims = dimsizes(dimsizes(ocnvar)) + ;print(ndims+" "+dimsizes(ocnvar)) + + if(vargrid .ne. "Ct")then + ; print error if the variable is not on the Ct grid + print("Variable is not on Ct grid") + exit + end if + + ; regrid to dsttype+dstgrd with method + ;print("remapping "+varname+" to grid "+dsttype+dstgrds(jj)) + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+"."+varmeth+".nc" + + rgtt = ESMF_regrid_with_weights(ocnvar,wgtsfile,False) + if(vardims .eq. "2")then + rgtt = where(ismissing(rgmask2d),ocnvar@_FillValue,rgtt) + rgtt=rgtt(:,::-1,:) + else + rgtt = where(ismissing(rgmask3d),ocnvar@_FillValue,rgtt) + rgtt=rgtt(:,:,::-1,:) + end if + + ; enter file definition mode to add variable attributes + setfileoption(outcdf,"DefineMode",True) + filevarattdef(outcdf, varname, rgtt) + setfileoption(outcdf,"DefineMode",False) + + outcdf->$varname$ = (/rgtt/) + + delete(ocnvar) + delete(rgtt) + + ; variable exists + end if + ; nv, loop over number of variables + end do + +;---------------------------------------------------------------------- +; +;---------------------------------------------------------------------- + + ;nv = 2 + do nv = 0,nvpairs-1 + vecnames = veclist(nv,0,:) + vecgrids = veclist(nv,1,:) + vecmeth = veclist(nv,2,:) + vecdims = veclist(nv,3,:) + ;print(nv+" "+vecnames+" "+vecgrids+" "+vecmeth) + + ; create a vector pair list + vecpairs = NewList("fifo") + n = 0 + uvel = ocnf[ii]->$vecnames(n)$ + vecfld = where(ismissing(uvel),0.0,uvel) + copy_VarAtts(uvel,vecfld) + ;print("unstagger "+vecnames(n)+" from "+vecgrids(n)+" to Ct") + wgtsfile = nemsrc+"/"+"tripole.mx025."+vecgrids(n)+".to.Ct.bilinear.nc" + ut = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + delete(ut@remap) + + n = 1 + vvel = ocnf[ii]->$vecnames(n)$ + vecfld = where(ismissing(vvel),0.0,vvel) + copy_VarAtts(vvel,vecfld) + ;print("unstagger "+vecnames(n)+" from "+vecgrids(n)+" to Ct") + wgtsfile = nemsrc+"/"+"tripole.mx025."+vecgrids(n)+".to.Ct.bilinear.nc" + vt = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + delete(vt@remap) + + ListAppend(vecpairs,ut) + ListAppend(vecpairs,vt) + ;print(vecpairs) + + ; rotate + ; first copy Metadata + urot = vecpairs[0] + vrot = vecpairs[1] + if(vecdims(0) .eq. "2")then + urot = ut*cosrot2d + vt*sinrot2d + vrot = vt*cosrot2d - ut*sinrot2d + else + urot = ut*cosrot3d + vt*sinrot3d + vrot = vt*cosrot3d - ut*sinrot3d + end if + ; change attribute to indicate these are now rotated velocities + urot@long_name=str_sub_str(urot@long_name,"X","Zonal") + vrot@long_name=str_sub_str(vrot@long_name,"Y","Meridional") + ; copy back + vecpairs[0] = urot + vecpairs[1] = vrot + delete([/urot, vrot/]) + + ; remap + do n = 0,1 + vecfld = vecpairs[n] + ; regrid to dsttype+dstgrd with method + ;print("remapping "+vecnames(n)+" to grid "+dsttype+dstgrds(jj)) + wgtsfile = nemsrc+"/"+"tripole.mx025.Ct.to."+dsttype+dstgrds(jj)+"."+vecmeth(n)+".nc" + + rgtt = ESMF_regrid_with_weights(vecfld,wgtsfile,False) + if(vecdims(n) .eq. "2")then + rgtt = where(ismissing(rgmask2d),vecfld@_FillValue,rgtt) + rgtt=rgtt(:,::-1,:) + else + rgtt = where(ismissing(rgmask3d),vecfld@_FillValue,rgtt) + rgtt=rgtt(:,:,::-1,:) + end if + + ; enter file definition mode to add variable attributes + setfileoption(outcdf,"DefineMode",True) + filevarattdef(outcdf, vecnames(n), rgtt) + setfileoption(outcdf,"DefineMode",False) + + outcdf->$vecnames(n)$ = (/rgtt/) + delete(rgtt) + end do + delete([/uvel,vvel,ut,vt,vecfld,vecpairs/]) + delete([/vecnames,vecgrids,vecmeth,vecdims/]) + ; nv, loop over number of vector pairs + end do + +;---------------------------------------------------------------------- +; close the outcdf and continue through filelist +;---------------------------------------------------------------------- + + delete(outcdf) + ; rename mld to ePBL if required + do nv = 0,nvars-1 + varname = varlist(nv,0) + ; if(varname .eq. "mld" .and. valid(nv))then + if(varname .eq. "MLD_003" .and. valid(nv))then + print("Renaming MLD_003 to mld") + ;print(ncocmd+" "+outfile) + system(ncocmd+" "+outfile) + end if + end do + + ; ii, loop over files +; + ;jj, loop over destination grids + delete([/lat,lon,nlon,nlat/]) + delete([/rgmask2d,rgmask3d/]) + end do + print("One complete ocn file in " + (get_cpu_time() - begTime) + " seconds") +exit +end diff --git a/ush/parsing_namelists_FV3.sh b/ush/parsing_namelists_FV3.sh index 202208cb4a..083c2e4015 100755 --- a/ush/parsing_namelists_FV3.sh +++ b/ush/parsing_namelists_FV3.sh @@ -50,7 +50,7 @@ cat > input.nml <> input.nml << EOF ccpp_suite = ${CCPP_SUITE:-"FV3_GFS_v15"} EOF @@ -209,9 +209,7 @@ if [ $CCPP_SUITE = "FV3_GFS_v15p2_coupled" ]; then oz_phys = .false. oz_phys_2015 = .true. EOF -fi - -if [ $CCPP_SUITE = "FV3_GSD_v0" ]; then +elif [ $CCPP_SUITE = "FV3_GSD_v0" ]; then cat >> input.nml << EOF ltaerosol = ${ltaerosol:-".F."} ! In config.fcst lradar = ${lradar:-".F."} ! In config.fcst @@ -289,9 +287,11 @@ cat >> input.nml <> input.nml << EOF cplflx = $cplflx + cplwav = ${cplwav} ! CROW configured + cplwav2atm = ${cplwav2atm} ! CROW configured EOF fi diff --git a/ush/wave_grib2_sbs.sh b/ush/wave_grib2_sbs.sh index 3e8bef351c..2afec2c65e 100755 --- a/ush/wave_grib2_sbs.sh +++ b/ush/wave_grib2_sbs.sh @@ -80,7 +80,7 @@ echo " Model ID : $WAV_MOD_TAG" [[ "$LOUD" = YES ]] && set -x - if [ -z "$CDATE" ] || [ -z "$cycle" ] || [ -z "$EXECwave" ] || [ -z "$EXECcode" ] || \ + if [ -z "$CDATE" ] || [ -z "$cycle" ] || [ -z "$EXECwave" ] || \ [ -z "$COMOUT" ] || [ -z "$WAV_MOD_TAG" ] || [ -z "$SENDCOM" ] || \ [ -z "$gribflags" ] || \ [ -z "$GRIDNR" ] || [ -z "$MODNR" ] || [ -z "$SENDDBN" ] @@ -134,12 +134,12 @@ set +x echo " Run ww3_grib2" - echo " Executing $EXECcode/ww3_grib" + echo " Executing $EXECwave/ww3_grib" [[ "$LOUD" = YES ]] && set -x ENSTAG="" if [ ${waveMEMB} ]; then ENSTAG=".${membTAG}${waveMEMB}" ; fi outfile=${WAV_MOD_TAG}.${cycle}${ENSTAG}.${grdnam}.${grdres}.f${FH3}.grib2 - $EXECcode/ww3_grib + $EXECwave/ww3_grib > grib2_${grdnam}_${FH3}.out 2>&1 $WGRIB2 gribfile -set_date $CDATE -set_ftime "$fhr hour fcst" -grib ${COMOUT}/gridded/${outfile} err=$? @@ -159,6 +159,18 @@ # Create index $WGRIB2 -s $COMOUT/gridded/${outfile} > $COMOUT/gridded/${outfile}.idx +# Create grib2 subgrid is this is the source grid + if [ "${grdID}" = "${WAV_SUBGRBSRC}" ]; then + for subgrb in ${WAV_SUBGRB}; do + subgrbref=`echo ${!subgrb} | cut -d " " -f 1-20` + subgrbnam=`echo ${!subgrb} | cut -d " " -f 21` + subgrbres=`echo ${!subgrb} | cut -d " " -f 22` + subfnam="${WAV_MOD_TAG}.${cycle}${ENSTAG}.${subgrbnam}.${subgrbres}.f${FH3}.grib2" + $COPYGB2 -g "${subgrbref}" -i0 -x ${COMOUT}/gridded/${outfile} ${COMOUT}/gridded/${subfnam} + $WGRIB2 -s $COMOUT/gridded/${subfnam} > $COMOUT/gridded/${subfnam}.idx + done + fi + # 1.e Save in /com if [ ! -s $COMOUT/gridded/${outfile} ] diff --git a/ush/wave_grid_interp.sh b/ush/wave_grid_interp.sh index bb91840212..273d8a813d 100755 --- a/ush/wave_grid_interp.sh +++ b/ush/wave_grid_interp.sh @@ -63,7 +63,7 @@ echo " Model ID : $WAV_MOD_TAG" [[ "$LOUD" = YES ]] && set -x - if [ -z "$YMDH" ] || [ -z "$cycle" ] || [ -z "$EXECcode" ] || \ + if [ -z "$YMDH" ] || [ -z "$cycle" ] || [ -z "$EXECwave" ] || \ [ -z "$COMOUT" ] || [ -z "$WAV_MOD_TAG" ] || [ -z "$SENDCOM" ] || \ [ -z "$SENDDBN" ] || [ -z "$waveGRD" ] then @@ -73,7 +73,7 @@ echo '*** EXPORTED VARIABLES IN postprocessor NOT SET ***' echo '***************************************************' echo ' ' - echo "$YMDH $cycle $EXECcode $COMOUT $WAV_MOD_TAG $SENDCOM $SENDDBN $waveGRD" + echo "$YMDH $cycle $EXECwave $COMOUT $WAV_MOD_TAG $SENDCOM $SENDDBN $waveGRD" [[ "$LOUD" = YES ]] && set -x postmsg "$jlogfile" "EXPORTED VARIABLES IN postprocessor NOT SET" exit 1 @@ -136,10 +136,10 @@ set +x echo " Run ww3_gint - echo " Executing $EXECcode/ww3_gint + echo " Executing $EXECwave/ww3_gint [[ "$LOUD" = YES ]] && set -x - $EXECcode/ww3_gint + $EXECwave/ww3_gint 1> gint.{$grdID}.out 2>&1 err=$? # Write interpolation file to main TEMP dir area if not there yet diff --git a/ush/wave_grid_interp_sbs.sh b/ush/wave_grid_interp_sbs.sh index 0f78d3f557..f2e0855331 100755 --- a/ush/wave_grid_interp_sbs.sh +++ b/ush/wave_grid_interp_sbs.sh @@ -71,7 +71,7 @@ echo " Model ID : $WAV_MOD_TAG" [[ "$LOUD" = YES ]] && set -x - if [ -z "$CDATE" ] || [ -z "$cycle" ] || [ -z "$EXECcode" ] || \ + if [ -z "$CDATE" ] || [ -z "$cycle" ] || [ -z "$EXECwave" ] || \ [ -z "$COMOUT" ] || [ -z "$WAV_MOD_TAG" ] || [ -z "$SENDCOM" ] || \ [ -z "$SENDDBN" ] || [ -z "$waveGRD" ] then @@ -81,7 +81,7 @@ echo '*** EXPORTED VARIABLES IN postprocessor NOT SET ***' echo '***************************************************' echo ' ' - echo "$CDATE $cycle $EXECcode $COMOUT $WAV_MOD_TAG $SENDCOM $SENDDBN $waveGRD" + echo "$CDATE $cycle $EXECwave $COMOUT $WAV_MOD_TAG $SENDCOM $SENDDBN $waveGRD" [[ "$LOUD" = YES ]] && set -x postmsg "$jlogfile" "EXPORTED VARIABLES IN postprocessor NOT SET" exit 1 @@ -144,10 +144,10 @@ set +x echo " Run ww3_gint - echo " Executing $EXECcode/ww3_gint + echo " Executing $EXECwave/ww3_gint [[ "$LOUD" = YES ]] && set -x - $EXECcode/ww3_gint + $EXECwave/ww3_gint 1> gint.${grdID}.out 2>&1 err=$? # Write interpolation file to main TEMP dir area if not there yet diff --git a/ush/wave_outp_spec.sh b/ush/wave_outp_spec.sh index 9cbcf6e9a6..2e902bc212 100755 --- a/ush/wave_outp_spec.sh +++ b/ush/wave_outp_spec.sh @@ -112,7 +112,7 @@ # 0.c Define directories and the search path. # The tested variables should be exported by the postprocessor script. - if [ -z "$CDATE" ] || [ -z "$dtspec" ] || [ -z "$EXECcode" ] || \ + if [ -z "$CDATE" ] || [ -z "$dtspec" ] || [ -z "$EXECwave" ] || \ [ -z "$WAV_MOD_TAG" ] || [ -z "${STA_DIR}" ] then set +x @@ -179,10 +179,10 @@ # 2.b Run the postprocessor set +x - echo " Executing $EXECcode/ww3_outp" + echo " Executing $EXECwave/ww3_outp" [[ "$LOUD" = YES ]] && set -x - $EXECcode/ww3_outp + $EXECwave/ww3_outp 1> outp_${specdir}_${buoy}.out 2>&1 err=$? if [ "$err" != '0' ] @@ -211,7 +211,6 @@ cat $outfile | sed -e '9,$d' >> ${STA_DIR}/${specdir}/$WAV_MOD_TAG.$buoy.bull cat $coutfile | sed -e '8,$d' >> ${STA_DIR}/c${specdir}/$WAV_MOD_TAG.$buoy.cbull else - #cat $outfile | sed -e '15,$d' >> ${STA_DIR}/${specdir}/$WAV_MOD_TAG.$buoy.spec cat $outfile >> ${STA_DIR}/${specdir}/$WAV_MOD_TAG.$buoy.spec fi elif [ "${ymdh}" = "${YMDHE}" ] @@ -250,7 +249,7 @@ # rm -f mod_def.ww3 out_pnt.ww3 cd .. - mv -f $specdir_$buoy done.$specdir_$buoy + rm -rf ${specdir}_${bloc} set +x echo ' ' diff --git a/ush/wave_prnc_cur.sh b/ush/wave_prnc_cur.sh index 1a6dd3608f..848088540c 100755 --- a/ush/wave_prnc_cur.sh +++ b/ush/wave_prnc_cur.sh @@ -23,40 +23,43 @@ set -x ymdh_rtofs=$1 curfile=$2 +fhr=$3 +fh3=`printf "%03d" "${fhr#0}"` # Timing has to be made relative to the single 00z RTOFS cycle for that PDY mkdir -p rtofs_${ymdh_rtofs} cd rtofs_${ymdh_rtofs} -ncks -x -v sst,sss,layer_density $curfile cur_uv_${PDY}_${fext}${fhr}.nc -ncks -O -a -h -x -v Layer cur_uv_${PDY}_${fext}${fhr}.nc cur_temp1.nc +ncks -x -v sst,sss,layer_density $curfile cur_uv_${PDY}_${fext}${fh3}.nc +ncks -O -a -h -x -v Layer cur_uv_${PDY}_${fext}${fh3}.nc cur_temp1.nc ncwa -h -O -a Layer cur_temp1.nc cur_temp2.nc ncrename -h -O -v MT,time cur_temp2.nc ncrename -h -O -d MT,time cur_temp2.nc ncks -v u_velocity,v_velocity cur_temp2.nc cur_temp3.nc -mv -f cur_temp3.nc cur_uv_${PDY}_${fext}${fhr}_flat.nc +mv -f cur_temp3.nc cur_uv_${PDY}_${fext}${fh3}_flat.nc # Convert to regular lat lon file - +# If weights need to be regenerated due to CDO ver change, use: +# $CDO genbil,r4320x2160 rtofs_glo_2ds_f000_3hrly_prog.nc weights.nc cp ${FIXwave}/weights_rtofs_to_r4320x2160.nc ./weights.nc # Interpolate to regular 5 min grid -$CDO remap,r4320x2160,weights.nc cur_uv_${PDY}_${fext}${fhr}_flat.nc cur_5min_01.nc +$CDO remap,r4320x2160,weights.nc cur_uv_${PDY}_${fext}${fh3}_flat.nc cur_5min_01.nc # Perform 9-point smoothing twice to make RTOFS data less noisy when # interpolating from 1/12 deg RTOFS grid to 1/6 deg wave grid if [ "WAV_CUR_CDO_SMOOTH" = "YES" ]; then $CDO -f nc -smooth9 cur_5min_01.nc cur_5min_02.nc - $CDO -f nc -smooth9 cur_5min_02.nc cur_glo_uv_${PDY}_${fext}${fhr}_5min.nc + $CDO -f nc -smooth9 cur_5min_02.nc cur_glo_uv_${PDY}_${fext}${fh3}_5min.nc else - mv cur_5min_01.nc cur_glo_uv_${PDY}_${fext}${fhr}_5min.nc + mv cur_5min_01.nc cur_glo_uv_${PDY}_${fext}${fh3}_5min.nc fi # Cleanup -rm -f cur_temp[123].nc cur_5min_??.nc cur_glo_uv_${PDY}_${fext}${fhr}.nc weights.nc +rm -f cur_temp[123].nc cur_5min_??.nc cur_glo_uv_${PDY}_${fext}${fh3}.nc weights.nc -if [ ${fhr_wave} -gt ${WAVHINDH} ] +if [ ${fhr} -gt 0 ] then sed -e "s/HDRFL/F/g" ${FIXwave}/ww3_prnc.cur.${WAVECUR_FID}.inp.tmpl > ww3_prnc.inp else @@ -64,12 +67,12 @@ else fi rm -f cur.nc -ln -s cur_glo_uv_${PDY}_${fext}${fhr}_5min.nc cur.nc -ln -s ${DATA}/mod_def.rtofs_5m ./mod_def.ww3 +ln -s cur_glo_uv_${PDY}_${fext}${fh3}_5min.nc cur.nc +ln -s ${DATA}/mod_def.${WAVECUR_FID} ./mod_def.ww3 -$EXECcode/ww3_prnc +$EXECwave/ww3_prnc 1> prnc_${WAVECUR_FID}_${ymdh_rtofs}.out 2>&1 -mv -f current.ww3 ${DATA}/${WAVECUR_FID}.${ymdh_rtofs} +mv -f current.ww3 ${DATA}/${WAVECUR_DID}.${ymdh_rtofs} cd ${DATA} diff --git a/ush/wave_prnc_ice.sh b/ush/wave_prnc_ice.sh index 328b8c6728..c7a0a1a720 100755 --- a/ush/wave_prnc_ice.sh +++ b/ush/wave_prnc_ice.sh @@ -53,7 +53,7 @@ postmsg "$jlogfile" "Making ice fields." if [ -z "$YMDH" ] || [ -z "$cycle" ] || \ - [ -z "$COMOUT" ] || [ -z "$FIXwave" ] || [ -z "$EXECcode" ] || \ + [ -z "$COMOUT" ] || [ -z "$FIXwave" ] || [ -z "$EXECwave" ] || \ [ -z "$WAV_MOD_TAG" ] || [ -z "$WAVEICE_FID" ] || [ -z "$SENDCOM" ] || \ [ -z "$COMIN_WAV_ICE" ] || [ -z "$COMPONENTwave" ] then @@ -141,7 +141,7 @@ cp -f ${DATA}/ww3_prnc.ice.$WAVEICE_FID.inp.tmpl ww3_prnc.inp - $EXECcode/ww3_prnc > wave_prnc.out + $EXECwave/ww3_prnc 1> prnc_${WAVEICE_FID}_${cycle}.out 2>&1 err=$? if [ "$err" != '0' ] diff --git a/ush/wave_tar.sh b/ush/wave_tar.sh index edeb1994d9..b5550b6fd4 100755 --- a/ush/wave_tar.sh +++ b/ush/wave_tar.sh @@ -30,7 +30,7 @@ # Use LOUD variable to turn on/off trace. Defaults to YES (on). export LOUD=${LOUD:-YES}; [[ $LOUD = yes ]] && export LOUD=YES - [[ "$LOUD" != YES ]] && set +x + [[ "$LOUD" != YES ]] && set -x cd $DATA postmsg "$jlogfile" "Making TAR FILE" @@ -98,6 +98,7 @@ set +x echo ' ' echo ' Making tar file ...' + set -x count=0 countMAX=5 @@ -106,14 +107,12 @@ while [ "$count" -lt "$countMAX" ] && [ "$tardone" = 'no' ] do - [[ "$LOUD" = YES ]] && set -v - # JY nf=`ls $ID.*.$type | wc -l | awk '{ print $1 }'` nf=`ls | awk '/'$ID.*.$filext'/ {a++} END {print a}'` - if [ "$nf" = "$nb" ] + nbm2=$(( $nb - 2 )) + if [ $nf -ge $nbm2 ] then tar -cf $ID.$cycle.${type}_tar ./$ID.*.$filext exit=$? - set +v; [[ "$LOUD" = YES ]] && set -x if [ "$exit" != '0' ] then diff --git a/workflow/cases/coupled_free_forecast.yaml b/workflow/cases/coupled_free_forecast.yaml index fe852cf499..f0af5bf0ab 100644 --- a/workflow/cases/coupled_free_forecast.yaml +++ b/workflow/cases/coupled_free_forecast.yaml @@ -21,7 +21,7 @@ case: OCN_INTERVAL: 24 FHOUT_GFS: 6 FHMIN_GFS: 0 - FHMAX_GFS: 840 + FHMAX_GFS: 48 FHMAX_HF_GFS: 0 FHOUT_HF_GFS: -1 @@ -42,9 +42,13 @@ case: WGRP: 1 WGRP_NTASKS: 24 WRTIOBUF: "32M" + cdmbgwd: "1.0,1.2" + + ocn_settings: OCNPETS: 120 + + ice_settings: ICEPETS: 48 - cdmbgwd: "1.0,1.2" post: downset: 2 diff --git a/workflow/cases/coupled_free_forecast_wave.yaml b/workflow/cases/coupled_free_forecast_wave.yaml new file mode 100644 index 0000000000..da8a054fb1 --- /dev/null +++ b/workflow/cases/coupled_free_forecast_wave.yaml @@ -0,0 +1,60 @@ +case: + places: + workflow_file: layout/free_forecast_gfs.yaml + #BASE_CPLIC: /scratch2/NCEPDEV/climate/climpara/S2S/IC + + settings: + SDATE: 2013-04-01t00:00:00 + EDATE: 2013-04-01t00:00:00 + + cplwav: .true. + cplwav2atm: .true. + cplflx: .true. + print_esmf: .true. + nems_temp: 'med_atm_ocn_ice_wav' + nems_temp_cold: 'medcold_atm_ocn_ice_wav' + mom6ic_prepared: .false. + KEEPDATA: YES + + nsst: + NST_MODEL: 0 + + output_settings: + OCN_INTERVAL: 24 + FHOUT_GFS: 6 + FHMIN_GFS: 0 + FHMAX_GFS: 48 + FHMAX_HF_GFS: 0 + FHOUT_HF_GFS: -1 + + fv3_gfs_settings: + CASE: C384 + LEVS: 65 + DELTIM: 450 + SEEDLET: 10 + CCPP_SUITE: FV3_GFS_v15p2_coupled + nst_anl: no + DO_SKEB: false + DO_SHUM: false + DO_SPPT: false + layout: + x: 6 + y: 8 + nth: 2 + WGRP: 1 + WGRP_NTASKS: 24 + WRTIOBUF: "32M" + cdmbgwd: "1.0,1.2" + + ocn_settings: + OCNPETS: 120 + + ice_settings: + ICEPETS: 48 + + wave_settings: + WAVPETS: 40 + + post: + downset: 2 + GOESF: no diff --git a/workflow/config/base.yaml b/workflow/config/base.yaml index 85464783c2..049c2f95b6 100644 --- a/workflow/config/base.yaml +++ b/workflow/config/base.yaml @@ -10,6 +10,8 @@ config_base: do: .true. - when: !calc doc.settings.cplice==".true." do: .true. + - when: !calc doc.settings.cplwav==".true." + do: .true. - otherwise: .false. content: !expand | #!/bin/ksh -x @@ -29,6 +31,8 @@ config_base: export cpl="{cpl}" export cplflx="{doc.settings.cplflx}" export cplice="{doc.settings.cplice}" + export cplwav="{doc.settings.cplwav}" + export cplwav2atm="{doc.settings.cplwav2atm}" export print_esmf="{doc.settings.print_esmf}" export nems_temp="{doc.settings.nems_temp}" @@ -57,7 +61,7 @@ config_base: export FIXcice=$HOMEgfs/fix/fix_cice5 export FIXmom=$HOMEgfs/fix/fix_mom6 export FIXgrid=$HOMEgfs/fix/fix_fv3grid - export OCNFIXDIR=$HOMEgfs/fix/fix_ocnice + export OCNFIXDIR=$HOMEgfs/fix/fix_reg2grb2 # GLOBAL static environment parameters export DMPDIR="{doc.places.DMPDIR}" @@ -95,7 +99,6 @@ config_base: # Build paths relative to $HOMEgfs export HOMEgsi="$HOMEgfs" export FIXgsi="{doc.places.get('FIXgsi','$HOMEgfs/fix/fix_gsi')}" - export HOMEfv3gfs="{doc.places.get('HOMEfv3gfs','$HOMEgfs/sorc/fv3gfs.fd')}" export HOMEpost="{doc.places.get('HOMEpost','$HOMEgfs')}" export HOMEobsproc_prep="{doc.places.get('HOMEobsproc_prep', '$BASE_GIT/obsproc/obsproc_prep_RB-5.2.0')}" export HOMEobsproc_network="{doc.places.get('HOMEobsproc_network', '$BASE_GIT/obsproc/obsproc_global_RB-3.2.0')}" @@ -110,9 +113,6 @@ config_base: export CHGRP_CMD="{doc.platform.CHGRP_RSTPROD_COMMAND}" export NEMSIOGET="$HOMEgfs/exec/nemsio_get" - # For coupled - export UGCSsrc="/scratch2/NCEPDEV/climate/Bin.Li/S2S/fix/ocean_ice_post" - # Machine environment, jobs, and other utility scripts export BASE_ENV="$HOMEgfs/env" export BASE_JOB="$HOMEgfs/jobs/rocoto" @@ -162,7 +162,9 @@ config_base: export LEVS={doc.fv3_gfs_settings.LEVS} export CASE="{doc.fv3_gfs_settings.CASE}" export CASE_ENKF="{doc.fv3_enkf_settings.CASE}" - + export ICERES={doc.ice_settings.ICERES} + export OCNRES={doc.ocn_settings.OCNRES} + # Surface cycle update frequency if [ "$CDUMP" == "gdas" ] ; then export FHCYC={doc.fv3_enkf_settings.FHCYC_GDAS} diff --git a/workflow/config/fcst.yaml b/workflow/config/fcst.yaml index e1568d5145..4265ffc74a 100644 --- a/workflow/config/fcst.yaml +++ b/workflow/config/fcst.yaml @@ -67,8 +67,9 @@ config_fcst: ####################################################################### # PE Mapping export ATMPETS="{doc.fv3_gfs_settings.ATMPETS}" - export OCNPETS="{doc.fv3_gfs_settings.OCNPETS}" - export ICEPETS="{doc.fv3_gfs_settings.ICEPETS}" + export OCNPETS="{doc.ocn_settings.OCNPETS}" + export ICEPETS="{doc.ice_settings.ICEPETS}" + export WAVPETS="{doc.wave_settings.WAVPETS}" # ####################################################################### diff --git a/workflow/config/fv3ic.yaml b/workflow/config/fv3ic.yaml index 72c8dd6bfd..61f280b434 100644 --- a/workflow/config/fv3ic.yaml +++ b/workflow/config/fv3ic.yaml @@ -21,6 +21,10 @@ config_fv3ic: export npe_node_fv3ic={doc.partition_common.resources.run_fv3ic[0].mpi_ranks} export ORIGIN_ROOT="{doc.places.BASE_CPLIC}" + export CPL_ATMIC={doc.fv3_gfs_settings.CPL_ATMIC} + export CPL_ICEIC={doc.ice_settings.CPL_ICEIC} + export CPL_OCNIC={doc.ocn_settings.CPL_OCNIC} + export CPL_WAVIC={doc.wave_settings.CPL_WAVIC} echo "END: config.fv3ic" diff --git a/workflow/config/ocnpost.yaml b/workflow/config/ocnpost.yaml index cd37e8b691..2995d1c1b5 100644 --- a/workflow/config/ocnpost.yaml +++ b/workflow/config/ocnpost.yaml @@ -20,5 +20,6 @@ config_ocnpost: # No. of concurrent post jobs [0 implies sequential] export NPOSTGRP={doc.ocnpost.NPOSTGRP} + export ncks="{doc.places.ncks}" echo "END: config.ocnpost" diff --git a/workflow/config/wave.yaml b/workflow/config/wave.yaml new file mode 100644 index 0000000000..4e6df17c6a --- /dev/null +++ b/workflow/config/wave.yaml @@ -0,0 +1,143 @@ +config_wave: + filename: config.wave + content: !expand | + #!/bin/ksh -x + # + # This file is automatically generated from the YAML-based system + # in ecf/ecfutils/. Any changes will be overwritten if + # setup_case.sh is rerun. + + ########## config.base ########## + # Common to all steps + + echo "BEGIN: config.wave" + export wave_sys_ver=v1.0.0 + + export COMPONENTRSTwave={doc.wave_settings.COMPONENTRSTwave} + export HOMEwave={doc.places.HOMEgfs} + export HOMEgefs={doc.places.HOMEgfs} + + # Grids for wave model + export waveGRD='{doc.wave_settings.waveGRD}' + export waveGRDN='{doc.wave_settings.waveGRDN}' # gridnumber for ww3_multi + export waveGRDG='10 20 30' # gridgroup for ww3_multi + export USE_WAV_RMP='{doc.wave_settings.USE_WAV_RMP}' #yes/no rmp grid remapping pre-processed coefficients + + # ESMF input grid + export waveesmfGRD='' # input grid + + # Grids for output fields + export waveuoutpGRD=points + export waveinterpGRD='' # Grids that need to be interpolated from native + # in POST will generate grib unless gribOK not set + export wavesbsGRD='' # side-by-side grids generated as wave model runs, writes to com + export wavepostGRD='gwes_30m' # Native grids that will be post-processed (grib2) + + if [ "$CDUMP" = "gdas" ] + then + export FHMAX_WAV={doc.output_settings.FHMAX_GDAS} #for coupled model, use GDAS + else + export FHMAX_WAV={doc.output_settings.FHMAX_GFS} #for coupled model, use GFS + fi + export WAVHINDH={doc.wave_settings.WAVHINDH} + export FHMIN_WAV={doc.output_settings.FHMIN_WAV} + export FHOUT_WAV={doc.output_settings.FHOUT_WAV} + export FHMAX_HF_WAV={doc.output_settings.FHMAX_HF_WAV} + export FHOUT_HF_WAV={doc.output_settings.FHOUT_HF_WAV} + + # Output stride + export WAV_WND_HOUR_INC=1 # This value should match with the one used in + # # the wind update script + # gridded and point output rate + export DTFLD_WAV=`expr $FHOUT_HF_WAV \* 3600` + export DTPNT_WAV=10800 + export FHINCP_WAV=`expr $DTPNT_WAV / 3600` + + # Grids for input fields + export WAVEICE_DID= + export WAVEICE_FID= + export WAVECUR_DID= + export WAVECUR_FID= + export WAVEWND_DID= + export WAVEWND_FID= + + # Selected output parameters (gridded) + export OUTPARS_WAV='{doc.output_settings.OUTPARS_WAV}' + + # Options for point output (switch on/off boundary point output) + # export DOIBP_WAV='NO' + # + # # Intake currents settings + export WAV_CUR_DT={doc.wave_settings.WAV_CUR_DT} + export WAV_CUR_HF_DT={doc.wave_settings.WAV_CUR_HF_DT} + export WAV_CUR_HF_FH={doc.wave_settings.WAV_CUR_HF_FH} + export WAV_CUR_CDO_SMOOTH="{doc.wave_settings.WAV_CUR_CDO_SMOOTH}" + + # Number of cycles to look back for restart files + export nback= + + if [ "$CDUMP" = "gdas" ] + then + WAVNCYC=4 + WAVHCYC=6 + FHMAX_WAV_CUR=48 # RTOFS forecasts only out to 8 days + elif [ $gfs_cyc -ne 0 ] + then + FHMAX_WAV_CUR=192 # RTOFS forecasts only out to 8 days + WAVHCYC=`expr 24 / $gfs_cyc` + else + WAVHCYC=0 + FHMAX_WAV_CUR=192 # RTOFS forecasts only out to 8 days + fi + export FHMAX_WAV_CUR WAVHCYC WAVNCYC + + export RSTTYPE_WAV='T' # generate second tier of restart files + export DT_1_RST_WAV=0 # time between restart files, set to DTRST=1 for a single restart file + export DT_2_RST_WAV=3024000 # restart stride for checkpointing restart + export RSTIOFF_WAV=0 # first restart file offset relative to model start + + export RUNMEM={doc.wave_settings.RUNMEM} + + if [ $RUNMEM = -1 ]; then + # No suffix added to model ID in case of deterministic run + export waveMEMB= + else + # Extract member number only + export waveMEMB=`echo $RUNMEM | grep -o '..$'` + fi + # + # Determine if wave component needs input and/or is coupled + export WW3ATMINP='CPL' + export WW3ICEINP='CPL' + export WW3CURINP='CPL' + # + if [ "$WW3ICEINP" = "YES" ]; then + export WAVICEFILE=$CDUMP.t$((cyc))z.seaice.5min.grib2 + fi + # + # Determine if input is from perturbed ensemble (T) or single input file (F) for all members + export WW3ATMIENS='F' + export WW3ICEIENS='F' + export WW3CURIENS='F' + # + + # Set resources to propagate NTASKS across cfp call + export NTASKS={doc.partition_common.resources.run_prep[0].mpi_ranks} + # + # # Path to HOME Directory + export CODEwave="{doc.places.HOMEgfs}/sorc/fv3_coupled.fd/WW3" + export EXECwave="{doc.places.HOMEgfs}/exec" + export FIXwave="{doc.places.FIXwave}" + export PARMwave="{doc.places.HOMEgfs}/parm/wave" + export USHwave="{doc.places.HOMEgfs}/ush" + export EXECcode="{doc.places.HOMEgfs}/sorc/fv3_coupled.fd/WW3/exec" + # + export MP_PULSE=0 + + export wavelog=$COMOUTwave/wave.log + + # # Set wave model ID tag to include member number + # # if ensemble; waveMEMB var empty in deterministic + export WAV_MOD_TAG=$COMPONENTwave$waveMEMB + # + echo "END: config.wave" diff --git a/workflow/config/waveprep.yaml b/workflow/config/waveprep.yaml new file mode 100644 index 0000000000..ea4ebd6ec2 --- /dev/null +++ b/workflow/config/waveprep.yaml @@ -0,0 +1,33 @@ +config_waveprep: + filename: config.waveprep + content: !expand | + #!/bin/ksh -x + # + # This file is automatically generated from the YAML-based system + # in ecf/ecfutils/. Any changes will be overwritten if + # setup_case.sh is rerun. + + ########## config.base ########## + # Common to all steps + + echo "BEGIN: config.waveprep" + + export sigMODE="{doc.wave_settings.sigMODE}" + + export HOUR_INC=3 # This value should match with the one used in + export GOFILETYPE=1 + export POFILETYPE=1 + + export FUNIPNT="T" + # Unified output server type (see ww3_multi.inp in WW3 repo) + export IOSRV="1" + # # Flag for dedicated output process for unified points + export FPNTPROC="T" + # # Flag for grids sharing dedicated output processes + export FGRDPROC="F" + # Flag for masking computation in two-way nesting + export FLAGMASKCOMP="F" + # Flag for masking at printout time. + export FLAGMASKOUT="F" + # + echo "END: config.waveprep" diff --git a/workflow/defaults/case.yaml b/workflow/defaults/case.yaml index e06cedfe0c..13c3abca66 100644 --- a/workflow/defaults/case.yaml +++ b/workflow/defaults/case.yaml @@ -17,6 +17,13 @@ ice_settings: !Immediate - !calc doc.ice_defaults - !calc doc.case.get('ice_settings',{}) +wave_settings: !Immediate + - !MergeMapping + - CDUMP: gfs + Template: *wave_settings_template + - !calc doc.wave_defaults + - !calc doc.case.get('wave_settings',{}) + schedvar: !Immediate - !MergeMapping - !calc doc.case.get('schedvar',{}) @@ -51,6 +58,7 @@ output_settings: !Immediate - !MergeMapping - Template: *output_settings_template - !calc doc.output_settings_defaults + - !calc doc.wave_output_settings_defaults - !calc doc.case.get('output_settings',{}) - !calc doc.get('user_output_settings',{}) diff --git a/workflow/defaults/default_resources.yaml b/workflow/defaults/default_resources.yaml index 68712908fd..1efe9d0cc0 100644 --- a/workflow/defaults/default_resources.yaml +++ b/workflow/defaults/default_resources.yaml @@ -306,7 +306,7 @@ default_resources: &default_resources run_coupled_fcst: !JobRequest - batch_memory: "1024M" - mpi_ranks: !icalc doc.fv3_gfs_settings.ATMPETS+doc.fv3_gfs_settings.OCNPETS+doc.fv3_gfs_settings.ICEPETS + mpi_ranks: !icalc doc.fv3_gfs_settings.ATMPETS+doc.ocn_settings.OCNPETS+doc.ice_settings.ICEPETS+doc.wave_settings.WAVPETS max_ppn: !icalc doc.gfs_resource_table.gfsfcst_ppn OMP_NUM_THREADS: !icalc doc.fv3_gfs_settings.layout.nth walltime: !icalc doc.gfs_resource_table.coupfcst_wall @@ -314,7 +314,7 @@ default_resources: &default_resources run_coupled_medcold: !JobRequest - batch_memory: "1024M" - mpi_ranks: !icalc doc.fv3_gfs_settings.ATMPETS+doc.fv3_gfs_settings.OCNPETS+doc.fv3_gfs_settings.ICEPETS + mpi_ranks: !icalc doc.fv3_gfs_settings.ATMPETS+doc.ocn_settings.OCNPETS+doc.ice_settings.ICEPETS+doc.wave_settings.WAVPETS max_ppn: !icalc doc.gfs_resource_table.gfsfcst_ppn OMP_NUM_THREADS: !icalc doc.fv3_gfs_settings.layout.nth walltime: !icalc doc.gfs_resource_table.coupfcst_medcold_wall diff --git a/workflow/defaults/fv3_enkf.yaml b/workflow/defaults/fv3_enkf.yaml index 3ea1137725..4e7e5bfd3a 100644 --- a/workflow/defaults/fv3_enkf.yaml +++ b/workflow/defaults/fv3_enkf.yaml @@ -2,7 +2,7 @@ fv3_enkf_defaults: &fv3_enkf_defaults CDUMP: gdas CASE: C192 LEVS: 65 - CCPP_SUITE: GFS + CCPP_SUITE: IPD FHCYC: 1 FHCYC_GDAS: 1 FHCYC_GFS: 24 diff --git a/workflow/defaults/fv3_gdas.yaml b/workflow/defaults/fv3_gdas.yaml index 586d5ab50c..1766a7313e 100644 --- a/workflow/defaults/fv3_gdas.yaml +++ b/workflow/defaults/fv3_gdas.yaml @@ -1,7 +1,7 @@ fv3_gdas_defaults: &fv3_gdas_defaults CDUMP: gdas CASE: C384 - CCPP_SUITE: GFS + CCPP_SUITE: IPD LEVS: 65 FHCYC: 24 FHCYC_GDAS: 1 diff --git a/workflow/defaults/fv3_gfs.yaml b/workflow/defaults/fv3_gfs.yaml index 723c5f787e..a9923e3279 100644 --- a/workflow/defaults/fv3_gfs.yaml +++ b/workflow/defaults/fv3_gfs.yaml @@ -2,7 +2,8 @@ fv3_gfs_defaults: &fv3_gfs_defaults CDUMP: gfs CASE: C384 LEVS: 65 - CCPP_SUITE: GFS + CCPP_SUITE: IPD + CPL_ATMIC: CFSR FHCYC: 24 FHCYC_GDAS: 1 FHCYC_GFS: 24 @@ -160,5 +161,3 @@ fv3_gfs_defaults: &fv3_gfs_defaults - when: !calc QUILTING do: !calc layout.x * layout.y * 6 + layout.WGRP * layout.WGRP_NTASKS - otherwise: !calc layout.x * layout.y * 6 - OCNPETS: 120 - ICEPETS: 48 diff --git a/workflow/defaults/ice.yaml b/workflow/defaults/ice.yaml index f885d70ce8..2feccddf20 100644 --- a/workflow/defaults/ice.yaml +++ b/workflow/defaults/ice.yaml @@ -2,4 +2,6 @@ ice_defaults: &ice_defaults MOD: cice5 + ICERES: "025" + CPL_ICEIC: CPC ICEPETS: 48 diff --git a/workflow/defaults/ocn.yaml b/workflow/defaults/ocn.yaml index 5ccb468d52..f77016c971 100644 --- a/workflow/defaults/ocn.yaml +++ b/workflow/defaults/ocn.yaml @@ -2,4 +2,6 @@ ocn_defaults: &ocn_defaults MOD: mom6 + CPL_OCNIC: CPC3Dvar + OCNRES: "025" OCNPETS: 120 diff --git a/workflow/defaults/output_settings.yaml b/workflow/defaults/output_settings.yaml index fad2f14d3f..83b3d58d31 100644 --- a/workflow/defaults/output_settings.yaml +++ b/workflow/defaults/output_settings.yaml @@ -17,6 +17,7 @@ output_settings_defaults: &output_settings_defaults FHMAX_HF_GFS: 0 FHOUT_HF_GFS: 1 + OCN_INTERVAL: 120 NCO_NAMING_CONV: YES @@ -36,3 +37,16 @@ output_settings_defaults: &output_settings_defaults tools.uniq(sorted( tools.seq(0,84,3) + tools.seq(90,240,6))) + +wave_output_settings_defaults: &wave_output_settings_defaults + FHMIN_WAV: 0 + FHMAX_WAV: 9 + FHMAX_WAV_GFS: 384 + FHOUT_WAV: 3 + + FHMAX_HF_WAV: 120 + FHOUT_HF_WAV: 3 + + FHMAX_WAV_CUR: 48 + + OUTPARS_WAV: "WND CUR ICE HS T01 T02 DIR FP DP PHS PTP PDIR CHAR" diff --git a/workflow/defaults/places.yaml b/workflow/defaults/places.yaml index 36fc127668..819a54583a 100644 --- a/workflow/defaults/places.yaml +++ b/workflow/defaults/places.yaml @@ -10,6 +10,8 @@ default_places: &default_places NWPROD: !calc doc.platform.NWPROD DMPDIR: !calc doc.platform.DMPDIR RTMFIX: !calc doc.platform.RTMFIX + + FIXwave: !expand "{HOMEgfs}/fix/fix_wav" EXPDIR: !expand "{doc.platform.EXPROOT}/{doc.names.experiment}" ROTDIR: !expand "{doc.platform.COMROOT}/{doc.names.experiment}" diff --git a/workflow/defaults/settings.yaml b/workflow/defaults/settings.yaml index 8fe93d1eed..21d0df2a62 100644 --- a/workflow/defaults/settings.yaml +++ b/workflow/defaults/settings.yaml @@ -44,6 +44,8 @@ default_settings: &default_settings cplflx: .false. cplice: !calc cplflx + cplwav: .false. + cplwav2atm: .false. cpl: .false. mom6ic_prepared: .false. # If set True, User need to manually copy mom6ic files into ROTIDR diff --git a/workflow/defaults/wave.yaml b/workflow/defaults/wave.yaml new file mode 100644 index 0000000000..063276cfa6 --- /dev/null +++ b/workflow/defaults/wave.yaml @@ -0,0 +1,19 @@ +wave_defaults: &wave_defaults + MOD: ww3 + CPL_WAVIC: CFSR + WAVPETS: !FirstTrue + - when: !calc doc.settings.cplwav + do: 24 + - otherwise: 0 + COMPONENTRSTwave: gdaswave + WAVHINDH: 0 + RUNMEM: -1 + sigMODE: prep + waveGRD: 'gwes_30m' + waveGRDN: '1 2 3' + waveGRDG: '10 20 30' + USE_WAV_RMP: 'NO' + WAV_CUR_DT: 3 + WAV_CUR_HF_DT: 1 + WAV_CUR_HF_FH: 72 + WAV_CUR_CDO_SMOOTH: NO diff --git a/workflow/layout/free_forecast_gfs.yaml b/workflow/layout/free_forecast_gfs.yaml index 51b4413122..90b0108553 100644 --- a/workflow/layout/free_forecast_gfs.yaml +++ b/workflow/layout/free_forecast_gfs.yaml @@ -52,11 +52,29 @@ suite: !Cycle RUN: 'gfs' ecflow_def: "edit RUN 'gfs'" + prep: !Family + Disable: !calc doc.settings.cplwav=='.false.' + + waveinit: !Task + <<: *exclusive_task_template + Trigger: !Depend up.up.ics.jgfs_emc_mom6ic + resources: !calc partition.resources.run_prep + J_JOB: rocoto/waveinit.sh + + waveprep: !Task + <<: *exclusive_task_template + Trigger: !Depend waveinit + resources: !calc partition.resources.run_prep + J_JOB: rocoto/waveprep.sh + forecast: !Family medcold: !Task <<: *exclusive_task_template Disable: !calc doc.settings.cplflx=='.false.' - Trigger: !Depend up.up.ics.jgfs_emc_mom6ic + Trigger: !FirstTrue + - when: !calc doc.settings.cplwav=='.true.' + do: !Depend up.prep + - otherwise: !Depend up.up.ics.jgfs_emc_mom6ic resources_remap: !JobRequest [ { <<: *remap_resource_template } ] resources: !calc partition.resources.run_coupled_medcold J_JOB: JGLOBAL_FORECAST_MEDCOLD @@ -152,7 +170,7 @@ suite: !Cycle resources: !calc partition.resources.run_one_node_downstream Name: !expand p_{dimval.fhr:03d} FHR: !expand "{dimval.fhr:03d}" - OCN_FHRLST: !calc "tools.seq(dimval.fhr, dimval.fhr+doc.output_settings.OCN_INTERVAL, 6)" + OCN_FHRLST: !calc "tools.seq(dimval.fhr, dimval.fhr+doc.output_settings.OCN_INTERVAL-6, 6)" J_JOB: rocoto/ocnpost.sh Trigger: !Depend up.forecast ecflow_def: !expand "edit FHR '{FHR}'" @@ -164,6 +182,28 @@ suite: !Cycle export OCN_FHRLST="{OCN_FHRLST}"; $HOMEgfs/jobs/{J_JOB} #endfamily ocnpost + wavepost: !TaskArray + RUN: !calc up.RUN + Disable: !calc doc.settings.cplwav=='.false.' + Dimensions: + fhr: !calc doc.output_settings.ocnpost_hours + jgfs_wavepost_fhr_el: !TaskElement + <<: *exclusive_task_template + Foreach: [ fhr ] + resources: !calc partition.resources.run_one_node_downstream + Name: !expand p_{dimval.fhr:03d} + FHR: !expand "{dimval.fhr:03d}" + OCN_FHRLST: !calc "tools.seq(dimval.fhr, dimval.fhr+doc.output_settings.OCN_INTERVAL, 6)" + J_JOB: rocoto/wavepostsbs.sh + Trigger: !Depend up.forecast + ecflow_def: !expand "edit FHR '{FHR}'" + ecflow_command: !expand | + export post_times=%FHR% FHRLST=%FHR% FHRGRP=%FHR% + $HOMEgfs/jobs/{J_JOB} + rocoto_command: !expand >- + {rocoto_load_modules} ; + export OCN_FHRLST="{OCN_FHRLST}"; $HOMEgfs/jobs/{J_JOB} + jgfs_emc_vrfy: !Task <<: *exclusive_task_template Trigger: !Depend post diff --git a/workflow/schema/ice.yaml b/workflow/schema/ice.yaml index afbcf32552..fdb6f5cfe2 100644 --- a/workflow/schema/ice.yaml +++ b/workflow/schema/ice.yaml @@ -4,3 +4,9 @@ ice_settings_template: !Template &ice_settings_template type: string allowed: [ cice5, cice6 ] description: "model selection for ice" + ICERES: + type: string + allowed: [ '025', '050' ] + CPL_ICEIC: + type: string + allowed: [ 'CFSR', 'CPC' ] diff --git a/workflow/schema/ocn.yaml b/workflow/schema/ocn.yaml index a349d09fd9..b9830e848b 100644 --- a/workflow/schema/ocn.yaml +++ b/workflow/schema/ocn.yaml @@ -7,3 +7,9 @@ ocn_settings_template: !Template &ocn_settings_template type: string allowed: [ mom6, hycom ] description: "model selection for ocean" + OCNRES: + type: string + allowed: [ '025', '050' ] + CPL_OCNIC: + type: string + allowed: [ 'CFSR', 'CPC3Dvar' ] diff --git a/workflow/schema/wave.yaml b/workflow/schema/wave.yaml new file mode 100644 index 0000000000..c6a4026c26 --- /dev/null +++ b/workflow/schema/wave.yaml @@ -0,0 +1,12 @@ +# fv3_settings_template - sets the namelist values for the fv3 +# forecast. See the physcs and model documentation for full +# information on these variables. + +wave_settings_template: !Template &wave_settings_template + MOD: + type: string + allowed: [ ww3, swan ] + description: "model selection for wave" + CPL_WAVIC: + type: string + allowed: [ 'CFSR' ]