- merge 23Jun20 port2orion branch, ed6b279, into gsd-ccpp-jet

- updated INFO file Squashed commit of the following: commit be717ce Merge: df8dc52 99277ae Author: Kate.Friedman <Kate.Friedman@noaa.gov> Date: Tue Jun 23 16:20:51 2020 +0000 Issue NOAA-EMC#5 - Sync merge branch 'feature/gfsv16b' into port2orion * feature/gfsv16b: Update README.md modified: Externals.cfg and sorc/checkout.sh to check out model tag GFS.v16.0.7 and UPP tag upp_gfsv16_release.v1.0.9. Changes include: Update version of upp in Externals.cfg to be consistent with sorc/checkout.sh commit 99277ae Merge: 14dd3c9 4f8d5a5 Author: fanglin.yang <fanglin.yang@noaa.gov> Date: Tue Jun 23 16:05:16 2020 +0000 Merge branch 'feature/gfsv16b' of https://github.com/NOAA-EMC/global-workflow into feature/gfsv16b commit 4f8d5a5 Author: Kate Friedman <kate.friedman@noaa.gov> Date: Tue Jun 23 11:57:26 2020 -0400 Update README.md Remove use/mention of manage_externals until checkout.sh is retired. commit 14dd3c9 Author: fanglin.yang <fanglin.yang@noaa.gov> Date: Tue Jun 23 15:53:11 2020 +0000 modified: Externals.cfg and sorc/checkout.sh to check out model tag GFS.v16.0.7 and UPP tag upp_gfsv16_release.v1.0.9. Changes include: 1) Inline POST Issues NOAA-EMC#136 and $142 * Update ceiling height calculation for global FV3. * add low,middle,high instantaneous cloud fraction * add radar reflectivity at model layers 1 and 2 , and radar reflectivities at 1 and 4-km height. * fix a bug in initializing DBZI * output mixed layer CAPE/CIN * remove simulated GOES-12 brightness temperature. * change the names of time averaged low/mid/high cloud fractions in grib2 files from "TCDC" to "LCDC/MCDC/HCDC", respectively. 2) Model Issue NOAA-EMC#152 * update in-line post control files * upgrade post library to 8.0.9 for hera and wcoss_dell_p3 commit dd76002 Merge: df89cc8 dd599ea Author: Kate Friedman <kate.friedman@noaa.gov> Date: Tue Jun 23 11:34:40 2020 -0400 Merge pull request NOAA-EMC#91 from christopherwharrop-noaa/feature/fix_externals Update version of upp in Externals.cfg to be consistent with sorc/che… commit dd599ea Author: Christopher Harrop <Christopher.W.Harrop@noaa.gov> Date: Tue Jun 23 15:25:08 2020 +0000 Update version of upp in Externals.cfg to be consistent with sorc/checkout.sh commit df8dc52 Merge: ff46607 df89cc8 Author: Kate.Friedman <Kate.Friedman@noaa.gov> Date: Mon Jun 22 14:41:17 2020 +0000 Issue NOAA-EMC#5 - Sync merge branch 'feature/gfsv16b' into port2orion * feature/gfsv16b: Update config.resources Issue NOAA-EMC#85: update DA checkout to point at release/gfsda.v16.0.0 Issue NOAA-EMC#85: (1) rename "chgresfcst" as "echgres", (2) add chgres variables to env Issue NOAA-EMC#85: add cfp option to chgresfcst; enable threads with chgresfcst add checkout.sh to test on Dell Commit changes from debugging addition of chgresfcst on hera First draft to add chgresfcst to rocoto workflow commit df89cc8 Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com> Date: Fri Jun 19 18:40:26 2020 -0400 Update config.resources Generalize setting of echgres threads to be maximum permitted on given platform instead of explicitly setting echgres threads on every platform. commit ff8cd28 Merge: 1dd83b8 965ff42 Author: russ.treadon <Russ.Treadon@noaa.gov> Date: Thu Jun 18 20:07:59 2020 +0000 Issue #1: merge branch 'feature/chgresfcst' at 965ff42 into feature/gfsv16b commit 965ff42 Author: russ.treadon <Russ.Treadon@noaa.gov> Date: Thu Jun 18 20:02:23 2020 +0000 Issue NOAA-EMC#85: update DA checkout to point at release/gfsda.v16.0.0 commit ff46607 Merge: ed6b279 1dd83b8 Author: Kate.Friedman <Kate.Friedman@noaa.gov> Date: Wed Jun 17 17:23:44 2020 +0000 Issue NOAA-EMC#5 - Sync merge branch 'feature/gfsv16b' into port2orion * feature/gfsv16b: modified: run_gfsmos_master.sh.dell to still set range=both as the default for running the real-time parallel update checkout.sh to switch back to post version upp_gfsv16_release.v1.0.8. 1.0.9 still has issues. Issue #1 - sync Externals.cfg with checkout.sh update for FV3 GFSv16.0.6 tag modified: scripts/run_gfsmos_master.sh.dell to set default verification type o short. modified: sorc/checkout.sh updated to model tag GFS.v16.0.6 to fix contrib issue on HERA Issue #1 - update Externals.cfg to match updates to checkout.sh Update exwave_post_sbs.sh Adapting wavepostsbs for running on Hera commit 71d7141 Author: russ.treadon <Russ.Treadon@noaa.gov> Date: Tue Jun 16 20:11:33 2020 +0000 Issue NOAA-EMC#85: (1) rename "chgresfcst" as "echgres", (2) add chgres variables to env commit 1dd83b8 Author: fanglin.yang <fanglin.yang@noaa.gov> Date: Tue Jun 16 16:07:45 2020 +0000 modified: run_gfsmos_master.sh.dell to still set range=both as the default for running the real-time parallel commit eb0e3b4 Author: fanglin.yang <fanglin.yang@noaa.gov> Date: Tue Jun 16 14:43:29 2020 +0000 update checkout.sh to switch back to post version upp_gfsv16_release.v1.0.8. 1.0.9 still has issues. commit dfc76f0 Author: Kate.Friedman <Kate.Friedman@noaa.gov> Date: Tue Jun 16 13:32:01 2020 +0000 Issue #1 - sync Externals.cfg with checkout.sh update for FV3 GFSv16.0.6 tag commit 7078bb5 Author: fanglin.yang <fanglin.yang@noaa.gov> Date: Tue Jun 16 13:22:43 2020 +0000 modified: scripts/run_gfsmos_master.sh.dell to set default verification type o short. modified: sorc/checkout.sh updated to model tag GFS.v16.0.6 to fix contrib issue on HERA commit 15d5bed Author: russ.treadon <Russ.Treadon@noaa.gov> Date: Mon Jun 15 20:47:47 2020 +0000 Issue NOAA-EMC#85: add cfp option to chgresfcst; enable threads with chgresfcst commit d878269 Author: Kate.Friedman <Kate.Friedman@noaa.gov> Date: Mon Jun 15 16:00:35 2020 +0000 Issue #1 - update Externals.cfg to match updates to checkout.sh commit a2bd621 Merge: 22b735d 295cd05 Author: Kate Friedman <kate.friedman@noaa.gov> Date: Mon Jun 15 11:57:14 2020 -0400 Merge pull request NOAA-EMC#84 from NOAA-EMC/feature/gfsv16b_herawavepost Adapting wavepostsbs for running on Hera commit 295cd05 Author: Jose-Henrique Alves <47567389+ajhenrique@users.noreply.github.com> Date: Mon Jun 15 11:53:13 2020 -0400 Update exwave_post_sbs.sh Removing obsolete nm variable entries commit 58d1139 Author: henrique.alves <Henrique.Alves@noaa.gov> Date: Thu Jun 11 02:44:41 2020 +0000 Adapting wavepostsbs for running on Hera commit c330e60 Author: CoryMartin-NOAA <cory.r.martin@noaa.gov> Date: Wed Jun 10 21:17:00 2020 +0000 add checkout.sh to test on Dell commit 201609b Author: CoryMartin-NOAA <cory.r.martin@noaa.gov> Date: Wed Jun 10 20:51:51 2020 +0000 Commit changes from debugging addition of chgresfcst on hera commit fc3066c Author: CoryMartin-NOAA <cory.r.martin@noaa.gov> Date: Wed Jun 10 14:19:07 2020 +0000 First draft to add chgresfcst to rocoto workflow
jkhender · Jul 1, 2020 · ea1aa3b · ea1aa3b
1 parent 3f96848
commit ea1aa3b
Show file tree

Hide file tree

Showing 24 changed files with 203 additions and 121 deletions.
diff --git a/Externals.cfg b/Externals.cfg
@@ -1,8 +1,8 @@
 # External sub-modules of global-workflow
 
 #JKH[FV3GFS]
-#JKHtag = orion_gfs.v16
-#JKHlocal_path = sorc/fv3gfs_emc.fd
+#JKHtag = GFS.v16.0.7
+#JKHlocal_path = sorc/fv3gfs.fd
 #JKHrepo_url = https://github.com/ufs-community/ufs-weather-model.git
 #JKHprotocol = git
 #JKHrequired = True
@@ -15,7 +15,7 @@ protocol = git
 required = True
 
 [GSI]
-tag = release/gfsda.v16.0.0
+branch = release/gfsda.v16.0.0
 local_path = sorc/gsi.fd
 repo_url = https://github.com/NOAA-EMC/GSI.git
 protocol = git
@@ -44,7 +44,7 @@ protocol = git
 required = True
 
 [EMC_verif-global]
-tag = verif_global_v1.8.1
+tag = verif_global_v1.8.0
 local_path = sorc/verif-global.fd
 repo_url = https://github.com/NOAA-EMC/EMC_verif-global.git
 protocol = git

diff --git a/INFO b/INFO
@@ -1,2 +1,5 @@
+01 July 2020
+    - merge 23Jun20 portorion branch, ed6b279, into gsd-ccpp-jet
+    - additional updates for running on jet
 17 June 2020
     - created new branch, gsd-ccpp-jet, from 15Jun20 port2orion branch, ed6b279
diff --git a/README.md b/README.md
@@ -6,7 +6,6 @@ The global-workflow depends on the following prerequisities to be available on t
 * workload management platform / scheduler - LSF or SLURM
 * workflow manager - ROCOTO (https://github.com/christopherwharrop/rocoto)
 * modules - NCEPLIBS (various), esmf v8.0.0bs48, hdf5, intel/ips v18, impi v18, wgrib2, netcdf v4.7.0, hpss, gempak (see module files under /modulefiles for additional details)
-* manage_externals - A utility from ESMCI to checkout external dependencies. Manage_externals can be obtained at the following address and should be in the users PATH: https://github.com/ESMCI/manage_externals
 
 The global-workflow current supports the following machines:
 
@@ -18,20 +17,11 @@ The global-workflow current supports the following machines:
 
 ### 1. Check out components
 
-The global-workflow uses the manage_externals utility to handle checking out its components. The manic-v1.1.8 manage_externals tag is supported. The manage_externals utility will be replacing the current checkout.sh script.
-
-Run manage_externals (checkout_externals) while at top of clone:
-
+While in /sorc folder:
 ```
-$ checkout_externals -e Externals.cfg
+$ sh checkout.sh
 ```
 
-If checkout_externals is not in your $PATH then use full path to it:
-
-* WCOSS-Dell: /gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
-* WCOSS-Cray: /gpfs/hps3/emc/global/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
-* Hera: /scratch1/NCEPDEV/global/glopara/git/manage_externals/manic-v1.1.8/checkout_externals
-
 ### 2. Build components
 
 While in /sorc folder:

diff --git a/env/HERA.env b/env/HERA.env
@@ -74,9 +74,15 @@ elif [ $step = "anal" ]; then
 
     export NTHREADS_GAUSFCANL=1
     npe_gausfcanl=${npe_gausfcanl:-1}
-
     export APRUN_GAUSFCANL="$launcher -n $npe_gausfcanl"
 
+    export NTHREADS_CHGRES=${nth_echgres:-1}
+    [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max
+    export APRUN_CHGRES=""
+
+    export USE_CFP=${USE_CFP:-"YES"}
+    export APRUNCFP="$launcher"
+
 elif [ $step = "gldas" ]; then
 
     nth_max=$(($npe_node_max / $npe_node_gldas))

diff --git a/env/JET.env b/env/JET.env
@@ -80,6 +80,10 @@ elif [ $step = "anal" ]; then
 
     export APRUN_GAUSFCANL="$launcher -n $npe_gausfcanl"
 
+    export NTHREADS_CHGRES=${nth_echgres:-1}
+    [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max
+    export APRUN_CHGRES=""
+
 elif [ $step = "gldas" ]; then
 
     nth_max=$(($npe_node_max / $npe_node_gldas))

diff --git a/env/WCOSS_C.env b/env/WCOSS_C.env
@@ -66,6 +66,10 @@ elif [ $step = "anal" ]; then
     [[ $npe_node_gausfcanl -gt $npe_gausfcanl ]] && npe_node_gausfcanl=$npe_gausfcanl
     export APRUN_GAUSFCANL="$launcher -j 1 -n $npe_gausfcanl -N $npe_node_gausfcanl -d $NTHREADS_GAUSFCANL -cc depth"
 
+    export NTHREADS_CHGRES=${nth_echgres:-1}
+    [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max
+    export APRUN_CHGRES="$launcher -j 1 -n 1 -N 1 -d $NTHREADS_CHGRES -cc depth"
+
 elif [ $step = "gldas" ]; then
 
     nth_max=$(($npe_node_max / $npe_node_gldas))

diff --git a/env/WCOSS_DELL_P3.env b/env/WCOSS_DELL_P3.env
@@ -66,6 +66,10 @@ elif [ $step = "anal" ]; then
     npe_gausfcanl=${npe_gausfcanl:-1}
     export APRUN_GAUSFCANL="$launcher $npe_gausfcanl"
 
+    export NTHREADS_CHGRES=${nth_echgres:-14}
+    [[ $NTHREADS_CHGRES -gt $npe_node_max ]] && export NTHREADS_CHGRES=$npe_node_max
+    export APRUN_CHGRES=""
+
     if [ ${USE_CFP:-"NO"} = "YES" ]; then
         export APRUNCFP="$launcher \$ncmd $mpmd"
     fi

diff --git a/jobs/rocoto/echgres.sh b/jobs/rocoto/echgres.sh
@@ -0,0 +1,13 @@
+#!/bin/ksh -x
+
+###############################################################
+# Source FV3GFS workflow modules
+. $HOMEgfs/ush/load_fv3gfs_modules.sh
+status=$?
+[[ $status -ne 0 ]] && exit $status
+
+###############################################################
+# Execute the JJOB
+$HOMEgfs/jobs/JGDAS_ENKF_CHGRES
+status=$?
+exit $status
diff --git a/parm/config/config.echgres b/parm/config/config.echgres
@@ -0,0 +1,13 @@
+#!/bin/ksh -x
+
+########## config.echgres ##########
+# regrid full-res forecast for use in ensemble-res analysis generation
+
+echo "BEGIN: config.echgres"
+
+# Get task specific resources
+. $EXPDIR/config.resources echgres
+
+export CHGRESFCSTSH=$HOMEgfs/scripts/exglobal_enkf_chgres_fv3gfs.sh.ecf
+
+echo "END: config.echgres"
diff --git a/parm/config/config.resources b/parm/config/config.resources
@@ -8,7 +8,7 @@ if [ $# -ne 1 ]; then
 
     echo "Must specify an input task argument to set resource variables!"
     echo "argument can be any one of the following:"
-    echo "anal analcalc analdiag gldas fcst post vrfy metp arch"
+    echo "anal analcalc analdiag gldas fcst post vrfy metp arch echgres"
     echo "eobs ediag eomg eupd ecen esfc efcs epos earc"
     echo "waveinit waveprep wavepostsbs wavegempaksbs waveawipssbs"
     echo "wavepost waveawips wavestat"
@@ -202,6 +202,13 @@ elif [ $step = "metp" ]; then
             export memory_metp="16384M"
     fi
 
+elif [ $step = "echgres" ]; then
+
+    export wtime_echgres="01:00:00"
+    export npe_echgres=3
+    export nth_echgres=$npe_node_max
+    export npe_node_echgres=1
+
 elif [ $step = "arch" -o $step = "earc" -o $step = "getic" ]; then
 
     eval "export wtime_$step='06:00:00'"

diff --git a/scripts/exwave_init.sh b/scripts/exwave_init.sh
@@ -14,6 +14,7 @@
 # Script history log:
 # 2019-05-06  J-Henrique Alves First Version.
 # 2019-11-02  J-Henrique Alves Ported to global-workflow.
+# 2020-06-10  J-Henrique Alves Ported to R&D machine Hera
 #
 # $Id$
 #