Skip to content

Commit

Permalink
merge 26Mar20 develop branch into gmtb_ccpp_hera
Browse files Browse the repository at this point in the history 
Squashed commit of the following:

commit 0e0dc87
Merge: b133700 95a6343
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Thu Mar 26 10:54:44 2020 -0400

    Merge pull request NOAA-EMC#42 from NOAA-EMC/hotfix/viewer

    Issue NOAA-EMC#41 - Update PRODUTIL paths for WCOSS in viewer

commit 95a6343
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Thu Mar 26 14:43:44 2020 +0000

    Issue NOAA-EMC#41 - Update PRODUTIL paths for WCOSS in viewer

commit b133700
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Thu Mar 26 13:01:38 2020 +0000

    Issue NOAA-EMC#21 - correct syntax for machine if-blocks in setup_expt scripts

commit f4ee509
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Mar 25 20:52:49 2020 +0000

    Issue NOAA-EMC#21 - remove unneeded logic from partition check

commit 4acc80f
Merge: accb6f4 fb1c79f
Author: Hang-Lei-NOAA <44901908+Hang-Lei-NOAA@users.noreply.github.com>
Date:   Wed Mar 25 15:22:51 2020 -0400

    Merge pull request NOAA-EMC#40 from NOAA-EMC/port2wcoss3p5

    Issue NOAA-EMC#21 - Add support for WCOSS phase 3.5

commit fb1c79f
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Mar 25 16:06:54 2020 +0000

    Issue NOAA-EMC#21 - updated WCOSS phase 3.5 queues

commit 08a5354
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Mar 25 14:58:14 2020 +0000

    Issue NOAA-EMC#21 - add phase 3.5 support

    - add partition option to setup scripts
    - remove machine if-blocks from config.base and add variable population to setup_expt*py scripts
    - add phase 3.5 ppn value to WCOSS_DELL_P3 env and config.resources files

commit accb6f4
Merge: 057b2a8 8b51b56
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Mon Mar 23 14:01:30 2020 -0400

    Merge pull request NOAA-EMC#39 from NOAA-EMC/feature/verif-tag

    Issue NOAA-EMC#38 - update EMC_verif-global pointer from VLab to GitHub

commit 8b51b56
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Mon Mar 23 17:40:18 2020 +0000

    Issue NOAA-EMC#38 - update EMC_verif-global pointer from VLab to GitHub

commit 057b2a8
Merge: 0377d20 622167d
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Wed Mar 11 12:00:58 2020 -0400

    Merge pull request NOAA-EMC#29 from NOAA-EMC/feature/manage_externals

    Issue NOAA-EMC#3 - Introduce manage_externals as replacement for checkout.sh

commit 622167d
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 18:20:31 2020 +0000

    Issue NOAA-EMC#3 - added explicit config flag example for checkout_externals in README and blurb about this replacing checkout.sh

commit e83b90d
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 17:00:15 2020 +0000

    Issue NOAA-EMC#3 - remove prod_util and grib_util sections from build_all.sh, removed elsewhere already

commit 8699b46
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 16:30:54 2020 +0000

    Issue NOAA-EMC#3 - updated README with new manic version

commit e602cd3
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 16:27:57 2020 +0000

    Issue NOAA-EMC#3 - updated link_fv3gfs.sh to adjust wafs links

commit 830c73f
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 15:21:45 2020 +0000

    Issue NOAA-EMC#3 - update EMC_verif-global tag in Externals.cfg after sync with develop

commit 40084e6
Merge: f662fff 0377d20
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Mar 6 15:18:52 2020 +0000

    Issue NOAA-EMC#3 - Merge branch 'develop' into feature/manage_externals

commit 0377d20
Merge: 1b359db 2552467
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Thu Mar 5 08:43:16 2020 -0500

    Merge pull request NOAA-EMC#28 from NOAA-EMC/feature/metplus2

    Issue NOAA-EMC#8 - add switch for MET+ jobs

commit 2552467
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Thu Mar 5 13:31:02 2020 +0000

    Issue NOAA-EMC#8 - add switch for MET+ jobs

commit 1b359db
Merge: 0359d34 31bb7d3
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Wed Mar 4 10:19:36 2020 -0500

    Merge pull request NOAA-EMC#26 from NOAA-EMC/feature/metplus

    Feature/metplus - refactored MET+ jobs to resolve timing issues

commit 31bb7d3
Merge: eb73e52 0359d34
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Wed Feb 19 15:24:42 2020 +0000

    Merge branch 'develop' into feature/metplus

commit f662fff
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Feb 14 15:57:05 2020 +0000

    Issue NOAA-EMC#3 - Updated README with new manic tag v1.1.7

commit e3196a8
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Thu Feb 13 15:59:06 2020 -0500

    Update README.md

commit e46b175
Merge: 4bd0e20 0359d34
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Thu Feb 13 20:38:04 2020 +0000

    Issue NOAA-EMC#3 - sync merge with develop branch

commit 0359d34
Merge: 1d9a1f0 bd00cb9
Author: Hang-Lei-NOAA <44901908+Hang-Lei-NOAA@users.noreply.github.com>
Date:   Thu Feb 13 09:53:59 2020 -0500

    Merge pull request NOAA-EMC#19 from NOAA-EMC/feature/remove_theia

    Feature/remove theia

commit eb73e52
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Fri Feb 7 14:04:37 2020 +0000

    Update EMC_verif-global checkout to verif_global_v1.5.0

commit bd00cb9
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Fri Feb 7 13:41:05 2020 +0000

    Issue NOAA-EMC#4 - bug fix in getic.sh for v15

commit 1c85197
Merge: 67dae40 1d9a1f0
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Fri Feb 7 13:27:18 2020 +0000

    Issue # 4 - Sync merge branch 'develop' into feature/remove_theia

commit 1d9a1f0
Merge: 3ed9267 bdbecaa
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Fri Feb 7 08:11:21 2020 -0500

    Merge pull request NOAA-EMC#18 from lgannoaa/exception_handling

    Exception handling

commit 4bd0e20
Author: Kate Friedman <kate.friedman@noaa.gov>
Date:   Thu Feb 6 11:55:31 2020 -0500

    Update README.md

commit d9ea1ac
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Thu Feb 6 16:03:11 2020 +0000

    Issue NOAA-EMC#3 - reduce hashes down to minimum 8 characters

commit bdbecaa
Author: Lin.Gan <lin.gan@noaa.gov>
Date:   Wed Jan 29 15:15:52 2020 +0000

    Display exception handling message for individual package with location of the log file

commit b64fd5f
Merge: cf00863 3ed9267
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Tue Jan 28 15:24:37 2020 +0000

    Merge branch 'develop' into feature/manage_externals

commit cf00863
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Tue Jan 28 15:23:14 2020 +0000

    Issue NOAA-EMC#3 - remove copy of manage_externals under util and add README.md file

commit c12e879
Author: Lin.Gan <lin.gan@noaa.gov>
Date:   Mon Jan 27 19:38:19 2020 +0000

    Implement exception handling in build_all script

commit bfc7bb0
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Mon Jan 27 19:06:17 2020 +0000

    Issue NOAA-EMC#3 - initial add of manage_externals and needed Externals.cfg. Also added .gitignore file and removed scripts/files associated with no-longer-used prod_util and grid_util builds.

commit 4d5713d
Author: Lin.Gan <lin.gan@noaa.gov>
Date:   Fri Jan 24 15:51:28 2020 +0000

    Testing github commit

commit 786806f
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Fri Jan 24 15:04:16 2020 +0000

    Missed file format updates in a few places in config.metp

commit d0a3b53
Author: Lin.Gan <lin.gan@noaa.gov>
Date:   Fri Jan 24 14:30:29 2020 +0000

    init commit for exception handling branch

commit c11dfef
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Fri Jan 24 14:07:29 2020 +0000

    Update EMC_verif-global tag to verif_global_v1.4.1

commit 0ea809c
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Thu Jan 23 13:29:56 2020 +0000

    Update file format variable in config.metp of online archive files

commit c0d7179
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Tue Jan 21 16:37:16 2020 +0000

    Add updated env machine files for gfsmetp

commit 82e6907
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Tue Jan 21 16:29:23 2020 +0000

    Update config.resources for gfsmetp

commit 72e8adf
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Tue Jan 21 16:19:31 2020 +0000

    Update EMC_verif-global tag checkout to 1.4.0

commit 6872f79
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Tue Jan 21 16:14:25 2020 +0000

    Add rocoto METplus job script

commit 9c94156
Author: Mallory Row <mallory.row@noaa.gov>
Date:   Tue Jan 21 16:09:13 2020 +0000

    Changes to setup_workflow.py for gfsmetp metatask

commit 3ed9267
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Tue Jan 14 19:32:36 2020 +0000

    Issue NOAA-EMC#10 - mid-year update to bufr station list (develop)

commit 1915aa9
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Fri Jan 10 18:36:32 2020 +0000

    Issue NOAA-EMC#8 - pulled in config.metp and modifications to two setup scripts

commit 67dae40
Merge: f78eb1b 091f4ba
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Jan 8 20:15:45 2020 +0000

    Merge branch 'develop' into feature/remove_theia

commit f78eb1b
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Jan 8 20:13:08 2020 +0000

    Issue NOAA-EMC#4 - removed references to Theia and Theia scripts

commit 091f4ba
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Wed Jan 8 19:37:45 2020 +0000

    Issue NOAA-EMC#7 - missed update to gdas transfer file from GFSv15.2.5 updates

commit 3fd4bcf
Author: kate.friedman <kate.friedman@noaa.gov>
Date:   Tue Dec 17 19:12:28 2019 +0000

    GitHub Issue NOAA-EMC#2 - GFSv15.2.6 obsproc version update, earc bug fix, and tracker path update

commit 5307952
Author: Kate.Friedman <Kate.Friedman@noaa.gov>
Date:   Wed Dec 11 20:47:44 2019 +0000

    HOTFIX - VLab Issue #72346 - fix to rocoto_viewer on Hera
  • Loading branch information
@jkhender
jkhender committed Mar 31, 2020
1 parent 6397f5e commit a847acc
Show file tree
Hide file tree
Showing 113 changed files with 708 additions and 2,065 deletions.
16 changes: 16 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Ignore all compiled files
*.pyc
*.o
*.mod

# Ignore exec folder
exec/

# Ignore sorc folders from externals
sorc/logs/
sorc/fv3gfs.fd/
sorc/gfs_post.fd/
sorc/gsi.fd/
sorc/ufs_utils.fd/
sorc/gfs_wafs.fd/
sorc/verif-global.fd/
46 changes: 46 additions & 0 deletions Externals.cfg
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# External sub-modules of global-workflow

[NEMSfv3gfs]
tag = gfs.v16_PhysicsUpdate
local_path = sorc/fv3gfs.fd
repo_url = ssh://vlab.ncep.noaa.gov:29418/NEMSfv3gfs
protocol = git
required = True

[GSI]
hash = cb8f69d8
local_path = sorc/gsi.fd
repo_url = ssh://vlab.ncep.noaa.gov:29418/ProdGSI
protocol = git
required = True

[EMC_post]
hash = ba7e59b2
local_path = sorc/gfs_post.fd
repo_url = https://github.com/NOAA-EMC/EMC_post.git
protocol = git
required = True

[UFS_UTILS]
tag = v1.1.0
local_path = sorc/ufs_utils.fd
repo_url = https://github.com/NOAA-EMC/UFS_UTILS.git
protocol = git
required = True

[EMC_verif-global]
tag = verif_global_v1.5.0
local_path = sorc/verif-global.fd
repo_url = https://github.com/NOAA-EMC/EMC_verif-global.git
protocol = git
required = True

[EMC_gfs_wafs]
tag = gfs_wafs.v5.0.11
local_path = sorc/gfs_wafs.fd
repo_url = https://github.com/NOAA-EMC/EMC_gfs_wafs.git
protocol = git
required = False

[externals_description]
schema_version = 1.0.0
49 changes: 49 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# global-workflow
Global Superstructure/Workflow currently supporting the Finite-Volume on a Cubed-Sphere Global Forecast System (FV3GFS)

The global-workflow depends on the following prerequisities to be available on the system:

* workload management platform / scheduler - LSF or SLURM
* workflow manager - ROCOTO (https://github.com/christopherwharrop/rocoto)
* modules - NCEPLIBS (various), esmf v8.0.0bs48, hdf5, intel/ips v18, impi v18, wgrib2, netcdf v4.7.0, hpss, gempak (see module files under /modulefiles for additional details)
* manage_externals - A utility from ESMCI to checkout external dependencies. Manage_externals can be obtained at the following address and should be in the users PATH: https://github.com/ESMCI/manage_externals

The global-workflow current supports the following machines:

* WCOSS-Dell
* WCOSS-Cray
* Hera

## Build global-workflow:

### 1. Check out components

The global-workflow uses the manage_externals utility to handle checking out its components. The manic-v1.1.8 manage_externals tag is supported. The manage_externals utility will be replacing the current checkout.sh script.

Run manage_externals (checkout_externals) while at top of clone:

```
$ checkout_externals -e Externals.cfg
```

If checkout_externals is not in your $PATH then use full path to it:

* WCOSS-Dell: /gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
* WCOSS-Cray: /gpfs/hps3/emc/global/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
* Hera: /scratch1/NCEPDEV/global/glopara/git/manage_externals/manic-v1.1.8/checkout_externals

### 2. Build components

While in /sorc folder:
```
$ sh build_all.sh
```

### 3. Link components

While in /sorc folder:
```
$ sh link_fv3gfs.sh emc $MACHINE
```

...where $MACHINE is "dell", "cray", or "hera".
4 changes: 2 additions & 2 deletions docs/doxygen/compile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
machine=${1:-${machine:-"WCOSS_C"}}
machine=$(echo $machine | tr '[a-z]' '[A-Z]')

if [ $machine = "THEIA" ]; then
doxygen=/scratch3/NCEPDEV/hwrf/save/Samuel.Trahan/doxygen-1.8.10/bin/doxygen
if [ $machine = "HERA" ]; then
doxygen=/usr/bin/doxygen
elif [ $machine = "WCOSS_C" ]; then
doxygen=/gpfs/hps3/emc/hwrf/noscrub/soft/doxygen-1.8.10/bin/doxygen
elif [ $machine = "WCOSS" ]; then
Expand Down
2 changes: 1 addition & 1 deletion docs/doxygen/mainpage.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ This is a very much a work in progress and any issues should be reported back an
To setup an experiment, a python script <b>\c setup_expt.py</b> (located in <b>\c fv3gfs/ush</b>) can be used:
$> setup_expt.py -h
usage: setup_expt.py [-h] [--machine {THEIA,WCOSS_C}] --pslot PSLOT
usage: setup_expt.py [-h] [--machine {HERA,WCOSS_C}] --pslot PSLOT
[--configdir CONFIGDIR] [--idate IDATE] [--icsdir ICSDIR]
[--resdet RESDET] [--resens RESENS] [--comrot COMROT]
[--expdir EXPDIR] [--nens NENS] [--cdump CDUMP]
Expand Down
2 changes: 1 addition & 1 deletion docs/note_fixfield.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ They are saved locally on all platforms

Venus/Mars: /gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/fv3gfs/fix
Surge/Luna: /gpfs/hps3/emc/global/noscrub/emc.glopara/git/fv3gfs/fix
THEIA: /scratch4/NCEPDEV/global/save/glopara/git/fv3gfs/fix
Hera: /scratch1/NCEPDEV/global/glopara/fix

------------------------------------------------------------------------------
09/28/2018
Expand Down
77 changes: 0 additions & 77 deletions driver/gdas/test_jgdas_verfozn_theia.sh
View file

This file was deleted.

77 changes: 0 additions & 77 deletions driver/gdas/test_jgdas_verfrad_theia.sh
View file

This file was deleted.

77 changes: 0 additions & 77 deletions driver/gdas/test_jgdas_vminmon_theia.sh
View file

This file was deleted.

Loading

0 comments on commit a847acc

Please sign in to comment.