Use fracoro data for all new UFS applications #1242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 tasks
|
@ChunxiZhang-NOAA is this still a draft? It also needs to be updated with current develop. |
|
@WalterKolczynski-NOAA Yes, it is still a draft. I will check with George tomorrow for the upstream PR. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Today is @ChunxiZhang-NOAA's last day. Do we know who is going to take over this work? It looks like the UFS Utils PR it was waiting for was merged last week. |
|
@WalterKolczynski-NOAA I don't know who will take over this work, but Fanglin will arrange it. This PR is basically ready, only needs to update the hash for UFS_UTILS to the latest one or 4e673bf. I can do it within the next few hours. |
|
@WalterKolczynski-NOAA This PR is ready for review. |
WalterKolczynski-NOAA
requested changes
Mar 10, 2023
|
@yangfanglin who is supposed to be taking this over? |
|
@WalterKolczynski-NOAA Can we get this PR committed first since Chunxi said it was ready ? We can come back to revisit the issue related to GDASINIT_DIR and GDAS_INIT_DIR. I am afraid if we wait longer this PR will become more difficult to merge. For now I do not have anyone can take over this PR. |
aerorahul
reviewed
Mar 20, 2023
| @@ -16,6 +16,7 @@ export GDASINIT_DIR=${UFS_DIR}/util/gdas_init | |||
|
|
|||
| export CRES_HIRES=$CASE | |||
| export CRES_ENKF=$CASE_ENKF | |||
| export FRAC_ORO="yes" | |||
There was a problem hiding this comment.
What is FRAC_ORO and how does it differ from FRAC_GRID=.true.?
There was a problem hiding this comment.
Do we need this? It has been introduced here, but I can't see where it is used.
There was a problem hiding this comment.
I see it is used in ufs-utils. The question still begs asking, how does it differ from FRAC_GRID
WalterKolczynski-NOAA
approved these changes
Mar 20, 2023
jkhender
added a commit
to jkhender/global-workflow
that referenced
this pull request
May 18, 2023
Squashed commit of the following:
commit 608ff650fbc902114203a8d03893e3d26fa05d70
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Mon May 1 22:19:39 2023 -0400
Remove gdas bump fix files (#1553)
* Remove bump_ver from versions/fix.ver
- No longer need a bump version variable in fix.ver.
Refs #1552
commit 810071bff4878b16e5b70113fff990a1b1e80a4f
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon May 1 22:18:31 2023 -0400
Remove remnant WAVE_RUN from archive (#1556)
`$WAVE_RUN` is no longer used in workflow, but one reference remained
in the archive job, which would cause failures. The conditional did
not need to be replaced by `$RUN` since `$RUN` has already been checked
at that point.
Fixes #1548
commit f2ea92bf7345be1669677affab401bf25dd917ff
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon May 1 17:08:30 2023 -0400
Update UFS to develop as of 2023 Apr 17 (#1509)
Updates the UFS model hash to the version as of 2023 Apr 17.
Some associated changes accompany this update
- Restart filenames for MOM6 (ufs-community/ufs-weather-model#1599)
- Remove store coriolis setting from MOM6 namelist (ufs-community/ufs-weather-model#1599)
- Change in atm 'log' file names (ufs-community/ufs-weather-model#1704)
- Additions to diag_table for frozen species (ufs-community/ufs-weather-model#1529)
- Restart quilting (ufs-community/ufs-weather-model#1633)
- Update to post itag (ufs-community/ufs-weather-model#1690)
The switch to restart quilting adds an additional constraint on the size of write groups to be divisible by the number of tiles, so all were increased to the next multiple of 6.
In the process of updating the diag tables, unused tables were removed.
Closes #1279 as moot
Closes #1445
Closes #1499
Partially addresses #1277
commit 6d3ed8ac4b71759fbace38f3dd544f7ee018b821
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Mon May 1 14:36:01 2023 -0400
Adapt the marine DA to the new COM structure (#1554)
commit 6c48e94b4f3c7cf180cd443a13b957c8ab87ab4c
Author: Cory Martin <cory.r.martin@noaa.gov>
Date: Mon May 1 02:02:37 2023 -0400
Update aerosol DA to use new COM structure (#1551)
This PR updates the j-jobs and python classes for aerosol DA to use the new COM directory structure.
This PR also includes removing of the chem history staging for the ICSDIR in setup_expt.py. The aerosol fields are treated as FV3 tracers, so they are either available (warm start) with the fv_tracer files or start at 0 (cold start) and must be spun up from emissions.
Fixes #1516
commit ec2dd3ab8fb78c4e0aff7d81cb64095fe6130ad6
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Fri Apr 28 14:45:09 2023 -0400
Update UFSDA ATM ens for new COM directory structure (#1538)
g-w PR #1421 changed the GFS COM directory structure. This PR updates UFSDA ATM ensemble analysis jobs and python script to work with the updated GFS COM directory structure.
Fixes #1518
commit 406d6900963ca9c5152ef4b84de0d2840b0c2fd3
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Apr 28 12:55:05 2023 -0400
Run an ensemble forecast of the coupled model (#1545)
commit ff37168eddd52e00b288cde4bdbdb1927369abe6
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Fri Apr 28 12:53:59 2023 -0400
Fix ocean anl path in staging (#1544)
The path for ocean analysis files was not properly updated after
analysis was moved out of model_data into its own directory.
commit 8506ec6977c53018a100fd4881c296016a7df630
Author: Cory Martin <cory.r.martin@noaa.gov>
Date: Thu Apr 27 15:48:47 2023 -0400
Two minor bugfixes (#1542)
Found some odd bugs in the aerosol DA changes that need to be fixed to work properly.
Not sure how they worked before with testing (by luck?) but these are straightforward fixes
to implement.
one has a missing / in a path for FileHandler
and the other has a missing .nc causing files to not be found
commit 5f66da919c2525e189a169bad1f3fcbab3a64739
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Thu Apr 27 14:04:55 2023 -0400
Move guts of ocean analysis post out of j-job (#1539)
Removes most of content of JGDAS_GLOBAL_OCEAN_ANALYSIS_POST to scripts/exgdas_global_marine_analysis_post.py in GDASApp, now just calls that script.
Addresses first bullet of #1480
commit 3dd6bbe7a77145ab31d00bd4c23af7649353cbd9
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Wed Apr 26 17:02:05 2023 -0400
Update UFSDA ATM anl for new COM directory structure (#1537)
g-w PR #1421 changed the GFS COM directory structure. This PR updates UFSDA ATM variational analysis jobs and python scripts to work with the updated GFS COM directory structure.
Fixes #1517
commit b5d173f6a95ccc5df55aecb9dc490fffb79b26ca
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Wed Apr 26 13:39:45 2023 -0400
Fix incorrect ocean history tmpl during workflow generation (#1533)
The trailing '_TMPL' was accidentally ommited from the ocean history
template used during workflow generation.
Refs: #1532
commit 789ae97a1210117b9b629c4a47d6cfa999b551ab
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Wed Apr 26 13:39:31 2023 -0400
Fix ocean stating from flat structure (#1531)
A typo led to the same variable being defined twice instead of the
variable that was supposed to be defined.
Fixes #1530
commit 1e4a24c35088a3db6348156c15bbc81b6f814e40
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Wed Apr 26 00:17:56 2023 -0400
Fix warm-start IC staging (#1529)
The incorrect path was used for staging coupled components during the COM refactor update (#1421). These are now corrected.
Fixes #1528
commit 7421d805e6f50a59cd0be611d3ad7568ae603985
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Tue Apr 25 15:37:04 2023 -0400
Quiet generate_com (#1526)
Turns off trace for the duration of the generate_com function unless
DEBUG_WORKFLOW is not set to "NO" (the default). In its place, the
function will now echo the assignment.
Closes #1524
commit 23e6cc22a456c4a4294216037673473db0a144af
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Tue Apr 25 11:45:16 2023 -0400
Add ocnanlvrfy job and bugfixes. (#1514)
commit efa5180462f71ec476aeb6c5de4ba074a9d38a29
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon Apr 24 15:37:52 2023 -0400
Reorganize COM and refactor to use templates (#1421)
Reorganizes the entire COM directory into a more hierarchical structure and uses centrally-defined templates to define COM paths.
## Hierarchical Structure
To organize output a lot better and not have 30000+ files in a single directory, all of the component COM directories are divided into a number of subdirectories for each type of output.
Sample directory trees
### Cycled atmosphere only
```
gdas.20211222/00
├── analysis
│ └── atmos
│ └── gsidiags
│ ├── dir.0000
│ ├── dir.0001
│ ├── (Additional dir.* directories omitted for brevity)
│ └── dir.0083
├── model_data
│ └── atmos
│ ├── history
│ ├── master
│ └── restart
├── obs
└── products
└── atmos
├── cyclone
│ └── tracks
└── grib2
├── 0p25
├── 0p50
└── 1p00
101 directories
```
```
enkfgdas.20211222/00
├── earc00
├── ensstat
│ ├── analysis
│ │ └── atmos
│ │ └── gsidiags
│ │ ├── dir.0000
│ │ ├── dir.0001
│ │ ├── (Additional dir.* omitted for brevity)
│ │ └── dir.0039
│ └── model_data
│ └── atmos
│ └── history
├── mem001
│ ├── analysis
│ │ └── atmos
│ └── model_data
│ └── atmos
│ ├── history
│ ├── master
│ └── restart
└── mem002
├── analysis
│ └── atmos
└── model_data
└── atmos
├── history
├── master
└── restart
64 directories
```
```
gfs.20211222/00
├── analysis
│ └── atmos
├── model_data
│ └── atmos
│ ├── history
│ ├── master
│ └── restart
├── obs
└── products
└── atmos
├── bufr
├── cyclone
│ ├── genesis_vital
│ └── tracks
├── gempak
│ ├── 0p25
│ ├── 0p50
│ ├── 1p00
│ ├── 35km_atl
│ ├── 35km_pac
│ └── 40km
├── grib2
│ ├── 0p25
│ ├── 0p50
│ └── 1p00
└── wmo
26 directories
```
```
enkfgfs.20211222/00
├── earc00
├── ensstat
│ ├── analysis
│ │ └── atmos
│ │ └── gsidiags
│ │ ├── dir.0000
│ │ ├── dir.0001
│ │ ├── (Additional dir.* directories removed for brevity)
│ │ └── dir.0039
│ └── model_data
│ └── atmos
│ └── history
├── mem001
│ ├── analysis
│ │ └── atmos
│ └── model_data
│ └── atmos
│ ├── history
│ ├── master
│ └── restart
└── mem002
├── analysis
│ └── atmos
└── model_data
└── atmos
├── history
├── master
└── restart
64 directories
```
### S2SWA coupled
prototype (forecast-only):
```
gfs.20130401/00/
├── model_data
│ ├── atmos
│ │ ├── history
│ │ ├── input
│ │ ├── master
│ │ └── restart
│ ├── chem
│ │ └── history
│ ├── ice
│ │ ├── history
│ │ ├── input
│ │ └── restart
│ ├── med
│ │ └── restart
│ ├── ocean
│ │ ├── history
│ │ ├── input
│ │ └── restart
│ └── wave
│ ├── history
│ ├── prep
│ └── restart
└── products
├── atmos
│ ├── cyclone
│ │ ├── genesis_vital
│ │ └── tracks
│ ├── gempak
│ │ ├── 0p25
│ │ ├── 0p50
│ │ ├── 1p00
│ │ ├── 35km_atl
│ │ ├── 35km_pac
│ │ └── 40km
│ ├── grib2
│ │ ├── 0p25
│ │ ├── 0p50
│ │ └── 1p00
│ └── wmo
├── ocean
│ ├── 2D
│ ├── 3D
│ ├── grib
│ │ ├── 0p25
│ │ └── 0p50
│ └── xsect
└── wave
├── gempak
├── gridded
├── station
└── wmo
51 directories
```
### Trees with files
gdas: https://gist.github.com/WalterKolczynski-NOAA/f1de04901e2703fd24d38146d2669789
gfs: https://gist.github.com/WalterKolczynski-NOAA/5d1b7c0a0f4b8cfff0be1ae54082316a
enkfgdas: https://gist.github.com/WalterKolczynski-NOAA/860aaa804e3e70e191e7cae2ebb1055b
enkfgfs: https://gist.github.com/WalterKolczynski-NOAA/130bfff4650ed8b07cf395079b65d318
S2SWA P8: https://gist.github.com/WalterKolczynski-NOAA/6ae90c6eafb573878f60682ce47179db
## Templating
All of the COM paths have been replaced with new variables that are derived from a set of templates centrally defined in `config.com`. Variables in the templates are then substituted at runtime to generate the COM paths via the use of `envsubst`. To facilitate this, there is a new function, `generate_com` (see below), provided to automatically generate the COM paths.
Where possible, COM paths are defined at the j-job level and made read-only. However, many of the EnKF scripts loop over the ensemble members, forcing the definitions to be made at the exscript level instead (and be mutable).
The arguments to `generate_com()` are the list of COM variables to generate, optionally accompanied by a template to use using a colon to separate them. When no template is specified, the variable will be generated using the ${varname}_TMPL template. Two options are accepted, `-r` and `-x`, which will mark the variable as read-only and for export, respectively (the same as with the `declare` builtin). It is best practice to define any additional variables needed by the template on the same line to avoid adding them to the calling script’s scope. Here are some examples used in the code:
Generate the path to the atmos analysis directory for the current cycle and `$RUN` (implicitly from the `$COM_ATMOS_ANALYSIS_TMPL` template) and mark as read-only and export:
```
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_ANALYSIS
```
Generate the path to the atmos history directory for the previous cycle's gdas from the `$COM_ATMOS_HISTORY_TMPL` template and mark as read-only and export:
```
RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} generate_com -rx \
COM_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL
```
Generate the path to the first ensemble member's history directory of the current cycle and `$RUN` and mark for export:
```
MEMDIR=’mem001’ YMD=${PDY} HH=${cyc} generate_com -x COM_ATMOS_HISTORY
```
## Additional information
The staging of initial conditions in `setup_expy.py` has been updated to stage in the new locations. The source of the initial conditions can **either** be in the new hierarchical structure or in the old flat structure and the script will stage the files in the new structure. The destination paths are hard-coded here, so if any changes are made to the analysis, input, or restart templates, they will need to be mirrored in `setup_expy.py`.
### Stipulations
All changes in this PR are subject to approval by several stakeholders, including NCO. Sample COM trees above are subject to revision based on feedback (for instance, file X isn't really an obs file).
File name updates are not included in this PR. File names (primarily for coupled components) will be updated to comply with NCO standards in a future PR.
AWIPS jobs are now almost working (they do not in current develop), but one last program is still ending with an error.
Work on fit2obs is deferred, so that portion of the verify job does not work.
WAFS scripts are all external and have not yet been updated. WAFS is expected to be packaged separately going forward, so will need to be updated like any other downstream package.
Some scripts that are not part of our normal development workflow have not yet been updated. I may be able to knock a few more off this list, but some just aren’t available in development mode currently:
- All UFSDA app jobs (to be handled separately)
- With associated dev jobs (may still modify and test)
- JGDAS_ATMOS_GLDAS
- ~~JGLOBAL_WAVE_GEMPAK~~
- ~~JGLOBAL_WAVE_POST_BNDPNT~~
- ~~JGLOBAL_WAVE_POST_BNDPNTBLL~~
- ~~JGLOBAL_WAVE_PRDGEN_BULLS~~
- ~~JGLOBAL_WAVE_PRDGEN_GRIDDED~~
- ~~JGLOBAL_WAVE_PREP~~
- With no associated dev job
- JGDAS_ATMOS_GEMPAK_META_NCDCJGFS_ATMOS_FBWIND
- JGFS_ATMOS_FSU_GENESIS
- JGFS_ATMOS_GEMPAK_META
- JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF
- JGLOBAL_ATMOS_EMCSFC_SFC_PREP
- JGLOBAL_ATMOS_POST_MANAGER
- JGLOBAL_ATMOS_TROPCY_QC_RELOC
+ All downstream scripts for the above
There are also a few scripts that are not available to the development workflow that I have already made a good-faith effort at updating:
- JGDAS_ATMOS_GEMPAK
- JGFS_ATMOS_PGRB2_SPEC_NPOESS
## Related Issues
Closes #761
Fixes #978
Fixes #999
Fixes #1207
Partially addresses #198
Partially addresses #289
Partially addresses #293
Partially addresses #1299
Partially addresses #1326
commit 408ef65a8e2318125ad61478746024b2d0ef463d
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon Apr 24 15:26:58 2023 -0400
Move GDASApp hash to stable version (#1508)
Changes the GDASApp hash to a more stable version than the tip of
develop.
See post-merge converastion in #1506
commit 699a759f80352aaade4203425c5df24842fcefa3
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Mon Apr 24 13:37:04 2023 -0400
update GDASApp hash to d34f616 (#1505) (#1506)
commit 44f5c28518a7d4b9e06658c6c21b9b1ee1d0918e
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Mon Apr 24 12:26:14 2023 -0400
ignore archiving sfluxgrbf00[124578] files when they are not present (#1498)
* ignore archiving certain sfluxgrbf??? files when they are not present.
Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>
commit 2e88dbfc5ff7a0bb2c6c1630b0c2783d76049d21
Author: Jiarui Dong <Jiarui.Dong@noaa.gov>
Date: Mon Apr 24 09:17:48 2023 -0400
Add initial land DA cycling scripts (#1351)
This PR adds rocoto jobs, jjobs, config files and updates to the machine.env to enable land-da cycling capability.
commit f159d39a3b28dfcc120cdcdf87d11a611c75061f
Author: TerrenceMcGuinness-NOAA <terrence.mcguinness@cox.net>
Date: Fri Apr 21 15:46:35 2023 -0400
Add CI cron jobs (#1476)
As a maintainer of the CI framework, I need a set of cron jobs that will fully automate the CI pipeline so that whenever the appropriate label on GitHub is created the PR gets cloned and built followed by a set of functional experiments that are executed and reported on.
commit 587e469a1be5e278326fc0cbceefedc90caf75bf
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Fri Apr 21 13:28:11 2023 -0400
Refactor UFS-DA ATM ens component to use python g-w (#1373)
This PR contains UFS-DA ATM ens changes originally in PR #1354.
Below is a list of changes in this PR
- rename UFS-DA ATM ens jobs atmensanalprep, atmensanalrun, and atmensanalpost as atmensanlinit, atmensanlrun, and atmensanlfinal, respectively
- replace UFS-DA ATM ens shell scripts with python scripts
- rename UFS-DA ATM ens j-jobs consistent with initialize, run, and finalize functions. Update j-jobs to execute python scripts instead of shell scripts
- rename UFS-DA ATM ens rocoto jobs to be consistent with initialize, run, and finalize functions. Update jobs to set python paths and execute renamed j-jobs
- update rocoto workflow generation to new names for UFS-DA ATM ens jobs
- update UFS-DA ATM ens job names in machine dependent env files to new job names
- rename UFS-DA ATM ens configuration files consistent with change in job names
- add python class for UFS-DA ATM ens analysis
- unify JEDIEXE link for UFS-DA Aerosol, ATM, and ENS
- properly set `cycledefs` for `gfsatmanlinit`
- remove unused `FV3JEDI_FIX` from atmanl and atmensanl config
The above changes are part of a larger g-w effort to transition from shell scripts to python. UFS-DA Aerosol was the first GDASApp system to be converted. PR #1372 converted UFS-DA atmospheric variational DA to the python based approach. This PR converts converts UFS-DA atmospheric local ensemble DA to the python based approach.
Fixes #1313
Depends (in part) on #1370 and #1372 and NOAA-EMC/GDASApp#388
commit 7db70496063fe32928cacb9790e45a1e987a3510
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Fri Apr 21 12:42:00 2023 -0400
Added Fit2Obs to S4. #1489 (#1497)
Adds Fit2Obs support for S4 by adding the module use/load commands to the module_base.s4.lua modulefile.
Fixes #1489.
commit fb236523140b09686a4c2961e0552e7bd5dbf04f
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Fri Apr 21 12:40:11 2023 -0400
Add new task to post-process marine DA (#1485)
The work in this PR is only meant to bring us closer to a viable WCDA system. The refactoring of the marine DA to the new standard introduced by @aerorahul and used by @RussTreadon-NOAA and @CoryMartin-NOAA will be addressed after this [Epic](https://github.com/noaa-emc/gdasapp/issues/416) is resolved.
### Motivation and context
This work adds a separate j-job ```JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT ``` that calls a script that will be in the GDASApp for the time being (PR to come once this is merged) and does the following:
- prepares the `SOCA` increment for `MOM6` IAU
- recursively apply the `SOCA2CICE` change of variable. A mapping from the 2D seaice analysis variable to the CICE6 dynamical and thermodynamic variables.
- merge the `Tref` increment from the `NSST` analysis with the `SOCA` increment
### Summary of the change
- HPC environment: the new j-job runs a `JEDI` executable twice and one python script. All are serial jobs but the JEDI exec need to be called as an MPI job with 1 pe.
- `jobs/JGDAS_GLOBAL_OCEAN_ANALYSIS_CHKPT`, that script point to a ex-script that is not in the GDASApp develop yet.
- addition of the option to merge the Tref NSST increment with the MOM6 increment. This is triggered with the `DO_MERGENSST` switch
- The new j-job dependency was added, with the option to wait for the surface analysis file `sfcanl.nc` if `do_mergensst` is true.
Refs: #1480.
Fixes NOAA-EMC/GDASApp/#418
commit 740daba8d6d34a327199701c1df7d6e10da73ec5
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Fri Apr 21 03:11:51 2023 -0400
Create fix file issue template (#1495)
Create fix_file.md template file for new fix file request issue. This should help formalize the process and document updates.
New issue will auto-assign to @KateFriedman-NOAA and @WalterKolczynski-NOAA (the developers with access to make fix files changes).
Fixes #1492
commit 35942896ca4eeef243fe35d47416be64fe0058ff
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Thu Apr 20 10:00:23 2023 -0400
Update TC_tracker version to v1.1.15.6
New ens_tracker.v1.1.15.6 tag installed on supported platforms. Adds Jet support and moves package to use new EPIC-installed hpc-stacks on R&Ds.
Refs #1463
commit 86c3923bf60b1ce39165070bf2e5c3d60193d6dd
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Thu Apr 20 09:39:56 2023 -0400
Update GSI-Monitor hash to reflect recent assimilation changes.
New GSI-Monitor hash to 45783e3 to update two fix files.
Refs #1483
commit 2f347f6ddc770f2524394af25561a8da0d8dfb50
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Wed Apr 19 15:54:19 2023 -0400
Fit2Obs updates for package reorganization and invocation via module (#1484)
The Fit2Obs repo has been reorganized to meet a few NCO standards and to add a module for invoking it from other packages.
Refs #1472
commit b2ed8648f80946de85983a51664b120540854cc9
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Tue Apr 18 16:21:14 2023 -0400
Refactor UFS-DA ATM var component to use python g-w (#1372)
This PR contains UFS-DA ATM var changes in PR https://github.com/NOAA-EMC/global-workflow/pull/1354.
Below is a list of changes in this PR
- rename UFS-DA ATM var jobs `atmanalprep`, `atmanalrun`, and `atmanalpost` as `atmanlinit`, `atmanlrun`, and `atmanlfinal`, respectively
- replace UFS-DA ATM var shell scripts with python scripts
- rename UFS-DA ATM var j-jobs consistent with initialize, run, and finalize functions. Update j-jobs to execute python scripts instead of shell scripts
- rename UFS-DA ATM var rocoto jobs to be consistent with initialize, run, and finalize functions. Update jobs to set python paths and execute renamed j-jobs
- update rocoto workflow generation to new names for UFS-DA ATM var jobs
- update UFS-DA ATM var job names in machine dependent `env` files to new job names
- rename UFS-DA ATM var configuration files consistent with change in job names
- add UFS-DA ATM yaml files for UFS-DA fix files, crtm files, and increments
- add python class for UFS-DA ATM var analysis
- link UFS-DA python increment conversion script from GDASApp to g-w `ush` directory
The above changes are part of a larger g-w effort to transition from shell scripts to python. UFS-DA Aerosol was the first GDASApp system to be converted. This PR represents the second GDASApp system, UFS-DA atmospheric variational DA, to be converted. PR #1373 contains changes to convert the UFS-DA ensemble DA to a python based approach.
Fixes (in part) #1313
Depends (in part) on PR #1370 and [GDASApp #388](https://github.com/NOAA-EMC/GDASApp/pull/388)
commit 70a7d99bb65fd7661d7d2bac10633c0dda5d39fd
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Tue Apr 18 10:26:00 2023 -0400
Port the global workflow to Jet (#1301)
Provides initial cycled and free-forecast support for the global workflow on Jet. References #357. Not included in this port is support for GLDAS, verif-global, and TC_tracker.
Jet will eventually support the following, with the currently tested options bolded, on xjet and kjet partitions
- uncoupled, cycled and free-forecast experiments at C48, C96, C192, C384, and C768 resolutions
- coupled, free-forecast experiments (ATMA, ATMW, S2S, and S2SW) at C384
~Note that currently coupled ICs are not available on Jet, so only ATM-only experiments can performed.~
Fixes #357
commit d2b268ab3d965c9a2cf998dd048ac7a3ee7dc36b
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon Apr 17 21:15:04 2023 -0400
Consolidate wave parm files (#1477)
When the new wave parm directory was created, the existing parm files
were not moved into the new directory. These files were used for AWIPS,
so not generally tested.
commit 8dcfaa6fbc5e0a94d44952f710f77c3b18ffa50d
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon Apr 17 13:37:26 2023 -0400
Split MPMD stdout into tasks on slurm (#1469)
It can be difficult to debug MPMD jobs because their logs are all written concurrently to a single file. While the use of tags to designate which task via the preamble and PS4 can help identify which line is from which task, it is still difficult to follow a single task through the log, particularly for larger MPMD jobs with dozens of tasks.
Individual stdout files are now created by using the `srun` `--output` option. These files are written to the working directory (in `$DATA`).
Fixes: #1468
commit 6e7e4f1db78bbb67d94d54bdcb0a021a626b01f1
Author: Henry R. Winterbottom <49202169+HenryWinterbottom-NOAA@users.noreply.github.com>
Date: Mon Apr 17 09:59:01 2023 -0600
GFS fixed-file YAMLs. (#1471)
Add a few fixed-file YAMLs used in the GFS.
commit 2ec4125f67e6b6c1d8dee6f6fcf1d0798f587a31
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Apr 14 20:55:57 2023 -0400
Initial blocks in place for forecast refactor work (#1466)
This PR is a first in a series of PR for transforming the forecast job.
This PR does not affect current function of the forecast job.
This PR:
- adds initial blocks to separate task specific and model configuration for the task blocks
commit d47f33f142824c0d2111f7e2f08c43f99b33bff4
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Fri Apr 14 15:51:14 2023 -0400
Update buoys file and fix boundary point jobs (#1465)
The buoy file used by the wave jobs ([wave_gfs.buoys](https://github.com/NOAA-EMC/global-workflow/blob/develop/parm/wave/wave_gfs.buoys)) was just a copy of [wave_gfs.buoys.dat](https://github.com/NOAA-EMC/global-workflow/blob/develop/parm/wave/wave_gfs.buoys.dat). In addition to being a duplicate, the file was a truncated version without any boundary points, causing boundary point jobs to fail.
The duplicate file has been removed and replaced by a symlink to the full buoy list [wave_gfs.buoys.full](https://github.com/NOAA-EMC/global-workflow/blob/develop/parm/wave/wave_gfs.buoys.full). This maintains the provenance of the file and prevents the former duplicate from becoming out-of-sync. Users who still want to use the truncated buoy can change the target of the symlink to wave_gfs.buoys.dat.
There are also a few minor bugs fixes that were necessary to get boundary point jobs to run:
- `FHMAX_WAV_IBP` had been set in the bndpnt config file but not used in the j-job. This was invisible unless a user changed the value, since the config and j-job used the same default.
- Checks against `FHMAX_WAV` would set the unused `FHMAX_WAV_IBP` to the max value instead of the `FHMAX_WAV_PNT` used for the loop. This is a problem when running for less than 180 h (the default value).
- The boundary point bulletin job was not in the env job list for Orion (other machines have it).
Now the boundary point jobs set `FHMAX_WAV_PNT` to `$FHMAX_WAV_IBP`. `FHMAX_WAV_IBP` was moved from the bndpnt config to config.wave so it is visible to bndpntbll as well.
Fixes #1464
commit e496e393b16565207c227f4b69a5691d97098624
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Wed Apr 12 15:59:00 2023 -0400
Move Fit2Obs to stand-alone job (#1456)
This PR moves the Fit2Obs invocation out of the vrfy job and into its own dedicated `fit2obs` job in the gdas suite. This new dedicated job uses the latest Fit2Obs tag `wflow.1.0`.
The Fit2Obs requires a type of spin-up. The job looks back `VBACKUP_FITS` hrs and needs available inputs for that lookback cycle in the `ROTDIR`. The `jobs/JGDAS_FIT2OBS` script will first check that `xdate` (`CDATE` - `VBACKUP_FITS`) > `SDATE` and then if met, will check that the needed inputs exist.
- If `xdate>SDATE` is not yet satisfied, the job will exit 0 with "Too early for FIT2OBS to run. Exiting.". The conditional is greater-than and not greater-than-or-equal since the first half cycle generally does not have some of the needed inputs (e.g. prepbufr). Thus the first half cycle is not included in the valid lookback cycles. This avoids erroneous job failures for the first cycle to run the Fit2Obs package. Additional logic could be introduced to include the half cycle if all available inputs are available.
- If any of the needed inputs are missing the job will abort with "FATAL ERROR: FILE MISSING: ${file}".
This spin-up means that the first cycles will run the job but exit 0 immediately. The 6th cycle (if `VBACKUP_FITS=24`) will be the first cycle to run the Fit2Obs package and produce output in the online archive.
Changes:
1. Remove fit2obs variables and settings from `config.vrfy` and into newly created `config.fit2obs` for `fit2obs` job.
2. Remove fit2obs submission/invocation from `jobs/rocoto/vrfy.sh`.
3. Create new `fit2obs` job scripts: `jobs/rocoto/fit2obs.sh` and `jobs/JGDAS_FIT2OBS`
4. Add new `fit2obs` job to setup scripts: `workflow/applications.py` and `workflow/rocoto/workflow_tasks.py`
5. Add new `fit2obs` job to all env files.
6. Add new `fit2obs` job into `config.resources` (use 1 node on WCOSS2 and 3 nodes elsewhere).
7. Add `export DO_FIT2OBS="YES"` to `config.base.emc.dyn`.
Resolves #1405
Resolves #1232
commit 363a2b47de11ab327408d2df20a3718f25062fa5
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Wed Apr 12 14:42:50 2023 -0400
Rework arch job dependencies (#1455)
This reworks the dependencies for arch tasks so that if there are no verification tasks selected and it is an uncoupled experiment, a dependency for the cycle's post jobs is added. It also fixes the dependency checks in rocoto.py to check for an empty list or string instead of checking the first element of a list, which may not exist if no dependencies are given. Lastly, it issues a warning to the user if there are no dependencies for any job.
Fixes #1451
commit 7e661f4cd00cf99b12e55574318bbd0eb1a5eaa0
Author: RussTreadon-NOAA <26926959+RussTreadon-NOAA@users.noreply.github.com>
Date: Wed Apr 12 10:36:58 2023 -0400
Stage bias files for UFSDA aerosols (#1370)
Updates UFSDA aerosols to stage bias files. Also adds in the threads and aprun commands for the run portion of the UFDA aero jobs.
Fixes (in part) #1313.
commit d5ae3328fa4041b177357b1133f6b92e81c859d7
Author: Henry R. Winterbottom <49202169+HenryWinterbottom-NOAA@users.noreply.github.com>
Date: Tue Apr 4 03:34:48 2023 -0600
Add logging level to logger (#1442)
This PR provides task-level support for the respective pygfs applications. As an example, logging-level may be defined within jobs/JGLOBAL_FORECAST as follows.
```
#! /usr/bin/env bash
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "fcst" -c "base fcst"
export LOGGING_LEVEL="DEBUG"
```
This feature allows a user to change the logging level from the run-time environment rather than requiring modification of the respective pygfs module or task.
Fixes #1438
commit de81c5911c308679366b97fda19f015b1388d9dc
Author: TerrenceMcGuinness-NOAA <terrence.mcguinness@cox.net>
Date: Mon Apr 3 17:53:12 2023 -0400
Update hash for GDASApp to db2f998 (#1443)
When an update described below made a change to the HASH in the `${HOMEgfs}/sorc/checkout.sh` script for the corresponding GDASApp it was entered in error:
_3e73038c - Use V2 version of fix files needed for Thompson MP (#1422) (7 days ago) <Rahul Mahajan>_
Examining the GDASApp repo it should be confirmed that the correct HASH should be **db2f998**
Fixes #1441
commit 0d1e993b2b5db6160c4a6b88b67899dc8e9754f8
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Fri Mar 31 09:58:22 2023 -0400
Remove para module paths for ncdiag on WCOSS2 (#1437)
The ncdiag/1.0.0 module moved from para to prod on WCOSS2 on March 27th (RFC 10769).
* Remove para module paths for ncdiag on WCOSS2 in global-workflow module_base.wcoss2.lua
* Update GSI-EnKF hash to update ncio and ncdiag
* Update GSI-Monitor hash to update ncdiag
Refs #1426
commit 88e091a8e92cbc813830b21562fa392f8dd2d3d9
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Mar 31 02:50:53 2023 -0400
Use P8 settings for C384 atm by default (#1440)
Switches the default C383 FV3 timestep to 300s and reduces the decomposition for gfs CDUMP to 8×8 with 48 write tasks per group. These are the settings used by P8. MDAB has advised these settings can be used for non-P8 runs.
Fixes #1439
commit 3cfdbe04e864847a3f89b6f8b89799b18b7e1f5e
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Wed Mar 29 10:46:35 2023 -0400
Enforce rstprod on relevant tarballs (#1436)
Ensures that tarballs that contain restricted data are properly restricted to the rstprod group.
Fixes #1433
commit 809b33bf50193a083c7ffdbd87bb83e0c78b2a9c
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Tue Mar 28 09:44:37 2023 -0400
GFSv16.3.5[6] GSI updates (#1404)
* Update GSI hash to 31b8b29
* Remove temporary hack that forced GSI to build with crtm/2.4.0
* Remove G18 ABI from exglobal_atmos_analysis.sh
* Remove GMI from processing in the GSI
Refs #1322, #1321
commit 222f055e1e082faf7f9e489297e5c46ccc582a20
Author: Cory Martin <cory.r.martin@noaa.gov>
Date: Mon Mar 27 17:48:39 2023 -0400
Have aerostat tar file extract to basename only (#1424)
This bugfix makes it so that when the aerosol diags are extracted, they are extracted to the directory directly and not a full directory tree to where the runtime directory was located.
Closes #1423
commit c549acb914c7ae479740c8f1007f5f3f36b91db5
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Mon Mar 27 10:00:18 2023 -0400
Updates in the aerosol tasks (#1420)
-removes the definition of environment variables in the shell script j-jobs
-eliminates the use of !ENV in the yaml files for aerosol jobs.
-eliminates use of CDATE in the python tasks. Use current_cycle and previous_cycle
-uses jinja templates where appropriate.
-uses the Executable class to run the variational analysis executable
-adds verbose logging to the actions in the aerosol analysis task
-uses string templates instead of string replace where looping over tiles is required.
-links aerosol ICs when cycling with the ATMA app. ICs courtesy of @CoryMartin-NOAA
Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
commit 3e73038c12f8261543c874cfffed11f8ce496399
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Mar 24 12:40:17 2023 -0400
Use V2 version of fix files needed for Thompson MP (#1422)
It has been noted by some developers who look at the run log in realtime that the model takes a while during the calculation of Thompson tables.
Specifically see this part of the output from the forecast log:
```
0: Calculating Thompson tables part 1 took 0.334 seconds.
0: Calling radar_init took 0.000 seconds.
0: creating rain collecting graupel table
0: ThompMP: computing qr_acr_qg
0: Writing qr_acr_qgV2.dat in Thompson MP init
0: Computing rain collecting graupel table took 203.539 seconds.
0: creating rain collecting snow table
0: ThompMP: computing qr_acr_qs
0: Writing qr_acr_qsV2.dat in Thompson MP init
0: Computing rain collecting snow table took 36.694 seconds.
0: creating freezing of water drops table
0: Computing freezing of water drops table took 2.084 seconds.
0: Calculating Thompson tables part 2 took 2.084 seconds.
0: ... DONE microphysical lookup tables
```
These tables are already available in the `fix` space and are being used in the ufs-weather-model regression tests.
Fixes #1411
commit 4ff622a36e3f9aa2f39e847a7b9e2b536ebcdc0a
Author: TerrenceMcGuinness-NOAA <terrence.mcguinness@cox.net>
Date: Thu Mar 23 01:55:42 2023 -0400
Add experiment creation for CI (#1388)
Adds a new python script to create an experiment on the fly for CI.
Also adds detect_machine.sh.
Closes #1375
commit 6bb2d64fa171604b9d5e8f58e4edf8bf946840e7
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Wed Mar 22 15:44:00 2023 -0400
Update state of operations in RTD to GFSv16.3.7 (#1417)
Update the "State of operations" blurb in index.rst to note the updated GFSv16.3.7 operational version.
Refs #1368
commit ea15b26585c0c804f44776cabb6ae3d08b76234f
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Wed Mar 22 14:07:06 2023 -0400
Fix typo in wave awips gridded task def (#1412)
The task name was misspelled in the task definition for gridded wave
awips.
commit 74b344ddb28fb40b8a65479f1584cf7daa376dfe
Author: Henry R. Winterbottom <49202169+HenryWinterbottom-NOAA@users.noreply.github.com>
Date: Wed Mar 22 09:05:42 2023 -0600
Make new methods in yaml_file be public.
commit 478f48ede4ee28e8193ff02befb0a954bd7eca61
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Tue Mar 21 17:18:46 2023 -0400
Fix groupsize for early cycle EnKF rocoto task (#1408)
When generating the rocoto tasks for enkfgfs, the different groupsize
for gfs would not be picked up because the cdump comparison was not
properly updated when the cdump/run was updated to include 'enkf'.
commit 49b96ed037c6919c651c6b7dd2b6c9fbe9866904
Author: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Date: Mon Mar 20 15:35:19 2023 -0400
Use fracoro data for all new UFS applications (#1242)
The new fracoro data should be used for all new UFS applications no matter if it uses frac_grid or not.
Most problems in Issue[#863](https://github.com/NOAA-EMC/global-workflow/issues/863) have been resolved. However, one problem remains, e.g., the latest fix, mask and oro datasets (fracoro) created by Shan/Mike/Helin should work for both fractional and non-fractional grid.
Note that it also requires the changes in UFS_UTILS. An corresponding PR[#741](https://github.com/ufs-community/UFS_UTILS/pull/741) in UFS_UTILS has been created.
Fixes: #863
Dependency: UFS_UTILS [PR#741](https://github.com/ufs-community/UFS_UTILS/pull/741)
commit e5af1b45f5aaa3acfe8e6ee37e690edff9aa608e
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Mar 17 13:02:54 2023 -0400
Updates to python tools for use in DA tasks (#1400)
Adds significant updates to pygw tools that expand the use of templated yaml files to make their use in the tasks clear and easier to use.
All changes come with associated tests.
New tests are added for timetools.py and jinja.py.
New methods to parse a "simple" $( ... ) templated as well as jinja2 {{ ... }} templated yaml files are added along with their tests.
commit 3fe3592338598ef6b957d29e809dc70df1f82cec
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Wed Mar 15 01:48:46 2023 -0400
Marine DA prep j-job needs more memory (#1393)
The concatenation step runs out of memory in the marine-gdas prep step. This is not optimized and probably overkill, but 24GB should cover all cases.
Fixes #1389
commit 6cf486190c83f64370f1c2dccabd85700fbc0a3a
Author: TerrenceMcGuinness-NOAA <terrence.mcguinness@cox.net>
Date: Tue Mar 14 15:50:18 2023 -0400
Update rocoto_viewer to replace deprecated getiterator call (#1397)
Rocoto viewer was using a deprecated function `getiterator` that caused it to fail on python 3.8+. The replacement method `iter` is now used.
Fixes #522
commit 2929430369b0e6b4e6d42b45637f552f8cb7e59e
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Mon Mar 13 16:29:51 2023 -0400
Reset modules properly at beginning of forecast job (#1394)
Following the PR last week that enabled ESMF threading, we had to replace `load_fv3gfs_modules.sh` with loading ufs-weather-model specific modules for the `fcst` and `efcs` jobs.
`module-setup.sh` is needed after `detect_machine.sh`. Previously, both these functions were performed in `load_fv3gfs_modules.sh`.
commit 995e2b0c819d2bf45e9cde7b3fb4dd638034f91d
Author: Henry R. Winterbottom <49202169+HenryWinterbottom-NOAA@users.noreply.github.com>
Date: Fri Mar 10 18:34:28 2023 -0700
Add base workflow exception class (#1392)
Adds a new WorkflowException that can serve as a base class for any new exceptions we wish to create to cover errors not well represented by the native python exceptions.
Also adds a test exception.
Closes #1391.
commit aa8175dc39bd64e895d2362dc0ec37ed76ababef
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Mar 10 19:50:28 2023 -0500
Enable ESMF threading in the ufs-weather-model forecast (#1371)
Transitions the workflow to use ESMF-managed threading for UFS. This allows for per-component specification of threads rather than a single value for all components.
The resource calculation is updated to handle the different thread counts for each component. The variable `NTHREADS_FV3` (which set the global thread count) is removed. Now each component has a `${COMPONENT}THREADS` variable.
In order to run properly, the launcher commands for each machine had to be modified so the number of processes is the number of CPUs on all nodes.
Also, the forecast job now uses the UFS modulefile rather than the typical workflow runtime module. `prod_util` is then loaded manually as it is needed to run the workflow. Additionally, on WCOSS2 `cray-pals` is also loaded manually as it is necessary. This arrangement is temporary until a more permanent solution is implemented.
Notes:
1. As a result of threading, the `WRTTASKS_PER_GROUP` in the `model_configure` will end up to be a multiple of the number of threads used in quilting. At present, they are assumed to be the same as the threads for FV3.
2. The `WCOSS2.env` file needs a look at as the sections for steps `fcst` and `efcs` are different and very different when compared to the [job card](https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/fv3_conf/fv3_qsub.IN_wcoss2) in the ufs-weather-model for WCOSS2.
Closes #1042
commit 8a2d5061da3c3067291e51680408339ec2efec5c
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Thu Mar 9 16:11:22 2023 -0500
Cleanup ocean, ice, and med directories when CDUMP=gdas (#1387)
commit 780a511e51c83b339b02709a782c123a16c5788d
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Thu Mar 9 15:59:40 2023 -0500
Archive ocean and ice files when CDUMP=gdas (#1384)
commit d8fdd29ab236798937fbd1c5e1aa9266db98dcc0
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Wed Mar 8 16:38:06 2023 -0500
Updated diag_table_da to allow output of ocean fields for SOCA at various resolutions (#1382)
commit 815823997e7886dd4c054fa2bf573dabd5d6047c
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Wed Mar 8 09:38:11 2023 -0500
Add mechanism to detect machine and clean module env. (#1381)
commit 4437181a72629e6f6f7214de749dc890e32a4994
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Tue Mar 7 08:57:10 2023 -0500
Update initial condition documentation and Orion BASE_CPLIC path (#1376)
* The initial conditions section of the Read-The-Docs documentation is updated to add information about staged initial conditions that were pulled into global account space on supported platforms. Some reformatting of the section is also done.
* The Orion BASE_CPLIC path is also updated after prototype ICs were copied from @WalterKolczynski-NOAA's personal area on Orion to the "glopara" area maintained by @KateFriedman-NOAA.
Refs #1345
commit 1a48aca822b1caf9325b37e9819243f91454f4e6
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Tue Mar 7 07:56:59 2023 -0500
Add a missing dollar sign in the earc script (#1378)
commit 33e8a8ea56920bca4e31bf961647ec989f24582b
Author: TerrenceMcGuinness-NOAA <terrence.mcguinness@cox.net>
Date: Mon Mar 6 22:52:33 2023 -0500
Add BASH scripts for initial CI testing system using GitHub Labels (#1362)
This PR is a set of high-level BASH scripts for creating a basis of a CI System using GitHub labels.
These scripts will first poll the **global-workflow** GitHub repo for open PR's with the label **${hostname}-CI**. The label is the designated RDHPCS system the CI functional tests will run. A second script will then clone and build from the designated PRs on the specified RDHPC.
Closes #1374
commit 47afc78568563bc4c55208e567bfb967684eae0a
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Mar 3 10:43:54 2023 -0500
Consolidate ocean, ice and wave task info into config.ufs (#1334)
config.fv3 is now renamed as config.ufs and contains ocean, ice, and wave task information
* config.fv3 is used to record FV3 and write grid component task decomposition based on resolution.
* Ocean (MOM6) and Ice (CICE6) decomposition, tasks and timesteps are added to config.ufs.
* Updates are made to config.fcst and config.efcs to source config.ufs based on the configuration of the (coupled) model.
* Several configurations were made deep in parsing_namelists_MOM.sh, which have now been elevated to config.ocn
* Some improvements are made to limit the export of variables from functions in nems_configure.sh.
* This will help with the ESMF threading work as it breaks down the work into multiple steps.
commit 0c523d9b75a6a5d24a55fcc3f56ed505dedac086
Author: Jessica Meixner <jessica.meixner@noaa.gov>
Date: Thu Mar 2 16:27:53 2023 -0500
Update to HR1 (#1197)
Updates settings for the HR1 prototype
Compilation is switched to "mixed-mode", with a 32-bit atmosphere and 64-bit for other components.
Atmosphere physics options are updated.
Changing the wave model to run on the outer loop and the resolution to ¼-deg.
The model version was already updated previously, but this commit updates some of the associated input files.
commit 5d6c71ab34667ef0dac777a89a19a37edd2652fc
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Thu Mar 2 11:53:10 2023 -0500
Add options while setting up Rocoto XML that are useful for CI (#1365)
Adds options for maximum tries (default is 2), cyclethrottle (default is 3), taskthrottle (default is 25) and verbosity (default is 10)
commit 6024e68d8f519d4d1de224de0d7d6799e02f2e07
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Thu Mar 2 11:06:17 2023 -0500
Revert "Add options while setting up Rocoto XML that are useful for CI (#1363)" (#1364)
This reverts commit c318cbdefc80b390a6bf897229fd2e206eb6873c.
commit 53952153eea6a9afc83512f84d860823b300bc97
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Thu Mar 2 10:02:14 2023 -0500
MOM6 backgrounds in cycled DA mode were hardcoded for IAU (#1355)
* fixed mom6 bkg output
Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>
commit c318cbdefc80b390a6bf897229fd2e206eb6873c
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Thu Mar 2 09:50:09 2023 -0500
Add options while setting up Rocoto XML that are useful for CI (#1363)
maxtries, cyclethrottle, taskthrottle and verbosity are command-line options to setup_xml.py
commit 0e1c753e7bd4d4e98b40372168578198c4300124
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Thu Mar 2 09:48:51 2023 -0500
Run executables or scripts from within python. (#1341)
- adds the ability of running executable (binary or shell scripts) via the subprocess.Popen call
- allows setting env. variables for the subprocess without having to modify the calling environment.
- allows passing custom arguments to the executable
- allows capturing stdout and stderr as well as pass stdin
commit 2e92b7c582f116434ca49af7f7e6b5ec48842f48
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Wed Mar 1 16:21:13 2023 -0500
Remove extra 'enkf' from folder/file names in archive scripts (#1360)
The archiving scripts have had preceding `enkf`s removed and `$CDUMP` has been replaced with `$RUN` throughout the scripts. This fixes #1353 and partially addresses #1299.
Fixes #1353
Refs #1299
commit 0a18568a58d4c33e63d728424223900e98ee0350
Author: Xianwu Xue - NOAA <48287866+XianwuXue-NOAA@users.noreply.github.com>
Date: Wed Mar 1 16:12:06 2023 -0500
Fix bugs to create analysis files for early cycle (#1343)
The cycled early enkf should generate analysis files, however, the CDUMP was changed from "gfs" to "enkfgfs" for enkfgfs* jobs, some of the scripts should do the corresponding changes. This PR is to fix this bug.
commit edbf8d955e04b54909c20379d4f674d0a9f3e1f7
Author: Cory Martin <cory.r.martin@noaa.gov>
Date: Wed Mar 1 16:10:59 2023 -0500
Make necessary bugfixes to get aerosol cycling going (#1349)
Makes a number of bugfixes that were overlooked in #1106 that now allow for 3DVar aerosol DA cycling on Hera/Orion.
Also updates the GDASApp hash to the most recent commit in develop.
commit c4d05e57d5e7192ac3b8a93e6880efe5bbad2e65
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Tue Feb 28 16:27:46 2023 -0500
Create production_update.md (#1348)
New template for operational production updates. Includes checklist for workflow side.
commit 8134f975d51905789f7a59b07d713306a91ba10b
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Mon Feb 27 14:36:03 2023 -0500
Update feature_request.md
commit a243b5c1c2bb21a47bcc9dee260ab5b90bb06843
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Mon Feb 27 12:21:27 2023 -0500
Create a template for requesting new features in the global workflow to separate from general issues (#1336)
commit 0c621d0b9ead8d46fb287d4eb547386b3335f9f8
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Fri Feb 24 19:30:00 2023 -0500
allow script to access eva yaml generators (#1331)
commit f69d3e54d92b257bf1102c13eb1197a967288c39
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Feb 24 12:06:46 2023 -0500
Add license and status badges for the CI (#1332)
commit ea414291341e77eb6eee64241a702141152efcfd
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Wed Feb 22 17:02:15 2023 -0500
Update ufs_utils hash that supports global_cycle with NoahMP. (#1315)
- Updates ufs_utils hash that contains upgrades to `global_cycle` that updates only the greenness fraction. This update expects a pre-existing surface restart file to be updated.
- Corressponding updates to scripts that call `global_cycle` to stage the surface restart file to update.
- Updates to `checkout.sh` and `Externals.cfg`. Temporary pointers to [this](https://github.com/GeorgeGayno-NOAA/UFS_UTILS/tree/feature/cycle_noahmp) branch until `develop` in `ufs_utils` is updated.
- Updates to `setup_expt.py` to force using the same `CCPP_SUITE` and `IMP_PHYSICS` for cycled and forecast-only modes and all apps.
- Only builds utilities from ufs_utils that are used in the GFS application.
@GeorgeGayno-NOAA still needs to work with the land team to determine which fields (other than greenness fraction) need to be updated through `global_cycle`.
Depends on https://github.com/ufs-community/UFS_UTILS/pull/774
Fixes #1314
Updating the UFS_utils hash will also resolve #1275
Fixes #1275
commit 6addad94b510b08a83e43236b9d8c430b8aeddce
Author: AndrewEichmann-NOAA <58948505+AndrewEichmann-NOAA@users.noreply.github.com>
Date: Wed Feb 22 15:02:07 2023 -0500
Save yaml for ocn analysis post and fix bug to save logs (#1318)
Temporarily saves var.yaml to comrot in ocean analysis post and fixes a bug that caused log files to not be saved. Necessary for https://github.com/NOAA-EMC/GDASApp/issues/202. This is temporary pending an evaluation of what needs to be saved and the appropriate place.
Refs https://github.com/NOAA-EMC/GDASApp/issues/202
commit dbbd8b19ce847136dbb6457a082e58ab17f5678f
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Wed Feb 22 14:53:29 2023 -0500
Update crtm to v2.4.0 (#1319)
* Update module_base modulefiles to use crtm/2.4.0.
* Remove line in module_base modulefiles to set CRTM_FIX; this variable is now set in the crtm module as of v2.4.0.
* Update ufs-weather-model hash to c22aaad
* Temporarily set crtm_ver in GSI build script to force GSI to build with crtm/2.4.0
Refs #1233
commit adae24cf6c08ca327f74c8bcd3aa9c89cbb28724
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Wed Feb 22 14:45:49 2023 -0500
Fix python style errors (#1330)
Some python style errors crept in to develop because we ignored pynorm failures during the PR process. These errors are now corrected.
commit e18a79036f16f60e7560771d17ab89d10ea39a96
Author: Xianwu Xue - NOAA <48287866+XianwuXue-NOAA@users.noreply.github.com>
Date: Wed Feb 22 13:17:37 2023 -0500
Fix workflow generation post times for early cycle (#1329)
In PR #1309 a setting was overlooked that controls what hours have post tasks created for the early cycle.
Fixes #1328
commit 07fedaa649e4576a2bd3d2fca32bb6c144fa2a85
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Fri Feb 17 16:43:16 2023 -0500
Change RUN and CDUMP for ensemble jobs (#1309)
Updates `$RUN` and `$CDUMP` for ensemble jobs to include `enkf`.
Previously, the `$RUN` for EnKF jobs had been set to `gdas` or `gfs`. However, this violates NCO policy and was also complicating Issue #761 . Now the `$RUN` for EnKF jobs is either `enkfgdas` or `enkfgfs`.
Theoretically, `$CDUMP` shouldn't need to change. However, `$CDUMP` and `$RUN` are used interchangeably throughout much of the workflow (Issue #1299), so for now the `$CDUMP` is kept identical to `$RUN`. This will be corrected in a future PR.
This change **changes the name** of enkf output files. Files now begin `enkfgdas` (or `enkfgfs`) rather than `gdas`.
Closes #1298
commit 383c8c3a25dbeedb3a4892d22d2531286e87389f
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Thu Feb 16 11:10:50 2023 -0500
Provide default for DEBUG_WORKFLOW in load_ufsda_modules.sh
commit 3bfcb8975acab6a14634d95189b6b10379e37afc
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Wed Feb 15 11:10:34 2023 -0500
Allow increments to be added for ocean and ice cycling. (#1308)
Removed forgotten commented out call to the B-mat j-job in jobs/rocoto/ocnanalbmat.sh ... oops
Provide ocean increments via `mom6_increment.nc`
Link to the JEDI/SOCA increment in ush/forecast_postdet.sh
Fixed a dependency bug in workflow/rocoto/workflow_tasks.py
MOM6 Increment is required in ROTDIR for the first 1/2 cycle, changes reflected in workflow/setup_expt.py
Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>
commit a1968e6cd7546151e670c5a0d6f1dd4b5d859c10
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Tue Feb 14 12:51:33 2023 -0500
Module base file reorder and reduction (#1306)
* Adjust the order of module loads in module_base modulefiles to conform with desired order: compilers, mpi, 3rd party, hdf5, netcdf, nceplibs.
* Remove modules that aren't needed for runtime.
* Checked dependencies of remaining modules to make sure prereqs are loaded beforehand as needed.
* Correct bug with hpss module in module_base.jet.lua
Refs #479
commit 5ac68361917e81555b13e0a2b160f2f7546b8fb4
Author: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
Date: Tue Feb 14 12:00:43 2023 -0500
Update S4 environment and module files (#1303)
Updates the S4 module file and environment file to keep up to date with the current develop branch. Fixes #1297. Addresses one bug in #1195.
This also increases the memory request for the ediag job. During testing, that job initially failed. For the 5 cycles run between 2022051500 - 2022051600, the largest memory footprint was ~26.5GB.
commit 1040216d8a4efb9955efecebf59775e91d8845e2
Author: Cory Martin <cory.r.martin@noaa.gov>
Date: Fri Feb 10 17:16:44 2023 -0500
Add in initial 3DVar aerosol DA cycling capability (#1106)
This PR adds an initial 3DVar aerosol DA cycling capability to support scientific development and testing towards an operationally viable candidate system for aerosol DA for GFSv17/GEFSv13.
This PR includes the following:
- Three new j-jobs and ex-scripts for aeroanlinit, aeroanlrun, and aeroanlfinal
- modifies the rocoto scripts that call the above j-jobs to actually call them
- makes modification to the config.resources and config.aeroanl files
- Introduces an object-oriented python structure to initialize/finalize the aerosol analysis with the intention of eventually using the top-level classes for other analysis jobs (soca, atm, land)
Closes #982
commit cc54b8cdab3b83b20c866bbaa5dba0004dae5425
Author: Neil Barton <103681022+NeilBarton-NOAA@users.noreply.github.com>
Date: Fri Feb 10 17:15:16 2023 -0500
Fix name of MOM6 n restarts for future cycle points (#1307)
Script fix for C384 O0.25 3DVAR cycling. Original scripts had wrong naming structure for MOM *res${n}.nc restart files.
Refs: #947, #1289
commit d8c1bd5dfb6b2654b5b8c5121af68f7473fac26e
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Fri Feb 10 12:26:30 2023 -0500
Update RTD GFS operational version to v16.3.6 (#1305)
Update the status of operations to the newly implemented v16.3.6 version on the read-the-docs main page.
Refs #1278
commit 9d11e1e871639523813fcc606aab40f1a6a8103f
Author: Guillaume Vernieres <guillaume.vernieres@noaa.gov>
Date: Tue Feb 7 21:58:29 2023 -0500
Stop wiping data at beginning of bmat vrfy j-job (#1302)
The ocean b-matrix verification job relies on data from the b-matrix job still residing in $DATA, but that directory was being wiped when this job began. Now setting WIPE_DATA to NO to prevent the deletion.
Closes GDASApp/issues/318
commit ae9d140273df9f78b4ae7d64e441e31f7b0d9e10
Author: Kate Friedman <kate.friedman@noaa.gov>
Date: Tue Feb 7 14:56:25 2023 -0500
Add back in module load block in coupled_ic.sh (#1300)
- Resolve bug introduced from removal of module load script block in coupled_ic.sh.
- Add block back in for now and then address errors produced by it more appropriately.
commit 2615fff62853769f1cc5beb5da49a70cb0ad267d
Author: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Date: Mon Feb 6 15:24:32 2023 -0500
Turn off differential pylint (#1296)
Turns off the differential pylint test until it can be evaluated further.
The YAML block is left in commented form in case we wish to turn it back
on later.
commit bdb0db77fdb3cc93a2eae981843499cd38724ba5
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Fri Feb 3 11:13:18 2023 -0500
Purge ICSDIR (#1295)
This PR:
- updates `coupled_ic.sh` job to copy initial conditions from `BASE_CPLIC` directly to `ROTDIR` in the names that the workflow expects and conforms to the naming convention within the workflow
- obsoletes the need and use of `ICSDIR` in forecast-only experiments which served as a intermediate space for staging initial conditions.
- updates the documentation section for `forecast-only`
This is a non-breaking change.
A change in documentation is required as the instructions for setting up the coupled forecast-only experiment no longer needs to pass the argument `--icsdir` to `setup_expt.py`.
Fixes #1276
commit 1b0905c4f25e793a5a210ee0bfdb737bb5136c04
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Thu Feb 2 14:42:18 2023 -0500
Deprecate `FDATE` (#1294)
There used to be a fringe functionality where the `gdas.tHHz.radstat` could be ignored in the first full cycle `gdasanal` task.
All cycled experiments that start with a cold-start or warm-start must have initial bias correction coefficients (the so called `gdas.tHHz.abias` files) and the `gdas.tHHz.radstat` file.
Experts may customize their configurations if they choose to setup without a `gdas.tHHz.radstat` file thereby initializing the radiance diagnostics. They would anyways know what they are doing.
There are no updates in documentation as this was never an advertised mode of starting an experiment.
Closes #1005
commit 219c23b182ef1aee845faad19f4b6c1c0fe817be
Author: Rahul Mahajan <aerorahul@users.noreply.github.com>
Date: Wed Feb 1 10:36:25 2023 -0500
Atmosphere cycling with a Coupled model (#1274)
This PR enables:
- cycling the atmosphere with GSI using the coupled model (S2S) intended for WCDA development
- allows staging of cycled IC's for coupled/atm-only model (in warm start and cold start)
- updates dependencies in the XML to account for coupled model tasks (currently disable in cycled mode due to inadequacies in `ocnpost` jobs)
This PR also:
- disables `GLDAS` by default as it is expected to be deprecated. Enthusiastic developers may turn it `ON`
- allows a tiny flexibility for DA developers to turn ON 3Dvar automatically if `nens = 0`.
Also in this PR:
- `diag_table_da` was updated to include instantaneous ocean fields. When running the model in `APP=ATM`, these entries from the `diag_table_da` are ignored.
- `diag_table` was also updated to include coupled fields from `diag_table_cpl`. This is a step towards unifying the `diag_table`. There will be more work done in this area in the near future.
- `MOM_input_template_500` is added for the 5 degree ocean configuration.
This PR was built on initial work from @guillaumevernieres and @NeilBarton-NOAA
@guillaumevernieres provided initial conditions
@NeilBarton-NOAA and @guillaumevernieres both provided updates to the scripts that were used in the creation of this PR.
The following commands are used to setup and configure the cycled experiment with S2S model configuration.
@guillaumevernieres @NeilBarton-NOAA @DeniseWorthen and @junwang-noaa provided valuable assistance in debugging the coupled model failures.
To setup and run a coupled model, cycled with atmosphere test at C48 atmosphere 5 degree ocean/ice resolution:
```
# Setup experiment and COMROT directories. Copy initial conditions to COMROT.
./setup_expt.py cycled --expdir <path_to_expdir> --comrot <path_to_comrot> --idate 202103231…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The new fracoro data should be used for all new UFS applications no matter if it uses frac_grid or not.
Most problems in Issue#863 have been resolved. However, one problem remains, e.g., the latest fix, mask and oro datasets (fracoro) created by Shan/Mike/Helin should work for both fractional and non-fractional grid.
Note that it also requires the changes in UFS_UTILS. An corresponding PR#741 in UFS_UTILS has been created.
Fixes: #863
Dependency: UFS_UTILS PR#741
Type of change
How Has This Been Tested?
The Forecast-only tests were successful on Orion. The tests are: Atmosphere-only C96 for FRAC_GRID on and off, respectively; Coupled c384 for GRAC_GRID on and off, respectively. However, the test for GLDAS is not done.