Add ability to use GEFS replay ICs #2559

NeilBarton-NOAA · 2024-05-01T17:27:10Z

Description

The PR allows the use of ICs from PSL's replay analysis. These replay ICs will be used for GEFS reforecasting and SFS. Two main changes are associated with these updates: (1) replay ICs being valid at 3Z, and (2) the use of warm starts.

This addresses #1838

Type of change

New feature

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

GEFS Forecast-only on Hera using replay ICs
GEFS Forecast-only on Hera using warm start ICs
GEFS Forecast-only on Hera using cold start ICs
GFS Forecast-only on Hera CI test

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing tests pass with my changes
I have made corresponding changes to the documentation if necessary

updates to have correct ICs in com for IAU after a cold start first half cycle

WalterKolczynski-NOAA · 2024-05-01T17:29:30Z

scripts/exgfs_wave_post_pnt.sh

@@ -247,8 +247,9 @@ source "${USHgfs}/preamble.sh"
        -e "s/FORMAT/F/g" \
                               ww3_outp_spec.inp.tmpl > ww3_outp.inp

-    ${NLN} mod_def.$waveuoutpGRD mod_def.ww3
-    HMS="${cyc}0000"
+    ln -s mod_def.$waveuoutpGRD mod_def.ww3


Please retain the use of ${NLN}

scripts/exglobal_stage_ic.sh

WalterKolczynski-NOAA · 2024-05-01T18:01:03Z

scripts/exglobal_stage_ic.sh

@@ -149,7 +171,7 @@ for MEMDIR in "${MEMDIR_ARRAY[@]}"; do

  # Stage the WW3 initial conditions to ROTDIR (warm start; TODO: these should be placed in $RUN.$gPDY/$gcyc)
  if [[ "${DO_WAVE:-}" = "YES" ]]; then
-    YMD=${PDY} HH=${cyc} declare_from_tmpl COM_WAVE_RESTART
+    YMD=${gPDY} HH=${gcyc} declare_from_tmpl COM_WAVE_RESTART


If this is now going to use the previous cycle's date, the variable name should be changed to COM_WAVE_RESTART_PREV to be consistent.

Also, I believe this will break the existing GEFS test.

WalterKolczynski-NOAA · 2024-05-01T18:03:48Z

scripts/exglobal_stage_ic.sh

+  # Atmosphere Perturbation Files (usually used with replay ICS)
+  # Extra zero on MEMDIR ensure we have a number even if the string is empty
+  if (( 0${MEMDIR:3} > 0 )) && [[ "${USE_ATM_PERTURB_FILES:-false}" == "true" ]]; then
+      src="${BASE_CPLIC}/${CPL_OCNIC:-}/${PDY}${cyc}/${MEMDIR}/atmos/${PDY}.${cyc}0000.fv3_perturbation.nc"


Why are atmosphere perturbations coming from the ocean IC directory? Doesn't matter right now since they are identical, but this should use CPL_ATMIC.

Error in coping and pasting. Updated

WalterKolczynski-NOAA · 2024-05-01T18:05:11Z

ush/forecast_postdet.sh

@@ -97,16 +100,28 @@ FV3_postdet() {
        local model_start_time="${current_cycle}"
        local model_current_time="${current_cycle}"
      fi
+      if [[ "${USE_REPLAY_ICS}" != "true" ]]; then
      rm -f "${DATA}/INPUT/coupler.res"


Please indent this block (except the EOF)

WalterKolczynski-NOAA · 2024-05-01T18:05:27Z

ush/forecast_postdet.sh

@@ -490,6 +505,7 @@ MOM6_out() {
  esac

  # Copy MOM6 restarts at the end of the forecast segment to COM for RUN=gfs|gefs
+  if [[ "${USE_REPLAY_ICS}" != "true" ]]; then
  local restart_file


Please indent this block.

WalterKolczynski-NOAA · 2024-05-01T18:05:39Z

ush/forecast_postdet.sh

@@ -590,6 +607,7 @@ CICE_out() {
  ${NCP} "${DATA}/ice_in" "${COM_CONF}/ufs.ice_in"

  # Copy CICE restarts at the end of the forecast segment to COM for RUN=gfs|gefs
+  if [[ "${USE_REPLAY_ICS}" != "true" ]]; then
  local seconds source_file target_file


Please indent this block.

WalterKolczynski-NOAA · 2024-05-01T18:05:56Z

ush/forecast_postdet.sh

@@ -729,6 +748,7 @@ CMEPS_out() {
  echo "SUB ${FUNCNAME[0]}: Copying output data for CMEPS mediator"

  # Copy mediator restarts at the end of the forecast segment to COM for RUN=gfs|gefs
+  if [[ "${USE_REPLAY_ICS}" != "true" ]]; then
  echo "Copying mediator restarts for 'RUN=${RUN}' at ${forecast_end_cycle}"


Please indent this block.

WalterKolczynski-NOAA · 2024-05-01T18:10:28Z

Does this resolve #1838 or is it only part of it? The details were never filled in to that issue to know what needs to be done.

Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>

emcbot · 2024-06-12T21:19:30Z

CI Passed Hera at
Built and ran in directory /scratch1/NCEPDEV/global/CI/2559

emcbot · 2024-06-12T21:40:20Z

CI Update on Wcoss2 at 06/12/24 09:40:18 PM
============================================
Cloning and Building global-workflow PR: 2559
with PID: 181360 on host: clogin05

emcbot · 2024-06-12T22:53:46Z

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Wed Jun 12 21:57:56 UTC 2024 on clogin05
---------------------------------------------------
Build: Completed at 06/12/24 10:53:28 PM
Case setup: Completed for experiment C48_ATM_7a383978
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_7a383978
Case setup: Skipped for experiment C48_S2SWA_gefs_7a383978
Case setup: Completed for experiment C48_S2SW_7a383978
Case setup: Completed for experiment C96_atm3DVar_extended_7a383978
Case setup: Skipped for experiment C96_atm3DVar_7a383978
Case setup: Skipped for experiment C96_atmaerosnowDA_7a383978
Case setup: Completed for experiment C96C48_hybatmDA_7a383978
Case setup: Completed for experiment C96C48_ufs_hybatmDA_7a383978

emcbot · 2024-06-13T00:36:16Z

Experiment C48_ATM_7a383978 SUCCESS on Wcoss2 at 06/13/24 12:36:15 AM

emcbot · 2024-06-13T01:06:11Z

Experiment C48_S2SW_7a383978 SUCCESS on Wcoss2 at 06/13/24 01:06:10 AM

emcbot · 2024-06-13T01:36:19Z

Experiment C96C48_hybatmDA_7a383978 SUCCESS on Wcoss2 at 06/13/24 01:36:18 AM

emcbot · 2024-06-13T01:48:18Z

Experiment C96C48_ufs_hybatmDA_7a383978 SUCCESS on Wcoss2 at 06/13/24 01:48:17 AM

emcbot · 2024-06-13T07:27:30Z

Experiment C96_atm3DVar_extended_7a383978 SUCCESS on Wcoss2 at 06/13/24 07:27:30 AM

emcbot · 2024-06-13T07:30:09Z

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_7a383978 *** SUCCESS *** at 06/13/24 12:36:15 AM
Experiment C48_S2SW_7a383978 *** SUCCESS *** at 06/13/24 01:06:10 AM
Experiment C96C48_hybatmDA_7a383978 *** SUCCESS *** at 06/13/24 01:36:18 AM
Experiment C96C48_ufs_hybatmDA_7a383978 *** SUCCESS *** at 06/13/24 01:48:17 AM
Experiment C96_atm3DVar_extended_7a383978 *** SUCCESS *** at 06/13/24 07:27:30 AM

* upstream/develop: Add Hercules-EMC to the Jenkins configurable parameter list (NOAA-EMC#2685) Update gdas.cd and gsi_utils hashes (NOAA-EMC#2641) Add ability to use GEFS replay ICs (NOAA-EMC#2559) Replace `sleep` with `wait_for_file` (NOAA-EMC#2586)

* origin/develop: Add observation preparation job for aerosols DA to workflow (NOAA-EMC#2624) Remove ocean daily files (NOAA-EMC#2689) Update Jenkinsfile Add Hercules-EMC to the Jenkins configurable parameter list (NOAA-EMC#2685) Update gdas.cd and gsi_utils hashes (NOAA-EMC#2641) Add ability to use GEFS replay ICs (NOAA-EMC#2559) Replace `sleep` with `wait_for_file` (NOAA-EMC#2586) Add COM template for JEDI obs (NOAA-EMC#2678) Link both global-nest fix files and non-nest ones at the same time (NOAA-EMC#2632) Update ufs-weather-model (NOAA-EMC#2663) Add ability to process ocean/ice products specific to GEFS (NOAA-EMC#2561) Update cleanup job to use COMIN/COMOUT (NOAA-EMC#2649) Add overwrite to creat experiment in BASH CI (NOAA-EMC#2676) Add handling to select CRTM cloud optical table based on cloud scheme and update calcanal_gfs.py (NOAA-EMC#2645) Refs NOAA-EMC#2475

NeilBarton-NOAA and others added 23 commits April 24, 2024 18:57

running with replay ICs

65f9449

update gdas app hash and turn on C48mx500_3DVarAOWCDA test on hera

36316e1

Bug fix for NOAA-EMC#2528 from aerorahul

06dae2f

add additional update from before

2cdc9aa

remove line that made things "work" but for the wrong reasons

1cfbe24

wave updates

2f09626

Merge remote-tracking branch 'origin/replay_ics' into SFS_C96X100

aa9862f

Merge remote-tracking branch 'origin/develop' into replay_ics

e9952b4

small updates for forecast_postdet.sh

9b5fe8b

Merge remote-tracking branch 'origin/replay_ics' into SFS_C96X100

b223067

updates to run with 3 instead of 4 mom restarts

1ba3cc1

updates to have correct ICs in com for IAU after a cold start first half cycle

Merge remote-tracking branch 'EMC/develop' into feature/c384wcda

caf09f6

update CICE restart copying

a9266f5

updates for replay ICS after merge of restart cap

8e11642

Merge remote-tracking branch 'origin/develop' into replay_ics

cee7a65

Merge remote-tracking branch 'origin/replay_ics' into SFS_C96X100

70abb0c

Merge remote-tracking branch 'EMC/develop' into feature/c384wcda

31b9481

update gdas app hash

a67d1b6

update gdas

1ffd164

updates for GFS CI

eb82d40

addition for GFS test

79046c1

fixes before PR

080cc9f

Merge remote-tracking branch 'origin/replay_ics' into SFS_C96X100

9e141ea

WalterKolczynski-NOAA requested changes May 1, 2024

View reviewed changes

WalterKolczynski-NOAA mentioned this pull request May 1, 2024

Stage initial conditions for cycled and forecast-only via stage_ic job. #2475

Open

WalterKolczynski-NOAA changed the title ~~Replay ics~~ Add ability to use GEFS replay ICs May 1, 2024

JessicaMeixner-NOAA and others added 3 commits May 1, 2024 15:37

Merge branch 'NOAA-EMC:develop' into feature/c384wcda

d709db5

Update scripts/exglobal_stage_ic.sh

71f89d7

Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>

addressing PR comments

8878810

WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Jun 12, 2024

emcbot added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Jun 12, 2024

emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Jun 13, 2024

TerrenceMcGuinness-NOAA removed the CI-Hercules-Running **Bot use only** CI testing on Hercules for this PR is in-progress label Jun 13, 2024

WalterKolczynski-NOAA merged commit 34155fb into NOAA-EMC:develop Jun 13, 2024
7 of 10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to use GEFS replay ICs #2559

Add ability to use GEFS replay ICs #2559

NeilBarton-NOAA commented May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

NeilBarton-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA May 1, 2024

WalterKolczynski-NOAA commented May 1, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

Add ability to use GEFS replay ICs #2559

Add ability to use GEFS replay ICs #2559

Conversation

NeilBarton-NOAA commented May 1, 2024

Description

Type of change

Change characteristics

How has this been tested?

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented May 1, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 12, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024

emcbot commented Jun 13, 2024