Fixing revert "Creating Yaml for GEFSv13 Restarts and increments"" #3723

AntonMFernando-NOAA · 2025-05-21T20:21:35Z

Fixes the revert #3722
Continue #3622

Resolves #3583
Resolves #3621

…OAA-EMC#3722)" This reverts commit fa84891.

…stage_ic

AntonMFernando-NOAA · 2025-05-22T15:10:56Z

Ran the stage_ic job for 12 available tests. All passed. Tests can be found at with a tag "all": /lfs/h2/emc/global/noscrub/anton.fernando/RUNTESTS/EXPDIR

KateFriedman-NOAA · 2025-05-22T15:38:43Z

dev/ci/cases/pr/C48_S2SWA_gefs_RT.yaml

+  start: cold
+  comroot: {{ 'RUNTESTS' | getenv }}/COMROOT
+  expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR
+  icsdir: /lfs/h2/emc/ens/noscrub/eric.sinsky/RETRO_ICS


This should not be a personal folder. Are these ICs that the GEFS team would like staged under the global group account on all supported platforms?

@KateFriedman-NOAA A decision is still being made on exactly what ICs are needed for the control member. An issue will be opened once we know exactly what needs to be staged. For now, I think icsdir can be removed from this yaml as this case is not running in the CI on any platform.

KateFriedman-NOAA · 2025-05-22T15:41:16Z

parm/stage/atmosphere_cold_RT.yaml.j2

+        - "{{ COMOUT_ATMOS_RESTART_PREV_MEM }}"
+        - "{{ COMOUT_ATMOS_ANALYSIS_MEM }}"
+    copy:
+        {% set DIAU_previous_file = ICSDIR ~ '/' ~ 'gdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/analysis/atmos/gdas.t' ~ previous_cycle_HH ~ 'z.atma003.ensres.nc' %}


What does DIAU mean there? DO_IAU?

DIAU_ is just a poor choice for a prefix. This confusion will get cleared out when a decision is made on the initialization of the control member

…thub.com/AntonMFernando-NOAA/global-workflow into revert-3722-revert-3622-feature/stage_ic

aerorahul · 2025-05-22T18:37:23Z

parm/stage/atmosphere_cold_RT.yaml.j2

+    copy:
+        {% set DIAU_previous_file = ICSDIR ~ '/' ~ 'gdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/analysis/atmos/gdas.t' ~ previous_cycle_HH ~ 'z.atma003.ensres.nc' %}
+        {% set DIAU_current_file = ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atminc.nc' %}
+        {% set DIAUNO_current_file= ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atminc.nc' %}


This needs to be

Suggested change

{% set DIAUNO_current_file= ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atminc.nc' %}

{% set DIAUNO_current_file= ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atmanl.nc' %}

aerorahul · 2025-05-22T18:38:21Z

parm/stage/atmosphere_cold_RT.yaml.j2

+        - "{{ COMOUT_ATMOS_RESTART_PREV_MEM }}"
+        - "{{ COMOUT_ATMOS_ANALYSIS_MEM }}"
+    copy:
+        {% set DIAU_previous_file = ICSDIR ~ '/' ~ 'gdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/analysis/atmos/gdas.t' ~ previous_cycle_HH ~ 'z.atma003.ensres.nc' %}


DIAU_ is just a poor choice for a prefix. This confusion will get cleared out when a decision is made on the initialization of the control member

aerorahul · 2025-05-22T18:47:29Z

parm/stage/master_gefs_RT.yaml.j2

+{% else %}
+    {% if REPLAY_ICS %}
+        {% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_end) %}
+    {% else %}
+        {% set model_start_date_current_cycle = current_cycle %}
+    {% endif %}
+{% endif %}


This block is redundant in a RT situation.
There should not be an if-else condition in real time.

aerorahul · 2025-05-22T18:49:30Z

parm/stage/master_gefs_RT.yaml.j2

+{% if DOIAU and MODE == "cycled" %}
+    {% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_begin) %}
+{% else %}
+    {% if REPLAY_ICS %}
+        {% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_end) %}
+    {% else %}
+        {% set model_start_date_current_cycle = current_cycle %}
+    {% endif %}
+{% endif %}


Suggested change

{% if DOIAU and MODE == "cycled" %}

{% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_begin) %}

{% else %}

{% if REPLAY_ICS %}

{% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_end) %}

{% else %}

{% set model_start_date_current_cycle = current_cycle %}

{% endif %}

{% endif %}

{% set model_start_date_current_cycle = current_cycle | add_to_datetime(half_window_begin) %}

Based on the above comment, in RT, GFS will be running w/ IAU and in cycled mode. There is no replay possible for running in RT

aerorahul · 2025-05-22T18:50:58Z

parm/stage/ocean_RT.yaml.j2

+            {% set mem = ENSMEM + offset %}
+            {% set imem = ENSMEM %}
+            {% set mid_cyc = ("%02d" | format(previous_cycle_HH | int + half_window)) ~ "0000" %}
+            {% set restart_mem_path = ICSDIR ~ '/' ~ 'enkfgdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/' ~ 'mem%03d' | format(mem)  ~ '/model/ocean/restart/' ~ previous_cycle_YMD ~ '.' ~ mid_cyc ~ '.MOM.' %}


Suggested change

{% set restart_mem_path = ICSDIR ~ '/' ~ 'enkfgdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/' ~ 'mem%03d' | format(mem) ~ '/model/ocean/restart/' ~ previous_cycle_YMD ~ '.' ~ mid_cyc ~ '.MOM.' %}

{% set restart_mem_path = ICSDIR ~ '/' ~ 'enkfgdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/' ~ 'mem%03d' | format(mem) ~ '/model/ocean/restart/' ~ previous_cycle_YMD ~ '.' ~ mid_cyc ~ '.' %}

Put MOM. in the loop like in the case of control for symmetry

aerorahul · 2025-05-22T18:51:43Z

parm/stage/ocean_RT.yaml.j2

+        - ["{{ file }}", "{{ increment_destination_path }}"]
+
+            {% if path_exists(ICSDIR) %}
+                {% for ftype in ["res.nc", "res_1.nc", "res_2.nc", "res_3.nc"] %}


Suggested change

{% for ftype in ["res.nc", "res_1.nc", "res_2.nc", "res_3.nc"] %}

{% for ftype in ["MOM.res.nc", "MOM.res_1.nc", "MOM.res_2.nc", "MOM.res_3.nc"] %}

Appending MOM. from above here, to look like the control set.

…stage_ic

EricSinsky-NOAA · 2025-05-22T18:39:01Z

dev/ci/cases/pr/C48_S2SWA_gefs_RT.yaml

+  start: cold
+  comroot: {{ 'RUNTESTS' | getenv }}/COMROOT
+  expdir: {{ 'RUNTESTS' | getenv }}/EXPDIR
+  icsdir: /lfs/h2/emc/ens/noscrub/eric.sinsky/RETRO_ICS


This line can be removed.

Suggested change

icsdir: /lfs/h2/emc/ens/noscrub/eric.sinsky/RETRO_ICS

…stage_ic

dev/workflow/rocoto/gefs_tasks.py

scripts/exglobal_stage_ic.py

EricSinsky-NOAA

Just one minor technical suggestion, but not essential. Otherwise, approved pending successful completion of CI testing.

dev/workflow/rocoto/gefs_tasks.py

AntonMFernando-NOAA · 2025-06-05T18:20:09Z

Just one minor technical suggestion, but not essential. Otherwise, approved pending successful completion of CI testing.

Thanks, Eric.

DavidHuber-NOAA

Looks good. Just one minor nitpick and then I think it's good to go for testing.

dev/workflow/rocoto/gefs_tasks.py

DavidHuber-NOAA

Looks good!

Revert "Revert "Creating Yaml for GEFSv13 Restarts and increments" (N…

ba65aac

…OAA-EMC#3722)" This reverts commit fa84891.

AntonMFernando-NOAA changed the title ~~Revert "Revert "Creating Yaml for GEFSv13 Restarts and increments""~~ Fixing revert "Creating Yaml for GEFSv13 Restarts and increments"" May 21, 2025

AntonMFernando-NOAA self-assigned this May 21, 2025

AntonMFernando-NOAA and others added 3 commits May 21, 2025 16:40

modified dev\parm\config\gfs\config.stage_ic.j2

efd8c12

modified dev\parm\config\gfs\config.stage_ic.j2

dde2d31

Merge branch 'NOAA-EMC:develop' into revert-3722-revert-3622-feature/…

b45d780

…stage_ic

AntonMFernando-NOAA marked this pull request as ready for review May 22, 2025 15:08

AntonMFernando-NOAA requested review from DavidHuber-NOAA, KateFriedman-NOAA, TerrenceMcGuinness-NOAA, WalterKolczynski-NOAA and aerorahul as code owners May 22, 2025 15:08

AntonMFernando-NOAA requested review from EricSinsky-NOAA and removed request for TerrenceMcGuinness-NOAA and WalterKolczynski-NOAA May 22, 2025 15:08

KateFriedman-NOAA mentioned this pull request May 22, 2025

Add an Empty Task for the GEFS to Change Resolution of the Control Member IC #3721

Merged

22 tasks

KateFriedman-NOAA reviewed May 22, 2025

View reviewed changes

AntonMFernando-NOAA added 2 commits May 22, 2025 12:14

modified dev\workflow\rocoto\gefs_tasks.py

251f45b

Merge branch 'revert-3722-revert-3622-feature/stage_ic' of https://gi…

ffa0350

…thub.com/AntonMFernando-NOAA/global-workflow into revert-3722-revert-3622-feature/stage_ic

AntonMFernando-NOAA force-pushed the revert-3722-revert-3622-feature/stage_ic branch from 93fa38e to ffa0350 Compare May 22, 2025 17:59

aerorahul requested changes May 22, 2025

View reviewed changes

Merge branch 'NOAA-EMC:develop' into revert-3722-revert-3622-feature/…

eeab472

…stage_ic

EricSinsky-NOAA reviewed May 22, 2025

View reviewed changes

AntonMFernando-NOAA and others added 6 commits May 23, 2025 11:37

Merge branch 'NOAA-EMC:develop' into revert-3722-revert-3622-feature/…

eb93da3

…stage_ic

change repetitive code

0ad4ec7

modified dev\workflow\rocoto\gefs_tasks.py

c9f39a5

modified dev\workflow\rocoto\gefs_tasks.py

a686de3

modified dev\workflow\rocoto\gefs_tasks.py

c1becd4

modifed dev\workflow\rocoto\gefs_tasks.py

bea0068

AntonMFernando-NOAA added 3 commits June 4, 2025 12:37

typo

6494571

change-offline gefs type

262be02

modified parm\stage\master_gefs_RT.yaml.j2

4fec2ee

DavidHuber-NOAA reviewed Jun 4, 2025

View reviewed changes

dev/workflow/rocoto/gefs_tasks.py Outdated Show resolved Hide resolved

modified gefs_tasks.py

7cb3465

DavidHuber-NOAA reviewed Jun 4, 2025

View reviewed changes

scripts/exglobal_stage_ic.py Outdated Show resolved Hide resolved

modified scripts\exglobal_stage_ic.py

e628641

AntonMFernando-NOAA requested review from EricSinsky-NOAA and aerorahul June 4, 2025 18:31

updated dev\ci\cases\pr\C48_S2SWA_gefs_RT.yaml

dd13a97

EricSinsky-NOAA previously approved these changes Jun 5, 2025

View reviewed changes

dev/workflow/rocoto/gefs_tasks.py Outdated Show resolved Hide resolved

modified dev\workflow\rocoto\gefs_tasks.py

15a4af4

AntonMFernando-NOAA dismissed EricSinsky-NOAA’s stale review via 15a4af4 June 5, 2025 18:08

EricSinsky-NOAA previously approved these changes Jun 5, 2025

View reviewed changes

DavidHuber-NOAA requested changes Jun 5, 2025

View reviewed changes

dev/workflow/rocoto/gefs_tasks.py Outdated Show resolved Hide resolved

dev/workflow/rocoto/gefs_tasks.py Outdated Show resolved Hide resolved

modified dev\workflow\rocoto\gefs_tasks.py

bb1a00c

AntonMFernando-NOAA dismissed EricSinsky-NOAA’s stale review via bb1a00c June 5, 2025 18:30

DavidHuber-NOAA approved these changes Jun 5, 2025

View reviewed changes

DavidHuber-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Jun 5, 2025

Merge branch 'develop' into revert-3722-revert-3622-feature/stage_ic

a0ad9da

aerorahul approved these changes Jun 6, 2025

View reviewed changes

Merge branch 'develop' into revert-3722-revert-3622-feature/stage_ic

d982b4f

DavidHuber-NOAA merged commit 2001f36 into NOAA-EMC:develop Jun 6, 2025
5 checks passed

	{% set DIAUNO_current_file= ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atminc.nc' %}
	{% set DIAUNO_current_file= ICSDIR ~ '/' ~ 'gfs.' ~ current_cycle_YMD ~ '/' ~ current_cycle_HH ~ '/analysis/atmos/gfs.t' ~ current_cycle_HH ~ 'z.atmanl.nc' %}

	{% set restart_mem_path = ICSDIR ~ '/' ~ 'enkfgdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/' ~ 'mem%03d' \| format(mem) ~ '/model/ocean/restart/' ~ previous_cycle_YMD ~ '.' ~ mid_cyc ~ '.MOM.' %}
	{% set restart_mem_path = ICSDIR ~ '/' ~ 'enkfgdas.' ~ previous_cycle_YMD ~ '/' ~ previous_cycle_HH ~ '/' ~ 'mem%03d' \| format(mem) ~ '/model/ocean/restart/' ~ previous_cycle_YMD ~ '.' ~ mid_cyc ~ '.' %}

	{% for ftype in ["res.nc", "res_1.nc", "res_2.nc", "res_3.nc"] %}
	{% for ftype in ["MOM.res.nc", "MOM.res_1.nc", "MOM.res_2.nc", "MOM.res_3.nc"] %}

Fixing revert "Creating Yaml for GEFSv13 Restarts and increments"" #3723

Fixing revert "Creating Yaml for GEFSv13 Restarts and increments"" #3723

Uh oh!

Conversation

AntonMFernando-NOAA commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AntonMFernando-NOAA commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EricSinsky-NOAA May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

EricSinsky-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AntonMFernando-NOAA commented Jun 5, 2025

Uh oh!

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AntonMFernando-NOAA commented May 21, 2025 •

edited

Loading

AntonMFernando-NOAA commented May 22, 2025 •

edited

Loading

EricSinsky-NOAA May 22, 2025 •

edited

Loading