Add ability to process ocean/ice products specific to GEFS #2561

EricSinsky-NOAA · 2024-05-02T13:59:15Z

Description

This PR begins to add the capability to produce GEFSv13 ocean and ice products in the global-workflow according to stakeholder requirements. The following features are added.

An oceanice prod yaml file has been added to address the ocean and ice products specific to GEFSv13.
The rocoto dependencies and config.base for GEFS have also been modified to allow for 24-hour averaged ocean and ice output.
Various scripts have been modified to allow for ocean and ice output frequencies of 24 hours.
FHOUT_OCNICE has been split into two variables called FHOUT_OCN and FHOUT_ICE. The same has been done for FHOUT_OCNICE_GFS.

Refs #1878

Type of change

New feature (adds functionality)

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

These changes have been tested using the EP5d ICs on WCOSS2. Additional testing will be performed using the replay ICs from the changes in PR #2559.

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing tests pass with my changes
I have made corresponding changes to the documentation if necessary

The config.base and gefs_tasks files have been modified so that ocean/ice GEFS hist files are 24-hour averages instead of 6-hour averages.

The ocean/ice_products yaml file has been modified so that 24-hour averaged products are post-processed instead of 6-hour averages.

The FHOUT_OCNICE_GFS variable has been added to the oceanice_products yaml file to allow for ocean/ice hist files of any time-averaged length to be inputted into the oceanice product program. Also, the yaml file has been updated so that the ocean/ice product netcdf files are saved for GEFS runs and grib2 files are saved for GFS runs.

A correction regarding the if statements have been added to the updated oceanice yaml file.

The original oceanice yaml file has been restored to a state prior to the changes related to GEFS.

A new oceanice_prod yaml file has been added for GEFS. The oceanice_prod config file for GEFS has been modified so that a GEFS-specific oceanice_prod yaml file is used.

A separate oceanice_prod config file has been added for GEFS.

…cts_config

GwenChen-NOAA · 2024-05-02T18:12:01Z

parm/post/oceanice_products_gefs.yaml

+    cosvar: "cos_rot"
+    angvar: ""
+    {% if model_grid == 'mx025' or model_grid == 'mx050' or model_grid == 'mx100' %}
+    ocean_levels: [5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195, 205, 215, 225.86945, 241.06255, 266.5239, 308.7874, 373.9288, 467.3998, 593.87915, 757.1453, 959.97325, 1204.059, 1489.9735, 1817.1455, 2183.879, 2587.3995, 3023.9285, 3488.7875, 3976.524, 4481.0625]


@EricSinsky-NOAA, ocean_levels will need to round up to integer. Users of grib2 files reported that they have problems plotting decimal depths in grads and plot_data_plane. So, I'm going to update it for GFS. Please change it to:

ocean_levels: [5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195, 205, 215, 226, 241, 267, 309, 374, 467, 594, 757, 960, 1204, 1490, 1817, 2184, 2587, 3024, 3489, 3977, 4481]

GwenChen-NOAA · 2024-05-02T18:14:26Z

parm/post/oceanice_products_gefs.yaml

+    {% if model_grid == 'mx025' or model_grid == 'mx050' or model_grid == 'mx100' %}
+    ocean_levels: [5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195, 205, 215, 225.86945, 241.06255, 266.5239, 308.7874, 373.9288, 467.3998, 593.87915, 757.1453, 959.97325, 1204.059, 1489.9735, 1817.1455, 2183.879, 2587.3995, 3023.9285, 3488.7875, 3976.524, 4481.0625]
+    {% elif model_grid == 'mx500' %}
+    ocean_levels: [5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195, 205, 215, 225.86945, 241.06255, 266.5239]


Same here, please use:

ocean_levels: [5, 15, 25, 35, 45, 55, 65, 75, 85, 95, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195, 205, 215, 226, 241, 267]

@GwenChen-NOAA Thank you for letting me know about this issue regarding the decimal depths. I will make these changes to parm/post/oceanice_products_gefs.yaml.

The oceanice_prod yaml file has been updated to fix an issue that users have encountered regarding plotting decimal depths. Decimal depths in the ocean_levels variable have been rounded up.

WalterKolczynski-NOAA · 2024-05-07T14:34:47Z

Do we really need to wait for #2559 for this? Seems independent.

EricSinsky-NOAA · 2024-05-07T14:49:46Z

Do we really need to wait for #2559 for this? Seems independent.

@WalterKolczynski-NOAA Waiting for PR #2559 is not required for this PR. However, further tests may be needed to make sure these changes are compatible with an f003 start. The current changes in this PR have been successfully tested with the EP5d ICs (f000 start). Should this PR be set to "ready for review", and then we can add any necessary changes in future PRs?

WalterKolczynski-NOAA · 2024-05-07T16:33:04Z

Do we really need to wait for #2559 for this? Seems independent.

@WalterKolczynski-NOAA Waiting for PR #2559 is not required for this PR. However, further tests may be needed to make sure these changes are compatible with an f003 start. The current changes in this PR have been successfully tested with the EP5d ICs (f000 start). Should this PR be set to "ready for review", and then we can add any necessary changes in future PRs?

Yes please.

EricSinsky-NOAA · 2024-05-07T16:49:31Z

Marking this PR as ready for review. Some scripts may need to be modified when PR #2559 is merged, but these changes can be addressed in future PRs.

WalterKolczynski-NOAA

Conditionally approved pending successful completion of CI tests.

aerorahul

thanks!

emcbot · 2024-05-08T20:08:14Z

CI Update on Wcoss2 at 05/08/24 08:08:13 PM
============================================
Cloning and Building global-workflow PR: 2561
with PID: 45053 on host: clogin05

emcbot · 2024-06-11T05:06:42Z

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Tue Jun 11 04:32:22 UTC 2024 on clogin05
---------------------------------------------------
Build: Completed at 06/11/24 05:06:28 AM
Case setup: Completed for experiment C48_ATM_80858bca
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_80858bca
Case setup: Skipped for experiment C48_S2SWA_gefs_80858bca
Case setup: Completed for experiment C48_S2SW_80858bca
Case setup: Completed for experiment C96_atm3DVar_extended_80858bca
Case setup: Skipped for experiment C96_atm3DVar_80858bca
Case setup: Skipped for experiment C96_atmaerosnowDA_80858bca
Case setup: Completed for experiment C96C48_hybatmDA_80858bca
Case setup: Completed for experiment C96C48_ufs_hybatmDA_80858bca

emcbot · 2024-06-11T06:21:15Z

Experiment C48_ATM_80858bca SUCCESS on Wcoss2 at 06/11/24 06:21:15 AM

emcbot · 2024-06-11T06:45:14Z

Experiment C48_S2SW_80858bca SUCCESS on Wcoss2 at 06/11/24 06:45:14 AM

emcbot · 2024-06-11T07:51:23Z

Experiment C96C48_hybatmDA_80858bca SUCCESS on Wcoss2 at 06/11/24 07:51:22 AM

emcbot · 2024-06-11T07:57:10Z

Experiment C96C48_ufs_hybatmDA_80858bca SUCCESS on Wcoss2 at 06/11/24 07:57:09 AM

emcbot · 2024-06-11T13:42:32Z

Experiment C96_atm3DVar_extended_80858bca SUCCESS on Wcoss2 at 06/11/24 01:42:31 PM

emcbot · 2024-06-11T13:51:09Z

All CI Test Cases Passed on Wcoss2:

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_80858bca *** SUCCESS *** at 06/11/24 06:21:15 AM
Experiment C48_S2SW_80858bca *** SUCCESS *** at 06/11/24 06:45:14 AM
Experiment C96C48_hybatmDA_80858bca *** SUCCESS *** at 06/11/24 07:51:22 AM
Experiment C96C48_ufs_hybatmDA_80858bca *** SUCCESS *** at 06/11/24 07:57:09 AM
Experiment C96_atm3DVar_extended_80858bca *** SUCCESS *** at 06/11/24 01:42:31 PM

TerrenceMcGuinness-NOAA · 2024-06-11T15:49:09Z

Connection to Hercules was lost from the controller after the build.

DavidHuber-NOAA · 2024-06-11T16:14:32Z

@TerrenceMcGuinness-NOAA Hercules had an issue with a handful of nodes which have now been taken offline. This may or may not help the issue you are seeing.

emcbot · 2024-06-12T01:51:26Z

CI Passed Hercules at
Built and ran in directory /work2/noaa/stmp/CI/HERCULES/2561

…bal-workflow into feature/move_jcb * 'feature/move_jcb' of https://github.com/danholdaway/global-workflow: Add COM template for JEDI obs (NOAA-EMC#2678) Link both global-nest fix files and non-nest ones at the same time (NOAA-EMC#2632) Update ufs-weather-model (NOAA-EMC#2663) Add ability to process ocean/ice products specific to GEFS (NOAA-EMC#2561) Update cleanup job to use COMIN/COMOUT (NOAA-EMC#2649) Add overwrite to creat experiment in BASH CI (NOAA-EMC#2676) Add handling to select CRTM cloud optical table based on cloud scheme and update calcanal_gfs.py (NOAA-EMC#2645) Update RDHPCS Hera resource for `eupd` task (NOAA-EMC#2636)

* origin/develop: Add observation preparation job for aerosols DA to workflow (NOAA-EMC#2624) Remove ocean daily files (NOAA-EMC#2689) Update Jenkinsfile Add Hercules-EMC to the Jenkins configurable parameter list (NOAA-EMC#2685) Update gdas.cd and gsi_utils hashes (NOAA-EMC#2641) Add ability to use GEFS replay ICs (NOAA-EMC#2559) Replace `sleep` with `wait_for_file` (NOAA-EMC#2586) Add COM template for JEDI obs (NOAA-EMC#2678) Link both global-nest fix files and non-nest ones at the same time (NOAA-EMC#2632) Update ufs-weather-model (NOAA-EMC#2663) Add ability to process ocean/ice products specific to GEFS (NOAA-EMC#2561) Update cleanup job to use COMIN/COMOUT (NOAA-EMC#2649) Add overwrite to creat experiment in BASH CI (NOAA-EMC#2676) Add handling to select CRTM cloud optical table based on cloud scheme and update calcanal_gfs.py (NOAA-EMC#2645) Refs NOAA-EMC#2475

aerorahul · 2024-06-26T15:34:13Z

ush/check_ice_netcdf.sh

@@ -0,0 +1,38 @@
+#! /usr/bin/env bash


@EricSinsky-NOAA
I just noticed this script was added after an approval review was made on this PR and it slipped from being reviewed.
The function implemented here is the datadep with age that Rocoto already provides.
Besides, this is completely hardwired for GEFS options.
The function is checking the age of a file.
The file happens to be a ice netcdf file.
We will need to implement this via the proper rocoto means of checking if a file exists and if it has been present for 2 mins.

EricSinsky-NOAA added 8 commits April 26, 2024 15:58

Make ocean/ice GEFS hist files 24-hr averages

eb8a425

The config.base and gefs_tasks files have been modified so that ocean/ice GEFS hist files are 24-hour averages instead of 6-hour averages.

Produce 24-hour ocean/ice products for GEFS

a6d31ba

The ocean/ice_products yaml file has been modified so that 24-hour averaged products are post-processed instead of 6-hour averages.

Add correction to updated oceanice yaml file

c2b4a58

A correction regarding the if statements have been added to the updated oceanice yaml file.

Restore original oceanice yaml file

c4d21dc

The original oceanice yaml file has been restored to a state prior to the changes related to GEFS.

Add oceanice_prod yaml file for GEFS

069cd86

A new oceanice_prod yaml file has been added for GEFS. The oceanice_prod config file for GEFS has been modified so that a GEFS-specific oceanice_prod yaml file is used.

Make oceanice_prod config files separate

a93bb34

A separate oceanice_prod config file has been added for GEFS.

Merge remote-tracking branch 'origin/develop' into feature/gefs_produ…

31042bc

…cts_config

EricSinsky-NOAA changed the title ~~Add ability to process ocean and ice products specific to GEFSv13~~ Add ability to process ocean/ice products specific to GEFS May 2, 2024

GwenChen-NOAA reviewed May 2, 2024

View reviewed changes

Update oceanice_prod yaml for GEFS

c6c24a9

The oceanice_prod yaml file has been updated to fix an issue that users have encountered regarding plotting decimal depths. Decimal depths in the ocean_levels variable have been rounded up.

EricSinsky-NOAA marked this pull request as ready for review May 7, 2024 16:49

WalterKolczynski-NOAA approved these changes May 7, 2024

View reviewed changes

WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels May 8, 2024

aerorahul approved these changes May 8, 2024

View reviewed changes

emcbot added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels May 8, 2024

emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Jun 11, 2024

EricSinsky-NOAA mentioned this pull request Jun 11, 2024

Missing CICE hist files when cycle is not t00z and FHOUT_GFS_ICE is 24 #2674

Open

TerrenceMcGuinness-NOAA added CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Jun 11, 2024

WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules and removed CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed labels Jun 11, 2024

WalterKolczynski-NOAA approved these changes Jun 12, 2024

View reviewed changes

WalterKolczynski-NOAA merged commit 15eaf35 into NOAA-EMC:develop Jun 12, 2024
9 of 10 checks passed

EricSinsky-NOAA deleted the feature/gefs_products_config branch June 12, 2024 12:10

aerorahul reviewed Jun 26, 2024

View reviewed changes

aerorahul mentioned this pull request Jun 26, 2024

Update dependency check for ice netcdf file for GEFS #2721

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to process ocean/ice products specific to GEFS #2561

Add ability to process ocean/ice products specific to GEFS #2561

EricSinsky-NOAA commented May 2, 2024 •

edited

Loading

GwenChen-NOAA May 2, 2024

GwenChen-NOAA May 2, 2024

EricSinsky-NOAA May 2, 2024 •

edited

Loading

WalterKolczynski-NOAA commented May 7, 2024

EricSinsky-NOAA commented May 7, 2024

WalterKolczynski-NOAA commented May 7, 2024

EricSinsky-NOAA commented May 7, 2024

WalterKolczynski-NOAA left a comment

aerorahul left a comment

emcbot commented May 8, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

TerrenceMcGuinness-NOAA commented Jun 11, 2024

DavidHuber-NOAA commented Jun 11, 2024

emcbot commented Jun 12, 2024

aerorahul Jun 26, 2024

Add ability to process ocean/ice products specific to GEFS #2561

Add ability to process ocean/ice products specific to GEFS #2561

Conversation

EricSinsky-NOAA commented May 2, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

GwenChen-NOAA May 2, 2024

Choose a reason for hiding this comment

GwenChen-NOAA May 2, 2024

Choose a reason for hiding this comment

EricSinsky-NOAA May 2, 2024 • edited Loading

Choose a reason for hiding this comment

WalterKolczynski-NOAA commented May 7, 2024

EricSinsky-NOAA commented May 7, 2024

WalterKolczynski-NOAA commented May 7, 2024

EricSinsky-NOAA commented May 7, 2024

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

aerorahul left a comment

Choose a reason for hiding this comment

emcbot commented May 8, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

emcbot commented Jun 11, 2024

TerrenceMcGuinness-NOAA commented Jun 11, 2024

DavidHuber-NOAA commented Jun 11, 2024

emcbot commented Jun 12, 2024

aerorahul Jun 26, 2024

Choose a reason for hiding this comment

EricSinsky-NOAA commented May 2, 2024 •

edited

Loading

EricSinsky-NOAA May 2, 2024 •

edited

Loading