[develop] Add process bufr task.

[danielabdi-noaa]

DESCRIPTION OF CHANGES:

Add process_bufr task from RRFS_dev1.
Note that there is no workflow so this PR can not be run independently until then.
However, it did successfully run for both spinup/production cycles in PR #540.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

TESTS CONDUCTED:

• hera.intel
  Run fundamental tests. Also run conus 3km test case from RRFS_dev1 successfully as shown in the run in PR [develop] Incorporate RRFS_dev1 workflow to SRW and import most tasks #540

• orion.intel

• cheyenne.intel

• cheyenne.gnu

• gaea.intel

• jet.intel

• wcoss2.intel

• NOAA Cloud (indicate which platform)

• Jenkins

• fundamental test suite

• comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

• BUFR Processing tasks are needed for RRFS #486

CHECKLIST

• My code follows the style guidelines in the Contributor's Guide
• I have performed a self-review of my own code using the Code Reviewer's Guide
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes do not require updates to the documentation (explain).
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

• Work In Progress
• bug
• enhancement
• documentation
• release
• high priority
• run_ci
• run_we2e_fundamental_tests
• run_we2e_comprehensive_tests
• Needs Cheyenne test
• Needs Jet test
• Needs Hera test
• Needs Orion test
• help wanted

CONTRIBUTORS (optional):

[MichaelLueken]

@danielabdi-noaa Thanks for creating a separate PR to integrate the changes you made for processing bufr files from PR #540! Overall, everything looks good to me!

I have noted one minor spelling issue in scripts/exregional_process_bufr.sh. I also left a comment in the same script about the placement of run_lightning (suggested moving it from line 144 to line 146 to bring it into alignment with run_cloud and run_metar further in the script).

[MichaelLueken]

@danielabdi-noaa Thank you very much for addressing my comments! Approving now.

[christinaholtNOAA]

Thanks for breaking this out, @danielabdi-noaa.

[MichaelLueken]

@danielabdi-noaa With the update from hash 10574e9, a minor spelling error was introduced in ush/config_defaults.yaml (loacated instead of located).

[MichaelLueken]

@danielabdi-noaa While looking at PR #540, it doesn't look like the GSD_RAP13km was added to ush/predef_grid_params.yaml in order to make the workflow run. Should we move forward with supporting this new grid, or should we go ahead and remove it from scripts/exregional_process_bufr.sh and ush/predef_grid_params.yaml?

[mkavulich]

A few questions and suggested changes

[danielabdi-noaa]

@mkavulich @christinaholtNOAA I believe i have addressed your concerns. Let me know if I missed something.

[mkavulich]

Thanks!

[christinaholtNOAA]

LGTM. Just one minor question below.

[MichaelLueken]

@danielabdi-noaa The Jenkins tests on Cheyenne, Gaea, and Orion successfully passed. Jenkins is unable to communicate with Jet, which led to the Jet tests being aborted. The WE2E fundamental tests were manually run on Jet and all the tests successfully passed. Is this work ready to be merged, or should it wait until PR #647 has been merged first (checking due to the dependency on PR #647 above)? Since this work doesn't affect the FV3LAM_wfow.xml file, I can move forward with the merge. Thanks!

[danielabdi-noaa]

@MichaelLueken I think merging PR #647 first is better. Also the yaml workflow PR would necessitate changes in this PR too. Thanks!

[christinaholtNOAA]

@MichaelLueken We discussed in our tag up yesterday that we would like to go ahead and get this PR through as one of our higher priorities. Let's not hold it waiting for #647 as we previously considered. Thanks!

[MichaelLueken]

@EdwardSnyder-NOAA Just checking to see if there are any other issues that you see with respect to moving forward with these changes. I have gone ahead and relaunched the Jenkins tests. So long as they continue to pass, then I will move forward with these changes. Thanks!

[EdwardSnyder-NOAA]

@EdwardSnyder-NOAA Just checking to see if there are any other issues that you see with respect to moving forward with these changes. I have gone ahead and relaunched the Jenkins tests. So long as they continue to pass, then I will move forward with these changes. Thanks!

No other issues I see. Proceed with testing and merging.

[venitahagerty]

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch1/BMC/zrtrr/rrfs_ci/autoci/pr/1264903189/20230412185015/ufs-srweather-app
Build was Successful
Rocoto jobs started
Experiment done: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
2023-04-12 19:33:47 +0000 :: hfe07 :: This cycle is complete: Success
Experiment failed: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-04-12 19:46:09 +0000 :: hfe07 :: Task run_MET_GridStat_vx_REFC_mem000, jobid=43764373, in state DEAD (FAILED), ran for 17.0 seconds, exit status=256, try=1 (of 1)
Experiment failed: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-04-12 19:46:09 +0000 :: hfe07 :: Task run_MET_GridStat_vx_RETOP_mem000, jobid=43764374, in state DEAD (FAILED), ran for 17.0 seconds, exit status=256, try=1 (of 1)
Experiment failed: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR
2023-04-12 19:46:25 +0000 :: hfe07 :: Task run_MET_PcpCombine_fcst_APCP01h_mem000, jobid=43764384, in state DEAD (FAILED), ran for 15.0 seconds, exit status=256, try=2 (of 2)
Experiment done: grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
2023-04-12 19:31:43 +0000 :: hfe07 :: This cycle is complete: Success
Experiment done: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
2023-04-12 19:28:01 +0000 :: hfe07 :: This cycle is complete: Success
Experiment done: grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional_plot
2023-04-12 19:29:02 +0000 :: hfe07 :: This cycle is complete: Success
Experiment done: pregen_grid_orog_sfc_climo
2023-04-12 19:27:15 +0000 :: hfe07 :: This cycle is complete: Success
Experiment done: grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v15p2
2023-04-12 19:26:28 +0000 :: hfe07 :: This cycle is complete: Success
If test failed, please make changes and add the following label back:
ci-hera-intel-WE

[MichaelLueken]

The Jenkins tests have successfully completed. For those who can access the Jenkins pipeline, it can be found - https://jenkins.epic.oarcloud.noaa.gov/blue/organizations/jenkins/ufs-srweather-app%2Fpipeline/detail/PR-636/2/pipeline. both manual and ci-hera-intel-WE tests were run and both passed (with the exception of PcpCombine due to issue #688). Moving forward with merging this work.