HR4 GWD update for FV3 #2290

Qingfu-Liu · 2024-05-19T15:52:54Z

Commit Queue Requirements:

Fill out all sections of this template.
All sub component pull requests have been reviewed by their code managers.
Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
Commit 'test_changes.list' from previous step

Description:

This PR#2290 depends on fv3atm PR##836 and ccpp-physics PR#207. These PRs are created for updating the code of the GWD.

Commit Message:

  * FV3 - 
    * ccpp-physics - 
  * NOAHMP -

Priority:

High: This PR#2290 is created for HR4 and updating GWD code

Git Tracking

UFSWM:

None

Sub component Pull Requests:

FV3: HR4 Gravity Wave Drag Update NOAA-EMC/fv3atm#836
- ccpp-physics: HR4 Gravity Wave Drag Update ccpp-physics#207

UFSWM Blocking Dependencies:

None

Changes

Regression Test Changes (Please commit test_changes.list):

PR Updates/Changes Baselines.

Input data Changes:

There is a new set of orographic data related to this PR#2290 and the new data is described in PR#2670 workflow:
HR4 GWD update NOAA-EMC/global-workflow#2670
This PR#2670 includes changes for four scripts, and new orographic data.
The new orographic data temporarily stored at:
/scratch1/NCEPDEV/global/Qingfu.Liu/git/GWD_SHong/APR2023_SSO
and this new data should replace the old data: /scratch1/NCEPDEV/global/glopara/fix/ugwd/20231027
New orographic input files on hera:
- /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data/INPUT_L127_gfsv17
- /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data48/INPUT_L127_gfsv17
- /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data192/INPUT_L127_gfsv17
- /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data384/INPUT_L127_gfsv17

Library Changes/Upgrades:

No Updates

Testing Log:

grantfirl · 2024-06-03T16:34:05Z

All (@Qingfu-Liu @barlage @JongilHan66 @mdtoyNOAA @cenlinhe @BoYang-NOAA), please look over this list copied from test_changes.list that comes out of the regression tests and confirm that all tests are expected to change results given the changes in NOAA-EMC/fv3atm#836 and ufs-community/ccpp-physics#207).

cpld_control_p8_mixedmode intel
cpld_control_gfsv17 intel
cpld_control_gfsv17_iau intel
cpld_restart_gfsv17 intel
cpld_mpi_gfsv17 intel
cpld_control_sfs intel
cpld_debug_gfsv17 intel
cpld_control_p8 intel
cpld_control_p8.v2.sfc intel
cpld_restart_p8 intel
cpld_control_qr_p8 intel
cpld_restart_qr_p8 intel
cpld_2threads_p8 intel
cpld_decomp_p8 intel
cpld_mpi_p8 intel
cpld_control_ciceC_p8 intel
cpld_control_c192_p8 intel
cpld_restart_c192_p8 intel
cpld_bmark_p8 intel
cpld_restart_bmark_p8 intel
cpld_s2sa_p8 intel
cpld_control_noaero_p8 intel
cpld_control_nowave_noaero_p8 intel
cpld_debug_p8 intel
cpld_debug_noaero_p8 intel
cpld_control_noaero_p8_agrid intel
cpld_control_c48 intel
cpld_warmstart_c48 intel
cpld_restart_c48 intel
cpld_control_p8_faster intel
cpld_control_pdlib_p8 intel
cpld_restart_pdlib_p8 intel
cpld_mpi_pdlib_p8 intel
cpld_debug_pdlib_p8 intel
control_CubedSphereGrid intel
control_CubedSphereGrid_parallel intel
control_latlon intel
control_wrtGauss_netcdf_parallel intel
control_c48 intel
control_c192 intel
control_c384 intel
control_p8 intel
control_p8.v2.sfc intel
control_p8_ugwpv1 intel
control_restart_p8 intel
control_noqr_p8 intel
control_restart_noqr_p8 intel
control_decomp_p8 intel
control_2threads_p8 intel
control_p8_lndp intel
control_p8_rrtmgp intel
control_p8_mynn intel
merra2_thompson intel
control_p8_faster intel
control_CubedSphereGrid_debug intel
control_wrtGauss_netcdf_parallel_debug intel
control_diag_debug intel
control_debug_p8 intel
rrfs_v1beta_debug intel
control_p8_atmlnd_sbs intel
control_p8_atmlnd intel
control_restart_p8_atmlnd intel
control_p8_atmlnd_debug intel
atmwav_control_noaero_p8 intel
atmaero_control_p8 intel
atmaero_control_p8_rad intel
atmaero_control_p8_rad_micro intel
control_c48 gnu
control_p8 gnu
control_p8_ugwpv1 gnu
control_diag_debug gnu
rrfs_v1beta_debug gnu
control_debug_p8 gnu
cpld_control_nowave_noaero_p8 gnu
cpld_control_pdlib_p8 gnu
cpld_debug_pdlib_p8 gnu

mdtoyNOAA · 2024-06-03T17:43:02Z

Looks good

grantfirl · 2024-06-03T18:21:52Z

@Qingfu-Liu Here is the PR into your branch to add RT logs: Qingfu-Liu#1 (also updates FV3 submodule pointer)

update RT logs and FV3 submodule pointer

grantfirl · 2024-06-04T15:41:37Z

@Qingfu-Liu Can we move this out of draft and fill out the description?

Qingfu-Liu · 2024-06-06T12:43:52Z

@Qingfu-Liu Can we move this out of draft and fill out the description?

@grantfirl Thank you very much. I just updated the description and change the draft to PR

jkbk2004 · 2024-06-18T19:11:02Z

@Qingfu-Liu can you sync up branch? We can start working on this pr.

grantfirl · 2024-06-20T07:52:26Z

@jkbk2004 Thanks very much for syncing this. I didn't have permissions to do this on Qingfu's branch. I'm still out of the country until mid next week, although I'll be available off and on throughout the day to merge if necessary if no one else is. Dustin is at the same conference as me.

jkbk2004 · 2024-06-20T12:03:27Z

@Qingfu-Liu @grantfirl @dustinswales @mdtoyNOAA can you check the following error message about mntvar !subgrid orographic statistics data ?

111: forrtl: severe (408): fort: (2): Subscript #2 of the array MNTVAR has value 15 which is greater than the upper bound of 14

It crashes with rrfs_v1beta_debug intel control_p8_atmlnd_sbs intel control_p8_atmlnd intel control_p8_atmlnd_debug intel rrfs_v1beta_debug gnu. An experiment output is available at /scratch2/NAGAPE/epic/Jong.Kim/stmp2/Jong.Kim/FV3_RT/rt_3529436/rrfs_v1beta_debug_intel/err

Qingfu-Liu · 2024-06-20T12:22:55Z

@Qingfu-Liu @grantfirl @dustinswales @mdtoyNOAA can you check the following error message about mntvar !subgrid orographic statistics data ?
111: forrtl: severe (408): fort: (2): Subscript #2 of the array MNTVAR has value 15 which is greater than the upper bound of 14
It crashes with rrfs_v1beta_debug intel control_p8_atmlnd_sbs intel control_p8_atmlnd intel control_p8_atmlnd_debug intel rrfs_v1beta_debug gnu. An experiment output is available at /scratch2/NAGAPE/epic/Jong.Kim/stmp2/Jong.Kim/FV3_RT/rt_3529436/rrfs_v1beta_debug_intel/err

@jkbk2004 There is a new set of orographic data related to this PR#2290 and the new data is described in PR#2670 workflow. Can you run the regression tests using the new data? Thank you very much.:
NOAA-EMC/global-workflow#2670
This PR#2670 includes changes for four scripts, and new orographic data.
The new orographic data temporarily stored at:
/scratch1/NCEPDEV/global/Qingfu.Liu/git/GWD_SHong/APR2023_SSO
and this new data should replace the old data: /scratch1/NCEPDEV/global/glopara/fix/ugwd/20231027

BrianCurtis-NOAA · 2024-06-20T12:29:15Z

If there are any anticipated changes to data in the input-data-YYYYMMDD directories, it should be very specifically addressed in the template. Please make changes there.

Qingfu-Liu · 2024-06-20T12:34:11Z

If there are any anticipated changes to data in the input-data-YYYYMMDD directories, it should be very specifically addressed in the template. Please make changes there.

OK. Thanks. After look the error, I am not sure the failed tests are related to the data change. I just add Jongil who works on the code to see if he know this.

Qingfu-Liu · 2024-06-20T12:35:02Z

@JongilHan66 Can you take a look of the errors @jkbk2004 mentioned in this PR#2290? Thanks

JongilHan66 · 2024-06-20T13:07:01Z

Currently we use gwd_opt=2 (unified ugwp GWD), which is defined in config.fcst. Then the dimension of MNTVAR is increased from 10 to 24, as described in 'GFS_typedefs.F90'.

JongilHan66 · 2024-06-20T13:14:10Z

@Qingfu-Liu If you haven't yet, please update the config.fcst files with those in the directories of gfs & gefs in /lfs/h2/emc/physics/noscrub/jongil.han/git_HR4_gwd/global-workflow/parm/config

Qingfu-Liu · 2024-06-20T13:15:58Z

@Qingfu-Liu If you haven't yet, please update the config.fcst files with those in the directories of gfs & gefs in /lfs/h2/emc/physics/noscrub/jongil.han/git_HR4_gwd/global-workflow/parm/config

@JongilHan66 Those files are updated in workflow PR#2670:
https://github.com/NOAA-EMC/global-workflow/pull/2670/files

JongilHan66 · 2024-06-20T13:23:41Z

@Qingfu-Liu If you haven't yet, please update the config.fcst files with those in the directories of gfs & gefs in /lfs/h2/emc/physics/noscrub/jongil.han/git_HR4_gwd/global-workflow/parm/config

@JongilHan66 Those files are updated in workflow PR#2670: https://github.com/NOAA-EMC/global-workflow/pull/2670/files

@Qingfu-Liu Did you also update the "parsing_namelists_FV3.sh" and "parsing_namelists_FV3_nest.sh" in the ush directory?

Qingfu-Liu · 2024-06-20T13:33:43Z

@Qingfu-Liu If you haven't yet, please update the config.fcst files with those in the directories of gfs & gefs in /lfs/h2/emc/physics/noscrub/jongil.han/git_HR4_gwd/global-workflow/parm/config

@JongilHan66 Those files are updated in workflow PR#2670: https://github.com/NOAA-EMC/global-workflow/pull/2670/files

@Qingfu-Liu Did you also update the "parsing_namelists_FV3.sh" and "parsing_namelists_FV3_nest.sh" in the ush directory?

@JongilHan66 Yes, both files are updated in the PR#2670. There are 4 files are updated in PR#2670

JongilHan66 · 2024-06-20T13:46:39Z

@Qingfu-Liu If you haven't yet, please update the config.fcst files with those in the directories of gfs & gefs in /lfs/h2/emc/physics/noscrub/jongil.han/git_HR4_gwd/global-workflow/parm/config

@JongilHan66 Those files are updated in workflow PR#2670: https://github.com/NOAA-EMC/global-workflow/pull/2670/files

@Qingfu-Liu Did you also update the "parsing_namelists_FV3.sh" and "parsing_namelists_FV3_nest.sh" in the ush directory?

@JongilHan66 Yes, both files are updated in the PR#2670. There are 4 files are updated in PR#2670

@Qingfu-Liu If gwd_opt=2 in config.fcst, the dimension of MNTVAR is 24 and the model should not complain the max of 14 which is for gwd_opt=1.

Qingfu-Liu · 2024-06-20T14:02:33Z

@JongilHan66 Thanks. I think I understand the problem.

DeniseWorthen · 2024-07-08T14:34:18Z

Why would the gnu version of these tests fail comparison but not the intel version?

zach1221 · 2024-07-08T14:40:03Z

Let me check the baselines again. It could be an issue with the baselines I synced.

DeniseWorthen · 2024-07-09T19:28:04Z

Thanks @BrianCurtis-NOAA. Does anyone know if those additional tests are expected to be failing comparison?

grantfirl · 2024-07-09T19:55:31Z

Thanks @BrianCurtis-NOAA. Does anyone know if those additional tests are expected to be failing comparison?

@DeniseWorthen @BrianCurtis-NOAA Something seems amiss between the test_changes.list and RegressionTests_hera.log. They don't show the same failures. FWIW, the failures listed in RegressionTests_hera.log match the failures that I had the last time I tested this PR there.

BrianCurtis-NOAA · 2024-07-09T19:59:05Z

@grantfirl apologies for not adding that, here's some other stats from that run_dir:

[18:54:25]Brian.Curtis@hfe03:/scratch1/NCEPDEV/nems/Brian.Curtis/git/Qingfu-Liu/ufs-weather-model/tests
-->grep -ril "NOT IDENTICAL" logs/log_hera/rt_* | wc -l
88

[18:54:35]Brian.Curtis@hfe03:/scratch1/NCEPDEV/nems/Brian.Curtis/git/Qingfu-Liu/ufs-weather-model/tests
-->cat test_changes.list | wc -l
106

[18:55:22]Brian.Curtis@hfe03:/scratch1/NCEPDEV/nems/Brian.Curtis/git/Qingfu-Liu/ufs-weather-model/tests
-->grep "aborted" rte.out | wc -l
18

the "aborted" search is for ecflow where it aborts a test as its parent test failed. The numbers add up.

grantfirl · 2024-07-09T20:32:33Z

@Qingfu-Liu @BrianCurtis-NOAA @DeniseWorthen @zach1221 Granted, this is not my work, but it appears to me that all additional RT failures to control_, rap_, etc. (since I ran them) are the result of the change in control_run.IN (changed input). So, in this sense, I guess that they are "expected", but whether these RTs should be using different input, I'll leave that up to others in this discussion.

grantfirl · 2024-07-10T16:22:03Z

@jkbk2004 @zach1221 Are we still waiting on results from some platforms, or is there another issue that needs to be addressed with this PR?

jkbk2004 · 2024-07-10T16:25:44Z

@jkbk2004 @zach1221 Are we still waiting on results from some platforms, or is there another issue that needs to be addressed with this PR?

@grantfirl We like to merge once wcoss2 tests are done.

BrianCurtis-NOAA · 2024-07-10T16:27:04Z

@jkbk2004 @zach1221 Are we still waiting on results from some platforms, or is there another issue that needs to be addressed with this PR?

@grantfirl We like to merge once wcoss2 tests are done.

113/189 tasks remaining with ecflow, currently.

grantfirl · 2024-07-10T16:42:10Z

@BrianCurtis-NOAA @jkbk2004 Thanks for the update!

jkbk2004 · 2024-07-10T19:05:43Z

@BrianCurtis-NOAA @FernandoAndrade-NOAA move on to merge? or wait for acorn and jet test results?

BrianCurtis-NOAA · 2024-07-10T19:27:03Z

@BrianCurtis-NOAA @FernandoAndrade-NOAA move on to merge? or wait for acorn and jet test results?

Acorn can be skipped it's down for a while.

jkbk2004 · 2024-07-10T19:51:18Z

I see jet queue is very busy. We will catch up jet baseline later. We can start merging process. @grantfirl @dustinswales we can merge ufs-community/ccpp-physics#207 first.

grantfirl · 2024-07-10T20:22:36Z

I see jet queue is very busy. We will catch up jet baseline later. We can start merging process. @grantfirl @dustinswales we can merge ufs-community/ccpp-physics#207 first.

@jkbk2004 CCPP physics and framework PRs were merged and .gitmodules/submodules were updated in NOAA-EMC/fv3atm#836

HR4 GWD update for FV3

c209f27

grantfirl mentioned this pull request May 31, 2024

Add explicit alpha_fd coefficient to TOFD in UGWPv1 drag suite #2297

Draft

14 tasks

update RT logs and FV3 submodule pointer

2bf0f5e

Merge pull request #1 from grantfirl/HR4-GWD-update_gjf

457e5ab

update RT logs and FV3 submodule pointer

Qingfu-Liu requested review from DusanJovic-NOAA and grantfirl June 6, 2024 12:42

Qingfu-Liu marked this pull request as ready for review June 6, 2024 12:42

jkbk2004 added 3 commits June 19, 2024 23:26

sync up FV3 feature branch

8a953dd

sync up against develop

54d1a40

sync up subcomponents

ca9fa83

Qingfu-Liu requested a review from JongilHan66 June 20, 2024 12:32

zach1221 and others added 4 commits July 8, 2024 18:28

add hercules RT logs: passed

edca3a3

add orion RT log: passed

77c0992

add derecho RT logs: passed

9086bb4

updated test_changes.list from hera testing using old baselines 20240614

5b1572b

hera log from comparisons against old baselines

3e59392

zach1221 added 2 commits July 9, 2024 20:59

add hera RT logs: passed

da579b9

add gaea RT logs: passed

a4ee6f0

WCOSS2 RT Log: Passed

24d179f

JessicaMeixner-NOAA mentioned this pull request Jul 10, 2024

HR4 GWD update NOAA-EMC/global-workflow#2732

Open

7 tasks

update FV3 hash and revert change in gitmodules

337801c

jkbk2004 requested review from BrianCurtis-NOAA, zach1221 and FernandoAndrade-NOAA July 10, 2024 23:43

BrianCurtis-NOAA approved these changes Jul 11, 2024

View reviewed changes

jkbk2004 approved these changes Jul 11, 2024

View reviewed changes

jkbk2004 merged commit 0b59ad3 into ufs-community:develop Jul 11, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HR4 GWD update for FV3 #2290

HR4 GWD update for FV3 #2290

Qingfu-Liu commented May 19, 2024 •

edited by jkbk2004

Loading

grantfirl commented Jun 3, 2024

mdtoyNOAA commented Jun 3, 2024

grantfirl commented Jun 3, 2024

grantfirl commented Jun 4, 2024

Qingfu-Liu commented Jun 6, 2024

jkbk2004 commented Jun 18, 2024

grantfirl commented Jun 20, 2024

jkbk2004 commented Jun 20, 2024 •

edited

Loading

Qingfu-Liu commented Jun 20, 2024 •

edited

Loading

BrianCurtis-NOAA commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

DeniseWorthen commented Jul 8, 2024 •

edited

Loading

zach1221 commented Jul 8, 2024

DeniseWorthen commented Jul 9, 2024

grantfirl commented Jul 9, 2024

BrianCurtis-NOAA commented Jul 9, 2024

grantfirl commented Jul 9, 2024

grantfirl commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024 •

edited

Loading

BrianCurtis-NOAA commented Jul 10, 2024

grantfirl commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024

BrianCurtis-NOAA commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024

grantfirl commented Jul 10, 2024

HR4 GWD update for FV3 #2290

HR4 GWD update for FV3 #2290

Conversation

Qingfu-Liu commented May 19, 2024 • edited by jkbk2004 Loading

Commit Queue Requirements:

Description:

Commit Message:

Priority:

Git Tracking

UFSWM:

Sub component Pull Requests:

UFSWM Blocking Dependencies:

Changes

Regression Test Changes (Please commit test_changes.list):

Input data Changes:

Library Changes/Upgrades:

Testing Log:

grantfirl commented Jun 3, 2024

mdtoyNOAA commented Jun 3, 2024

grantfirl commented Jun 3, 2024

grantfirl commented Jun 4, 2024

Qingfu-Liu commented Jun 6, 2024

jkbk2004 commented Jun 18, 2024

grantfirl commented Jun 20, 2024

jkbk2004 commented Jun 20, 2024 • edited Loading

Qingfu-Liu commented Jun 20, 2024 • edited Loading

BrianCurtis-NOAA commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

JongilHan66 commented Jun 20, 2024

Qingfu-Liu commented Jun 20, 2024

DeniseWorthen commented Jul 8, 2024 • edited Loading

zach1221 commented Jul 8, 2024

DeniseWorthen commented Jul 9, 2024

grantfirl commented Jul 9, 2024

BrianCurtis-NOAA commented Jul 9, 2024

grantfirl commented Jul 9, 2024

grantfirl commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024 • edited Loading

BrianCurtis-NOAA commented Jul 10, 2024

grantfirl commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024

BrianCurtis-NOAA commented Jul 10, 2024

jkbk2004 commented Jul 10, 2024

grantfirl commented Jul 10, 2024

Qingfu-Liu commented May 19, 2024 •

edited by jkbk2004

Loading

jkbk2004 commented Jun 20, 2024 •

edited

Loading

Qingfu-Liu commented Jun 20, 2024 •

edited

Loading

DeniseWorthen commented Jul 8, 2024 •

edited

Loading

jkbk2004 commented Jul 10, 2024 •

edited

Loading