Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RRFSv1.0 Product cleanup: removal of retired products from fv3lam_rrfs.xml #909

Merged
merged 18 commits into from
Apr 4, 2024

Conversation

AndrewBenjamin-NOAA
Copy link
Contributor

The following products were removed from fv3lam_rrfs.xml. They are no longer needed/will be retired in RRFSv1.0.

REFZR: Equivalent radar reflectivity factor from rain
REFZI: Equivalent radar reflectivity factor for snow
PBLREG Planetary Boundary Layer Regime
NCPCP Large-Scale Precipitation (non-convective)
LSPA Land Surface Precipitation Accumulation
WATR Water Runoff
GWLOWS Group Water Lower Storage
CDLYR Non-Convective Cloud Cover
BMIXL Blackadars Mixing Length Scale
POT Potential Temperature: 10 m above ground
SPFH Specific Humidity: 10m above ground
SWHR Solar Radiative Heating Rate
UFLX Momentum Flux, U-Component
VFLX Momentum Flux, V-Component
LWHR Long-Wave Radiative Heating Rate
BGRUN Baseflow-Groundwater Runoff
EVP
SNFALB
TMP:sigma levels
PRES:heights above ground level of 10 m and above
SPFH:heights above ground level of 10 m and above
SNOWC: Remove from native level file (all zero values)
TCDC: Remove sigma levels on Native file
HGT: Convective Cloud Top
ACPCP
VVEL: All pressure levels on prslev file: 250, 500, 700, 800mb on native level
RCS
RCSOL
RCQ
RCT

WCOSS2 tests show that products were removed from PRSLEV and NATLEV files. The test can be found here:
/lfs/h2/emc/ptmp/andrew.benjamin/post_rrfs_2023062800.54503

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA Please open an UPP issue linking to this PR.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA Please open an UPP issue linking to this PR.

@AndrewBenjamin-NOAA Never mind. I see your issue.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA Please sync your branch with the latest UPP branch.

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA my branch has synched with the latest UPP develop branch

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA, @HuiyaChuang-NOAA found that the SSRUN entry in PRSLEV dataset and NALET dataset have inconsistent precision settings. Could you combine a fix in RRFS control file fv3lam_rrfs.xml as:

Change the entry stating line 2309
  <param>
       <shortname>ACM_SNOM_ON_SURFACE</shortname>
       <pname>SNOM</pname>
       <scale>4.0</scale>
       </param>
into
<param>
       <shortname>ACM_SNOM_ON_SURFACE</shortname>
       <pname>SNOM</pname>
       <scale>6.0</scale>
       </param>

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA done.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA Can you also regenerate postxconfig-NT-fv3lam_rrfs.txt?

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA sorry about that,, I generated and committed the flat file.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA Please verify my test /u/wen.meng/ptmp/fv3r_2023062800 on Dogwwod to ensure number of record changes in datasets NATLEV and PRSLVE for RRFS are expected.

@MatthewPyle-NOAA
Copy link
Contributor

@AndrewBenjamin-NOAA I just opened an issue about LTNG data to see if it could be added for the PRSLEV files. Not sure if it would be easy to fold in this request in this PR, but throwing the idea out there.

@WenMeng-NOAA
Copy link
Collaborator

@FernandoAndrade-NOAA Please start the UPP RTs on Hera and Orion. There will be baseline updates for fv3r and 3drtma with this PR.

@WenMeng-NOAA
Copy link
Collaborator

WenMeng-NOAA commented Apr 4, 2024

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2:
Wen.Meng@hfe11 stmp2$ ls -ltr */*diff
-rw-r--r-- 1 Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA I apologize for the delay. I am still seeing LSPA and SNOWC on the NATLEV file. Those entries have been removed from fv3lam_rrfs.xml and the postxconfig-NT-fv3lam_rrfs.txt files in my commit, so I am unsure why they are still there.

Also, @MatthewPyle-NOAA asked that I add LTNG to the PRSLEV file, as per issue #916. May I add that product to this PR?

@WenMeng-NOAA
Copy link
Collaborator

@WenMeng-NOAA I apologize for the delay. I am still seeing LSPA and SNOWC on the NATLEV file. Those entries have been removed from fv3lam_rrfs.xml and the postxconfig-NT-fv3lam_rrfs.txt files in my commit, so I am unsure why they are still there.

Also, @MatthewPyle-NOAA asked that I add LTNG to the PRSLEV file, as per issue #916. May I add that product to this PR?

@AndrewBenjamin-NOAA Thanks for the update. You might be focused on verifying the test. I will submit another PR to address issue #916.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA In your fv3lam_rrfs.xml, the LSPA entry starts at line 2763 and the SNOWC entry starts at line 3519.

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA I just submitted a commit to address this. I must have missed it when I was addressing conflicts. This has fixed the issue, you may proceed.

@WenMeng-NOAA
Copy link
Collaborator

@WenMeng-NOAA I just submitted a commit to address this. I must have missed it when I was addressing conflicts. This has fixed the issue, you may proceed.

@AndrewBenjamin-NOAA We will wrap your PR for final testing.

@WenMeng-NOAA
Copy link
Collaborator

@AndrewBenjamin-NOAA I assume my testing on 04/02 was fine, except SNOWC and LSPA, correct?

@AndrewBenjamin-NOAA
Copy link
Contributor Author

@WenMeng-NOAA yes that is correct

@FernandoAndrade-NOAA
Copy link
Collaborator

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2:

Wen.Meng@hfe11 stmp2$ ls -ltr */*diff
-rw-r--r-- 1Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

Hera and Orion RTs match your results, Hercules had different changes in fv3r/PRSLEV, rtma/NATLEV, and rtma/PRSLEV if you could review at /work2/noaa/epic/nandoam/regression-testing/upp/hercules/909/UPP/ci/rundir/upp-HERCULES

I did not see any issues in the run itself on Hercules.

@WenMeng-NOAA
Copy link
Collaborator

The UPP RTs were completed on WCOSS2 and Hera. @FernandoAndrade-NOAA Please refer to my testing on Hera at /home/Wen.Meng/stmp2:

Wen.Meng@hfe11 stmp2$ ls -ltr */*diff
-rw-r--r-- 1Wen.Meng stmp 75 Apr 4 13:52 rap_2020072316_pe_test/WRFPRS.GrbF16.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:53 fv3r_2023062800/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60363 Apr 4 13:54 fv3r_2023062800_pe_test/PRSLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400_pe_test/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 34910 Apr 4 13:54 rtma_2023040400/NATLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400_pe_test/PRSLEV00.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 35056 Apr 4 13:56 fv3r_2023062800_pe_test/NATLEV10.tm00.diff
-rw-r--r-- 1 Wen.Meng stmp 60522 Apr 4 13:56 rtma_2023040400/PRSLEV00.tm00.diff

Hera and Orion RTs match your results, Hercules had different changes in fv3r/PRSLEV, rtma/NATLEV, and rtma/PRSLEV if you could review at /work2/noaa/epic/nandoam/regression-testing/upp/hercules/909/UPP/ci/rundir/upp-HERCULES

I did not see any issues in the run itself on Hercules.

The tests on Hercules look good to me.

@WenMeng-NOAA
Copy link
Collaborator

This PR is ready for merging.

@WenMeng-NOAA WenMeng-NOAA merged commit 62ecf8a into NOAA-EMC:develop Apr 4, 2024
5 checks passed
@WenMeng-NOAA WenMeng-NOAA added Baseline Change The baselines of the UPP regression tests are changed. Ready for Review This PR is ready for code review. RRFS labels Apr 4, 2024
zhanglikate added a commit to zhanglikate/UPP that referenced this pull request Apr 30, 2024
commit 13d5b6d
Author: EricJames-NOAA <eric.james@noaa.gov>
Date:   Mon Apr 29 15:28:47 2024 -0600

    RRFS updates (NOAA-EMC#938)

    * Initial commit to rename RRFS fix files, update EBB to be vertically integrated from 3D ebu_smoke, and switch to use 1-h average HWP.

    * Getting rid of vertically integrated hydrometeor fields.

    * Updating postxconfig-NT-rrfs.txt

commit 88493f3
Author: Wei Huang <wei.huang@noaa.gov>
Date:   Fri Apr 26 21:29:51 2024 -0600

    add module for AWS (NOAA-EMC#934)

    * add module for AWS

    * comment that this is for NOAA Cloud

commit be0410e
Author: hsinmulin-NOAA <48441737+hsinmulin-NOAA@users.noreply.github.com>
Date:   Tue Apr 23 15:46:47 2024 -0400

    Add files via upload (NOAA-EMC#937)

    add
    1. 80 m for SPFH_ON_SPEC_HGT_LVL_ABOVE_GRND_FDHGT
    2. 80 m for PRES_ON_SPEC_HGT_LVL_ABOVE_GRND_FDHGT

commit 79d74d9
Author: Edward Hartnett <38856240+edwardhartnett@users.noreply.github.com>
Date:   Mon Apr 22 13:34:39 2024 -0600

    turn on doxygen build in CI (NOAA-EMC#932)

    * turn on doxygen build in CI

    * turn on doxygen build in CI

    * turn on doxygen build in CI

    * turn on doxygen build in CI

    * fixed cmake file

    * fixed doc directory

commit 5d6d179
Author: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Date:   Fri Apr 19 14:17:10 2024 -0400

    Remove dependency on sfcio and use sigio only when nemsio is enabled (NOAA-EMC#930)

    * Remove sfcio and use sigio only when numsio is enabled

    * Remove sfcio from modulefiles/wcoss2.lua

    * Compile GFSPOSTSIG.F only if nemsio is enabled

commit 91921f4
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Mon Apr 15 21:04:39 2024 -0400

    Bump idna from 3.6 to 3.7 in /doc/UserGuide (NOAA-EMC#928)

    Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
    - [Release notes](https://github.com/kjd/idna/releases)
    - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
    - [Commits](kjd/idna@v3.6...v3.7)

    ---
    updated-dependencies:
    - dependency-name: idna
      dependency-type: indirect
    ...

    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: Wen Meng <Wen.Meng@noaa.gov>

commit 8bc3fc9
Author: EricJames-NOAA <eric.james@noaa.gov>
Date:   Fri Apr 12 16:06:36 2024 -0600

    Bug fix for blowing snow effect on visibility (NOAA-EMC#915)

    * Correcting bug in CALVIS_GSD.f

    * Introducing roughness length impact on BLSN (no BLSN effect at z0>0.7).

    * Removing some gridpoint print statements for testing.

    * Checking change to compile_upp.sh

    * Reverting change.

commit fd93933
Author: YaliMao-NOAA <53870326+YaliMao-NOAA@users.noreply.github.com>
Date:   Thu Apr 11 17:14:24 2024 +0000

    For RRFS, change the mnemonics of column max of EDPARM from EDPARM on 0m level to MXEDPRM of the entire atmosphere. (NOAA-EMC#926)

    For this purpose, MXEDPRM is added to both post_avblflds.xml and control file fv3lam_rrfs.xml

commit 5faac75
Author: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Date:   Tue Apr 9 18:25:18 2024 -0400

    Correct total soil moisture from SOIMOI to CISOIM (NOAA-EMC#922)

    * Correct total soil moisture from SOIMOI to CISOIM

    * Update shortname of UPP ID 36

    * update CMake debug mode

commit 83e83a9
Author: Innocent Souopgui <162634017+InnocentSouopgui-NOAA@users.noreply.github.com>
Date:   Fri Apr 5 15:10:14 2024 -0500

    Remove the detection of Jet/Centos7 nodes (NOAA-EMC#920)

    * Remove the detection of Jet/Centos7 nodes and removes the associated module file jet_c.lua

    * Update detect_machine.sh remove hera 12

    ---------

    Co-authored-by: Fernando Andrade - NOAA <121964418+FernandoAndrade-NOAA@users.noreply.github.com>

commit 1d1e58a
Author: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Date:   Fri Apr 5 13:10:20 2024 -0400

    Add BUILD_WITH_NEMSIO build option to allow building UPP without nemsio (NOAA-EMC#911)

    * Add BUILD_WITH_NEMSIO build option to allow building UPP without nemsio

    * Specify w3emc and bacio as dependencies of nemsio library

    * Add compile_upp.sh option to build without nemsio (-n)

commit 7d39b5c
Author: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Date:   Thu Apr 4 17:07:14 2024 -0400

    Add LTNG in PRSLEV dataset of RRFS (NOAA-EMC#918)

commit 62ecf8a
Author: AndrewBenjamin-NOAA <103067657+AndrewBenjamin-NOAA@users.noreply.github.com>
Date:   Thu Apr 4 14:50:08 2024 -0400

    RRFSv1.0 Product cleanup: removal of retired products from fv3lam_rrfs.xml (NOAA-EMC#909)

    * removed  <table_info>NCEP</table_info> for ULWRF to use the official WMO ID rather than the dupilcate NCEP local use ID

    * removed <table_info>NCEP</table_info> from REFZR, REFZI, BMIXL, and CNWAT to use WMO grib2 ids

    * removed <table_info>NCEP</table_info> from SNOWC, SFEXC, SSRUN, BGRUN, FRICV, CD, SNOHF, PEVAP, TCOLI in order to use WMO id over NCEP local use ID.

    * removed <table_info>NCEP<table_info> from SNFALB, CDLYR, USWRF, DSWRF, DLWRF, VWSH, TCOLR, TCOLS, UPHL

    * removed <table_info>NCEP<table_info> from MSTAV in post_avblflds.xml to use WMO grib2 id for RRFS

    * removed duplicate CNWAT NATLEV entry in fv3lam_rrfs.xml.

    * updated fv3lam_rrfs.xml and parm/postxconfig-NT-fv3lam_rrfs.txt to remove products no longer needed in the RRFS.

    * changed precision in SSRUN

    * fixed SSRUN precision in flat file

    * removed LSPA and SNOWC from NATLEV

commit df9b16b
Author: EricJames-NOAA <eric.james@noaa.gov>
Date:   Mon Apr 1 15:51:56 2024 -0600

    Add surface albedo, accumulated snow melt, and total soil moisture output in RRFS (NOAA-EMC#908)

    * Updating CLDRAD and INITPOST_NETCDF

    * Reverting change in CLDRAD.f

    * Changes to output CISOILM and SNOM.

    * Cleaning up syntax.

    * Adding change logs.

    * Small updates.

    * Removing extcof55 from visibility calculation (it's all zeroes).

    * Reverting change in INITPOST_NETCDF.f setting smstot to spval

    * Using a single level "0 m underground" for CISOILM

commit 92b080d
Author: KarinaAsmar-NOAA <148993962+KarinaAsmar-NOAA@users.noreply.github.com>
Date:   Sat Mar 30 20:03:49 2024 -0400

    Add additional SFS variables and isobaric levels (NOAA-EMC#903)

    * added additional isobaric levels

    * adding HLCY, SSRUN, VBDSF

    * added HLCY, SSRUN, VBDSF

    * replacing id 772 with 401 and 403 (vbdsf and nbdsf)

    * vbdsf and nbdsf with info from avblflds

    * removed id 772 and added 401 and 403 (vbdsf and nbdsf)

    * vbdsf and nbdsf copied from gfs flux

    * vbdsf and nbdsf copied from gfs flux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Baseline Change The baselines of the UPP regression tests are changed. Ready for Review This PR is ready for code review. RRFS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RRFSv1.0 Product Cleanup: Removal of retired products from fv3lam_rrfs.xml
4 participants