feature/gfs_wmo

[malloryprow]

Pull Request Testing

This adds the WMO required verification for the GFS-atmos. It does the grid2grid_upperair, grid2obs_upperair, and grid2obs_sfc. This adds the archiving of the GFS native grid files to global_det atmos prep. There is the daily WMO job that will run in operations that creates a daily stats file and a daily station information file. There is also the monthly job that will run in dev due to its runtime varying, which is not acceptable for operations (end of the month = more days = more stats to calculate and go through = longer runtime). It generates the required reports to send to the WMO at the end of every month.

• Describe testing already performed for this Pull Request:
  

I have been running the daily job in my own space for the past month or so. I ran the monthly job a few times.

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  

Set-up

1. Clone my fork and checkout branch feature/gfs_wmo
2. ln -sf /lfs/h2/emc/vpppg/noscrub/emc.vpppg/verification/EVS_fix fix
3. cd sorc; ./build.sh

✅ global_det atmos prep

4. cd dev/drivers/scripts/prep/global_det
5. In jevs_global_det_atmos_prep.sh, set HOMEevs to the location of the clone. Run. (Let me know when you are going to run and I can tell you if you need to adjust INITDATE or not.)

🔲 global_det atmos stats

6. Copy my prep data. We will need to link or copy my prep data to your prep data location, so yours has all the days of the GFS native grid files in prep. Or we can just point COMIN to my data in step 8.
7. cd dev/drivers/scripts/stats/global_det
8. In jevs_global_det_gfs_atmos_wmo_daily_stats.sh, set HOMEevs to the location of the clone. Run. (Let me know when you are going to run and I can tell you if you need to adjust VDATE or not.)
9. In jevs_global_det_gfs_atmos_wmo_monthly_stats.sh, set HOMEevs to the location of the clone, set export COMIN=/lfs/h2/emc/vpppg/noscrub/mallory.row/$NET/$evs_ver_2d, and export VDATE=20240531.

• Has the code been checked to ensure that no errors occur during the execution? Yes

When the GFS atmos daily stats job is run there is a WARNING message coming from MET

WARNING: VL1L2Info::compute_stats() -> Skipping 1 of 1 vector pairs for which the direction difference is undefined.
WARNING: Set the "wind_thresh" and "wind_logic" configuration options to exclude zero vectors.

DTC is aware of this and made a fix to not have this WARNING message (dtcenter/MET#2395).

• Do these updates/additions include sufficient testing updates? Yes

• Please complete this pull request review by 06/28/2024.
  

Pull Request Checklist

• Review the source issue metadata (required labels, projects, and milestone).

• Complete the PR description above.

• Ensure the PR title matches the feature branch name.

• Check the following:

• Instructions provided on how to run

• Developer's name is replaced by ${user} where necessary throughout the code

• Check that the ecf file has all the proper definitions of variables

• Check that the jobs file has all the proper settings of COMIN and COMOUT and other input variables

• Check to see that the output directory structure is followed

• Be sure that you are not using MET utilities outside the METplus wrapper structure

• After submitting the PR, select Development issue with the original issue number.

• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.

• Close the linked issue.

[PerryShafran-NOAA]

@malloryprow I am going to work on this PR starting now. Your instructions suggest that I need to inquire first about adjusting INITDATE. Let me know if that is the case and what it should be.

[malloryprow]

Wonderful! Yes, let's run with INITDATE=20240618 so that I can compare your testing from my run from yesterday.

[PerryShafran-NOAA]

Thank you. The prep step is underway.

[PerryShafran-NOAA]

@malloryprow The job is finished.

Check out the .o file here: /lfs/h2/emc/vpppg/noscrub/perry.shafran/pr491test/EVS/dev/drivers/scripts/prep/global_det

and the output directory here:

/lfs/h2/emc/vpppg/noscrub/perry.shafran/evs/v2.0/prep/global_det/atmos.20240618

[malloryprow]

✅ All is well with global_det atmos prep! Nothing bad in the log files and the output matches mine.

For testing the stats, you'll need to copy my prep data directories in/lfs/h2/emc/vpppg/noscrub/mallory.row/evs/v2.0/prep/global_det to /lfs/h2/emc/vpppg/noscrub/perry.shafran/evs/v2.0/prep/global_det, so that we have the full dataset. Though, you don't need to copy atmos.20240618 since you ran that in the testing of the prep!

We will also want to test the daily job (jevs_global_det_gfs_atmos_wmo_daily_stats.sh) for VDATE=20240618.

[PerryShafran-NOAA]

@malloryprow Can I just set COMIN to your prep directory? That would seem easier, and your instructions say that I can do that.

[malloryprow]

That'll work too!

[PerryShafran-NOAA]

Both stats jobs are underway. You can follow along the .o files here; /lfs/h2/emc/vpppg/noscrub/perry.shafran/pr491test/EVS/dev/drivers/scripts/stats/global_det

[PerryShafran-NOAA]

@malloryprow The daily job seems to be complete. Looks like the monthly is still running.

[malloryprow]

The daily job output matches mine. Log is good too.

Sounds like things are running on track for the monthly as it does take longer to run than the daily job.

[PerryShafran-NOAA]

The monthly is now complete if you want to look at the output.

[malloryprow]

The monthly looks good too. The output matches mine and no problems with the log files (though this necessarily won't be running in ops, but still good to see!).

This is all good for me. It would be good to get this running in the parallel, pending merge. My prep data for the GFS can be copied into the parallel output and the daily job can be set to run in a cron at 1845.

[malloryprow]

I'll be on leave tomorrow so if you want to wait until Monday to set up the GFS WMO job in the emc.vpppg parallel, that is fine with me. It is running in mine.

[PerryShafran-NOAA]

Thanks, Mallory! I can probably copy (or at least link) your data to emc.vpppg and get that started in the next day or two. You can check things upon your return.

[PerryShafran-NOAA]

Code works as expected. Approved for merge.