inflated RMSE values with coarse regridding

[michellefrazer]

I am using GridStat with MetPlus.

RMSE at 0 lead time should be approximately 0, but it's not when I use coarse resolutions, which I want to do to match previous analysis using other tools.

I'm using: GRID_STAT_REGRID_METHOD = AW_MEAN.

I have model forecasts at 1-deg resolution:

DEBUG 4: Grid information:
DEBUG 4:    Projection: Lat/Lon
DEBUG 4:    Nx: 360
DEBUG 4:    Ny: 181
DEBUG 4:    lat_ll: -90.000
DEBUG 4:    lon_ll: 180.000
DEBUG 4:    delta_lat: 1.000
DEBUG 4:    delta_lon: 1.000
DEBUG 4:    wrapLon: true

My observation data (analysis) is at 0.25-deg resolution:

DEBUG 4: Grid information:
DEBUG 4:    Projection: Lat/Lon
DEBUG 4:    Nx: 1440
DEBUG 4:    Ny: 721
DEBUG 4:    lat_ll: -90.000
DEBUG 4:    lon_ll: 180.000
DEBUG 4:    delta_lat: 0.250
DEBUG 4:    delta_lon: 0.250
DEBUG 4:    wrapLon: true

Examples with height at 500mb:
When I regrid to 2.0x2.5-deg res (GRID_STAT_REGRID_TO_GRID = "latlon 144 91 -90.000 -180.000 2.000 2.500")
When I regrid to 1-deg res (either GRID_STAT_REGRID_TO_GRID = FCST or GRID_STAT_REGRID_TO_GRID = "latlon 360 181 -90.000 -180.000 1.000 1.000"), RMSE is ~2
When I regrid to 0.25-deg res (GRID_STAT_REGRID_TO_GRID = "latlon 1440 721 -90.000 -180.000 0.250 0.250"), RMSE is ~0.005 (what I'd expect based out calculations outside of MET)

Note I am getting RMSE=~0 at coarse resolutions when working with forecasts from a different model that is 0.25x0.3125-deg res.

I'm wondering what I might be doing wrong in my regridding process?

Thanks!
Michelle

[michellefrazer]

Update:

I remembered why AW_MEAN might be failing... when I used GRID_STAT_GRID_WEIGHT_FLAG = AREA and looked at the weights output in the pairs netcdf files, they are off. 0deg lat is not the greatest weight as everything is shifted north 1 grid box such that the for 2deg spacing lat=90, lat=88, and lat=-90 have the same min weight and lat=0 and lat=-2 have the same max weight. So now I'm regularly using GRID_STAT_GRID_WEIGHT_FLAG = COS_LAT.

Returning to regridding, I tested my 2x2.5-deg coarse grid (GRID_STAT_REGRID_TO_GRID = "latlon 144 91 -90.000 -180.000 2.000 2.500") with various GRID_STAT_REGRID_METHOD options. While AW_MEAN leads to RMSE values of ~6, LS_FIT/BILIN/DW_MEAN/BUDGET all give the same values of ~2, which is still too high but better than AW_MEAN. Again, if I regrid to a finer grid, RMSE values improve.

So two issues here:

1. Why am I still getting too high of RMSE values at coarse grids?
2. You may want to look into how the AREA weighting and AW_MEAN regridding options interpret Lat/Lon projections... it feels like a bug.

[JohnHalleyGotway]

@michellefrazer, I took a look at the area weighting using sample GFS data at 0.25 and 1.00 degree resolutions. I ran them through Grid-Stat, regridding to your 2x2.5 degree grid ("latlon 144 91 -90.000 -180.000 2.000 2.500"), writing the area weights to the matched pairs NetCDF file. I do see the behavior you describe where the weights for lat = 0 and lat = -2 are the same value of 61947.6. Zooming up to the equator, here's the pattern I see:

Where the double-bars of red in the center are for latitudes 0 and -2.

Let me step back to explain what's happening. By convention, MET identifies each grid box by the (lat, lon) of the lower-left corner of that grid box. With that in mind, in this example, grid boxes with lat = 0 degrees include latitudes from 0 to 2. And grid boxes with lat = -2 degrees include latitudes from -2 to 0. So they do (and mathematically should) have the same area weight, as shown in the image.

I do see the internal inconsistency here between setting grid_weight_flag to AREA vs COS_LAT. With AREA the weights for lat = 0 and lat = -2 are the same. Whereas with COS_LAT they differ.

This has come up over the years as to how each grid box should be identified, using it's center or lower-left corner. Frankly, I agree that using the center of the grid box would be more intuitive. For high res grids, the difference is minimized. But for coarse grids like this one, it's not.

Rerunning with your grid shifted by 1 degree of latitude ("latlon 144 89 -89.000 -180.000 2.000 2.500") creates grid boxes that extend from -1 to 1 degree and produces a single row of maximized weights, like this:

So I don't think there's a coding bug in the computation of grid box areas to be fixed.

[michellefrazer]

Thanks, @JohnHalleyGotway! I tested your proposed grid GRID_STAT_REGRID_TO_GRID = "latlon 144 89 -89.000 -180.000 2.000 2.500" with GRID_STAT_GRID_WEIGHT_FLAG = AREA and while there is still a single row of maximized weights, its at -1 deg. I understand that "latlon" grids identify grid boxes by their lower-left corner, but that's not correctly interpreting input data which uses the center. My input data has lats of (-90, -88, ..., 0, ..., 88, 90). So when I regrid as you specify with GRID_STAT_REGRID_METHOD = AW_MEAN, I still get terrible RMSE values at 0z (when they should be 0).

Do you by change have another solution or creative workaround for working with grid-box-centered data? It'd really be nice if there was a centered grid option (like a setting within "latlon" or a different projection type to regrid to) or a flag within the regridding methods to tell MET to interpret the incoming grid as centered.

[michellefrazer]

Hi, @JohnHalleyGotway,
Sorry to be annoying but I wanted to bump this as I'm feeling very stuck. I'm not sure how to effectively make use of MET if its always going to interpret all my data (F/O/C) as on grids specified by the lower-left point instead of the center... it will always lead to errors. :(
The ability of MET to regrid on the fly (instead of us having to make re-gridded counterparts of each of my F/O/C files first) is a huge draw for my group's workflow.
Surely someone has a workaround as I can't imagine all operational MET users have lower-left identified grids?
Thanks,
Michelle

[jprestop]

Hi @michellefrazer. Thank you for all of this information. Unfortunately, I believe that @JohnHalleyGotway is best suited to answer your questions, but he is currently out of the office. I've tagged him here so that he can follow up upon his return next week. Thank you for your patience during this time.

[JohnHalleyGotway]

@michellefrazer, I wanted to let you know that we did have some additional discussion about this issue you raised. I was definitely able to replicate an unexpected pattern shift when regridding 1.0 vs 0.25 degree GFS data to your 2x2.5 degree. And that pattern shift is very likely the cause of the unexpectedly large RMSE value. I did briefly try tweaking the AW_MEAN regridding algorithm, shifting by 1/2 grid cell in both directions and modifying rounding that's done, but no obvious solution emerged. It'll take a bit more work to resolve it.

I did document this in a new issue, dtcenter/MET#3178. Since the US Air Force originally requested the area-weighted regridding option, we think they may also be interested in straightening this out.

For the time-being, I'd recommend using bilinear interpolation instead of area-weighted means. That should produce much lower RMSE differences.

Thanks for pointing out and helping clarify this issue!

Please feel free to comment on dtcenter/MET#3178 with any other details or information you'd like to provide.

[michellefrazer]

Thanks so much for this update, @JohnHalleyGotway. I can confirm that bilinear interpolation has been giving me the lowest RMSE values (though still too high!) among the regridding options. I'll follow MET#3178. :)