Missing message names from HRRR GRIB2 file.

[rustyconover]

Hi there,

I'm using GRIB.jl to parse a HRRR forecast Grib2 file i.e.

https://pando-rgw01.chpc.utah.edu/hrrr/sfc/20200911/hrrr.t00z.wrfsfcf00.grib2

But it seems that the file isn't fully understandable as I'm missing some message names vs wgrib2.

Table of messages using GRIB.jl

Maximum/Composite radar reflectivity
unknown
Vertically-integrated liquid
Visibility
Derived radar reflectivity
Derived radar reflectivity
Derived radar reflectivity
Wind speed (gust)
U component of wind
V component of wind
U component of wind
V component of wind
Geopotential Height
Temperature
Dew point temperature
U component of wind
V component of wind
Geopotential Height
Temperature
Dew point temperature
U component of wind
V component of wind
Geopotential Height
Temperature
Dew point temperature
U component of wind
V component of wind
Temperature
Dew point temperature
U component of wind
V component of wind
Temperature
Dew point temperature
U component of wind
V component of wind
unknown
unknown
Geometric vertical velocity
MSLP (MAPS System Reduction)
Geopotential Height
unknown
Derived radar reflectivity
unknown
unknown
unknown
unknown
unknown
unknown
Vorticity (relative)
Vorticity (relative)
Hail
Hail
unknown
Lightning
U component of wind
V component of wind
Surface pressure
Orography
Temperature
Total snowfall
Moisture availability
Plant canopy surface water
Water equivalent of accumulated snow depth (deprecated)
Snow cover
Snow depth
2 metre temperature
Potential temperature
Specific humidity
2 metre dewpoint temperature
2 metre relative humidity
10 metre U wind component
10 metre V wind component
10 metre wind speed
unknown
unknown
Percent frozen precipitation
Precipitation rate
Total Precipitation
Water equivalent of accumulated snow depth (deprecated)
unknown
Freezing Rain
Storm surface runoff
Baseflow-groundwater runoff
Categorical snow
Categorical ice pellets
Categorical freezing rain
Categorical rain
Surface roughness
Frictional velocity
Sensible heat net flux
Latent heat net flux
Ground heat flux
Vegetation Type
Surface lifted index
Convective available potential energy
Convective inhibition
Precipitable water
Low cloud cover
Medium cloud cover
High cloud cover
Total Cloud Cover
Pressure
Geopotential Height
Geopotential Height
Pressure
Geopotential Height
Upward long-wave radiation flux
Downward short-wave radiation flux
Downward long-wave radiation flux
Upward short-wave radiation flux
Upward long-wave radiation flux
Visible Beam Downward Solar Flux
Visible Diffuse Downward Solar Flux
Upward short-wave radiation flux
Storm relative helicity
Storm relative helicity
U-component storm motion
V-component storm motion
Vertical u-component shear
Vertical v-component shear
Vertical u-component shear
Vertical v-component shear
Geopotential Height
Relative humidity
Pressure
Geopotential Height
Relative humidity
Pressure
Geopotential Height
Geopotential Height
Best (4-layer) lifted index
Convective available potential energy
Convective inhibition
Planetary boundary layer height
Geopotential Height
Convective available potential energy
Convective inhibition
Convective available potential energy
Convective inhibition
Geopotential Height
Pressure of level from which parcel was lifted
unknown
Land-sea mask
Sea ice area fraction
unknown
unknown
unknown
unknown

When using wgrib2 I can decode the entire file and every message has a name.

Table of contents from wgrib2.

1:0:D=20200911000000:REFC Composite reflectivity [dB]:entire atmosphere:anl:
2:569171:D=20200911000000:RETOP Echo Top [m]:cloud top:anl:
3:831923:D=20200911000000:VIL Vertically-Integrated Liquid Water [kg/m^2]:entire atmosphere:anl:
4:1214910:D=20200911000000:VIS Visibility [m]:surface:anl:
5:2405765:D=20200911000000:REFD Reflectivity [dB]:1000 m above ground:anl:
6:2613416:D=20200911000000:REFD Reflectivity [dB]:4000 m above ground:anl:
7:2809924:D=20200911000000:REFD Reflectivity [dB]:263 K level:anl:
8:3000776:D=20200911000000:GUST Wind Speed (Gust) [m/s]:surface:anl:
9:4039149:D=20200911000000:UGRD U-Component of Wind [m/s]:250 mb:anl:
10:4674949:D=20200911000000:VGRD V-Component of Wind [m/s]:250 mb:anl:
11:5285812:D=20200911000000:UGRD U-Component of Wind [m/s]:300 mb:anl:
12:5936591:D=20200911000000:VGRD V-Component of Wind [m/s]:300 mb:anl:
13:6559735:D=20200911000000:HGT Geopotential Height [gpm]:500 mb:anl:
14:7236238:D=20200911000000:TMP Temperature [K]:500 mb:anl:
15:7771059:D=20200911000000:DPT Dew Point Temperature [K]:500 mb:anl:
16:8628233:D=20200911000000:UGRD U-Component of Wind [m/s]:500 mb:anl:
17:9207521:D=20200911000000:VGRD V-Component of Wind [m/s]:500 mb:anl:
18:9784266:D=20200911000000:HGT Geopotential Height [gpm]:700 mb:anl:
19:10462982:D=20200911000000:TMP Temperature [K]:700 mb:anl:
20:11020633:D=20200911000000:DPT Dew Point Temperature [K]:700 mb:anl:
21:11977993:D=20200911000000:UGRD U-Component of Wind [m/s]:700 mb:anl:
22:12573691:D=20200911000000:VGRD V-Component of Wind [m/s]:700 mb:anl:
23:13159372:D=20200911000000:HGT Geopotential Height [gpm]:850 mb:anl:
24:13865160:D=20200911000000:TMP Temperature [K]:850 mb:anl:
25:14444833:D=20200911000000:DPT Dew Point Temperature [K]:850 mb:anl:
26:15547693:D=20200911000000:UGRD U-Component of Wind [m/s]:850 mb:anl:
27:16162963:D=20200911000000:VGRD V-Component of Wind [m/s]:850 mb:anl:
28:16766183:D=20200911000000:TMP Temperature [K]:925 mb:anl:
29:17357438:D=20200911000000:DPT Dew Point Temperature [K]:925 mb:anl:
30:18425304:D=20200911000000:UGRD U-Component of Wind [m/s]:925 mb:anl:
31:19044084:D=20200911000000:VGRD V-Component of Wind [m/s]:925 mb:anl:
32:19649770:D=20200911000000:TMP Temperature [K]:1000 mb:anl:
33:20254323:D=20200911000000:DPT Dew Point Temperature [K]:1000 mb:anl:
34:21294820:D=20200911000000:UGRD U-Component of Wind [m/s]:1000 mb:anl:
35:21908929:D=20200911000000:VGRD V-Component of Wind [m/s]:1000 mb:anl:
36:22512567:D=20200911000000:MAXUVV Hourly Maximum of Upward Vertical Velocity in the lowest 400hPa [m/s]:100-1000 mb above ground:0-0 day max fcst:
37:22761604:D=20200911000000:MAXDVV Hourly Maximum of Downward Vertical Velocity in the lowest 400hPa [m/s]:100-1000 mb above ground:0-0 day max fcst:
38:22989103:D=20200911000000:DZDT Vertical Velocity (Geometric) [m/s]:0.5-0.8 sigma layer:0-0 day ave fcst:
39:23190170:D=20200911000000:MSLMA MSLP (MAPS System Reduction) [Pa]:mean sea level:anl:
40:23810793:D=20200911000000:HGT Geopotential Height [gpm]:1000 mb:anl:
41:24520595:D=20200911000000:MAXREF Hourly Maximum of Simulated Reflectivity at 1 km AGL [dB]:1000 m above ground:0-0 day max fcst:
42:24768927:D=20200911000000:REFD Reflectivity [dB]:263 K level:0-0 day max fcst:
43:24953892:D=20200911000000:MXUPHL Hourly Maximum of Updraft Helicity over Layer 2km to 5 km AGL [m^2/s^2]:5000-2000 m above ground:0-0 day max fcst:
44:24954104:D=20200911000000:MNUPHL Hourly Minimum of Updraft Helicity [m^2/s^2]:5000-2000 m above ground:0-0 day min fcst:
45:24954316:D=20200911000000:MXUPHL Hourly Maximum of Updraft Helicity over Layer 2km to 5 km AGL [m^2/s^2]:2000-0 m above ground:0-0 day max fcst:
46:24954528:D=20200911000000:MNUPHL Hourly Minimum of Updraft Helicity [m^2/s^2]:2000-0 m above ground:0-0 day min fcst:
47:24954740:D=20200911000000:MXUPHL Hourly Maximum of Updraft Helicity over Layer 2km to 5 km AGL [m^2/s^2]:3000-0 m above ground:0-0 day max fcst:
48:24954952:D=20200911000000:MNUPHL Hourly Minimum of Updraft Helicity [m^2/s^2]:3000-0 m above ground:0-0 day min fcst:
49:24955164:D=20200911000000:RELV Relative Vorticity [1/s]:2000-0 m above ground:0-0 day max fcst:
50:24955376:D=20200911000000:RELV Relative Vorticity [1/s]:1000-0 m above ground:0-0 day max fcst:
51:24955588:D=20200911000000:HAIL Hail [m]:entire atmosphere:0-0 day max fcst:
52:25022089:D=20200911000000:HAIL Hail [m]:0.1 sigma level:0-0 day max fcst:
53:25028360:D=20200911000000:TCOLG Total Column Integrate Graupel [kg/m^2]:entire atmosphere (considered as a single layer):0-0 day max fcst:
54:25039588:D=20200911000000:LTNG Lightning [non-dim]:entire atmosphere:anl:
55:25039776:D=20200911000000:UGRD U-Component of Wind [m/s]:80 m above ground:anl:
56:26026614:D=20200911000000:VGRD V-Component of Wind [m/s]:80 m above ground:anl:
57:26980281:D=20200911000000:PRES Pressure [Pa]:surface:anl:
58:28557752:D=20200911000000:HGT Geopotential Height [gpm]:surface:anl:
59:30792529:D=20200911000000:TMP Temperature [K]:surface:anl:
60:32062137:D=20200911000000:ASNOW Total Snowfall [m]:surface:0-0 day acc fcst:
61:32062349:D=20200911000000:MSTAV Moisture Availability [%]:0 m underground:anl:
62:33486538:D=20200911000000:CNWAT Plant Canopy Surface Water [kg/m^2]:surface:anl:
63:33627333:D=20200911000000:WEASD Water Equivalent of Accumulated Snow Depth [kg/m^2]:surface:anl:
64:33665038:D=20200911000000:SNOWC Snow Cover [%]:surface:anl:
65:33689314:D=20200911000000:SNOD Snow Depth [m]:surface:anl:
66:33719835:D=20200911000000:TMP Temperature [K]:2 m above ground:anl:
67:34918028:D=20200911000000:POT Potential Temperature [K]:2 m above ground:anl:
68:35911572:D=20200911000000:SPFH Specific Humidity [kg/kg]:2 m above ground:anl:
69:37223834:D=20200911000000:DPT Dew Point Temperature [K]:2 m above ground:anl:
70:38281206:D=20200911000000:RH Relative Humidity [%]:2 m above ground:anl:
71:39672738:D=20200911000000:UGRD U-Component of Wind [m/s]:10 m above ground:anl:
72:40679141:D=20200911000000:VGRD V-Component of Wind [m/s]:10 m above ground:anl:
73:41660035:D=20200911000000:WIND Wind Speed [m/s]:10 m above ground:0-0 day max fcst:
74:42680300:D=20200911000000:MAXUW U Component of Hourly Maximum 10m Wind Speed [m/s]:10 m above ground:0-0 day max fcst:
75:43686727:D=20200911000000:MAXVW V Component of Hourly Maximum 10m Wind Speed [m/s]:10 m above ground:0-0 day max fcst:
76:44667645:D=20200911000000:CPOFP Percent frozen precipitation [%]:surface:anl:
77:44667833:D=20200911000000:PRATE Precipitation Rate [kg/m^2/s]:surface:anl:
78:44668021:D=20200911000000:APCP Total Precipitation [kg/m^2]:surface:0-0 day acc fcst:
79:44668233:D=20200911000000:WEASD Water Equivalent of Accumulated Snow Depth [kg/m^2]:surface:0-0 day acc fcst:
80:44668445:D=20200911000000:FROZR Frozen Rain [kg/m^2]:surface:0-0 day acc fcst:
81:44668657:D=20200911000000:FRZR Freezing Rain [kg/m^2]:surface:0-0 day acc fcst:
82:44676422:D=20200911000000:SSRUN Storm Surface Runoff [kg/m^2]:surface:0-0 day acc fcst:
83:44676634:D=20200911000000:BGRUN Baseflow-Groundwater Runoff [kg/m^2]:surface:0-0 day acc fcst:
84:44676846:D=20200911000000:CSNOW Categorical Snow [-]:surface:anl:
85:44677034:D=20200911000000:CICEP Categorical Ice Pellets [-]:surface:anl:
86:44677222:D=20200911000000:CFRZR Categorical Freezing Rain [-]:surface:anl:
87:44677410:D=20200911000000:CRAIN Categorical Rain [-]:surface:anl:
88:44677598:D=20200911000000:SFCR Surface Roughness [m]:surface:anl:
89:46534705:D=20200911000000:FRICV Frictional Velocity [m/s]:surface:anl:
90:47439768:D=20200911000000:SHTFL Sensible Heat Net Flux [W/m^2]:surface:anl:
91:48553481:D=20200911000000:LHTFL Latent Heat Net Flux [W/m^2]:surface:anl:
92:49206767:D=20200911000000:GFLUX Ground Heat Flux [W/m^2]:surface:anl:
93:49778966:D=20200911000000:VGTYP Vegetation Type [Integer(0-13)]:surface:anl:
94:50560145:D=20200911000000:LFTX Surface Lifted Index [K]:500-1000 mb:anl:
95:51308486:D=20200911000000:CAPE Convective Available Potential Energy [J/kg]:surface:anl:
96:51851212:D=20200911000000:CIN Convective Inhibition [J/kg]:surface:anl:
97:52084788:D=20200911000000:PWAT Precipitable Water [kg/m^2]:entire atmosphere (considered as a single layer):anl:
98:53106726:D=20200911000000:LCDC Low Cloud Cover [%]:low cloud layer:anl:
99:53797506:D=20200911000000:MCDC Medium Cloud Cover [%]:middle cloud layer:anl:
100:54044999:D=20200911000000:HCDC High Cloud Cover [%]:high cloud layer:anl:
101:54189941:D=20200911000000:TCDC Total Cloud Cover [%]:entire atmosphere:anl:
102:54904589:D=20200911000000:PRES Pressure [Pa]:cloud base:anl:
103:55676372:D=20200911000000:HGT Geopotential Height [gpm]:cloud base:anl:
104:56934084:D=20200911000000:HGT Geopotential Height [gpm]:cloud ceiling:anl:
105:58281385:D=20200911000000:PRES Pressure [Pa]:cloud top:anl:
106:59088630:D=20200911000000:HGT Geopotential Height [gpm]:cloud top:anl:
107:60355709:D=20200911000000:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:top of atmosphere:anl:
108:62012936:D=20200911000000:DSWRF Downward Short-Wave Radiation Flux [W/m^2]:surface:anl:
109:63159919:D=20200911000000:DLWRF Downward Long-Wave Rad. Flux [W/m^2]:surface:anl:
110:64939723:D=20200911000000:USWRF Upward Short-Wave Radiation Flux [W/m^2]:surface:anl:
111:65769636:D=20200911000000:ULWRF Upward Long-Wave Rad. Flux [W/m^2]:surface:anl:
112:67414596:D=20200911000000:VBDSF Visible Beam Downward Solar Flux [W/m^2]:surface:anl:
113:68105037:D=20200911000000:VDDSF Visible Diffuse Downward Solar Flux [W/m^2]:surface:anl:
114:69133047:D=20200911000000:USWRF Upward Short-Wave Radiation Flux [W/m^2]:top of atmosphere:anl:
115:70175888:D=20200911000000:HLCY Storm Relative Helicity [m^2/s^2]:3000-0 m above ground:anl:
116:71874960:D=20200911000000:HLCY Storm Relative Helicity [m^2/s^2]:1000-0 m above ground:anl:
117:73565110:D=20200911000000:USTM U-Component Storm Motion [m/s]:0-6000 m above ground:anl:
118:74515624:D=20200911000000:VSTM V-Component Storm Motion [m/s]:0-6000 m above ground:anl:
119:75470567:D=20200911000000:VUCSH Vertical U-Component Shear [1/s]:0-1000 m above ground:anl:
120:76574414:D=20200911000000:VVCSH Vertical V-Component Shear [1/s]:0-1000 m above ground:anl:
121:77654426:D=20200911000000:VUCSH Vertical U-Component Shear [1/s]:0-6000 m above ground:anl:
122:78655270:D=20200911000000:VVCSH Vertical V-Component Shear [1/s]:0-6000 m above ground:anl:
123:79658071:D=20200911000000:HGT Geopotential Height [gpm]:0C isotherm:anl:
124:81701692:D=20200911000000:RH Relative Humidity [%]:0C isotherm:anl:
125:82357827:D=20200911000000:PRES Pressure [Pa]:0C isotherm:anl:
126:83045604:D=20200911000000:HGT Geopotential Height [gpm]:highest tropospheric freezing level:anl:
127:83748981:D=20200911000000:RH Relative Humidity [%]:highest tropospheric freezing level:anl:
128:84402237:D=20200911000000:PRES Pressure [Pa]:highest tropospheric freezing level:anl:
129:85073856:D=20200911000000:HGT Geopotential Height [gpm]:263 K level:anl:
130:85702622:D=20200911000000:HGT Geopotential Height [gpm]:253 K level:anl:
131:86287577:D=20200911000000:4LFTX Best (4 layer) Lifted Index [K]:180-0 mb above ground:anl:
132:87239308:D=20200911000000:CAPE Convective Available Potential Energy [J/kg]:180-0 mb above ground:anl:
133:87865366:D=20200911000000:CIN Convective Inhibition [J/kg]:180-0 mb above ground:anl:
134:88198479:D=20200911000000:HPBL Planetary Boundary Layer Height [m]:surface:anl:
135:91069323:D=20200911000000:HGT Geopotential Height [gpm]:level of adiabatic condensation from sfc:anl:
136:93769410:D=20200911000000:CAPE Convective Available Potential Energy [J/kg]:90-0 mb above ground:anl:
137:94268232:D=20200911000000:CIN Convective Inhibition [J/kg]:90-0 mb above ground:anl:
138:94482824:D=20200911000000:CAPE Convective Available Potential Energy [J/kg]:255-0 mb above ground:anl:
139:95134732:D=20200911000000:CIN Convective Inhibition [J/kg]:255-0 mb above ground:anl:
140:95484215:D=20200911000000:HGT Geopotential Height [gpm]:equilibrium level:anl:
141:97807227:D=20200911000000:PLPL Pressure of level from which parcel was lifted [Pa]:255-0 mb above ground:anl:
142:98911623:D=20200911000000:RHPW Relative Humidity with Respect to Precipitable Water [%]:entire atmosphere:anl:
143:100093729:D=20200911000000:LAND Land Cover (0=sea, 1=land) [Proportion]:surface:anl:
144:100144205:D=20200911000000:ICEC Ice Cover [Proportion]:surface:anl:
145:100144393:D=20200911000000:SBT123 Simulated Brightness Temperature for GOES 12, Channel 3 [K]:top of atmosphere:anl:
146:101602413:D=20200911000000:SBT124 Simulated Brightness Temperature for GOES 12, Channel 4 [K]:top of atmosphere:anl:
147:103272606:D=20200911000000:SBT113 Simulated Brightness Temperature for GOES 11, Channel 3 [K]:top of atmosphere:anl:
148:104599363:D=20200911000000:SBT114 Simulated Brightness Temperature for GOES 11, Channel 4 [K]:top of atmosphere:anl:

Thank you for writing your module, can you please let me know what you recommend to get the expected message names?

Rusty

[weech]

Hi,
Those "unknown" messages means that NCEP is using non-standard products in these files. The GRIB format allows centers to create "local" products that are not meant for use outside of those centers. wgrib2 can read those messages fully because it is also NCEP software so it knows the codes. This library uses ecCodes, so it can only understand the standard codes and the local codes at ECMWF natively. Long term, it'd be great to get definition files for NCEP's local codes into either GRIB.jl or eccodes_jll so that things just work for this library.

In the short term, GRIB.jl should still be able to get the values, latitude, and longitude correctly, even if it can't provide all of the metadata at the moment. If it can't, let me know, and I'll see if I can figure out what other weird things are going on with those files.

[rustyconover]

@weech Thanks for fast reply.

I'd be happy to help in any way to get the local codes supported, without knowing the inventory of each file (some products fail to be included randomly), I'd still have to make a call to wgrib2. If you point me to an interface I might even work up a PR.

I'd love to have a 100% Julia GRIB workflow without the need for wgrib2 at all.

Rusty

[weech]

I would be glad for some help with this! I put together a quick contributors' guide in the definitions folder, and I made sure that it worked with the Echo Top message from that file from Brian's archive. I'll spend some time this weekend adding some of the other missing products in the HRRR, and eventually I'd like definitions for all of the new NCEP products in this folder.