Current weather

[stefan-hofer]

I've recently used MetPy's wx_code_map functionality to work with METAR data. From what I've seen I think that there are some entries missing in the dictionary this includes (but there might potentially be a few more):

• "-SG" - Snow grains (light intensity)
• "FZBR" - Freezing mist

This can easily be reproduced by importing the wx_code_map as a dictionary interactively and then trying call these keys:

wx_code_map = {'': 0, 'M': 0, 'TSNO': 0, 'TS': 0, 'VA': 4, 'FU': 4, 'HZ': 5,
               'DU': 6, 'BLDU': 7, 'PO': 8, 'VCSS': 9, 'BR': 10,
               'MIFG': 11, 'VCTS': 13, 'VIRGA': 14, 'VCSH': 16,
               '-VCTSRA': 17, 'VCTSRA': 17, '+VCTSRA': 17,
               'THDR': 17, 'SQ': 18, 'FC': 19, 'DS': 31, 'SS': 31,
               '+DS': 34, '+SS': 34, 'DRSN': 36, '+DRSN': 37, 'BLSN': 38,
               '+BLSN': 39, 'VCFG': 40, 'BCFG': 41, 'PRFG': 44, 'FG': 45,
               'FZFG': 49, '-DZ': 51, 'DZ': 53, '+DZ': 55, '-FZDZ': 56,
               'FZDZ': 57, '+FZDZ': 57, '-DZRA': 58, 'DZRA': 59, '-RA': 61,
               'RA': 63, '+RA': 65, '-FZRA': 66, 'FZRA': 67, '+FZRA': 67,
               '-RASN': 68, 'RASN': 69, '+RASN': 69, '-SN': 71, 'SN': 73,
               '+SN': 75, 'IN': 76, '-UP': 76, 'UP': 76, '+UP': 76, 'SG': 77,
               'IC': 78, '-PL': 79, 'PL': 79, '-SH': 80, '-SHRA': 80,
               'SH': 81, 'SHRA': 81, '+SH': 81, '+SHRA': 81, '-SHRASN': 83,
               '-SHSNRA': 83, 'SHRASN': 84, '+SHRASN': 84, 'SHSNRA': 84,
               '+SHSNRA': 84, '-SHSN': 85, 'SHSN': 86, '+SHSN': 86, '-GS': 87,
               '-SHGS': 87, 'GS': 88, 'SHGS': 88, '+GS': 88, '+SHGS': 88,
               '-GR': 89, '-SHGR': 89, 'GR': 90, 'SHGR': 90, '+GR': 90,
               '+SHGR': 90, '-TSRA': 95, 'TSRA': 95, 'TSSN': 95, 'TSPL': 95,
               'TSGS': 96, 'TSGR': 96, '+TSRA': 97, '+TSSN': 97, '+TSPL': 97,
               'TSSA': 98, 'TSDS': 98, '+TSGS': 99, '+TSGR': 99}

So when calling this is the error message that I get:

In [10]: wx_code_map['FZBR']
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-10-9fbf8a8a864b> in <module>()
----> 1 wx_code_map['FZBR']

KeyError: 'FZBR'

In [6]: wx_code_map['-SG']
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-6-a308ba4149ce> in <module>()
----> 1 wx_code_map['-SG']

KeyError: '-SG'

My current workaround is to replace these entries in my dataset with something similar in the dictionary, for example I replace FZBR with freezing fog (FZFG instead of FZBR) and snow grains with light intensity with snow grains with moderate intensity (SG instead of -SG):

    df['weather'] = df['weather'].replace('-SG','SG')
    df['weather'] = df['weather'].replace('FZBR','FZFG')

This is my first "contribution" to a project on GitHub so every comment on how to contribute more efficiently in the future is welcome. I'd be happy to check and add entries to the dictionary if you can point me to a link with the decoding instructions used by MetPy for METARs.

[akrherz]

Perhaps useful, I did a database query against my entire 2017 archive counting up present weather code usages. Here's the entire listing
codes.txt

and the top 10 for your quick viewing enjoyment

     CODE    |  count
-------------+---------
 BR          | 9578107
 -RA         | 6741052
 -SN         | 2599722
 HZ          | 1468016
 FG          | 1349656
 //          | 1187559
 RA          | 1078717
 -DZ         |  388599
 -SHRA       |  383392
 +RA         |  362285

Of course, some of these codes could be false-positives and bugs with the METAR parsing.

[dopplershift]

This comes back to these being human-encoded reports, which aren't necessarily valid. For instance, according to the WMO manual:

The descriptor FZ shall be used only in combination with the letter abbreviations FG, DZ, RA and UP for example FZRA.

So based on that, 'FZBR' is an invalid report. I don't see any similar language for "SG" and intensity, but it's pretty clear in the WMO code listing that you only have 77 -> "Snow grains".

So the question I have is: Is this a problem with MetPy's code listing? Or is it really a METAR problem? MetPy is going to have its own METAR parser eventually, so I wonder if the "real" solution is to fix it when parsing METARs. On the other hand, adding a couple more entries to the table to work around data-encoding problems isn't such a big deal (provided such work arounds are documented as such).

Anyone have opinions on the matter?

[jrleeman]

I have no problems adding "aliases" to the dictionary if it's a relatively common problem and if it's reasonable to map it that way. The other option would be to have the aliases separately that can be used to update that dictionary to a "strict" or "approximate" interpretation of the METAR.

[stefan-hofer]

Today I ran into a problem with '-DRSN'. Currently there are

'DRSN': 36, '+DRSN': 37, 'BLSN': 38,'+BLSN': 39,

Does this not already contradict the following quotes (http://www.moratech.com/aviation/metar-class/metar-pg9-ww.html#LDRIFT)

Intensity shall be coded with precipitation types, except ice crystals and hail, including those associated with a thunderstorm (TS) and those of a showery nature (SH). Tornadoes and waterspouts shall be coded as +FC.

No intensity shall be ascribed to the obscurations of blowing dust (BLDU), blowing sand (BLSA), and blowing snow (BLSN). Only moderate or heavy intensity shall be ascribed to sandstorm (SS) and duststorm (DS).

[dopplershift]

Are you accusing me of being inconsistent? 😉

I will say one difference is that WMO code table 4677 lists

36 Slight or moderate drifting snow
37 Heavy drifting snow
38 Slight or moderate blowing snow
39 Heavy blowing snow

Upon further reflection, if you can figure out a sane WMO code to use for the reports you need, I'd merge a PR adding them.

[stefan-hofer]

Sorry, I didn't want to give that impression, I should have phrased it differently :-) I just want to understand how you developed the wx_code_map (to maybe contribute something with a PR). I'll try to figure out how I can help and I'll have a look at the WMO codes.

[jrleeman]

If you get stuck anywhere, just holler and we'll be happy to help! 😄

[dopplershift]

@shofer16450 No worries at all, I was completely joking. 😁

[eliteuser26]

I did create a Metar parser in python in my individual weather program and created a table in an XML file to verify the precipitation. In some cases I had to modify the text to make valid as there was an error in the text. However I was able to assume the weather reported but it is not always certain in some cases. I know the code was not built in the same structure as Metpy but it could be adapted if you have a need for it. I am using the WMO weather codes but I have noted that the Canadian version can be slightly different. It depends on which version of the code as been used over the years. We do use SG (snow grain) in Canada as it does occur in my location. For drifting or blowing snow, I will need to verify this.

[dopplershift]

Noting another code seen in the wild that isn't in our dictionary: '-FZRAGR'.

Still not sure whether the table enumerating all the possibilities or some kind of algorithmic approach is better. I mean, an algorithm in general would be better, but I have no idea where to begin with something like that in this case since there's no actual robust, documented procedure that humans do for these.

[eliteuser26]

I do see those metar reports indicating freezing rain and ice pellets. This would be normal for our area in winter. According to definition for metars, the precipitation goes from most important on the left to least important to the right. So FZRA freezing rain most important combined with GR ice pellets least important. This is how we interpret this obs.

[VORTEXJeff]

I am struggling to have the parser correctly handle smoke (FU) as a sensible weather condition. Parsing

KFLG 252353Z AUTO 27005KT 10SM FU BKN036 BKN085 22/03 A3018 RMK AO2 SLP130 T02220033 10250 20217 55007=

yields NaN for all fields except station ID, date/time, wind direction/speed, and a 10 for sky cover, meaning that the sky cover was not correctly parsed.

If I parse

KSYR 252354Z 18011KT 10SM FEW180 BKN250 27/12 A3012 RMK AO2 SLP193 T02670122 10294 20267 55002 $=

it parses correctly. but if I parse (same METAR, only with the manual addition of smoke)

KSYR 252354Z 18011KT 10SM FU FEW180 BKN250 27/12 A3012 RMK AO2 SLP193 T02670122 10294 20267 55002 $=

it fails as with the FLG example above.

[dopplershift]

Thanks for the report @VORTEXJeff ! I've gone ahead and moved that to it's own issue in #1951, because that's really a problem with the METAR parser.

As far as the original issue here, we now have wx_code_to_numeric in metpy.plots (should be in metar, but that's a 2.0 issue at this point). If there are specific failing on how wx_code_to_numeric is operating, I encourage anyone posting here to open a new issue.

[dopplershift]

(Fixed by #1238)