Adds more combinations for wx symbols

[kgoebber]

I've add a 37 new METAR combinations to the options in the wx symbol map for what I assume are not well structured METAR. Most of the additions are where there are more than two combinations of current weather (e.g., -FZRASN). I chose to always go with the combination of the first two (if possible) or just the first one if no other logical option was available. Additionally, there were a number of combos (e.g., RASN, RAPL, etc) that I set to be a combination of rain and solid precipitation (numbers 68 to 69, depending on intensity).

Comments welcome, likely will need more as there seems to be a lot of "bad" METAR out there.

[eliteuser26]

I don't think it is bad Metars. In Canada we do observe a combination of freezing and solid precipitations or liquid and solid precipitations with temperatures near the freezing mark. I observed this winter in Kingston rasn or fzrasn or plsn or sgsn of different intensities.

[eliteuser26]

I built a python decoder that takes into account those precipitation combinations. I use it to highlight in colour the precipitation type based on the state (liquid, freezing and solid) in Metar observations that I display on my personal website. I find this useful for viewing purposes.

[jrleeman]

Overall I'm +1 here, checking into what's going on with the build, but pint is the likely culprit and unrelated.

[dopplershift]

Is there any rhyme or reason to the ordering of the symbols? If not, can we find one?

[jrleeman]

Hmm - maybe alphabetical by key or numerical by value sorting? I think I lean slightly towards numerical by value.

[jrleeman]

@dopplershift was good with numeric by value, so I sorted them real quick:

{'': 0, 'M': 0, 'TSNO': 0, 'TS': 0, 'VA': 4, 
'FU': 4, 'HZ': 5, 'DU': 6, 'BLDU': 7, 
'VCBLDU': 7, 'PO': 8, 'VCSS': 9, 'BR': 10, 
'BCBR': 10, 'MIFG': 11, 'BC': 11, 'VCTS': 13, 
'VIRGA': 14, 'VCSH': 16, '-VCTSRA': 17, 'VCTSRA': 17, 
'+VCTSRA': 17, 'THDR': 17, '-VCTSDZ': 17, 'VCTSDZ': 17, 
'-VCTSSN': 17, 'VCTSHZ': 17, 'VCTSUP': 17, 'VCTSSN': 17, 
'+VCTSSN': 17, 'SQ': 18, 'FC': 19, 'DS': 31, 
'SS': 31, '+DS': 34, '+SS': 34, 'DRSN': 36, 
'+DRSN': 37, '-BLSN': 38, 'BLSN': 39, '+BLSN': 39, 
'VCFG': 40, 'BCFG': 41, 'PRFG': 44, 'FG': 45, 
'FZFG': 49, '-DZ': 51, '-DZSN': 51, '-DZBR': 51, 
'DZ': 53, '+DZ': 55, '-FZDZ': 56, '-FZDZSN': 56, 
'-FZDZPL': 56, '-FZDZPLSN': 56, 'FZDZ': 57, '+FZDZ': 57, 
'FZDZPL': 57, 'FZDZSN': 57, '-DZRA': 58, 'DZRA': 59, 
'-RA': 61, '-RABR': 61, 'RA': 63, 'RABR': 63, 
'RAFG': 63, '+RA': 65, '-FZRA': 66, '-FZRASN': 66, 
'-FZRAPL': 66, '-FZRASNPL': 66, '-FZRABR': 66, '-FZRAPLSN': 66, 
'FZRA': 67, '+FZRA': 67, 'FZRASN': 67, 'FZRAPL': 67, 
'+FZRAPL': 67, 'FZRAPLGS': 67, '-RASN': 68, '-SNRA': 68, 
'-RAPL': 68, '-RASNPL': 68, '-RAPLSN': 68, 'RASN': 69, 
'+RASN': 69, 'SNRA': 69, '+RAPL': 69, 'RAPL': 69, 
'-SNDZ': 69, '-SN': 71, '-SNPL': 71, '-SNGS': 71, 
'-SNSG': 72, 'SN': 73, '-SNBR': 73, '-SNGR': 73, 
'SNPL': 73, '+SN': 75, 'IN': 76, '-UP': 76, 
'UP': 76, '+UP': 76, 'SG': 77, '-SG': 77, 
'GSPLSN': 77, 'GSPL': 77, 'IC': 78, '-PL': 79, 
'PL': 79, '-PLSN': 79, '-PLRA': 79, 'PLRA': 79, 
'-PLDZ': 79, '+PL': 79, 'PLSN': 79, 'PLUP': 79, 
'+PLSN': 79, 'PLGSSN': 79, '-SH': 80, '-SHRA': 80, 
'SH': 81, 'SHRA': 81, '+SH': 81, '+SHRA': 81, 
'+SHRABR': 81, '-SHRASN': 83, '-SHSNRA': 83, 'SHRASN': 84, 
'+SHRASN': 84, 'SHSNRA': 84, '+SHSNRA': 84, '-SHSN': 85, 
'SHSN': 86, '+SHSN': 86, '-GS': 87, '-SHGS': 87, 
'GS': 88, 'SHGS': 88, '+GS': 88, '+SHGS': 88, 
'-GR': 89, '-SHGR': 89, 'GR': 90, 'SHGR': 90, 
'+GR': 90, '+SHGR': 90, '-TSRA': 95, 'TSRA': 95, 
'TSSN': 95, 'TSPL': 95, '-TSDZ': 95, '-TSSN': 95, 
'TSFZFG': 95, 'TSUP': 95, '-TSFZRA': 95, '-TSPL': 95, 
'TSPLSN': 95, 'TSSNPL': 95, 'TSRAGS': 95, 'TSBR': 95, 
'TSFZRA': 95, 'TSFZRAPL': 95, '-TSSNPL': 95, 'TSDZ': 95, 
'TSGS': 96, 'TSGR': 96, '+TSRA': 97, '+TSSN': 97, 
'+TSPL': 97, '+TSPLSN': 97, 'TSSA': 98, 'TSDS': 98, 
'+TSGS': 99, '+TSGR': 99,}

For reference, hacky sort with okay wrapping:

sorted_by_value = sorted(wx_code_map.items(), key=lambda kv: kv[1])

dict_str = '{'
for i, (k, v) in enumerate(sorted_by_value):
    dict_str += '\'' + str(k) + '\': '
    dict_str += str(v) + ', '
    if i%4 == 0 and i > 2:
        dict_str += '\n'
dict_str += '}'

[kgoebber]

I like the numerical ordering too (thanks for the nice code to do that and get wrap it relatively easily).

[sgdecker]

Those numbers look like they come from the official WMO table. I'll see if I can find it.

[sgdecker]

Based on this and that (p. 16 ff.), why are all the VCTS variations not code 13, and TS not code 17?

[kgoebber]

@sgdecker Thanks for your comment - the second link is amazing and I wish I had that when I was trying to compile my additional list. Some of it was me guessing and trying to get a map to run with using symbols. I'll update and fully review the list in light of second link with a nice table to symbols and METAR code!

[kgoebber]

Okay, I went through the table that @sgdecker posted to this issue and went through all of the symbols. Someone may want to double check me. The basis for adding some of the new ones is that I am using a source of surface data from Iowa State that smashes a number of different types together. I have used the same document to determine what the symbol should be based on importance values. For example, a number of the new combinations include pellets (PL), which is one of the more important (important value 20) symbols.

[akrherz]

The basis for adding some of the new ones is that I am using a source of surface data from Iowa State that smashes a number of different types together.

I shouldn't be doing more smashing than what is provided by the METARs. Is there some summation or other report I could generate to help out with your PR?

[kgoebber]

@akrherz I just looked at my code - it may not be anything in particular that you are doing. My guess is that it is "bad" METAR. For example, when I was plotting surface data for the Groundhogs Day Blizzard (31 January 2 February 2011) here is one of the current wx codes (-FZDZPLSN) that comes out that is not "standard" according to the document provided by @sgdecker. Right now we are trying to tackle this with only a dictionary, probably need (as is indicated in this section of the MetPy code) a more robust method of deconstructing the combinations if not found to determine what whether should be plotted. I've submitted this PR to help stem the tide until such time that we can implement such a structure.

[sgdecker]

I consider the text-to-symbol document, however useful it may be, to be unofficial, and incomplete. Indeed, looking at this document of METAR standards, section 12.6.8(c): "Up to three types of precipitation may be coded in a single present weather group." So I think -FZDZPLSN is a standard report, but the folks that designed the text-to-symbol matching document couldn't cover every single combination of three precip types, so they punted.

[kgoebber]

@sgdecker Ah, thanks! METAR, what a lovely data format.

[eliteuser26]

I wanted to chime in into this discussion. According to the aviation weather rules, Metar can have up to 3 types of precipitation in order of decreasing dominance or intensity. I observed during the last winter, where I am located in Ontario, Canada, different precipitation types or mixed precipitations in major storms. I frequently see precipitation start as snow then changing to ice pellets to freezing rain and to rain when temperatures hover above the freezing mark. Sometimes there is a mix of rain/snow or freezing rain/ice pellets or other mix. So it is allowed to have a mixture of precipitation during a storm. Usually the first precipitation type is usually the dominant type. This can also occur in thunderstorms during the summer season with a mixture of rain and hail. This has nothing to do with bad Metars. It is allowed according to the aviation rules. Even seen thunder and snow in winter in my area.

Here are some combination I have seen in previous winters: SHSNGS, PLSN, SNPL, RAPL, RASN, SNRA, DZSN, SHRASN, SHRAGS, SHRASNPL, RASG. This can be seen anywhere in the northern U.S. or in Canada. Even in the WMO present weather symbols, there can be a mixture of precipitation but the first type is the dominant one. We can use this to figure out which symbol to use.

[dopplershift]

It's not ideal in terms of being robust, but it doesn't make the current situation worse. I'm open to someone opening PR to replace this monstrosity with something that encodes a programmatic algorithm.

For now, let's just put this in.

[dopplershift]

Merging over Travis, since just the docs fail for some unrelated reason.