Relative Humidity Inconsistency

[sgdecker]

What should we add?

MetPy is inconsistent in its definition of relative humidity. For review, there are two definitions:

1. The WMO favors $w/w_s$
2. Most thermo textbooks (e.g., Curry and Webster p. 113) and the AMS prefer $e/e_s$.

Although the differences between these definitions are small, it would be good to be consistent, and perhaps give the user the power to decide which definition to use (perhaps with an rcparams-like option or by giving each existing RH-related function a twin?).

As examples, relative_humidity_from_dewpoint and dewpoint_from_relative_humidity use the AMS definition, but relative_humidity_from_mixing_ratio and relative_humidity_from_specific_humidity use the WMO definition. In general, it appears the current code picks the definition by expediency (i.e., functions easier to write with the WMO definition use that definition, and functions easier to write with the AMS definition use that definition).

Reference

No response

[sgdecker]

Example:

from metpy.units import units
import metpy.calc as mpcalc
from metpy.constants import epsilon

T = 25 * units.degC
RH = .5
p = 975 * units.hPa

es = mpcalc.saturation_vapor_pressure(T)
ws = mpcalc.saturation_mixing_ratio(p, T)

# AMS Calculations
e_ams = RH * es
w_ams = mpcalc.mixing_ratio(e_ams, p)
Td_ams = mpcalc.dewpoint_from_relative_humidity(T, RH)

# WMO Calculations
w_wmo = RH * ws
w_wmo2 = mpcalc.mixing_ratio_from_relative_humidity(p, T, RH)
assert(w_wmo == w_wmo2)
e_wmo = mpcalc.vapor_pressure(p, w_wmo)
Td_wmo = mpcalc.dewpoint(e_wmo)

print('  Quantity          AMS      WMO  ')
print('-'*32)
print(f'Dewpoint(C)      {Td_ams.m_as("degC"):7.2f} {Td_wmo.m_as("degC"):7.2f}')
print(f'Vap. Pres. (mb)  {e_ams.m_as("hPa"):7.2f} {e_wmo.m_as("hPa"):7.2f}')
print(f'Mix Ratio (g/kg) {w_ams.m_as("g/kg"):7.2f} {w_wmo.m_as("g/kg"):7.2f}')

Output:

  Quantity          AMS      WMO  
--------------------------------
Dewpoint(C)        13.87   14.12
Vap. Pres. (mb)    15.84   16.10
Mix Ratio (g/kg)   10.27   10.44

So the feature request is a suggestion, for instance, that there should be a version of dewpoint_from_relative_humidty that returns 14.12... degC, a version of mixing_ratio_from_relative_humidity that returns 10.27... g/kg, etc.

[sgdecker]

As far as I can tell, GEMPAK exclusively uses the AMS definition.

[dopplershift]

Definite 👍 to making everything consistent around the AMS definition.

I'm not sure what to do about the WMO definition. I don't want even more ugly-named functions laying around, but I'm not sure how to elegantly swap. I think my lean would be to wait until an actual compelling use case beyond "completeness" arose.

[kgoebber]

Proposal:

To use the nomenclature of

$RH = \frac{w}{w_s}$

$S = \text{saturation ratio} = \frac{e}{e_s}$

Which is shown in detail at the following link:
https://www.e-education.psu.edu/meteo300/node/519

Create new functions:

• mixing_ratio_from_vapor_pressure
  • This would get at the direct conversion between the two measures
• dewpoint_from_vapor_pressure
  • This is the current dewpoint_from_relative_humidity, which we can keep around until 2.0 API break.
• dewpoint_from_mixing_ratio
  • This would add the functionality desired and more accurately represent how dewpoint is being calculated. In our current implementation of dewpoint, we explicitly use the vapor pressure and therefore not really the RH, but the saturation ratio.

[sgdecker]

@kgoebber So to be clear, you are proposing that the MetPy functions (at least by default) should all be consistent with the WMO definition of RH as opposed to the AMS definition of RH?

[kgoebber]

Yes. But I am very open to other suggestions. The challenge of course is that relative humidity has the multiple definitions and we have typically shied away from boolean arguments to alter behavior (e.g., calc_type = 'AMS' or calc_type='WMO'). This is one path I could see us moving forward that would allow us to make a change before 2.0 where it would be more appropriate to do an API break. Although the dewpoint_from_vapor_pressure name doesn't really imply that an RH could be submitted and the dewpoint function is already set to use the vapor pressure to calculate the dew point.

Maybe another possibility on naming would be:

dewpoint_from_vapor_pressure_rh and dewpoint_from_mixing_ratio_rh

We have also typically shied away from using abbreviations in our functions, although we do have a few (e.g., lcl, lfc, cape, etc.). Then at the 2.0 mark, when it would be appropriate more holistically review our relative humidity naming convention throughout the calculations.

[dopplershift]

Given that GEMPAK uses the e/e_s definition, do we have any idea what's being used by NOAA/NCEP products (e.g. NBM, GFS, NARR, etc.)? What about WRF (I assume that's at least somewhere in some pre-/post-processing code). I think that would be illuminating to the discussion.

[sgdecker]

Here is some ancient-looking code from WRF-VAR that uses the WMO definition. Similarly, this particular shallow convection scheme is using the WMO definition. (I think q tends to be used for mixing ratio in WRF-world even though I associate specific humidtiy with that letter.) But I can't find the right place in the WRF preprocessor code to see what they do there.

[deeplycloudy]

On the dev call today, we reviewed a few more sources from the literature and I wanted to memorialize that discussion here.

• NIST uses the mixing ratio definition. See eq. 4 in their calibration white paper).
  • Vaisala's RH calibration document for their current sondes mentions they are NIST traceable.
• Pruppacher and Klett's microphysics text uses mixing ratio (eq. 4-34) and saturation ratio (eq. 4-36).
• Emanuel's atmospheric convection text uses vapor pressure (eq. 4.1.5) but mentions mixing ratio as also valid in a footnote.

In my own estimation, the NIST and cloud physics communities have a greater commitment to formal thermodynamic consistency (a sort of heritage from chemistry), which would have me favor WMO.

[sgdecker]

It does seem like going with WMO is consistent with what more of the community is doing. I take it that at version 2.0 the problematic functions (the RH/dewpoint conversions) can be changed to match WMO, but if I want to calculate the dewpoint (given T, RH, and p) now, it isn't clear to me how the functions Kevin proposed are any better than the current workaround of calling the three functions I used in my example (mixing_ratio_from_relative_humidity, vapor_pressure, and dewpoint) in sequence. How would my code look with the new functions?

On the flip side, if at some point dewpoint_from_relative_humidity were changed to match the WMO definition, but the user wanted the AMS definition, how would they accomplish that?

[dopplershift]

One way or another we're going to swap something so that we're internally consistent, and I'm happy to do that without worrying about 2.0. We've only every promised that you won't have to update code to remain working. (Yes everyone's definition of bug vs. "working" is different but satisfying everyone in that is the path to madness).

Once we pick one (I lean WMO), there will be no way to get the other behavior in the short term. I'm open to allowing that at some point, but it's MUCH bigger lift, and would have to be considered with things like #142. For this particular issue, though, it's not like you'd be switching out some base calculation. You'd have to swap out a bunch of calculations and how they work--instead of going f1(x) -> f2(x), you're going f1(e, e_s) -> f2(r, r_s).

That's a lot of complexity that would personally require an impressive use case to really want to embark upon it.

[dcamron]

After realizing that we'd take on a pressure dependency in relative_humidity_from_dewpoint and vice-versa to use the WMO definition in those two functions, we've changed course slightly to avoid API break. The plan:

• add optional pressure kwarg to these two functions to allow for "more correct" WMO pathway if you have pressure to provide. Default to AMS simplification if not provided.
• document this behavior clearly
• fix dewpoint_from_specific_humidity which should be the only pre-existing function that does mix these two formulations. It already requires pressure, so no API break there.

This should satisfy both use-cases in a physically meaningful way. @sgdecker would you be willing to contribute a modified version of your comparison above as an example explaining the difference? Or allow us to use it with credit in the PR to address this?

[sgdecker]

@dcamron Sure, I could contribute something! When you say example, you mean a new entry in the Calculations section of the Example Gallery, correct?

[dcamron]

Yes! I envision your comparison above with a bit of explanation of the e/e_s simplification. This would need to go in after the actual code changes go in, showing how to perform either calculation by using the new optional pressure kwarg.

[dopplershift]

So, after @dcamron did some digging, we've found this in the WMO No. 8 Vol 1; the NIST citation above which uses the "WMO definition" cites this Metrologia article, which cites WMO No. 8:

I'll note the Metrologia paper also says 😆:

This de facto standard definition (as it will be referred to in this paper) has been authorized by the WMO9 since 1950 and by many other organizations (see sec- tion 3.2). However, the standard definition is valid only over a limited range, and attempts to create full-range definitions have lacked both clear theoretical basis and official sanction. Finally, a variety of other non-standard definitions (e.g., with z = q and z = r) continue to propagate in particular in clima- tological and meteorological textbooks or research articles.

So does anyone actually have a heavy weight citation for the mixing ratio-based definition?

[sgdecker]

I haven't had a chance to read through everything, but I get that the gist is that the NIST citation claims something that isn't true. Fun!

Yes, I see that the conclusion of the Metrologia article is that using mixing ratio and calling that the "WMO definition" is a mistake, for instance, footnote 18: "For example, the on-line ‘Glossary of Meteorology’ of the American Meteorological Society (AMS) incorrectly states that relative humidity ‘is alternatively defined by the World Meteorological Organization as the ratio of the mixing ratio to the saturation mixing ratio’"

[dopplershift]

I haven't had a chance to read through everything, but I get that the gist is that the NIST citation claims something that isn't true. Fun!

Well, the gist is that it was asserted at the start of the issue that:

The WMO favors $w/w_s$

And the only documentation we can find from WMO says:

Ratio in per cent of the observed vapour pressure to the saturation vapour pressure with respect to water at the same temperature and pressure;

So right now there's a definite inconsistency in MetPy that needs fixing, but there's no evidence so that we should be giving any weight to the idea that w/ws is a desirable relation to use and support.

[dcamron]

Importantly, that BIPM document refers to RH as the "ratio of some non-negative humidity quantity, z, to the same quantity at saturation, zsat, at the same temperature", and does discuss a variety of quantities to plug in, including true vapor pressure $e$, partial pressure of vapor $P_V$, mixing ratio $r$, specific humidity $q$, and more.

It then goes on to refer to the definition from WMO No. 8, the isobaric ratio of partial pressure of vapor $P_V$ to the same at saturation as the de facto standard definition. Unfortunately, we in meteorology often call this quantity vapor pressure 😵‍💫. The document continues to talk about some of the shortcomings and where that definition is most usable, but still starts with acknowledging this as the definition of RH for many.

I'm going to keep this branch I've been working on, create a new branch getting these standardized on pressure ratios, and we'll eventually figure out where this should end up!

[deeplycloudy]

So does anyone actually have a heavy weight citation for the mixing ratio-based definition?

I would consider Pruppacher and Klett (p. 107) a heavyweight, but favor implementing the community consensus and standard practice that is emerging based on the recent scholarship you've described above.

[sgdecker]

I can't find anything supporting the mixing ratio-based definition. I looked at the fifth edition of WMO No. 8. It is no different from more recent editions, thus supporting the contention that what I and the AMS call the "WMO definition" is not really the WMO definition. The Bohren and Albrecht textbook supports the vapor-pressure definition:

They go on to show that, with the e/es definition, RH can be interpreted as the ratio of the rate of condensation of water vapor to the rate of evaporation of water vapor (assuming there is something which water vapor can condense on and evaporate from).