Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow smarter weights (cubes, coordinates, cell measures, or ancillary variables) for aggregation #5084

Merged
merged 25 commits into from
Mar 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
4656b39
First implementation for smarter weighted aggregation
schlunma Nov 23, 2022
fcde563
Allowed arbitrary _DimensionalMetadata for weights
schlunma Nov 24, 2022
ae4efbb
Added tests
schlunma Nov 24, 2022
c3dd61d
Added units argument to constructor of Weights
schlunma Nov 25, 2022
401d49d
Added documentation
schlunma Nov 25, 2022
546f204
Fixed tests and added Weights class to API doc
schlunma Nov 25, 2022
c694c4b
Fixed tests again
schlunma Nov 25, 2022
6dff4a8
Fixed bug which lead to errors when weights was explicitly set to None
schlunma Nov 25, 2022
c820183
Fixed bug in doc
schlunma Nov 25, 2022
b4e2e83
Try 1 to fix doctest
schlunma Nov 25, 2022
43ff2db
Do not import inherited functions for Weights anymore to avoid doctes…
schlunma Nov 25, 2022
8670788
Made Weights class private
schlunma Mar 6, 2023
1b458f9
Suggestions from code review
schlunma Mar 6, 2023
2505fb8
Split tests into smaller parts
schlunma Mar 6, 2023
768a0c4
Re-wrote TestWeights as pytest test
schlunma Mar 6, 2023
0352241
Spellcheck
schlunma Mar 6, 2023
af9aaab
Added test for _sum_units_func
schlunma Mar 6, 2023
69a854f
Ensure backwards-compatibility of Aggregator.units_func
schlunma Mar 6, 2023
0741081
Restored import order of test
schlunma Mar 6, 2023
fca72f8
Added What's new entry
schlunma Mar 6, 2023
8232c9d
Split further tests
schlunma Mar 6, 2023
9d8a87a
Optimized What's new entry
schlunma Mar 6, 2023
d1b9c56
Apply suggestions from code review
schlunma Mar 10, 2023
17885c4
obj cannot be None in _Weights.__array_finalize__
schlunma Mar 10, 2023
ab0a20f
Removed whitespace
schlunma Mar 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/gallery_code/general/plot_custom_aggregation.py
Expand Up @@ -72,7 +72,7 @@ def main():

# Make an aggregator from the user function.
SPELL_COUNT = Aggregator(
"spell_count", count_spells, units_func=lambda units: 1
"spell_count", count_spells, units_func=lambda units, **kwargs: 1
lbdreyer marked this conversation as resolved.
Show resolved Hide resolved
)

# Define the parameters of the test.
Expand Down
93 changes: 89 additions & 4 deletions docs/src/userguide/cube_statistics.rst
Expand Up @@ -14,7 +14,7 @@ Cube Statistics
Collapsing Entire Data Dimensions
---------------------------------

.. testsetup::
.. testsetup:: collapsing

import iris
filename = iris.sample_data_path('uk_hires.pp')
Expand Down Expand Up @@ -125,7 +125,7 @@ in order to calculate the area of the grid boxes::

These areas can now be passed to the ``collapsed`` method as weights:

.. doctest::
.. doctest:: collapsing

>>> new_cube = cube.collapsed(['grid_longitude', 'grid_latitude'], iris.analysis.MEAN, weights=grid_areas)
>>> print(new_cube)
Expand All @@ -141,8 +141,8 @@ These areas can now be passed to the ``collapsed`` method as weights:
altitude - x
Scalar coordinates:
forecast_reference_time 2009-11-19 04:00:00
grid_latitude 1.5145501 degrees, bound=(0.14430022, 2.8848) degrees
grid_longitude 358.74948 degrees, bound=(357.494, 360.00497) degrees
grid_latitude 1.5145501 degrees, bound=(0.13755022, 2.89155) degrees
grid_longitude 358.74948 degrees, bound=(357.48724, 360.01172) degrees
surface_altitude 399.625 m, bound=(-14.0, 813.25) m
Cell methods:
mean grid_longitude, grid_latitude
Expand All @@ -155,6 +155,50 @@ Several examples of area averaging exist in the gallery which may be of interest
including an example on taking a :ref:`global area-weighted mean
<sphx_glr_generated_gallery_meteorology_plot_COP_1d.py>`.

In addition to plain arrays, weights can also be given as cubes or (names of)
:meth:`~iris.cube.Cube.coords`, :meth:`~iris.cube.Cube.cell_measures`, or
:meth:`~iris.cube.Cube.ancillary_variables`.
This has the advantage of correct unit handling, e.g., for area-weighted sums
the units of the resulting cube are multiplied by an area unit:

.. doctest:: collapsing

>>> from iris.coords import CellMeasure
>>> cell_areas = CellMeasure(
... grid_areas,
... standard_name='cell_area',
... units='m2',
... measure='area',
... )
>>> cube.add_cell_measure(cell_areas, (0, 1, 2, 3))
>>> area_weighted_sum = cube.collapsed(
... ['grid_longitude', 'grid_latitude'],
... iris.analysis.SUM,
... weights='cell_area'
... )
>>> print(area_weighted_sum)
air_potential_temperature / (m2.K) (time: 3; model_level_number: 7)
Dimension coordinates:
time x -
model_level_number - x
Auxiliary coordinates:
forecast_period x -
level_height - x
sigma - x
Derived coordinates:
altitude - x
Scalar coordinates:
forecast_reference_time 2009-11-19 04:00:00
grid_latitude 1.5145501 degrees, bound=(0.13755022, 2.89155) degrees
grid_longitude 358.74948 degrees, bound=(357.48724, 360.01172) degrees
surface_altitude 399.625 m, bound=(-14.0, 813.25) m
Cell methods:
sum grid_longitude, grid_latitude
Attributes:
STASH m01s00i004
source 'Data from Met Office Unified Model'
um_version '7.3'

.. _cube-statistics-aggregated-by:

Partially Reducing Data Dimensions
Expand Down Expand Up @@ -338,3 +382,44 @@ from jja-2006 to jja-2010:
mam 2010
jja 2010

Moreover, :meth:`Cube.aggregated_by <iris.cube.Cube.aggregated_by>` supports
weighted aggregation.
For example, this is helpful for an aggregation over a monthly time
coordinate that consists of months with different numbers of days.
Similar to :meth:`Cube.collapsed <iris.cube.Cube.collapsed>`, weights can be
given as arrays, cubes, or as (names of) :meth:`~iris.cube.Cube.coords`,
:meth:`~iris.cube.Cube.cell_measures`, or
:meth:`~iris.cube.Cube.ancillary_variables`.
When weights are not given as arrays, units are correctly handled for weighted
sums, i.e., the original unit of the cube is multiplied by the units of the
weights.
The following example shows a weighted sum (notice the change of the units):

.. doctest:: aggregation

>>> from iris.coords import AncillaryVariable
>>> time_weights = AncillaryVariable(
... cube.coord("time").bounds[:, 1] - cube.coord("time").bounds[:, 0],
... long_name="Time Weights",
... units="hours",
... )
>>> cube.add_ancillary_variable(time_weights, 0)
>>> seasonal_sum = cube.aggregated_by("clim_season", iris.analysis.SUM, weights="Time Weights")
>>> print(seasonal_sum)
surface_temperature / (3600 s.K) (-- : 4; latitude: 18; longitude: 432)
Dimension coordinates:
latitude - x -
longitude - - x
Auxiliary coordinates:
clim_season x - -
forecast_reference_time x - -
season_year x - -
time x - -
Scalar coordinates:
forecast_period 0 hours
Cell methods:
mean month, year
sum clim_season
Attributes:
Conventions 'CF-1.5'
STASH m01s00i024
6 changes: 6 additions & 0 deletions docs/src/whatsnew/latest.rst
Expand Up @@ -39,6 +39,12 @@ This document explains the changes made to Iris for this release
#. `@rcomer`_ enabled lazy evaluation of :obj:`~iris.analysis.RMS` calcuations
with weights. (:pull:`5017`)

#. `@schlunma`_ allowed the usage of cubes, coordinates, cell measures, or
ancillary variables as weights for cube aggregations
(:meth:`iris.cube.Cube.collapsed`, :meth:`iris.cube.Cube.aggregated_by`, and
:meth:`iris.cube.Cube.rolling_window`). This automatically adapts cube units
if necessary. (:pull:`5084`)


🐛 Bugs Fixed
=============
Expand Down