error analysis + update prepro gdirs (#1345)

docs/contributing.rst

@@ -210,18 +210,19 @@ Requirements
 ~~~~~~~~~~~~
 
 There are some extra requirements to build the docs: you will need to
-have ``sphinx``, ``sphinx_rtd_theme``, ``numpydoc`` and ``ipython`` installed.
+have ``sphinx``, ``sphinx-togglebutton``, ``sphinx-book-theme``, ``numpydoc``
+and ``ipython`` installed.
 
 If you have a conda environment named ``oggm_env``, you can install the extra
 requirements with::
 
-      conda install -n oggm_env sphinx sphinx_rtd_theme ipython numpydoc
+      conda install -n oggm_env sphinx ipython numpydoc
+      conda install -c conda-forge sphinx-togglebutton sphinx-book-theme
 
 If you use pip, activate your python environment and install the requirements
 with::
 
-      pip install sphinx sphinx_rtd_theme ipython numpydoc
-
+      pip install sphinx sphinx-togglebutton sphinx-book-theme ipython numpydoc
 
 Building the documentation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -367,3 +368,4 @@ branch has not actually been merged.
 The branch will still exist on GitHub, so to delete it there do::
 
     git push origin --delete shiny-new-feature
+

docs/input-data.rst

@@ -171,7 +171,7 @@ Available pre-processed configurations
     argument in your call to :py:func:`workflow.init_glacier_directories`,
     and set it to one of the urls listed below.
 
-    See `this tutorial <https://oggm.org/tutorials/notebooks/elevation_bands_vs_centerlines.html>`_
+    See `this tutorial <https://oggm.org/tutorials/stable/notebooks/elevation_bands_vs_centerlines.html>`_
     for an example.
 
 
@@ -184,10 +184,12 @@ directories from the default urls. Here is a summary of the default configuratio
 
 - model parameters as of the ``oggm/params.cfg`` file at the published model version
 - flowline glaciers computed from the geometrical centerlines (including tributaries)
-- baseline climate from CRU (not available for Antarctica)
-- baseline climate quality checked with :py:func:`tasks.historical_climate_qc` with ``N=3``
-- mass-balance parameters calibrated with the standard OGGM procedure. No calibration
-  against geodetic MB (see options below for regional calibration)
+- baseline climate from CRU (not available for Antarctica) using a global precipitation factor of 2.5 (path index: *pcp2.5*)
+- baseline climate quality checked and corrected if needed with :py:func:`tasks.historical_climate_qc` with ``N=3``.
+  If the condition of at least 3 months of melt per year at the terminus and 3 months of accumulation
+  at the glacier top is not reached, temperatures are shifted (path index: *qc3*).
+- mass-balance parameters calibrated with the standard OGGM procedure (:ref:`mass-balance-2012`, path index: *no_match*).
+  No calibration against geodetic MB (see options below for regional calibration).
 - ice volume inversion calibrated to match the ice volume from [Farinotti_etal_2019]_
   **at the RGI region level**, i.e. glacier estimates might differ. If not specified otherwise,
   it's also the precalibrated parameters that will be used for the dynamical run.
@@ -239,21 +241,72 @@ flowlines:
 - `L3-L5_files/ERA5/centerlines/qc3/pcp1.6/no_match <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/centerlines/qc3/pcp1.6/no_match/>`_ for ERA5 + centerlines
 - `L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/no_match <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/no_match/>`_ for ERA5 + elevation bands
 
-D. Further set-ups
+Note that the globally calibrated multiplicative precipitation factor (pcp) depends on the used baseline climate
+(e.g. pcp is 2.5 for CRU and 1.6 for ERA5). If you want to use another baseline climate, you have to calibrate the
+precipitation factor yourself. Please get in touch with us in that case!
+
+D. Option: Mass-balance calibration method
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+There are different mass-balance calibration options available in the preprocessed directories:
+
+-   **no_match**: This is the default calibration option. For calibration, the direct glaciological WGMS data is used
+    and the `Marzeion et al., 2012`_ *tstar* method is applied to interpolate to glaciers without measurements
+    (see also :ref:`mass-balance-2012`). With this method, the geodetic estimates are not matched.
+-   **match_geod**: The default calibration with direct glaciological WGMS data is still applied on the glacier per
+    glacier level, but on the regional level the epsilon (residual) is corrected to match the geodetic estimates
+    from `Hugonnet et al., 2021`_ (using :py:func:`oggm.workflow.match_regional_geodetic_mb`, see also
+    :ref:`regional-match-calib`). For example:
+
+    - `L3-L5_files/CRU/elev_bands/qc3/pcp2.5/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CRU/elev_bands/qc3/pcp2.5/match_geod/>`_ for CRU + elevation bands flowlines + matched on regional geodetic mass-balances
+    - `L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod/>`_ for ERA5 + elevation bands flowlines + matched on regional geodetic mass-balances
+
+-   **match_geod_pergla**: Only the per-glacier geodetic estimates from `Hugonnet et al., 2021`_
+    (mean mass-balance change between 2000 and 2020) are used for calibration. The mass-balance model parameter
+    :math:`\mu ^{*}` of each glacier is calibrated to match individually the geodetic estimates (using
+    :py:func:`oggm.core.climate.mustar_calibration_from_geodetic_mb`, more in :ref:`mass-balance-2012-pergla`). It
+    only works for elevation band flowlines at the moment. match_geod_pergla makes only sense without "external" climate
+    quality check and correction (i.e. qc0) as this is already done internally. For example:
+
+    - `L3-L5_files/CRU/elev_bands/qc3/pcp2.5/match_geod_pergla <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CRU/elev_bands/qc0/pcp2.5/match_geod_pergla/>`_ for CRU + elevation bands flowlines + matched geodetic mass-balances on individual glacier level
+
+.. _Marzeion et al., 2012: http://www.the-cryosphere.net/6/1295/2012/tc-6-1295-2012.html
+
+E. Further set-ups
 ^^^^^^^^^^^^^^^^^^
 
-Here is the current list of available configurations at the time of writing (explore the server for more!):
+Here is a list of other available configurations at the time of writing (explore the server for more!):
 
-- `L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/no_match <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/no_match/>`_ for CERA+ERA5 + elevation bands
-- `L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/match_geod/>`_ for CERA+ERA5 + elevation bands + matched on regional geodetic mass-balances
-- `L3-L5_files/CRU/elev_bands/qc3/pcp2.5/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CRU/elev_bands/qc3/pcp2.5/match_geod/>`_ for CRU + elevation bands + matched on regional geodetic mass-balances
-- `L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod/>`_ for ERA5 + elevation bands + matched on regional geodetic mass-balances
+- `L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/no_match <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/no_match/>`_ for CERA+ERA5 + elevation bands flowlines
+- `L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CERA+ERA5/elev_bands/qc3/pcp1.6/match_geod/>`_ for CERA+ERA5 + elevation bands flowlines + matched on regional geodetic mass-balances
 - `L3-L5_files/ERA5/elev_bands/qc3/pcp1.8/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.8/match_geod/>`_ for ERA5 + elevation bands flowlines + matched on regional geodetic mass-balances + precipitation factor 1.8
 - `L3-L5_files/CRU/elev_bands/qc0/pcp2.5/match_geod <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CRU/elev_bands/qc0/pcp2.5/match_geod/>`_ for CRU + elevation bands flowlines + matched on regional geodetic mass-balances + no climate quality check
 - `L3-L5_files/CRU/elev_bands/qc0/pcp2.5/no_match <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/CRU/elev_bands/qc0/pcp2.5/no_match/>`_ for CRU + elevation bands flowlines + no climate quality check
+- `L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod_pergla <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod_pergla/>`_ for ERA5 + elevation bands flowlines + matched geodetic mass-balances on individual glacier level + "wrong" qc3
+- `L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod_pergla_massredis <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.4/L3-L5_files/ERA5/elev_bands/qc3/pcp1.6/match_geod_pergla_massredis/>`_ for ERA5 + elevation bands flowlines + mass redistribution instead of SIA (see: :ref:`mass-redistribution`)
 
 Note: the additional set-ups might not always have all map sizes available. Please
-get in touch if you have interest in a specific set-up.
+get in touch if you have interest in a specific set-up. Remember that per default, the climate quality check
+and correction (:py:func:`oggm.tasks.historical_climate_qc`) is applied (*qc3*). However, if the pre-processed directory
+has the path index "*qc0*", it was not applied. At the moment, there are still preprocessed gdirs using
+match_geod_pergla and qc3, this makes not really sense as it is reset anyways internally. We will replace them with
+"qc0" soon, but do not expect any significant differences.
+
+
+F. Error analysis and further volume and mass change comparison for different pre-processed glacier directories
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. figure:: _static/relative_failing_glacier_area.png
+    :width: 100%
+    :align: left
+
+A more detailed analysis about the type, amount and relative failing glacier area (in total and per RGI region) can be
+found in this `error analysis jupyter notebook <https://nbviewer.jupyter.org/urls/cluster.klima.uni-bremen.de/~lschuster/error_analysis/error_analysis_v1.ipynb?flush_cache=true>`_.
+
+If you are also interested in how the "common" non-failing glaciers differ in terms of historical volume change, total
+mass change and specific mass balance between different pre-processed glacier directories,
+you can check out this
+`jupyter notebook <https://nbviewer.org/urls/cluster.klima.uni-bremen.de/~lschuster/error_analysis/working_glacier_gdirs_comparison.ipynb?flush_cache=true>`_.
 
 Climate data
 ------------
@@ -412,7 +465,7 @@ Geodetic MB data
 OGGM ships with a geodetic mass-balance table containing MB information for all
 of the world's glaciers as obtained from `Hugonnet et al., 2021`_.
 
-The original, raw data have been modified in three ways (`code <https://nbviewer.jupyter.org/urls/cluster.klima.uni-bremen.de/~oggm/geodetic_ref_mb/convert.ipynb>`_):
+The original, raw data have been modified in three ways (`code <https://nbviewer.jupyter.org/urls/cluster.klima.uni-bremen.de/~oggm/geodetic_ref_mb/convert.ipynb?flush_cache=true>`_):
 
 - the glaciers in RGI region 12 (Caucasus) had to be manually linked to the product by
   Hugonnet because of large errors in the RGI outlines. The resulting product

docs/mass-balance-2012-pergla.rst

@@ -7,8 +7,10 @@ Temperature index model calibrated on geodetic MB data
 
 As of OGGM v1.5.3, we implemented a simple mass-balance model that can be
 calibrated on the geodetic mass-balance data from
-`Hugonnet et al., 2021 <https://www.nature.com/articles/s41586-021-03436-z>_`
-(see also: :ref:`shop-geod`).
+`Hugonnet et al., 2021`_ (see also: :ref:`shop-geod`).
+
+.. _Hugonnet et al., 2021: https://www.nature.com/articles/s41586-021-03436-z
+
 
 This can be seen as a temporary improvement of the :ref:`mass-balance-2012`
 until the `sandbox models <https://github.com/OGGM/massbalance-sandbox>`_ become
@@ -40,7 +42,7 @@ The glacier per glacier calibration is done for two parameters:
   glacier, and is calibrated to match the geodetic mass-balance data over the
   reference period.
 - if the resulting :math:`\mu ^{*}` is outside of predefined bounds
-  (for example [20, 600] :math:`mm w.e. K^{-1} mth^{-1}`), then the temperature
+  (for example [20, 600] :math:`mm\,w.e.\,K^{-1}\,mth^{-1}`), then the temperature
   is bias corrected until a physically reasonable :math:`\mu ^{*}` is found.
 
 Like for the previous model, several parameters ("hyper parameters") are
@@ -50,10 +52,12 @@ and the precipitation correction factor :math:`P_f`.
 How can I use this model instead of the old one?
 ------------------------------------------------
 
-Since the model equations are the same, it can be used readily within OGGM.
-Only the calibration steps are different (and simpler!). Have a look at our
-tutorials for more information, or use our :ref:`preprodir` with precalibrated
-parameters.
+Since the model equations are the same, it can be used readily within OGGM with
+:py:func:`oggm.core.climate.mu_star_calibration_from_geodetic_mb`. Only the calibration steps
+are different (and simpler!). Check out the
+`run_prepro_levels <https://docs.oggm.org/en/latest/_modules/oggm/cli/prepro_levels.html#run_prepro_levels>`_
+under level 3 when match_geodetic_mb_per_glacier is set to True or use our :ref:`preprodir` with precalibrated
+parameters (path index: *match_geod_pergla*).
 
 Notes
 -----

docs/mass-balance-2012.rst

@@ -167,21 +167,24 @@ resulting changes in calibrated :math:`\mu^*` will be comparatively small
 (again, because of the local constraints on :math:`\mu`). The MB observations,
 however, play a major role for the assessment of model uncertainty.
 
+.. _regional-match-calib:
+
 Regional calibration
 --------------------
 
 .. admonition:: **New in version 1.4!**
 
    As of version 1.4, we now also offer to calibrate the mass-balance at the
    regional level (RGI regions) based on geodetic mass-balance products
-   ([Zemp_et_al_2019]_ or [Hugonnet_et_al_2020]_). This is done by correcting
-   (shifting) the residual for each glacier (:math:`\epsilon` in the equation
+   ([Zemp_et_al_2019]_ or [Hugonnet_et_al_2021]_, see :py:func:`oggm.workflow.match_regional_geodetic_mb`).
+   This is done by correcting (shifting) the residual for each glacier (:math:`\epsilon` in the equation
    above) by a constant value so that the regional estimates match the
    observations. This is not applied per default, as it might lead to
    unrealistic results at the single glacier scale (but it is very useful
    for global studies). An overview of the regional shifts for all regions
    and presently available set-ups can be assessed via
-   `this notebook <https://cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.4/compare_with_geod.html>`_.
+   `this notebook <https://nbviewer.jupyter.org/urls/cluster.klima.uni-bremen.de/~oggm/ref_mb_params/oggm_v1.4/compare_with_geod.ipynb?flush_cache=true>`_.
+   In the preprocessed directories, it is available under the path index *match_geod*.
 
 References
 ----------
@@ -195,7 +198,8 @@ References
    - the CRU TS3.10 Dataset. International Journal of Climatology, 34(3),
    623–642. https://doi.org/10.1002/joc.3711
 
-.. [Hugonnet_et_al_2020] Hugonnet et al., accepted.
+.. [Hugonnet_et_al_2021] Hugonnet, R., McNabb, R., Berthier, E. et al. Accelerated global glacier mass loss in the
+   early twenty-first century, Nature 592, 726–731 (2021). https://doi.org/10.1038/s41586-021-03436-z
 
 .. [Zemp_et_al_2019] Zemp, M., Huss, M., Thibert, E., Eckert, N., McNabb, R.,
    Huber, J., Barandun, M., Machguth, H., Nussbaumer, S. U., Gärtner-Roer, I.,