Skip to content

Commit

Permalink
Update shop doc (#1622)
Browse files Browse the repository at this point in the history 
* list of edits to shop documentation

* removes repeated info on RGI-Topo

* add back by accident removed section

* add explanation on the new folder structure

* add citation info

* Small add-ons

---------

Co-authored-by: Fabien Maussion <fabien.maussion@gmail.com>
  • Loading branch information
@anoukvlug @fmaussion
anoukvlug and fmaussion committed Aug 23, 2023
1 parent 84a4d77 commit 04847e5
Showing 1 changed file with 88 additions and 66 deletions.
154 changes: 88 additions & 66 deletions docs/shop.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,23 @@ OGGM Shop
:width: 100%

OGGM needs various data files to run. **We rely exclusively on
open-access data that can be downloaded automatically for the user**. We like
to call this service a "shop", allowing users to define a shopping list
of data that they wish to add to their :ref:`glacierdir`.

This page describes the various products you will find in the shop.
open-access data that can be downloaded automatically for the user**. This
data needs to be extracted and pre-processed for each individual glacier. To
avoid that everyone needs to repeat these steps, we have added a
service that we like to call a "shop", allowing users to define a shopping list
of data that they wish to add to their :ref:`glacierdir`. The data that can be
included your glacier directories range from essentials for standard OGGM workflow
to other data, like velocity provided as gridded_data for each respective glacier,
that might be of interest to you.

This page describes the various products you will find in the shop. Don't forget to
cite the original data sources of the data that you use (all details can be found
the glacier directories, but details are also listed in this document).

.. important::

Don't forget to set-up or check your system (:ref:`system-settings`) before
downloading new data! (you'll need to
do this only once per computer)
downloading new data! You'll need to do this only once per computer.

.. _preprodir:

Expand All @@ -30,7 +36,7 @@ you can start from various stages in the processing chain, map sizes,
and model set-ups.

The directories have been generated with the standard parameters
of the current stable OGGM version (and a few alternative combinations).
of the respective OGGM version (and a few alternative combinations).
If you want to change some of these parameters, you may have to start a
run from a lower processing level and re-run the processing tasks.
Whether or not this is necessary depends on the stage of the workflow
Expand All @@ -41,7 +47,11 @@ To start from a pre-processed state, simply use the
:py:func:`workflow.init_glacier_directories` function with the
``prepro_base_url``, ``from_prepro_level`` and ``prepro_border`` keyword arguments
set to the values of your choice. This will fetch the desired directories:
there are more options to these, which we explain below.
there are more options to these, which we explain in detail below.
If you like to start using the pre-processed directories right away, with out
reading about all the different options and details first, you can go to the
[10 minutes to… a preprocessed directory](https://oggm.org/tutorials/stable/notebooks/10minutes/preprocessed_directories.html)
tutorial.

Processing levels
~~~~~~~~~~~~~~~~~
Expand All @@ -56,49 +66,50 @@ There are six available levels of pre-processing:
- **Level 0**: the lowest level, with directories containing the glacier
outlines only.
- **Level 1**: directories now contain the glacier topography data as well.
- **Level 2**: at this stage, the flowlines and their downstream lines are
- **Level 2**: at this stage, the glacier flowlines and their downstream lines are
computed and ready to be used.
- **Level 3**: adding the baseline climate timeseries (e.g. W5E5, CRU or ERA5)
to the directories. Adding all necessary pre-processing tasks
- **Level 3**: has the baseline climate timeseries (e.g. W5E5, CRU or ERA5)
added to the directories. It also contains all necessary pre-processing tasks
for a dynamical run, including the mass balance calibration, bed inversion,
up to the :py:func:`tasks.init_present_time_glacier` task included.
These directories still contain all data that were necessary for the processing,
i.e. large in size but also the most flexible since
up to the :py:func:`tasks.init_present_time_glacier` task.
These directories still contain all data that were necessary for the processing.
Therefore they are large in size but also the most flexible since
the processing chain can be re-run from them.
- **Level 4**: includes a historical simulation from
the RGI date to the last possible date of the baseline climate file
(currently January 1st 2020 at 00H for most datasets), stored with the file suffix
``_historical``. Moreover, some configurations (called ``dynspin``) may include
(currently January 1<sup>st</sup> 2020 at 00H for most datasets), stored with the file suffix
``_historical``. Moreover, some configurations (called ``spinup``) may include
a simulation running a spinup from 1979 to the last possible date of the baseline climate file,
stored with the file suffix ``_spinup_historical``. This spinup attempts to conduct a
dynamic mu star calibration and a dynamic spinup matching the RGI area.
dynamic melt factor calibration and a dynamic spinup matching the RGI area.
If this fails, only a dynamic spinup is carried out. If this also fails, a
fixed geometry spinup is conducted. To learn more about these different spinup types,
check out :ref:`dynamic-spinup`.
- **Level 5**: same as level 4 but with all intermediate output files removed.
The strong advantage of level 5 files is that their size is considerably
- **Level 5**: is same as level 4 but with all intermediate output files removed.
The strong advantage of level 5 directories is that their size is considerably
reduced, at the cost that certain operations (like plotting on maps or
running the bed inversion algorithm) are not possible anymore.
re-running the bed inversion algorithm) are not possible anymore.

In practice, most users are going to use level 2, level 3 or level 5 files. To
save space on our servers, level 4 data might be unavailable for some
experiments (but easily recovered if needed).
experiments (but are easily recovered if needed).

.. admonition:: **Changes to the version 1.4 directories**
:class: note, dropdown

In previous versions, level 4 files were the "reduced" directories with intermediate
files removed. Level 5 was very similar, but without the dynamic spinup files.
I practice, most users wont really see a change.
In practice, most users won't really see a change.

Here are some example use cases for glacier directories, and recommendations on which
level to pick:

1. Running OGGM from GCM / RCM data with the default settings: **start at level 5**
1. Running OGGM from GCM / RCM data with the default settings: **start from level 5**
2. Using OGGM's flowlines but running your own baseline climate,
mass balance or ice thickness inversion models: **start at level 2** (and maybe
use OGGM's workflow again for glacier dynamic evolution?). This is the workflow used
by associated model `PyGEM <https://github.com/drounce/PyGEM>`_ for example.
mass balance or ice thickness inversion models: **start at level 2**.
When using an own module, for instance for the mass balance, one can still decide to
use OGGM again further on in the workflow, for instance for the glacier dynamics. This is
the workflow used by associated model `PyGEM <https://github.com/drounce/PyGEM>`_ for example.
3. Run sensitivity experiments for the ice thickness inversion: start at level
3 (with climate data available) and re-run the inversion steps.

Expand All @@ -107,7 +118,7 @@ Glacier map size: the prepro_border argument
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The size of the local glacier map is given in number of grid points *outside*
the glacier boundaries. The larger the map, the largest the glacier can
the glacier boundaries. The larger the domain, the larger the glacier can
become. Here is an example with Hintereisferner in the Alps:

.. ipython:: python
Expand Down Expand Up @@ -152,14 +163,16 @@ become. Here is an example with Hintereisferner in the Alps:
:width: 100%
:align: left

Users should choose the map border parameter depending
Users should choose the border parameter depending
on the expected glacier growth in their simulations. For simulations into
the 21st century, a border value of 80 is sufficient.
For runs into the Little Ice Age, a border value of 160 or 240 is recommended.
the 21<sup>st</sup> century, a border value of 80 is sufficient.
For runs including the Little Ice Age, a border value of 160 or 240 is recommended.

Users should be aware that the amount of data to download isn't small,
especially for full directories at processing level 3 and 4. Here is an indicative
table for the total amount of data with ERA5 centerlines for all 19 RGI regions:
especially for full directories at processing level 3 and 4. It is recommended
to always pick the smallest border value suitable for your research question,
and to start your runs from level 5 if possible. Here is an indicative table for
the total amount of data with ERA5 centerlines for all 19 RGI regions:

====== ===== ===== ===== =====
Level B 10 B 80 B 160 B 240
Expand All @@ -179,8 +192,6 @@ Certain regions are much smaller than others of course. As an indication,
with prepro level 3 and a map border of 160, the Alps are ~2.1G large, Greenland
~21G, and Iceland ~660M.

Therefore, it is recommended to always pick the smallest border value suitable
for your research question, and to start your runs from level 5 if possible.

.. note::

Expand All @@ -199,7 +210,7 @@ to choose from!

To choose from a specific preprocessed configuration, use the ``prepro_base_url``
argument in your call to :py:func:`workflow.init_glacier_directories`,
and set it to one of the urls listed below.
and set it to the url of your choice.

The recommended ``prepro_base_url`` for a standard OGGM run is:

Expand All @@ -209,7 +220,7 @@ The recommended ``prepro_base_url`` for a standard OGGM run is:
from oggm import DEFAULT_BASE_URL
DEFAULT_BASE_URL
This is the setup that was used to generate the OGGM 1.6.1 standard projections
This is the set-up that was used to generate the OGGM 1.6.1 standard projections
(:doc:`download-projections`).
The basic set-up is following:

Expand All @@ -219,10 +230,22 @@ The basic set-up is following:
- "informed three-step" mass balance model calibration (see :ref:`mb-calib`)
- :ref:`dynamic-spinup` with dynamic melt factor calibration

There are however multiple options to choose from. Our `tutorials <https://oggm.org/tutorials>`_
showcase example of applications for some of them. One can explore
`cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6 <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6>`_
for more options. Here follows a brief guide through the folder structure:

- Step 1:
- L1_L2_files: here the directories with pre-processing level 1 and 2 can be found.
- L3_L5_files: here the directories with pre-processing level 3 to 5 can be found.
- Step 2: one can select a version of the directories (e.g. 2023.3)
- Step 3: select the flowline type, centerlines or elevation band flowlines (elev_bands), optionally with the extension of you choice in when using L1_L2_files.
- Step 4: This is only needed when taking the L3_L5_files route. The folder name starts with the name of the baseline climate (e.g. w5e5) that has been used, optionally followed by one or more extensions.

Explanation of the naming convention for the folder name extensions:

Explore
`cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6 <https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6>`_ for more options. Our `tutorials <https://oggm.org/tutorials>`_ showcase
example of applications for some of them.
- `_spinup` indicates that the dynamic spin-up has been used for the calibration, if left out the calibration was done without the dynamic spin-up.
- `w_data` indicates that additional data has been added to the directories: ITS-LIVE, Millan et al. ice velocity product and the consensus ice thickness estimate (all described in more detail later).

.. admonition:: Deprecated: **version 1.4 and 1.5 directories (before v1.6)**
:class: note, dropdown
Expand Down Expand Up @@ -365,10 +388,21 @@ example of applications for some of them.

.. _Marzeion et al., 2012: http://www.the-cryosphere.net/6/1295/2012/tc-6-1295-2012.html


Climate data
------------

Here are the various climate datasets that OGGM handles automatically.
Here are the various climate datasets that OGGM can handle automatically, using the for instance
one of the following functions to pre-process the climate data:

.. code-block:: python
from oggm.tasks import process_w5e5_data
from oggm.tasks import import process_cru_data
from oggm.tasks import import process_histalp_data
from oggm.tasks import import process_ecmwf_data
.. warning::

.. _climate-w5e5:

Expand Down Expand Up @@ -406,27 +440,35 @@ following convenience functions:
after decompression!

The raw, coarse (0.5°) dataset is then downscaled to a higher resolution grid
(CRU CL v2.0 at 10' resolution) following the anomaly mapping approach
(CRU CL v2.0 at 10' resolution [New_et_al_2002]_) following the anomaly mapping approach
described by Tim Mitchell in his `CRU faq`_ (Q25). Note that we don't expect
this downscaling to add any new information than already available at the
original resolution, but this allows us to have an elevation-dependent dataset
based on a presumably better climatology. The monthly anomalies are computed
following [Harris_et_al_2010]_ : we use standard anomalies for temperature and
scaled (fractional) anomalies for precipitation.

**When using these data, please refer to the original provider:**
**When using these data, please refer to the original providers:**

Harris, I., Jones, P. D., Osborn, T. J., & Lister, D. H. (2014). Updated
high-resolution grids of monthly climatic observations - the CRU TS3.10 Dataset.
International Journal of Climatology, 34(3), 623–642. https://doi.org/10.1002/joc.3711

New, M., Lister, D., Hulme, M., & Makin, I (2002). A high-resolution data
set of surface climate over global land areas. Climate Research, 21(715), 1–25.
https://doi.org/10.3354/cr021001

.. _CRU faq: https://crudata.uea.ac.uk/~timm/grid/faq.html

.. [Harris_et_al_2010] Harris, I., Jones, P. D., Osborn, T. J., & Lister,
D. H. (2014). Updated high-resolution grids of monthly climatic observations
- the CRU TS3.10 Dataset. International Journal of Climatology, 34(3),
623–642. https://doi.org/10.1002/joc.3711
.. [New_et_al_2002] New, M., Lister, D., Hulme, M., & Makin, I (2002). A high-resolution
data set of surface climate over global land areas. Climate Research, 21(715),
1–25. https://doi.org/10.3354/cr021001
ERA5 and CERA-20C
~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -477,19 +519,13 @@ recommend to use data from 1850 onwards.
:okwarning:
@savefig plot_temp_ts.png width=100%
example_plot_temp_ts() # the code for these examples is posted below
Any other climate dataset
~~~~~~~~~~~~~~~~~~~~~~~~~

It is fairly easy to add a dataset to OGGM. Recent publications have used
It is fairly easy to force OGGM with other datasets too. Recent publications have used
plenty of options, from ERA5-Land to regional reanalyses or more.

You can provide any other dataset to OGGM. See the `HISTALP_oetztal.nc` data
file in the OGGM `sample-data`_ folder for an example format.

.. _sample-data: https://github.com/OGGM/oggm-sample-data/tree/master/test-workflow


GCM data
~~~~~~~~
Expand All @@ -503,21 +539,6 @@ period. This method is often called the
Visit our online tutorials to see how this can be done
(`OGGM run with GCM tutorial <https://oggm.org/tutorials/master/notebooks/run_with_gcm.html>`_).

RGI-TOPO
--------

The `RGI-TOPO <https://rgitools.readthedocs.io/en/latest/dems.html>`_ dataset
provides a local topography map for each single glacier in the RGI. It was
generated with OGGM, and can be used very easily from the OGGM-Shop (visit
our `tutorials`_ if you are interested!).

.. figure:: _static/malaspina_topo.png
:width: 100%
:align: left

Example of the various RGI-TOPO products at Malaspina glacier

.. _tutorials: https://oggm.org/tutorials

Reference mass balance data
---------------------------
Expand Down Expand Up @@ -748,7 +769,7 @@ run the task: ``workflow.init_glacier_directories``.
``'Lake-terminating'``, ``'Dry calving'``, ``'Regenerated'``,
``'Shelf-terminating'``, ``'Not assigned'``. Marine and Lake
terminating are classified as "tidewater" in OGGM and cannot advance
- they "calve" instead, using a very simple parameterisation.
- they "calve" instead, using a very simple parameterization.
.. [#f6] Glacier status: ``'Glacier or ice cap'``, ``'Glacier complex'``,
``'Nominal glacier'``, ``'Not assigned'``. Nominal glaciers fail at
the "Glacier Mask" processing step in OGGM.
Expand Down Expand Up @@ -831,3 +852,4 @@ reproduce this information
requirements a DEM must fulfill to be helpful to OGGM. And we also explain
why and how we preprocess some DEMs before we make them available to the
OGGM workflow.

0 comments on commit 04847e5

Please sign in to comment.