notebooks/bmi_era5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://github.com/gantian127/bmi_era5/blob/master/docs/source/_static/bmi_era5_logo.png?raw=true\" width='600' align='center'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quick Start Tutorial "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook will demonstrate how to use the bmi_era5 package to download the [ERA5](https://confluence.ecmwf.int/display/CKB/ERA5) datasets. \n",
    "\n",
    "This tutorial includes the following sections:\n",
    "\n",
    "1. [Brief Introduction](#section1)\n",
    "\n",
    "   This section provides basic information about bmi_era5 package. \n",
    "   <br>\n",
    "   \n",
    "2. [Start with Examples](#section2)\n",
    "   \n",
    "   This section provides two examples to demonstrate how to use the CDS API and the bmi_era5 package to download the same ERA5 dataset for visualization.\n",
    "   <br>\n",
    "   \n",
    "\n",
    "\n",
    "**Suggested citation**: Gan, T. (2023). Jupyter Notebooks for the ERA5 Data Component, HydroShare, https://www.hydroshare.org/resource/765e4e8bebab4eea8d72ee447a27c2fa/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='section1'></a>\n",
    "## 1. Brief Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "bmi_era5 package is an implementation of the Basic Model Interface ([BMI](https://bmi.readthedocs.io/en/latest/)) for the [ERA5](https://confluence.ecmwf.int/display/CKB/ERA5) dataset. This package uses the [CDS API](https://cds.climate.copernicus.eu/api-how-to) to download the ERA5 dataset and wraps the dataset with BMI for data control and query (currently support 3 dimensional ERA5 dataset). This package is not implemented for people to use but is the key element to convert the ERA5 dataset into a data component for the [PyMT](https://pymt.readthedocs.io/en/latest/?badge=latest) modeling framework developed by Community Surface Dynamics Modeling System ([CSDMS](https://csdms.colorado.edu/wiki/Main_Page)).\n",
    "\n",
    "   \n",
    "To install bmi_era5 package, please follow the instructions [here](https://github.com/gantian127/bmi_era5/#install-package)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can uncomment the code below to use install_cds( ) to install the CDS API Key file. This file is required for data download. So please make sure you have already created an account at the [CDS registration page](https://cds.climate.copernicus.eu/#!/home) and obtained your CDS API Key. For more details please check [here](https://cds.climate.copernicus.eu/api-how-to)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from installation import install_cds\n",
    "\n",
    "# install_cds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='section2'></a>\n",
    "## 2. Start with Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CDS API (Climate Data Store Application Program Interface) is a service providing Python based programmatic access to CDS data. ERA5 is one of the CDS data products that can be accessed using the CDS API.  \n",
    "\n",
    "In bmi_era5 package, BmiEra5 class is designed to download the ERA5 dataset and wrap the dataset with BMI (currently support 3 dimensional ERA5 dataset). The following examples demonstrate how to download the same dataset using the CDS API and BmiEra5 for data visualization. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1: use CDS API to download the ERA5 data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import cdsapi module and create an object from the \"Client\" class. Use the retrieve( ) method to make a request for data download. You can check the details of the parameter settings for a request from the [CDS web interface](https://confluence.ecmwf.int/display/CKB/How+to+download+ERA5#HowtodownloadERA5-3-DownloadingonlineERA5familydatathroughtheCDSwebinterface). \n",
    "\n",
    "In this example, it downloads the ERA5 hourly reanalysis data on single level for 2 metre temperature and total precipitation in Colorado on Jan 1st, 2021. The grid resolution is 0.25 degree. The dataset is saved in a netCDF file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cdsapi\n",
    "import matplotlib.pyplot as plt\n",
    "import xarray\n",
    "\n",
    "c = cdsapi.Client()\n",
    "\n",
    "c.retrieve(\n",
    "    \"reanalysis-era5-single-levels\",\n",
    "    {\n",
    "        \"product_type\": \"reanalysis\",\n",
    "        \"format\": \"netcdf\",\n",
    "        \"variable\": [\"2m_temperature\", \"total_precipitation\"],\n",
    "        \"year\": \"2021\",\n",
    "        \"month\": \"01\",\n",
    "        \"day\": \"01\",\n",
    "        \"time\": [\"00:00\", \"01:00\", \"02:00\"],\n",
    "        \"area\": [41, -109, 36, -102],\n",
    "        \"grid\": [0.25, 0.25],\n",
    "    },\n",
    "    \"download.nc\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use xarray to load the netCDF file and select the data for the 2 metre temperature on 2021-01-01 at 00:00. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load netCDF data\n",
    "dataset = xarray.open_dataset(\"download.nc\")\n",
    "\n",
    "# select 2 metre temperature on 2021-01-01 at 00:00\n",
    "air_temp = dataset.t2m.isel(time=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The temperature dataset is loaded as a DataArray object (xarray). You can directly plot the data using its plot( ) method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot data\n",
    "air_temp.plot(figsize=(9, 5))\n",
    "plt.title(\"2 metre temperature in Colorado on Jan 1st, 2021 at 00:00\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 2: use BmiEra5 class to download the ERA5 data (Demonstration of how to use BMI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import BmiEra5 class and instantiate it. A configuration file (yaml file) is required to provide the parameter settings for data download. An example config_file.yaml file is provided in the same folder with this Jupyter Notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "from bmi_era5 import BmiEra5\n",
    "\n",
    "data_comp = BmiEra5()\n",
    "data_comp.initialize(\"config_file.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you check the contents in the config_file.yaml, you will find out that the parameters are the same as the parameters specified in the retrieve( ) method from the CDS API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cat config_file.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use variable related methods from BmiEra5 class to check the variable information of the ERA5 dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get variable info\n",
    "for var_name in data_comp.get_output_var_names():\n",
    "    var_unit = data_comp.get_var_units(var_name)\n",
    "    var_location = data_comp.get_var_location(var_name)\n",
    "    var_type = data_comp.get_var_type(var_name)\n",
    "    var_grid = data_comp.get_var_grid(var_name)\n",
    "    var_itemsize = data_comp.get_var_itemsize(var_name)\n",
    "    var_nbytes = data_comp.get_var_nbytes(var_name)\n",
    "\n",
    "    print(f\"{var_name=}\")\n",
    "    print(f\"{var_unit=}\")\n",
    "    print(f\"{var_location=}\")\n",
    "    print(f\"{var_type=}\")\n",
    "    print(f\"{var_grid=}\")\n",
    "    print(f\"{var_itemsize=}\")\n",
    "    print(f\"{var_nbytes=}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use time related methods from BmiEra5 class to check the time information of the ERA5 dataset. The time values are stored in a format which follows [CF convention](http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get time info\n",
    "start_time = data_comp.get_start_time()\n",
    "end_time = data_comp.get_end_time()\n",
    "time_step = data_comp.get_time_step()\n",
    "time_unit = data_comp.get_time_units()\n",
    "time_steps = int((end_time - start_time) / time_step) + 1\n",
    "\n",
    "print(f\"{start_time=}\")\n",
    "print(f\"{end_time=}\")\n",
    "print(f\"{time_step=}\")\n",
    "print(f\"{time_unit=}\")\n",
    "print(f\"{time_steps=}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use grid related methods of BmiEra5 class to check the grid information of the ERA5 dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get variable grid info\n",
    "grid_rank = data_comp.get_grid_rank(var_grid)\n",
    "grid_size = data_comp.get_grid_size(var_grid)\n",
    "\n",
    "grid_shape = np.empty(grid_rank, int)\n",
    "data_comp.get_grid_shape(var_grid, grid_shape)\n",
    "\n",
    "grid_spacing = np.empty(grid_rank)\n",
    "data_comp.get_grid_spacing(var_grid, grid_spacing)\n",
    "\n",
    "grid_origin = np.empty(grid_rank)\n",
    "data_comp.get_grid_origin(var_grid, grid_origin)\n",
    "\n",
    "print(f\"{grid_rank=}\")\n",
    "print(f\"{grid_size=}\")\n",
    "print(f\"{grid_shape=}\")\n",
    "print(f\"{grid_spacing=}\")\n",
    "print(f\"{grid_origin=}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use get_value( ) method to get the 2 meter temperature data at the first time step (2021-01-01 00:00) as a numpy 2D array. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get variable data\n",
    "data = np.empty(grid_size, var_type)\n",
    "data_comp.get_value(\"2 metre temperature\", data)\n",
    "data_2D = data.reshape(grid_shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the 2 meter temperature data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get X, Y extent for plot\n",
    "min_y, min_x = grid_origin\n",
    "max_y = min_y + grid_spacing[0] * (grid_shape[0] - 1)\n",
    "max_x = min_x + grid_spacing[1] * (grid_shape[1] - 1)\n",
    "dy = grid_spacing[0] / 2\n",
    "dx = grid_spacing[1] / 2\n",
    "extent = [min_x - dx, max_x + dx, min_y - dy, max_y + dy]\n",
    "\n",
    "# plot data\n",
    "fig, ax = plt.subplots(1, 1, figsize=(9, 5))\n",
    "im = ax.imshow(data_2D, extent=extent)\n",
    "cbar = fig.colorbar(im)\n",
    "cbar.set_label(\"2 metre temperature [K]\")\n",
    "plt.xlabel(\"longitude [degree_east]\")\n",
    "plt.ylabel(\"latitude [degree_north]\")\n",
    "plt.title(\"2 metre temperature in Colorado on Jan 1st, 2021 at 00:00\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complete the example by finalizing the component. finalize( ) method performs tasks that take place after using the data component, such as deallocating memory and closing files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_comp.finalize()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}