docs/notebooks/cli.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "# Workaround for determining the notebook folder within a running notebook\n",
    "# This cell is not visible when the documentation is built.\n",
    "\n",
    "from __future__ import annotations\n",
    "\n",
    "try:\n",
    "    from _finder import _find_current_folder\n",
    "\n",
    "    notebook_folder = _find_current_folder()\n",
    "except ImportError:\n",
    "    from pathlib import Path\n",
    "\n",
    "    notebook_folder = Path().cwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Command Line Interface\n",
    "\n",
    "xclim provides the `xclim` command line executable to perform basic indicator\n",
    "computation easily without having to start up a full Python environment. However, not\n",
    "all indicators listed in [Climate Indicators](../indicators.rst) are available through this tool.\n",
    "\n",
    "Its use is simple; Type the following to see the usage message:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To list all available indicators, use the \"indices\" subcommand:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim indices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information about a specific indicator, you can either use the `info` sub-command or directly access the `--help` message of the indicator. The former gives more information about the metadata, while the latter only prints the usage. Note that the module name (`atmos`, `land` or `seaIce`) is mandatory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim info liquidprcptot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the usage message, `VAR_NAME` indicates that the passed argument must match a variable in the input dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import annotations\n",
    "\n",
    "import warnings\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import xarray as xr\n",
    "from pandas.plotting import register_matplotlib_converters\n",
    "\n",
    "register_matplotlib_converters()\n",
    "warnings.filterwarnings(\"ignore\", \"implicitly registered datetime converter\")\n",
    "%matplotlib inline\n",
    "xr.set_options(display_style=\"html\")\n",
    "\n",
    "\n",
    "time = pd.date_range(\"2000-01-01\", periods=366)\n",
    "tasmin = xr.DataArray(\n",
    "    -5 * np.cos(2 * np.pi * time.dayofyear / 365) + 273.15,\n",
    "    dims=\"time\",\n",
    "    coords={\"time\": time},\n",
    "    attrs={\"units\": \"K\"},\n",
    ")\n",
    "tasmax = xr.DataArray(\n",
    "    -5 * np.cos(2 * np.pi * time.dayofyear / 365) + 283.15,\n",
    "    dims=\"time\",\n",
    "    coords={\"time\": time},\n",
    "    attrs={\"units\": \"K\"},\n",
    ")\n",
    "pr = xr.DataArray(\n",
    "    np.clip(10 * np.sin(18 * np.pi * time.dayofyear / 365), 0, None),\n",
    "    dims=\"time\",\n",
    "    coords={\"time\": time},\n",
    "    attrs={\"units\": \"mm/d\"},\n",
    ")\n",
    "ds = xr.Dataset({\"tasmin\": tasmin, \"tasmax\": tasmax, \"pr\": pr})\n",
    "\n",
    "data_folder = notebook_folder / \"data\"\n",
    "data_folder.mkdir(exist_ok=True)\n",
    "ds.to_netcdf(data_folder / \"example_data.nc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Computing indicators\n",
    "--------------------\n",
    "\n",
    "Let's say we have the following toy dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "ds = xr.open_dataset(data_folder.joinpath(\"example_data.nc\"))\n",
    "display(ds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig1, (ax_tas, ax_pr) = plt.subplots(1, 2, figsize=(10, 5))\n",
    "ds.tasmin.plot(label=\"tasmin\", ax=ax_tas)\n",
    "ds.tasmax.plot(label=\"tasmax\", ax=ax_tas)\n",
    "ds.pr.plot(ax=ax_pr)\n",
    "ax_tas.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute an **indicator**, say the monthly solid precipitation accumulation, we simply call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim -i data/example_data.nc -o data/out1.nc solidprcptot --pr pr --tas tasmin --freq MS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we decided to use `tasmin` for the `tas` variable. We didn't need to provide the `--pr` parameter, as our data has the same name.\n",
    "\n",
    "Finally, more than one indicator can be computed and written to the output dataset by simply chaining the calls:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim -i data/example_data.nc -o data/out2.nc liquidprcptot --tas tasmin --freq MS tropical_nights --thresh \"2 degC\" --freq MS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds1 = xr.open_dataset(data_folder / \"out1.nc\")\n",
    "ds2 = xr.open_dataset(data_folder / \"out2.nc\", decode_timedelta=False)\n",
    "\n",
    "fig2, (ax_prcptot, ax_tropical_nights) = plt.subplots(1, 2, figsize=(10, 5))\n",
    "ds1.solidprcptot.plot(ax=ax_prcptot, label=ds1.solidprcptot.long_name)\n",
    "ds2.liquidprcptot.plot(ax=ax_prcptot, label=ds2.liquidprcptot.long_name)\n",
    "ds2.tropical_nights.plot(ax=ax_tropical_nights, marker=\"o\")\n",
    "ax_prcptot.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds1.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds2.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data Quality Checks\n",
    "-------------------\n",
    "\n",
    "As of version 0.30.0, `xclim` now also provides a command-line utility for performing data quality control checks on existing NetCDF files.\n",
    "\n",
    "These checks examine the values of data_variables for suspicious value patterns (e.g. values that repeat for many days) or erroneous values (e.g. humidity percentages outside 0-100, minimum temperatures exceeding maximum temperatures, etc.). The checks (called ``dataflags``) are based on the ECAD ICCLIM quality control checks (https://www.ecad.eu/documents/atbd.pdf).\n",
    "\n",
    "The full list of checks performed for each variable are listed in `xclim/core/data/variables.yml`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim dataflags --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When running the ``dataflags`` CLI checks, you must either set an output file (`-o filename.nc`) or set the checks to raise if there are any failed checks (`-r`).\n",
    "\n",
    "By default, when setting an output file, the returned file will only contain the flag value\n",
    "(`True` if no flags were raised, `False` otherwise). To append the flag to a copy of the dataset, we use the `-a` option.\n",
    "\n",
    "The default behaviour is to raise a flag if any element of the array resolves to `True` (i.e. aggregated across all dimensions), but we can specify the level of aggregation by dimension with the `-d` or `--dims` option."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an output file with just the flag value and no aggregation (dims=None)\n",
    "\n",
    "!xclim -i data/example_data.nc -o data/flag_output.nc dataflags -d none\n",
    "\n",
    "# Need to wait until the file is written\n",
    "\n",
    "!sleep 2s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "ds1 = xr.open_dataset(data_folder / \"flag_output.nc\")\n",
    "display(ds1.data_vars, ds1.ecad_qc_flag)\n",
    "ds1.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an output file with values appended to the original dataset.\n",
    "\n",
    "!xclim -i data/example_data.nc -o data/flag_output_appended.nc dataflags -a\n",
    "\n",
    "# Need to wait until the file is written\n",
    "!sleep 2s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "ds2 = xr.open_dataset(data_folder / \"flag_output_appended.nc\")\n",
    "display(ds2.data_vars, ds2.ecad_qc_flag)\n",
    "ds2.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Raise an error if any quality control checks fail. Passing example:\n",
    "\n",
    "!xclim -i data/example_data.nc dataflags -r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "# Create some bad data with minimum temperatures exceeding max temperatures\n",
    "bad_ds = xr.open_dataset(data_folder / \"example_data.nc\")\n",
    "\n",
    "# Swap entire variable arrays\n",
    "bad_ds[\"tasmin\"].values, bad_ds[\"tasmax\"].values = (\n",
    "    bad_ds.tasmax.values,\n",
    "    bad_ds.tasmin.values,\n",
    ")\n",
    "bad_ds.to_netcdf(data_folder / \"suspicious_data.nc\")\n",
    "bad_ds.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Raise an error if any quality control checks fail. Failing example:\n",
    "\n",
    "!xclim -i data/suspicious_data.nc dataflags -r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These checks can also be set to examine a specific variable within a NetCDF file, with more descriptive information for each check performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xclim -i data/example_data.nc -o data/flag_output_pr.nc dataflags pr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "ds3 = xr.open_dataset(data_folder / \"flag_output_pr.nc\")\n",
    "display(ds3.data_vars)\n",
    "for dv in ds3.data_vars:\n",
    "    display(ds3[dv])"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}