doc/jupyter/Demo/Demo_1a_compute_climatologies.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "de6a850f",
   "metadata": {},
   "source": [
    "# Compute Climatologies"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de63a73f",
   "metadata": {},
   "source": [
    "Model output and observations must be converted into annual climatologies for use with the mean climate driver. This has already been done for the observations provided by PCMDI. PMP provides a script for generating these input climatologies from other data. For more information about using the script, see [the documentation](https://github.com/PCMDI/pcmdi_metrics/tree/main/sample_setups/pcmdi_parameter_files/mean_climate/make_clims). The basic use is shown here:   \n",
    "```\n",
    "python pcmdi_compute_climatologies.py \\  \n",
    "-p clim_calc_cmip_inparam.py --options\n",
    "``` \n",
    "Examples of parameter files can be found under [sample_setups](https://github.com/PCMDI/pcmdi_metrics/tree/main/sample_setups/pcmdi_parameter_files/mean_climate/make_clims).  \n",
    "\n",
    "Five netcdf files are produced by this script: an annual climatology (AC) and seasonal climatologies (DJF, MAM, JJA, SON)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18e67f6e",
   "metadata": {},
   "source": [
    "## Passing parameters via parameter file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b68abfb",
   "metadata": {},
   "source": [
    "First, load custom demo directories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "863f326d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from user_choices import demo_data_directory, demo_output_directory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "284afd7d",
   "metadata": {},
   "source": [
    "The parameter file for this demo is shown here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6d96daa7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import os\n",
      "\n",
      "#\n",
      "#  OPTIONS ARE SET BY USER IN THIS FILE AS INDICATED BELOW BY:\n",
      "#\n",
      "#\n",
      "\n",
      "# VARIABLES TO USE\n",
      "vars = ['rlut']\n",
      "\n",
      "# START AND END DATES FOR CLIMATOLOGY\n",
      "start = '2003-01'\n",
      "end = '2018-12'\n",
      "\n",
      "# INPUT DATASET - CAN BE MODEL OR OBSERVATIONS\n",
      "infile = 'demo_data/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc'\n",
      "\n",
      "# DIRECTORY WHERE TO PUT RESULTS\n",
      "outfile = 'demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc'\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(\"basic_annual_cycle_param.py\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdbb358a",
   "metadata": {},
   "source": [
    "In this example, all of the parameters are set in the parameter file shown above. The climatology script is then run from the command line with the parameter file as the sole input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5bff62ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start and end are  2003-01   2018-12\n",
      "variable list:  ['rlut']\n",
      "infilename is  rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "outdir is  None\n",
      "start_yr_str is  2003\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "(12, 180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "pcmdi_compute_climatologies.py -p basic_annual_cycle_param.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39a0611c",
   "metadata": {},
   "source": [
    "## Passing parameters via command line"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a299a47a",
   "metadata": {},
   "source": [
    "The following example uses the command line to pass arguments rather than a parameter file. Bash cell magic is used to pass the directory names (referenced as `$1` and `$2`) and run the command as a subprocess.\n",
    "\n",
    "Similar to the mean climate metrics, filenames can use chain notation to substitute variables into the file name. In this case, the `%(variable)` placeholder shows the climatology script where to substitute the variable name into the file name. This functionality is useful for model data where there are separate timeseries files to read in for each variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "75f301b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start and end are  2003-01   2018-12\n",
      "variable list:  ['rlut']\n",
      "infilename is  rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "outdir is  None\n",
      "start_yr_str is  2003\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "(12, 180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n"
     ]
    }
   ],
   "source": [
    "%%bash -s \"$demo_data_directory\" \"$demo_output_directory\"\n",
    "pcmdi_compute_climatologies.py \\\n",
    "--var rlut \\\n",
    "--start 2003-01 --end 2018-12 \\\n",
    "--outfile $2'/climo/%(variable)_mon_CERES-EBAF-4-1_BE_gn.nc' \\\n",
    "--infile $1/obs4MIPs_PCMDI_monthly/NASA-LaRC/CERES-EBAF-4-1/mon/rlut/gn/v20210727/rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a529d8b",
   "metadata": {},
   "source": [
    "## Mixing inputs and specifying the output directory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e107f8d",
   "metadata": {},
   "source": [
    "Parameters that are specified on the command line override those in the parameter file when both are provided. In this case the output directory and file name are specified separately on the command line using the variables `outpath` and `outfilename`. This functionality is usefuly for creating batch climatologies scripts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ede12487",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start and end are  2003-01   2018-12\n",
      "variable list:  ['rlut']\n",
      "infilename is  rlut_mon_CERES-EBAF-4-1_PCMDI_gn_200301-201812.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "outdir is  demo_output/climo/\n",
      "start_yr_str is  2003\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "(12, 180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.AC.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.DJF.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.MAM.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.JJA.v20220420.nc\n",
      "outfd is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.nc\n",
      "out is  demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n",
      "(180, 360)   (12, 180, 360)   demo_output/climo/rlut_mon_CERES-EBAF-4-1_BE_gn.200301-201812.SON.v20220420.nc\n"
     ]
    }
   ],
   "source": [
    "%%bash -s \"$demo_data_directory\" \"$demo_output_directory\"\n",
    "pcmdi_compute_climatologies.py \\\n",
    "-p basic_annual_cycle_param.py \\\n",
    "--outpath $2/climo/ \\\n",
    "--outfilename rlut_mon_CERES-EBAF-4-1_BE_gn.nc \\"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}