SUMMEDupTutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6L1RpyfbPO1l"
   },
   "source": [
    "# Tutorial for the SUMup Dataset Explorer (SUMMEDup) tool\n",
    "\n",
    "Created by: Megan Thompson-Munson (metm9666@colorado.edu)\n",
    "\n",
    "Last updated: 27 January 2022"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lfeSRBl8QMEJ"
   },
   "source": [
    "## 1) Set-up and data downloading"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4rcSOm6mFJn2"
   },
   "source": [
    "### Google Colab Users: Uncomment and run the following cell\n",
    "\n",
    "To comment and uncomment, select the text you wish to modify and use the shortcut ```CTRL + /```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gzaJDmnTHeNj"
   },
   "outputs": [],
   "source": [
    "# # Hide unnecessary output\n",
    "# %%capture\n",
    "\n",
    "# # Mount drive so files on Google Drive can be accessed by this script.\n",
    "# # When this cell is run, it may output a link and pause running. \n",
    "# # Click the link, sign in, copy the code, and paste in the box that appears.\n",
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive/')\n",
    " \n",
    "#  # Set working directory\n",
    "# %cd '/content/drive/MyDrive/SUMMEDup/'\n",
    "\n",
    "# # Install required libraries and use correct versions of imgaug and shapely\n",
    "# !pip install simplekml\n",
    "# !pip install cartopy\n",
    "# !pip uninstall imgaug -y\n",
    "# !pip install imgaug==0.2.5\n",
    "# !pip uninstall shapely -y\n",
    "# !pip install shapely --no-binary shapely"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J5v4k96TFUjH"
   },
   "source": [
    "### All Users: Run the following cells to import SUMMEDup and download SUMup (if not already downloaded)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Kd0IDdPunv3o"
   },
   "outputs": [],
   "source": [
    "# Import the sumup library, which contains all other necessary libraries (e.g., numpy, matplotlib)\n",
    "import summedup as su"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only run the cell below if you do not have SUMup downloaded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NsvFLC71Fdxh"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "%%bash\n",
    "wget https://arcticdata.io/metacat/d1/mn/v2/object/urn%3Auuid%3A2512397e-effe-4b9d-9c5d-49c4b6ffdac6\n",
    "mv urn\\:uuid\\:2512397e-effe-4b9d-9c5d-49c4b6ffdac6 sumup_density_2020_v060121.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U1JnfdTCRxba"
   },
   "source": [
    "## 2) Use `ReadNetcdf` to create a dataframe from the NetCDF file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WdbTGoIYakZc"
   },
   "source": [
    "This function simply reads in the NetCDF and turns it into a dataframe of the raw data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "executionInfo": {
     "elapsed": 788,
     "status": "ok",
     "timestamp": 1639244439484,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "QCsZOwt2HBpb",
    "outputId": "784142a0-260f-410c-a049-1bae961ece68"
   },
   "outputs": [],
   "source": [
    "# Create dataframe called 'dfSumup'\n",
    "dfRaw = su.ReadNetcdf('sumup_density_2020_v060121.nc')\n",
    "\n",
    "# Show dataframe\n",
    "dfRaw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EiinKWhmYIpn"
   },
   "source": [
    "## 3) Use `Reformat` to add a unique core index to the dataframe and standardize units"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6xoWJsEnat_C"
   },
   "source": [
    "This function creates a dataframe with the same data but in a more useful format, and accomplishes the following tasks:\n",
    "* Fixes dates where only the year or only the year and month are given\n",
    "* Adds a unique \"CoreID\" to each core and sorts dataframe by ice sheet\n",
    "    * Antarctica: CoreID 0-886\n",
    "    * Greenland: CoreID 887-1689\n",
    "* Calculates the midpoint and thickness for each measurement\n",
    "* Standardizes units\n",
    "* Fixes any erroneous data points\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 461
    },
    "executionInfo": {
     "elapsed": 34185,
     "status": "ok",
     "timestamp": 1639244473665,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "D6tKYnrhSRp4",
    "outputId": "a349eace-8910-4c74-ab56-99ebd701bab5"
   },
   "outputs": [],
   "source": [
    "# Create new dataframe from raw data\n",
    "dfData = su.Reformat(dfRaw)\n",
    "\n",
    "# Show new dataframe\n",
    "dfData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cY2x6FKkJWyU"
   },
   "source": [
    "## 4) Use `GetInfo` to create a dataframe of information about each core and sort by a value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6r1blfkGcOOY"
   },
   "source": [
    "This function reads in the full processed dataframe and outputs a dataframe of information about the measurements. It also allows you to sort the dataframe by several different values:\n",
    "* `'CoreID'`\n",
    "* `'Citation'`\n",
    "* `'Method'`\n",
    "* `'Timestamp'`\n",
    "* `'Latitude'`\n",
    "* `'Longitude'`\n",
    "* `'Elevation (m)'`\n",
    "* `'Core Depth (m)'`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "executionInfo": {
     "elapsed": 9094,
     "status": "ok",
     "timestamp": 1639244482752,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "TYl4eN-KWTB_",
    "outputId": "9cacdc27-a119-4f3e-ae38-6fb317d7d028"
   },
   "outputs": [],
   "source": [
    "# Create a dataframe of locations sorted by depth\n",
    "# dfPoints = su.GetInfo(df=dfData, sort='Core Depth (m)')\n",
    "dfPoints = su.GetInfo(df=dfData, sort='CoreID')\n",
    "\n",
    "# Show new dataframe\n",
    "dfPoints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BopVU4FWBB6G"
   },
   "source": [
    "## 5) Use `FilterPoints` to search within the dataframe for cores that meet given conditions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rSiBKwAQBhN6"
   },
   "source": [
    "This function reads in the dataframe of information about the measurements and lets you select certain conditions. Essentially, it's a filter for the metadata so you can find entried in the dataframe that meet desired conditions. Available filters include:\n",
    "* `icesheet`: Default is `'both'`. Input either `'Antarctica'` or `'Greenland'`.\n",
    "* `citation`: Default includes all citations. Input an integer to filter for that citation. All citations can be found in the dataset readme.\n",
    "* `method`: Default includes all methods. Input an integer to filter for that method. All method codes can be found in the dataset readme.\n",
    "* `startDate` and `endDate`: Default includs all dates. Input date in the format `'YYYY-MM-DD'`.\n",
    "* `minLat` and `maxLat`: Default includes all latitudes. Input a value to filter latitudes.\n",
    "* `minLon` and `maxLon`: Default includes all longitudes. Input a value to filter longitudes.\n",
    "* `minElev` and `maxElev`: Default includes all elevations. Input a value to filter elevations.\n",
    "* `minDepth` and `maxDepth`: Default includes all core depths. Input a value to filter depths.\n",
    "\n",
    "All filters are optional."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 98
    },
    "executionInfo": {
     "elapsed": 13,
     "status": "ok",
     "timestamp": 1639244482754,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "UxoXvo6BBAeB",
    "outputId": "3fc91eed-a65b-4c11-9766-330be2e6f9fa"
   },
   "outputs": [],
   "source": [
    "# Try uncommenting each of these to filter by different values\n",
    "\n",
    "# dfFiltered = su.FilterPoints(df=dfPoints,startDate='2000-01-01')\n",
    "# dfFiltered = su.FilterPoints(df=dfPoints,icesheet='Greenland',minElev=1000,maxElev=1500)\n",
    "# dfFiltered = su.FilterPoints(df=dfPoints,minDepth=20,method=4)\n",
    "dfFiltered = su.FilterPoints(df=dfPoints,icesheet='Antarctica',endDate='1989-12-31',minDepth=10)\n",
    "\n",
    "# Print new dataframe\n",
    "dfFiltered"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UVxxyVAsmd_N"
   },
   "source": [
    "## 6) Use `PlotLocs` to plot locations of SUMup observations on both ice sheets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "168ebjoIkyz_"
   },
   "source": [
    "You can color the points by a value or a single color.\n",
    "\n",
    "At the minimum, provide a dataframe of locations: `su.PlotLocs(df=dfPoints)`\n",
    "\n",
    "Other optional arguments:\n",
    "* `color_by`: Select a color (e.g., `'blue'`) or a value to color by (e.g., `dfPoints['Max Depth (m)']`)\n",
    "* `color_map`: If you selected a value to color by, you can choose a color map for the shading (e.g., `'plasma'`)\n",
    "* `vmin` and `vmax`: If you selected a value to color by, you can set the minimum and maximum values for the color bar\n",
    "* `'save'`: Default is `'no'`. Choose `'yes'` to save the figure in the `figures` folder.\n",
    "\n",
    "Colors: https://matplotlib.org/stable/gallery/color/named_colors.html\n",
    "\n",
    "Color maps: https://matplotlib.org/stable/gallery/color/colormap_reference.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 486
    },
    "executionInfo": {
     "elapsed": 7619,
     "status": "ok",
     "timestamp": 1639244490366,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "gK67ZQ56iVR8",
    "outputId": "bc3ba441-ec82-4960-861b-629d93f52e09"
   },
   "outputs": [],
   "source": [
    "# Try uncommenting each of these to make different plots, or change the arguments to anything you'd like\n",
    "\n",
    "# su.PlotLocs(df=dfPoints)\n",
    "# su.PlotLocs(df=dfPoints, color_by='limegreen')\n",
    "# su.PlotLocs(df=dfPoints, color_by=dfPoints['Method'], color_map='tab20b')\n",
    "su.PlotLocs(df=dfPoints, color_by=dfPoints['Core Depth (m)'], color_map='plasma', vmin=0, vmax=100, save='no')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sE_LYc_SBDh9"
   },
   "source": [
    "## 7) Use `SavePoints` to save location data as `.csv` or `.kmz`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z_RlIgiyBUHG"
   },
   "source": [
    "Provide a dataframe, the file type, and whether you'd like it separated into files by ice sheet.\n",
    "*   `ftype`: choose `'csv'` or `'kmz'`\n",
    "*   `by_icesheet`: choose `'yes'` or `'no'`\n",
    "\n",
    "The output file(s) will be saved in the `output` folder. It can also be called on directly from the web version of [Google Earth](https://earth.google.com/web/). \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xqIhOUOJA1HV"
   },
   "outputs": [],
   "source": [
    "# Save a single kml file with both ice sheets included\n",
    "su.SavePoints(df=dfPoints, ftype='kmz', by_icesheet='no')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cSg-IASRMzba"
   },
   "source": [
    "## 8) Use `PlotDensity` to create a density profile figure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1YG8MfOq98L6"
   },
   "source": [
    "This function reads in the processed dataframe and given CoreID's and then plots density profiles.\n",
    "\n",
    "At the minimum provide a dataframe a 1 to 6 CoreIDs: `su.PlotDensity(df=df, CoreID=[180,839,1294]`\n",
    "\n",
    "Other optional arguments:\n",
    "* `color`: Default color is `'m'` for a single plot, and `['m','c','k','y','r','b']` for a plot with multiple lines. Use brackets if you specify a color (e.g., `color=['red']`)\n",
    "* `compare`: Default is `'no'`, but you can choose `'yes'` to show the profiles on the same plot rather than individual ones. Note that if you choose `'yes'`, all of the CoreID's should be from the same ice sheet\n",
    "    * Antarctica: 0-886\n",
    "    * Greenland: 887-1689\n",
    "* `'save'`: Default is `'no'`. Choose `'yes'` to save the figure in the `figures` folder.\n",
    "\n",
    "Note that many of the measurements in Antarctica are just one or two measurements of surface density so the plots are less interesting. Try plotting ones with CoreID's less than 26."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 586
    },
    "executionInfo": {
     "elapsed": 11,
     "status": "ok",
     "timestamp": 1639244490366,
     "user": {
      "displayName": "Megan Thompson-Munson",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "03738559122241243453"
     },
     "user_tz": 420
    },
    "id": "faLTrnonPw2f",
    "outputId": "5ccc0458-0988-4a7d-b6d4-aa2b709884c8"
   },
   "outputs": [],
   "source": [
    "# Try uncommenting these to view density profiles and different ways to visualize them\n",
    "\n",
    "# su.PlotDensity(df=dfData, CoreID=[1412])\n",
    "# su.PlotDensity(df=dfData, CoreID=[1135], color=['b'], save='yes')\n",
    "# su.PlotDensity(df=dfData, CoreID=[10,23,191,201], compare='yes', save='yes')\n",
    "# su.PlotDensity(df=dfData, CoreID=[913,914,915], color=['slateblue'])\n",
    "su.PlotDensity(df=dfData, CoreID=[2,6,8], color=['mediumslateblue','indigo','deeppink'], compare='yes')"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyN0WnQhiYRLckG5BZqjJs45",
   "collapsed_sections": [],
   "mount_file_id": "1KH3nwCQvyf7dhAhg1GUU17rw9-LxhNrY",
   "name": "SUMMEDupTutorial.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "analysis",
   "language": "python",
   "name": "analysis"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}