03.Time-series-analysis/2.1.time-series-analysis_data-download.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5e8631b7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "<img src=\"../images/5thNRST_logo.png\" alt=\"NOWPAP\" width=\"900\" height=\"700\">\n",
    "\n",
    "## NOWPAP REMOTE SENSING TRAINING COURSE   \n",
    "#### Webinar 2: Monitoring and Assessment of Water Quality by Ocean Color Remote Sensing\n",
    "<span style=\"color:brown; font-size:1.1em;font-style:italic\">Time series analysis: data download</span>\n",
    "\n",
    "<hr style=\"border:2px solid rgb(30,70,125)\"> </hr>\n",
    "\n",
    "<span style=\"font-family:Helvieta; font-size:2em;\">Eligio Maure (maure@npec.or.jp)</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6213e392",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "***\n",
    "**Note 1**: Execute each cell through the <button class=\"btn btn-default btn-xs\"><i class=\"icon-play fa fa-play\"></i></button> button from the top MENU (or keyboard shortcut `Shift` + `Enter`).<br>\n",
    "<br>\n",
    "**Note 2**: If, for any reason, the kernel is not working anymore, in the top MENU, click on the <button class=\"btn btn-default btn-xs\"><i class=\"fa fa-repeat icon-repeat\"></i></button> button. Then, in the top MENU, click on \"Cell\" and select \"Run All Above Selected Cell\".<br>\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "357bedc8",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Table of contents\n",
    "- [1. Purpose](#1.-Purpose)\n",
    "- [2. Library import](#2.-Library-import)\n",
    "- [3. Define download parameters](#3.-Define-download-parameters)\n",
    "- [4. Retrieve data](#4.-Retrieve-data)\n",
    "- [5. Resources](#5.-Resources)\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "855dd9f5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 1. Purpose\n",
    "[Return to the \"Table of contents\"](#Table-of-contents)\n",
    "\n",
    "The objective of this exercise is to download satellite data products from the [Marine Environmental Watch](https://ocean.nowpap3.go.jp/?page_id=862) of Northwest Pacific Action Plan (NOWPAP).\n",
    "\n",
    "You will learn how to retrieve PNG and NetCDF files and custom subareas\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "faa21d18",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 2. Library import\n",
    "[Return to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "89f10c9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# System\n",
    "import os\n",
    "import sys\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9dcafdf4",
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.path.append(str(Path('.').absolute().parent)) \n",
    "import sutils as util"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f2625ed",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 3. Define download parameters\n",
    "[Return to the \"Table of contents\"](#Table-of-contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6e6b4bc",
   "metadata": {},
   "source": [
    "| [Subarea acronym](https://ocean.nowpap3.go.jp/?page_id=2297#211_Naming_Convection) | Subarea name |  \n",
    "|:-- |:-------------|\n",
    "| NW | NOWPAP Sea Area |  \n",
    "| AS | Ariake Sea |  \n",
    "| JB | Jinhae Bay |\n",
    "| KS | North Kyushu Sea Area-Genkai Sea |  \n",
    "| PB | Peter the Great Bay |  \n",
    "| SP | Northern Shandong Peninsula |  \n",
    "| SS | Southern Korean Peninsula |  \n",
    "| TB | Toyama Bay |  \n",
    "| YR | Yangtze River Estuary |  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4c9ecb1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# =================================================================\n",
    "SBA = 'TB'     # Other subareas (TB-Toyama Bay, AS-Ariake Sea, etc.)\n",
    "VAR = 'CHL'    # variable name ('CHL', 'SST', 'CDOM', 'TSM')\n",
    "COMP = 'month'   # composite period ('day', 'month', 'year')\n",
    "\n",
    "SY = 2020      # start year\n",
    "SM = 1         # start month\n",
    "SD = 1        # start day of month (only needed if `comp` == 'day')\n",
    "\n",
    "EY = 2020      # end year \n",
    "EM = 12        # end month\n",
    "ED = 31        # end day (only needed if `comp` == 'day')\n",
    "\n",
    "EXT = ('nc',)  # file extension to download, ('nc',), ('png',) or ('nc','png') \n",
    "\n",
    "# Sensor(s) to download, sens = ('SeaWiFS', 'MODIS-Aqua', 'MERIS', 'VIIRS-SNPP', 'SGLI', 'YOC') or sens = ('SeaWiFS',)\n",
    "SEN = ('SGLI',)\n",
    "# sens = 'SeaWiFS', 'MODIS-Aqua', 'MERIS', 'VIIRS-SNPP', 'YOC'\n",
    "OUTPUT_DIR = f'data/{{sen}}/{COMP}'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b623497",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 4. Retrieve data\n",
    "[Return to the \"Table of contents\"](#Table-of-contents)\n",
    "- retrieving data from NMEW"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b666ac1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|                                                                                          | 0/276 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "YOC\n",
      "C:\\Users\\Eligio\\Documents\\NPEC\\2021\\5th-NOWPAP-Training-Course-on-Remote-Sensing-Data-Analysis\\03.Time-series-analysis\\data\\YOC\\month\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading: Y202012_CHL_TB_month.nc: 100%|██████████████████████████████████████████| 276/276 [14:49<00:00,  3.22s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "done!\n",
      "OutputDir: C:\\Users\\Eligio\\Documents\\NPEC\\2021\\5th-NOWPAP-Training-Course-on-Remote-Sensing-Data-Analysis\\03.Time-series-analysis\\data\\YOC\\month\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "for sen in SEN:\n",
    "    print(f'\\n{sen}')\n",
    "\n",
    "    opath = Path(OUTPUT_DIR.format(sen=sen)).absolute()\n",
    "    if opath.is_dir() is False:\n",
    "        opath.mkdir(parents=True)\n",
    "        print(opath)\n",
    "    util.download(variable=VAR, subarea=SBA, sensor=sen\n",
    "                  , start_year=SY, end_year=EY\n",
    "                  , composite_period=COMP, file_type=EXT\n",
    "                  , start_month=SM, end_month=EM\n",
    "                  , start_day=SD, end_day=ED, output_dir=opath)\n",
    "\n",
    "print(f'\\ndone!\\nOutputDir: {str(opath)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f294a41",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div class=\"alert alert-block alert-success\"/div>\n",
    "    <b>Done!</b><br>\n",
    "  \n",
    "--- \n",
    "\n",
    "#### And thank you for your attention! :) \n",
    "    Now try the same exercise with other months or sensors, etc.\n",
    "    We hope you enjoyed the training provided by NOWPAP CEARAC."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c02e847",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 5. Resources\n",
    "[Go back to the \"Table of contents\"](#Table-of-contents)\n",
    "1. https://ocean.nowpap3.go.jp/\n",
    "2. https://github.com/npec/NMEW.demos/blob/master/NMEW_bulk_download_demo.ipynb\n",
    "3. https://github.com/npec/5th-NOWPAP-Training-Course-on-Remote-Sensing-Data-Analysis"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}