docs/1_PearlRiverIntro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![UCL](https://github.com/profLewis/Geog2021_Coursework/blob/master/images/ucl_logo.png?raw=true)\n",
    "\n",
    "# Geog0027 Coursework"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepared by:\n",
    "\n",
    "[Dr Qingling Wu](http://www.geog.ucl.ac.uk/about-the-department/people/academic-staff/qingling-wu), [Prof Philip Lewis](http://www2.geog.ucl.ac.uk/~plewis/), [Dr Mathias Disney](http://www.geog.ucl.ac.uk/about-the-department/people/academic-staff/mat-disney)\n",
    "\n",
    "Contact: [Prof Lewis](<mailto: p.lewis@ucl.ac.uk>)\n",
    "\n",
    "------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Pearl River Delta, and particularly Shenzhen City, in Southern China has been going through a dramatic urbanization process since 1978 due to decentralization policies and market-oriented reforms. \n",
    "\n",
    "![Shenzhen 1970](https://encounteringurbanization.files.wordpress.com/2011/06/picture3.png)\n",
    "[**Shenzhen City 1970**](https://encounteringurbanization.wordpress.com/2011/06/22/shenzhen-the-instant-city/)\n",
    "\n",
    "![Shenzhen 2011](https://encounteringurbanization.files.wordpress.com/2011/06/picture1.png)\n",
    "[**Shenzhen City 2011**](https://encounteringurbanization.wordpress.com/2011/06/22/shenzhen-the-instant-city/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"100%\"\n",
       "            height=\"490\"\n",
       "            src=\"http://svs.gsfc.nasa.gov/stories/Landsat/pearl_river.html\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x10910a2b0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "from IPython.display import IFrame\n",
    "IFrame('http://svs.gsfc.nasa.gov/stories/Landsat/pearl_river.html','100%',490)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"100%\"\n",
       "            height=\"490\"\n",
       "            src=\"https://earthengine.google.org/timelapse/player?c=https%3A%2F%2Fearthengine.google.org%2Ftimelapse%2Fdata&v=22.5500,114.1000,8.5&r=.5&p=true\"\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x10910a4e0>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "IFrame('https://earthengine.google.org/timelapse/player?c=https%3A%2F%2Fearthengine.google.org%2Ftimelapse%2Fdata&v=22.5500,114.1000,8.5&r=.5&p=true\"','100%',490)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "![encounteringurbanization.files.wordpress.com: Shenzhen 2011](https://encounteringurbanization.files.wordpress.com/2011/06/picture7.png\n",
    ")\n",
    "[**Shenzhen Plan of 2005**](https://encounteringurbanization.wordpress.com/2011/06/22/shenzhen-the-instant-city/)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Purpose of the practical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, we aim to quantify the change of land use due to urbanization, and attempt to explain such change by a number of important socioeconomical drivers. Yearly Landsat TM images are given from 1986 to present. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview of task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The project is most clearly phrased in two parts:\n",
    "    \n",
    "**1. Classification of land cover from a series of Landsat images**\n",
    "\n",
    "As a first step, we need to quantify at least *three* types of land use for each given year, namely, **urban**, **cropland**, and **other**. In many circumstances, we would want to have cartegories of other as well, but this is not strictly necessary for this practical. Then, we can identify where changes have happened and how much land area has been transformed into urban. See example the land use change map below from [Seto et al. 2002](reading/Setoetal2002.pdf) as anexample of this.\n",
    " \n",
    "![seto](images/seto.png)\n",
    "\n",
    "We want to be able to classify quite a large number of images, so we need to bear that in mind when considering what approach to take.\n",
    "  \n",
    "A number of classification methods have been taught in this module, and Maximum Likelihood is often a good starting point for multispectral TM imagery. There are also other image processing methods could help to identify land use classes (e.g. vegetation indices, filters, segmentation, etc.). However, if we need to apply training for each scene, the task soon becomes very tedious. We could try to use the same training data for each scene, but this might not work well across multiple sensors.\n",
    "\n",
    "Instead, we shall use a method of unsupervised classification (ISOdata clustering) to process the bulk of the data. This shifts the problem from one of large manual effort in providing the training datasets to a smaller manual effort making sure we interpret the machine-generated clusters appropriately. This is a basic trade-off of unsupervised versus supervised methods. \n",
    "\n",
    "In this work then, you will apply a supervised method and an unsupervised method to *one* scene, then assess the classification accuracy of your approaches. Because of the large number of images involved in the project, we will use an envi program (provided to you) to automate the process of unsupervised classification for the rest of the datasets. \n",
    "\n",
    "After you have performed the classifications, you will process each dataset to estimate the area of urban land cover per year. Again, because of the large number of images involved, we will use an envi program to do this.\n",
    "\n",
    "**2. Modelling Land Use change with socio-economic drivers**\n",
    "\n",
    "Having derived a dataset of urban land use, we can calculate urban land use change per year.\n",
    "\n",
    "Following the general approach of [Seto & Kaufmann (2003)](reading/SetoandKaufmann2003.pdf), we will build a multi-linear model to attempt to describe the urban land use change per year (the 'y' variable) as a function of a number of key socioeconomic factors (e.g. capital investment, land productivity, population, wage rates, etc) (the '$x$' variables). \n",
    "\n",
    "Following the example in the paper, we will use the following in $x$:\n",
    "\n",
    "* $x_1$: Investment in capital construction / population\n",
    "* $x_2$: value of gross agricultural output / agriculture population\n",
    "* $x_3$: log(wages in non-state, non-collective units)\n",
    "* $x_4$: log(average total wage)\n",
    "* $x_5$: (value of gross agricultural output/Agricultural land) / (value of gross industrial output / Urban land)\n",
    "\n",
    "This will give a model with 6 parameters that we need to estimate (i.e. 6 unknowns) that we could call $p_0, p_1, p_2, p_3, p_4, p_5$. We could then write the model more specifically as:\n",
    "\n",
    "**Equation 1:**\n",
    "\n",
    "$$\n",
    "y = p_0 + p_1 x_1 + p_2 x_2 + + p_3 x_3 + p_4 x_4 + p_5 x_5\n",
    "$$\n",
    "\n",
    "Note that the final term ($x_5$) requires that we have data for Agricultural and Urban land, which we will need to derive from the remote sensing data for each year of observation.\n",
    "\n",
    "The rest of the data will come from official statistics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "The project involves the following tasks:\n",
    "\n",
    "### Data processing\n",
    "\n",
    "* Download Landsat annual datasets for 1986 to present (or some suitable subset). \n",
    "* For one year (your choice), perform a supervised classification and an unsupervised classification using `envi` and assess the accuracy of the classifications; \n",
    "* Perform unsupervised classifications (clustering) of the time series of Landsat data, using an `envi` program that you will be provided with;\n",
    "    * apply suitable class labels, and modify the number of classes as appropriate; \n",
    "* Calculate the area of urban land use for Shenzhen for each year\n",
    "* Estimate the area of agricultural land use for Shenzhen for each year (if possible, not critical)\n",
    "* Try to assign a value of uncertainty to the derived data (from earlier accuracy assessment)\n",
    "\n",
    "### Modelling\n",
    "\n",
    "Following the general approach of [Seto & Kaufmann (2003)](reading/SetoandKaufmann2003.pdf), we will build a multi-linear model to attempt to describe the urban land use change per year (the 'y' variable) as a function of a number of key socioeconomic factors (e.g. capital investment, land productivity, population, wage rates, etc) (the '$x$' variables). \n",
    "\n",
    "**Equation 1:**\n",
    "\n",
    "$$\n",
    "y = p_0 + p_1 x_1 + p_2 x_2 + + p_3 x_3 + p_4 x_4 + p_5 x_5\n",
    "$$\n",
    "\n",
    "The model relates socio-economic variables (constant, plus `x1, x2, x3, x4, x5`), weighted by model parameters (`p0, p1, p2, p3, p4, p5`) to predict the rate if change of urban area per year (`du_dy`).\n",
    "\n",
    "We have taken a set of observations of `du_dy`, derived from Landsat land cover classifications for the years 1986 to present (or a subset). Along with estimates of the `x` variables from the Guangdong yearbook, we have then seen how to produce an esrimate of the model parameters (the `p` terms).\n",
    "\n",
    "Using the data derived above, calibrate a model that describes urban land use change as a function of a set of socioeconomic factors, following the approach of Seto et al. (2002, 2003). You are provided with `R` [code and appropriate datasets to achieve this](http://www2.geog.ucl.ac.uk/~plewis/GEOG0027/UrbanModel.zip).\n",
    "\n",
    "Analyse the statistics of the model and experiment to try to find an improved model with fewer parameters.\n",
    "\n",
    "\n",
    "You are free to perform additional experiment, with the expectation of higher marks, provided (i) you have done the basic requirements well enough, and (ii) you show clarity of thoiught and understanding of what you are doing in your experiments.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Introduction](1_PearlRiverIntro.ipynb)\n",
    "* [ENVI Setup](2_RunningENVIOnYourOwnComputer.ipynb)\n",
    "* [Modelling Setup](3_RunningROnYourOwnComputer.ipynb)\n",
    "* [Google Download](4_DownloadLandsatFromGEE.ipynb)\n",
    "* [Classification](5_PearlRiverClassification.ipynb)\n",
    "* [Modelling](5_UrbanModel.ipynb)\n",
    "* [Project Write up](7_WriteUp.ipynb)\n",
    "* [Project Advice](<8_ProjectAdvice.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further Reading"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Hao, P. (2012) Spatial evolution of urban villages in Shenzhen. PhD thesis University of Twente; Summaries in Dutch and English. ITC Dissertation 205, ISBN: 978-90-6266-295-1.](https://webapps.itc.utwente.nl/librarywww/papers_2012/phd/puhao.pdf)\n",
    "\n",
    "[Seto, K.C., Woodcock, C.E., Song, C., Huand, X., Lu, J., and Kaufmann, R.K. (2002) Monitoring land-use change in the Pearl River Delta using Landsat TM, Int J. Rem. Sens, 23(10).](reading/Setoetal2002.pdf)\n",
    "\n",
    "[Seto, K.C., and Kaufmann, R.K., (2003) Modeling the Drivers of Urban Land Use Change in the Pearl River Delta, China: Integrating Remote Sensing with Socioeconomic Data, Land Economics, 79(1): 106-121](reading/SetoandKaufmann2003.pdf)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}