08. Data Visualization with Python/Week 1: Introduction to Data Visualization Tools/DV0101EN-Exercise-Dataset-Preprocessing-Exploring-with-Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"text-align:center\">\n",
    "    <a href=\"https://skills.network\" target=\"_blank\">\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\">\n",
    "    </a>\n",
    "</p>\n",
    "\n",
    "\n",
    "# Exploring and pre-processing a dataset using Pandas \n",
    "\n",
    "\n",
    "Estimated time needed: **30** minutes\n",
    "    \n",
    "\n",
    "## Objectives\n",
    "\n",
    "After completing this lab you will be able to:\n",
    "\n",
    "* Explore the dataset\n",
    "* Pre-process dataset as required (may be for visualization)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "The aim of this lab is to provide you a refresher on the **Pandas** library, so that you can pre-process and anlyse the datasets before applying data visualization techniques on it. This lab will work as acrash course on *pandas*. if you are interested in learning more about the *pandas* library, detailed description and explanation of how to use it and how to clean, munge, and process data stored in a *pandas* dataframe are provided in our course [**Data Analysis with Python**](https://www.coursera.org/learn/data-analysis-with-python?specialization=ibm-data-analyst) and [**Python for Applied Data Science**](https://www.coursera.org/learn/python-for-applied-data-science-ai?specialization=ibm-data-analyst)\n",
    "\n",
    "------------\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents\n",
    "\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "\n",
    "1. [Exploring Datasets with *pandas*](#0)<br>\n",
    "2. [The Dataset: Immigration to Canada from 1980 to 2013](#1)<br>\n",
    "3. [*pandas* Basics](#2) <br>\n",
    "4. [*pandas* Intermediate: Indexing and Selection](#3) <br>\n",
    "5. [*pandas* Filtering based on a criteria](#4)<br>\n",
    "6. [*pandas* Sorting Values](#5)\n",
    "\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring Datasets with *pandas* <a id=\"0\"></a>\n",
    "\n",
    "*pandas* is an essential data analysis toolkit for Python. From their [website](http://pandas.pydata.org/):\n",
    ">*pandas* is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, **real world** data analysis in Python.\n",
    "\n",
    "The course heavily relies on *pandas* for data wrangling, analysis, and visualization. We encourage you to spend some time and familiarize yourself with the *pandas* API Reference: http://pandas.pydata.org/pandas-docs/stable/api.html.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Dataset: Immigration to Canada from 1980 to 2013 <a id=\"1\"></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset Source: [International migration flows to and from selected countries - The 2015 revision](https://www.un.org/development/desa/pd/data/international-migration-flows).\n",
    "\n",
    "The dataset contains annual data on the flows of international immigrants as recorded by the countries of destination. The data presents both inflows and outflows according to the place of birth, citizenship or place of previous / next residence both for foreigners and nationals. The current version presents data pertaining to 45 countries.\n",
    "\n",
    "In this lab, we will focus on the Canadian immigration data.\n",
    "\n",
    "![Data Preview](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DV0101EN-SkillsNetwork/labs/Module%201/images/DataSnapshot.png)\n",
    "\n",
    " The Canada Immigration dataset can be fetched from <a href=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DV0101EN-SkillsNetwork/Data%20Files/Canada.xlsx\">here</a>.\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *pandas* Basics<a id=\"2\"></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing we'll do is install **openpyxl** (formerly **xlrd**), a module that *pandas* requires to read Excel files.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "                  __    __    __    __\n",
      "                 /  \\  /  \\  /  \\  /  \\\n",
      "                /    \\/    \\/    \\/    \\\n",
      "███████████████/  /██/  /██/  /██/  /████████████████████████\n",
      "              /  / \\   / \\   / \\   / \\  \\____\n",
      "             /  /   \\_/   \\_/   \\_/   \\    o \\__,\n",
      "            / _/                       \\_____/  `\n",
      "            |/\n",
      "        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗\n",
      "        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗\n",
      "        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║\n",
      "        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║\n",
      "        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║\n",
      "        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝\n",
      "\n",
      "        mamba (1.4.2) supported by @QuantStack\n",
      "\n",
      "        GitHub:  https://github.com/mamba-org/mamba\n",
      "        Twitter: https://twitter.com/QuantStack\n",
      "\n",
      "█████████████████████████████████████████████████████████████\n",
      "\n",
      "\n",
      "Looking for: ['openpyxl==3.0.9']\n",
      "\n",
      "\u001b[?25l\u001b[2K\u001b[0G[+] 0.0s\n",
      "\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.1s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.1s\n",
      "pkgs/main/noarch   \u001b[33m━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.1s\n",
      "pkgs/r/linux-64    \u001b[90m━━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.1s\n",
      "pkgs/r/noarch      \u001b[90m━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.1s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.2s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.2s\n",
      "pkgs/main/noarch   \u001b[33m━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━\u001b[0m  41.0kB /  ??.?MB @ 266.5kB/s  0.2s\n",
      "pkgs/r/linux-64    \u001b[90m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.2s\n",
      "pkgs/r/noarch      \u001b[90m━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━\u001b[0m   0.0 B /  ??.?MB @  ??.?MB/s  0.2s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.3s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━\u001b[0m 413.7kB /  ??.?MB @   1.6MB/s  0.3s\n",
      "pkgs/main/noarch   \u001b[33m━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━\u001b[0m 479.2kB /  ??.?MB @   1.9MB/s  0.3s\n",
      "pkgs/r/linux-64    \u001b[33m━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━\u001b[0m 417.8kB /  ??.?MB @   1.6MB/s  0.3s\n",
      "pkgs/r/noarch      \u001b[90m━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━\u001b[0m 503.8kB /  ??.?MB @   2.0MB/s  0.3s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0Gpkgs/main/noarch                                   871.2kB @   2.5MB/s  0.4s\n",
      "[+] 0.4s\n",
      "pkgs/main/linux-64 \u001b[90m╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━\u001b[0m   1.0MB /  ??.?MB @   2.7MB/s  0.4s\n",
      "pkgs/r/linux-64    \u001b[33m━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━\u001b[0m 913.4kB /  ??.?MB @   2.4MB/s  0.4s\n",
      "pkgs/r/noarch      \u001b[90m━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━━\u001b[0m   1.1MB /  ??.?MB @   2.8MB/s  0.4s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.5s\n",
      "pkgs/main/linux-64 \u001b[90m━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━\u001b[0m   1.5MB /  ??.?MB @   3.2MB/s  0.5s\n",
      "pkgs/r/linux-64    \u001b[33m━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━\u001b[0m   1.4MB /  ??.?MB @   2.9MB/s  0.5s\n",
      "pkgs/r/noarch      \u001b[90m━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━\u001b[0m   1.5MB /  ??.?MB @   3.1MB/s  0.5s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.6s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━\u001b[0m   2.1MB /  ??.?MB @   3.6MB/s  0.6s\n",
      "pkgs/r/linux-64    \u001b[90m╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━\u001b[0m   1.8MB /  ??.?MB @   3.1MB/s  0.6s\n",
      "pkgs/r/noarch      \u001b[90m━━━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━\u001b[0m   2.0MB /  ??.?MB @   3.4MB/s  0.6s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0Gpkgs/r/linux-64                                      1.9MB @   3.0MB/s  0.7s\n",
      "[+] 0.7s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━\u001b[0m   2.3MB @   3.6MB/s             0.7s\n",
      "pkgs/r/noarch      ━━━━━━━━━━━━━━━━━━━━━━━━   2.3MB @   3.5MB/s Finalizing  0.7s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0Gpkgs/r/noarch                                      @   3.5MB/s  0.7s\n",
      "[+] 0.8s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━\u001b[0m   3.0MB /  ??.?MB @   3.8MB/s  0.8s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.9s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━\u001b[0m   3.5MB /  ??.?MB @   3.9MB/s  0.9s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.0s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━\u001b[0m   3.8MB /  ??.?MB @   3.8MB/s  1.0s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.1s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━\u001b[0m   4.4MB /  ??.?MB @   4.0MB/s  1.1s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.2s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━\u001b[0m   4.6MB /  ??.?MB @   4.0MB/s  1.2s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.3s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━\u001b[0m   5.1MB /  ??.?MB @   4.1MB/s  1.3s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.4s\n",
      "pkgs/main/linux-64 \u001b[33m━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━\u001b[0m   5.5MB /  ??.?MB @   4.0MB/s  1.4s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.5s\n",
      "pkgs/main/linux-64 \u001b[90m╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━\u001b[0m   5.9MB /  ??.?MB @   4.1MB/s  1.5s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.6s\n",
      "pkgs/main/linux-64 \u001b[90m━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━\u001b[0m   6.3MB /  ??.?MB @   4.0MB/s  1.6s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.7s\n",
      "pkgs/main/linux-64 \u001b[90m━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━\u001b[0m   6.6MB /  ??.?MB @   4.0MB/s  1.7s\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.8s\n",
      "pkgs/main/linux-64 ━━━━━━━━━━━━━━━━━━━━━━━━   6.9MB @   4.0MB/s Finalizing  1.8s\u001b[2K\u001b[1A\u001b[2K\u001b[0Gpkgs/main/linux-64                                 @   4.0MB/s  1.9s\n",
      "\u001b[?25h\n",
      "Pinned packages:\n",
      "  - python 3.7.*\n",
      "\n",
      "\n",
      "Transaction\n",
      "\n",
      "  Prefix: /home/jupyterlab/conda/envs/python\n",
      "\n",
      "  Updating specs:\n",
      "\n",
      "   - openpyxl==3.0.9\n",
      "   - ca-certificates\n",
      "   - certifi\n",
      "   - openssl\n",
      "\n",
      "\n",
      "  Package              Version  Build           Channel                 Size\n",
      "──────────────────────────────────────────────────────────────────────────────\n",
      "  Install:\n",
      "──────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "  \u001b[32m+ et_xmlfile     \u001b[0m      1.1.0  py37h06a4308_0  pkgs/main/linux-64      10kB\n",
      "  \u001b[32m+ openpyxl       \u001b[0m      3.0.9  pyhd3eb1b0_0    pkgs/main/noarch       168kB\n",
      "\n",
      "  Upgrade:\n",
      "──────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "  \u001b[31m- ca-certificates\u001b[0m   2023.5.7  hbcca054_0      conda-forge                 \n",
      "  \u001b[32m+ ca-certificates\u001b[0m  2024.3.11  h06a4308_0      pkgs/main/linux-64     130kB\n",
      "  \u001b[31m- openssl        \u001b[0m     1.1.1t  h0b41bf4_0      conda-forge                 \n",
      "  \u001b[32m+ openssl        \u001b[0m     1.1.1w  h7f8727e_0      pkgs/main/linux-64       4MB\n",
      "\n",
      "  Summary:\n",
      "\n",
      "  Install: 2 packages\n",
      "  Upgrade: 2 packages\n",
      "\n",
      "  Total download: 4MB\n",
      "\n",
      "──────────────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\n",
      "\u001b[?25l\u001b[2K\u001b[0G[+] 0.0s\n",
      "Downloading      \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m   0.0 B                            0.0s\n",
      "Extracting       \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m       0                            0.0s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.1s\n",
      "Downloading  (4) \u001b[33m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m   0.0 B ca-certificates            0.0s\n",
      "Extracting       \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m       0                            0.0s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.2s\n",
      "Downloading  (4) \u001b[33m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m   0.0 B ca-certificates            0.1s\n",
      "Extracting       \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m       0                            0.0s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0Gca-certificates                                    130.4kB @ 510.8kB/s  0.3s\n",
      "et_xmlfile                                          10.2kB @  39.9kB/s  0.3s\n",
      "openpyxl                                           167.6kB @ 648.6kB/s  0.3s\n",
      "[+] 0.3s\n",
      "Downloading  (1) ━╸\u001b[33m━━━━━━━━━━━━━━━━━━━━━\u001b[0m 512.1kB openssl                    0.2s\n",
      "Extracting   (3) \u001b[90m━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━\u001b[0m       0 ca-certificates            0.0s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0Gopenssl                                              3.9MB @  11.4MB/s  0.3s\n",
      "[+] 0.4s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━━━\u001b[0m       0 ca-certificates            0.1s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.5s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━━\u001b[0m       0 ca-certificates            0.2s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.6s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━━\u001b[0m       0 ca-certificates            0.3s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.7s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━━\u001b[0m       0 et_xmlfile                 0.4s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.8s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━━\u001b[0m       0 et_xmlfile                 0.5s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 0.9s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[90m━━━━━━━━━━━━╸\u001b[0m\u001b[33m━━━━━━━━━━\u001b[0m       0 et_xmlfile                 0.6s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.0s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━\u001b[0m       0 et_xmlfile                 0.7s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.1s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━\u001b[0m       0 openpyxl                   0.8s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.2s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━━\u001b[0m       0 openpyxl                   0.9s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.3s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━━\u001b[0m       0 openpyxl                   1.0s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.4s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━━━\u001b[0m       0 openpyxl                   1.1s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.5s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━━\u001b[0m       0 openssl                    1.2s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.6s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (4) \u001b[33m━━━━━━━━━━━━━━╸\u001b[0m\u001b[90m━━━━━━━━\u001b[0m       0 openssl                    1.3s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.7s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (3) ━━━━╸\u001b[33m━━━━━━━━━━━━━━━━━━\u001b[0m       1 openssl                    1.4s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G[+] 1.8s\n",
      "Downloading      ━━━━━━━━━━━━━━━━━━━━━━━   4.2MB                            0.3s\n",
      "Extracting   (2) ━━━━━━━━━━╸\u001b[33m━━━━━━━━━━━━\u001b[0m       2 openssl                    1.5s\u001b[2K\u001b[1A\u001b[2K\u001b[1A\u001b[2K\u001b[0G\u001b[?25h\n",
      "Downloading and Extracting Packages\n",
      "\n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n"
     ]
    }
   ],
   "source": [
    "!mamba install openpyxl==3.0.9 -y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll do is import two key data analysis modules: *pandas* and *numpy*.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np  # useful for many scientific computing in Python\n",
    "import pandas as pd # primary data structure library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's download and import our primary Canadian Immigration dataset using *pandas*'s `read_excel()` method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data read into a pandas dataframe!\n"
     ]
    }
   ],
   "source": [
    "df_can = pd.read_excel(\n",
    "    'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DV0101EN-SkillsNetwork/Data%20Files/Canada.xlsx',\n",
    "    sheet_name='Canada by Citizenship',\n",
    "    skiprows=range(20),\n",
    "    skipfooter=2)\n",
    "\n",
    "print('Data read into a pandas dataframe!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's view the top 5 rows of the dataset using the `head()` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Type</th>\n",
       "      <th>Coverage</th>\n",
       "      <th>OdName</th>\n",
       "      <th>AREA</th>\n",
       "      <th>AreaName</th>\n",
       "      <th>REG</th>\n",
       "      <th>RegName</th>\n",
       "      <th>DEV</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>...</th>\n",
       "      <th>2004</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>935</td>\n",
       "      <td>Asia</td>\n",
       "      <td>5501</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>16</td>\n",
       "      <td>...</td>\n",
       "      <td>2978</td>\n",
       "      <td>3436</td>\n",
       "      <td>3009</td>\n",
       "      <td>2652</td>\n",
       "      <td>2111</td>\n",
       "      <td>1746</td>\n",
       "      <td>1758</td>\n",
       "      <td>2203</td>\n",
       "      <td>2635</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Albania</td>\n",
       "      <td>908</td>\n",
       "      <td>Europe</td>\n",
       "      <td>925</td>\n",
       "      <td>Southern Europe</td>\n",
       "      <td>901</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1450</td>\n",
       "      <td>1223</td>\n",
       "      <td>856</td>\n",
       "      <td>702</td>\n",
       "      <td>560</td>\n",
       "      <td>716</td>\n",
       "      <td>561</td>\n",
       "      <td>539</td>\n",
       "      <td>620</td>\n",
       "      <td>603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Algeria</td>\n",
       "      <td>903</td>\n",
       "      <td>Africa</td>\n",
       "      <td>912</td>\n",
       "      <td>Northern Africa</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>80</td>\n",
       "      <td>...</td>\n",
       "      <td>3616</td>\n",
       "      <td>3626</td>\n",
       "      <td>4807</td>\n",
       "      <td>3623</td>\n",
       "      <td>4005</td>\n",
       "      <td>5393</td>\n",
       "      <td>4752</td>\n",
       "      <td>4325</td>\n",
       "      <td>3774</td>\n",
       "      <td>4331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>American Samoa</td>\n",
       "      <td>909</td>\n",
       "      <td>Oceania</td>\n",
       "      <td>957</td>\n",
       "      <td>Polynesia</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Andorra</td>\n",
       "      <td>908</td>\n",
       "      <td>Europe</td>\n",
       "      <td>925</td>\n",
       "      <td>Southern Europe</td>\n",
       "      <td>901</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 43 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         Type    Coverage          OdName  AREA AreaName   REG  \\\n",
       "0  Immigrants  Foreigners     Afghanistan   935     Asia  5501   \n",
       "1  Immigrants  Foreigners         Albania   908   Europe   925   \n",
       "2  Immigrants  Foreigners         Algeria   903   Africa   912   \n",
       "3  Immigrants  Foreigners  American Samoa   909  Oceania   957   \n",
       "4  Immigrants  Foreigners         Andorra   908   Europe   925   \n",
       "\n",
       "           RegName  DEV             DevName  1980  ...  2004  2005  2006  \\\n",
       "0    Southern Asia  902  Developing regions    16  ...  2978  3436  3009   \n",
       "1  Southern Europe  901   Developed regions     1  ...  1450  1223   856   \n",
       "2  Northern Africa  902  Developing regions    80  ...  3616  3626  4807   \n",
       "3        Polynesia  902  Developing regions     0  ...     0     0     1   \n",
       "4  Southern Europe  901   Developed regions     0  ...     0     0     1   \n",
       "\n",
       "   2007  2008  2009  2010  2011  2012  2013  \n",
       "0  2652  2111  1746  1758  2203  2635  2004  \n",
       "1   702   560   716   561   539   620   603  \n",
       "2  3623  4005  5393  4752  4325  3774  4331  \n",
       "3     0     0     0     0     0     0     0  \n",
       "4     1     0     0     0     0     1     1  \n",
       "\n",
       "[5 rows x 43 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.head()\n",
    "# tip: You can specify the number of rows you'd like to see as follows: df_can.head(10) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also view the bottom 5 rows of the dataset using the `tail()` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Type</th>\n",
       "      <th>Coverage</th>\n",
       "      <th>OdName</th>\n",
       "      <th>AREA</th>\n",
       "      <th>AreaName</th>\n",
       "      <th>REG</th>\n",
       "      <th>RegName</th>\n",
       "      <th>DEV</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>...</th>\n",
       "      <th>2004</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>190</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Viet Nam</td>\n",
       "      <td>935</td>\n",
       "      <td>Asia</td>\n",
       "      <td>920</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1191</td>\n",
       "      <td>...</td>\n",
       "      <td>1816</td>\n",
       "      <td>1852</td>\n",
       "      <td>3153</td>\n",
       "      <td>2574</td>\n",
       "      <td>1784</td>\n",
       "      <td>2171</td>\n",
       "      <td>1942</td>\n",
       "      <td>1723</td>\n",
       "      <td>1731</td>\n",
       "      <td>2112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>191</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Western Sahara</td>\n",
       "      <td>903</td>\n",
       "      <td>Africa</td>\n",
       "      <td>912</td>\n",
       "      <td>Northern Africa</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Yemen</td>\n",
       "      <td>935</td>\n",
       "      <td>Asia</td>\n",
       "      <td>922</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>124</td>\n",
       "      <td>161</td>\n",
       "      <td>140</td>\n",
       "      <td>122</td>\n",
       "      <td>133</td>\n",
       "      <td>128</td>\n",
       "      <td>211</td>\n",
       "      <td>160</td>\n",
       "      <td>174</td>\n",
       "      <td>217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Zambia</td>\n",
       "      <td>903</td>\n",
       "      <td>Africa</td>\n",
       "      <td>910</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>56</td>\n",
       "      <td>91</td>\n",
       "      <td>77</td>\n",
       "      <td>71</td>\n",
       "      <td>64</td>\n",
       "      <td>60</td>\n",
       "      <td>102</td>\n",
       "      <td>69</td>\n",
       "      <td>46</td>\n",
       "      <td>59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>194</th>\n",
       "      <td>Immigrants</td>\n",
       "      <td>Foreigners</td>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>903</td>\n",
       "      <td>Africa</td>\n",
       "      <td>910</td>\n",
       "      <td>Eastern Africa</td>\n",
       "      <td>902</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>72</td>\n",
       "      <td>...</td>\n",
       "      <td>1450</td>\n",
       "      <td>615</td>\n",
       "      <td>454</td>\n",
       "      <td>663</td>\n",
       "      <td>611</td>\n",
       "      <td>508</td>\n",
       "      <td>494</td>\n",
       "      <td>434</td>\n",
       "      <td>437</td>\n",
       "      <td>407</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 43 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           Type    Coverage          OdName  AREA AreaName  REG  \\\n",
       "190  Immigrants  Foreigners        Viet Nam   935     Asia  920   \n",
       "191  Immigrants  Foreigners  Western Sahara   903   Africa  912   \n",
       "192  Immigrants  Foreigners           Yemen   935     Asia  922   \n",
       "193  Immigrants  Foreigners          Zambia   903   Africa  910   \n",
       "194  Immigrants  Foreigners        Zimbabwe   903   Africa  910   \n",
       "\n",
       "                RegName  DEV             DevName  1980  ...  2004  2005  2006  \\\n",
       "190  South-Eastern Asia  902  Developing regions  1191  ...  1816  1852  3153   \n",
       "191     Northern Africa  902  Developing regions     0  ...     0     0     1   \n",
       "192        Western Asia  902  Developing regions     1  ...   124   161   140   \n",
       "193      Eastern Africa  902  Developing regions    11  ...    56    91    77   \n",
       "194      Eastern Africa  902  Developing regions    72  ...  1450   615   454   \n",
       "\n",
       "     2007  2008  2009  2010  2011  2012  2013  \n",
       "190  2574  1784  2171  1942  1723  1731  2112  \n",
       "191     0     0     0     0     0     0     0  \n",
       "192   122   133   128   211   160   174   217  \n",
       "193    71    64    60   102    69    46    59  \n",
       "194   663   611   508   494   434   437   407  \n",
       "\n",
       "[5 rows x 43 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When analyzing a dataset, it's always a good idea to start by getting basic information about your dataframe. We can do this by using the `info()` method.\n",
    "\n",
    "This method can be used to get a short summary of the dataframe.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 195 entries, 0 to 194\n",
      "Columns: 43 entries, Type to 2013\n",
      "dtypes: int64(37), object(6)\n",
      "memory usage: 65.6+ KB\n"
     ]
    }
   ],
   "source": [
    "df_can.info(verbose=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the list of column headers we can call upon the data frame's `columns` instance variable.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([    'Type', 'Coverage',   'OdName',     'AREA', 'AreaName',      'REG',\n",
       "        'RegName',      'DEV',  'DevName',       1980,       1981,       1982,\n",
       "             1983,       1984,       1985,       1986,       1987,       1988,\n",
       "             1989,       1990,       1991,       1992,       1993,       1994,\n",
       "             1995,       1996,       1997,       1998,       1999,       2000,\n",
       "             2001,       2002,       2003,       2004,       2005,       2006,\n",
       "             2007,       2008,       2009,       2010,       2011,       2012,\n",
       "             2013],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, to get the list of indices we use the `.index` instance variables.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=195, step=1)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: The default type of intance variables `index` and `columns` are **NOT** `list`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.indexes.base.Index'>\n",
      "<class 'pandas.core.indexes.range.RangeIndex'>\n"
     ]
    }
   ],
   "source": [
    "print(type(df_can.columns))\n",
    "print(type(df_can.index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the index and columns as lists, we can use the `tolist()` method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Type',\n",
       " 'Coverage',\n",
       " 'OdName',\n",
       " 'AREA',\n",
       " 'AreaName',\n",
       " 'REG',\n",
       " 'RegName',\n",
       " 'DEV',\n",
       " 'DevName',\n",
       " 1980,\n",
       " 1981,\n",
       " 1982,\n",
       " 1983,\n",
       " 1984,\n",
       " 1985,\n",
       " 1986,\n",
       " 1987,\n",
       " 1988,\n",
       " 1989,\n",
       " 1990,\n",
       " 1991,\n",
       " 1992,\n",
       " 1993,\n",
       " 1994,\n",
       " 1995,\n",
       " 1996,\n",
       " 1997,\n",
       " 1998,\n",
       " 1999,\n",
       " 2000,\n",
       " 2001,\n",
       " 2002,\n",
       " 2003,\n",
       " 2004,\n",
       " 2005,\n",
       " 2006,\n",
       " 2007,\n",
       " 2008,\n",
       " 2009,\n",
       " 2010,\n",
       " 2011,\n",
       " 2012,\n",
       " 2013]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.columns.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0,\n",
       " 1,\n",
       " 2,\n",
       " 3,\n",
       " 4,\n",
       " 5,\n",
       " 6,\n",
       " 7,\n",
       " 8,\n",
       " 9,\n",
       " 10,\n",
       " 11,\n",
       " 12,\n",
       " 13,\n",
       " 14,\n",
       " 15,\n",
       " 16,\n",
       " 17,\n",
       " 18,\n",
       " 19,\n",
       " 20,\n",
       " 21,\n",
       " 22,\n",
       " 23,\n",
       " 24,\n",
       " 25,\n",
       " 26,\n",
       " 27,\n",
       " 28,\n",
       " 29,\n",
       " 30,\n",
       " 31,\n",
       " 32,\n",
       " 33,\n",
       " 34,\n",
       " 35,\n",
       " 36,\n",
       " 37,\n",
       " 38,\n",
       " 39,\n",
       " 40,\n",
       " 41,\n",
       " 42,\n",
       " 43,\n",
       " 44,\n",
       " 45,\n",
       " 46,\n",
       " 47,\n",
       " 48,\n",
       " 49,\n",
       " 50,\n",
       " 51,\n",
       " 52,\n",
       " 53,\n",
       " 54,\n",
       " 55,\n",
       " 56,\n",
       " 57,\n",
       " 58,\n",
       " 59,\n",
       " 60,\n",
       " 61,\n",
       " 62,\n",
       " 63,\n",
       " 64,\n",
       " 65,\n",
       " 66,\n",
       " 67,\n",
       " 68,\n",
       " 69,\n",
       " 70,\n",
       " 71,\n",
       " 72,\n",
       " 73,\n",
       " 74,\n",
       " 75,\n",
       " 76,\n",
       " 77,\n",
       " 78,\n",
       " 79,\n",
       " 80,\n",
       " 81,\n",
       " 82,\n",
       " 83,\n",
       " 84,\n",
       " 85,\n",
       " 86,\n",
       " 87,\n",
       " 88,\n",
       " 89,\n",
       " 90,\n",
       " 91,\n",
       " 92,\n",
       " 93,\n",
       " 94,\n",
       " 95,\n",
       " 96,\n",
       " 97,\n",
       " 98,\n",
       " 99,\n",
       " 100,\n",
       " 101,\n",
       " 102,\n",
       " 103,\n",
       " 104,\n",
       " 105,\n",
       " 106,\n",
       " 107,\n",
       " 108,\n",
       " 109,\n",
       " 110,\n",
       " 111,\n",
       " 112,\n",
       " 113,\n",
       " 114,\n",
       " 115,\n",
       " 116,\n",
       " 117,\n",
       " 118,\n",
       " 119,\n",
       " 120,\n",
       " 121,\n",
       " 122,\n",
       " 123,\n",
       " 124,\n",
       " 125,\n",
       " 126,\n",
       " 127,\n",
       " 128,\n",
       " 129,\n",
       " 130,\n",
       " 131,\n",
       " 132,\n",
       " 133,\n",
       " 134,\n",
       " 135,\n",
       " 136,\n",
       " 137,\n",
       " 138,\n",
       " 139,\n",
       " 140,\n",
       " 141,\n",
       " 142,\n",
       " 143,\n",
       " 144,\n",
       " 145,\n",
       " 146,\n",
       " 147,\n",
       " 148,\n",
       " 149,\n",
       " 150,\n",
       " 151,\n",
       " 152,\n",
       " 153,\n",
       " 154,\n",
       " 155,\n",
       " 156,\n",
       " 157,\n",
       " 158,\n",
       " 159,\n",
       " 160,\n",
       " 161,\n",
       " 162,\n",
       " 163,\n",
       " 164,\n",
       " 165,\n",
       " 166,\n",
       " 167,\n",
       " 168,\n",
       " 169,\n",
       " 170,\n",
       " 171,\n",
       " 172,\n",
       " 173,\n",
       " 174,\n",
       " 175,\n",
       " 176,\n",
       " 177,\n",
       " 178,\n",
       " 179,\n",
       " 180,\n",
       " 181,\n",
       " 182,\n",
       " 183,\n",
       " 184,\n",
       " 185,\n",
       " 186,\n",
       " 187,\n",
       " 188,\n",
       " 189,\n",
       " 190,\n",
       " 191,\n",
       " 192,\n",
       " 193,\n",
       " 194]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.index.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'list'>\n",
      "<class 'list'>\n"
     ]
    }
   ],
   "source": [
    "print(type(df_can.columns.tolist()))\n",
    "print(type(df_can.index.tolist()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To view the dimensions of the dataframe, we use the `shape` instance variable of it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(195, 43)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# size of dataframe (rows, columns)\n",
    "df_can.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: The main types stored in *pandas* objects are `float`, `int`, `bool`, `datetime64[ns]`, `datetime64[ns, tz]`, `timedelta[ns]`, `category`, and `object` (string). In addition, these dtypes have item sizes, e.g. `int64` and `int32`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's clean the data set to remove a few unnecessary columns. We can use *pandas* `drop()` method as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>OdName</th>\n",
       "      <th>AreaName</th>\n",
       "      <th>RegName</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>...</th>\n",
       "      <th>2004</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>16</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>47</td>\n",
       "      <td>71</td>\n",
       "      <td>340</td>\n",
       "      <td>...</td>\n",
       "      <td>2978</td>\n",
       "      <td>3436</td>\n",
       "      <td>3009</td>\n",
       "      <td>2652</td>\n",
       "      <td>2111</td>\n",
       "      <td>1746</td>\n",
       "      <td>1758</td>\n",
       "      <td>2203</td>\n",
       "      <td>2635</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Albania</td>\n",
       "      <td>Europe</td>\n",
       "      <td>Southern Europe</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1450</td>\n",
       "      <td>1223</td>\n",
       "      <td>856</td>\n",
       "      <td>702</td>\n",
       "      <td>560</td>\n",
       "      <td>716</td>\n",
       "      <td>561</td>\n",
       "      <td>539</td>\n",
       "      <td>620</td>\n",
       "      <td>603</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        OdName AreaName          RegName             DevName  1980  1981  \\\n",
       "0  Afghanistan     Asia    Southern Asia  Developing regions    16    39   \n",
       "1      Albania   Europe  Southern Europe   Developed regions     1     0   \n",
       "\n",
       "   1982  1983  1984  1985  ...  2004  2005  2006  2007  2008  2009  2010  \\\n",
       "0    39    47    71   340  ...  2978  3436  3009  2652  2111  1746  1758   \n",
       "1     0     0     0     0  ...  1450  1223   856   702   560   716   561   \n",
       "\n",
       "   2011  2012  2013  \n",
       "0  2203  2635  2004  \n",
       "1   539   620   603  \n",
       "\n",
       "[2 rows x 38 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# in pandas axis=0 represents rows (default) and axis=1 represents columns.\n",
    "df_can.drop(['AREA','REG','DEV','Type','Coverage'], axis=1, inplace=True)\n",
    "df_can.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's rename the columns so that they make sense. We can use `rename()` method by passing in a dictionary of old and new names as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([  'Country', 'Continent',    'Region',   'DevName',        1980,\n",
       "              1981,        1982,        1983,        1984,        1985,\n",
       "              1986,        1987,        1988,        1989,        1990,\n",
       "              1991,        1992,        1993,        1994,        1995,\n",
       "              1996,        1997,        1998,        1999,        2000,\n",
       "              2001,        2002,        2003,        2004,        2005,\n",
       "              2006,        2007,        2008,        2009,        2010,\n",
       "              2011,        2012,        2013],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.rename(columns={'OdName':'Country', 'AreaName':'Continent', 'RegName':'Region'}, inplace=True)\n",
    "df_can.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also add a 'Total' column that sums up the total immigrants by country over the entire period 1980 - 2013, as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jupyterlab/conda/envs/python/lib/python3.7/site-packages/ipykernel_launcher.py:1: FutureWarning: Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0      58639\n",
       "1      15699\n",
       "2      69439\n",
       "3          6\n",
       "4         15\n",
       "       ...  \n",
       "190    97146\n",
       "191        2\n",
       "192     2985\n",
       "193     1677\n",
       "194     8598\n",
       "Name: Total, Length: 195, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can['Total'] = df_can.sum(axis=1)\n",
    "df_can['Total']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check to see how many null objects we have in the dataset as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Country      0\n",
       "Continent    0\n",
       "Region       0\n",
       "DevName      0\n",
       "1980         0\n",
       "1981         0\n",
       "1982         0\n",
       "1983         0\n",
       "1984         0\n",
       "1985         0\n",
       "1986         0\n",
       "1987         0\n",
       "1988         0\n",
       "1989         0\n",
       "1990         0\n",
       "1991         0\n",
       "1992         0\n",
       "1993         0\n",
       "1994         0\n",
       "1995         0\n",
       "1996         0\n",
       "1997         0\n",
       "1998         0\n",
       "1999         0\n",
       "2000         0\n",
       "2001         0\n",
       "2002         0\n",
       "2003         0\n",
       "2004         0\n",
       "2005         0\n",
       "2006         0\n",
       "2007         0\n",
       "2008         0\n",
       "2009         0\n",
       "2010         0\n",
       "2011         0\n",
       "2012         0\n",
       "2013         0\n",
       "Total        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's view a quick summary of each column in our dataframe using the `describe()` method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>1987</th>\n",
       "      <th>1988</th>\n",
       "      <th>1989</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "      <td>195.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>508.394872</td>\n",
       "      <td>566.989744</td>\n",
       "      <td>534.723077</td>\n",
       "      <td>387.435897</td>\n",
       "      <td>376.497436</td>\n",
       "      <td>358.861538</td>\n",
       "      <td>441.271795</td>\n",
       "      <td>691.133333</td>\n",
       "      <td>714.389744</td>\n",
       "      <td>843.241026</td>\n",
       "      <td>...</td>\n",
       "      <td>1320.292308</td>\n",
       "      <td>1266.958974</td>\n",
       "      <td>1191.820513</td>\n",
       "      <td>1246.394872</td>\n",
       "      <td>1275.733333</td>\n",
       "      <td>1420.287179</td>\n",
       "      <td>1262.533333</td>\n",
       "      <td>1313.958974</td>\n",
       "      <td>1320.702564</td>\n",
       "      <td>32867.451282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1949.588546</td>\n",
       "      <td>2152.643752</td>\n",
       "      <td>1866.997511</td>\n",
       "      <td>1204.333597</td>\n",
       "      <td>1198.246371</td>\n",
       "      <td>1079.309600</td>\n",
       "      <td>1225.576630</td>\n",
       "      <td>2109.205607</td>\n",
       "      <td>2443.606788</td>\n",
       "      <td>2555.048874</td>\n",
       "      <td>...</td>\n",
       "      <td>4425.957828</td>\n",
       "      <td>3926.717747</td>\n",
       "      <td>3443.542409</td>\n",
       "      <td>3694.573544</td>\n",
       "      <td>3829.630424</td>\n",
       "      <td>4462.946328</td>\n",
       "      <td>4030.084313</td>\n",
       "      <td>4247.555161</td>\n",
       "      <td>4237.951988</td>\n",
       "      <td>91785.498686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>28.500000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>36.000000</td>\n",
       "      <td>40.500000</td>\n",
       "      <td>37.500000</td>\n",
       "      <td>42.500000</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>952.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>13.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>210.000000</td>\n",
       "      <td>218.000000</td>\n",
       "      <td>198.000000</td>\n",
       "      <td>205.000000</td>\n",
       "      <td>214.000000</td>\n",
       "      <td>211.000000</td>\n",
       "      <td>179.000000</td>\n",
       "      <td>233.000000</td>\n",
       "      <td>213.000000</td>\n",
       "      <td>5018.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>251.500000</td>\n",
       "      <td>295.500000</td>\n",
       "      <td>275.000000</td>\n",
       "      <td>173.000000</td>\n",
       "      <td>181.000000</td>\n",
       "      <td>197.000000</td>\n",
       "      <td>254.000000</td>\n",
       "      <td>434.000000</td>\n",
       "      <td>409.000000</td>\n",
       "      <td>508.500000</td>\n",
       "      <td>...</td>\n",
       "      <td>832.000000</td>\n",
       "      <td>842.000000</td>\n",
       "      <td>899.000000</td>\n",
       "      <td>934.500000</td>\n",
       "      <td>888.000000</td>\n",
       "      <td>932.000000</td>\n",
       "      <td>772.000000</td>\n",
       "      <td>783.000000</td>\n",
       "      <td>796.000000</td>\n",
       "      <td>22239.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>22045.000000</td>\n",
       "      <td>24796.000000</td>\n",
       "      <td>20620.000000</td>\n",
       "      <td>10015.000000</td>\n",
       "      <td>10170.000000</td>\n",
       "      <td>9564.000000</td>\n",
       "      <td>9470.000000</td>\n",
       "      <td>21337.000000</td>\n",
       "      <td>27359.000000</td>\n",
       "      <td>23795.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>42584.000000</td>\n",
       "      <td>33848.000000</td>\n",
       "      <td>28742.000000</td>\n",
       "      <td>30037.000000</td>\n",
       "      <td>29622.000000</td>\n",
       "      <td>38617.000000</td>\n",
       "      <td>36765.000000</td>\n",
       "      <td>34315.000000</td>\n",
       "      <td>34129.000000</td>\n",
       "      <td>691904.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               1980          1981          1982          1983          1984  \\\n",
       "count    195.000000    195.000000    195.000000    195.000000    195.000000   \n",
       "mean     508.394872    566.989744    534.723077    387.435897    376.497436   \n",
       "std     1949.588546   2152.643752   1866.997511   1204.333597   1198.246371   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "50%       13.000000     10.000000     11.000000     12.000000     13.000000   \n",
       "75%      251.500000    295.500000    275.000000    173.000000    181.000000   \n",
       "max    22045.000000  24796.000000  20620.000000  10015.000000  10170.000000   \n",
       "\n",
       "              1985         1986          1987          1988          1989  \\\n",
       "count   195.000000   195.000000    195.000000    195.000000    195.000000   \n",
       "mean    358.861538   441.271795    691.133333    714.389744    843.241026   \n",
       "std    1079.309600  1225.576630   2109.205607   2443.606788   2555.048874   \n",
       "min       0.000000     0.000000      0.000000      0.000000      0.000000   \n",
       "25%       0.000000     0.500000      0.500000      1.000000      1.000000   \n",
       "50%      17.000000    18.000000     26.000000     34.000000     44.000000   \n",
       "75%     197.000000   254.000000    434.000000    409.000000    508.500000   \n",
       "max    9564.000000  9470.000000  21337.000000  27359.000000  23795.000000   \n",
       "\n",
       "       ...          2005          2006          2007          2008  \\\n",
       "count  ...    195.000000    195.000000    195.000000    195.000000   \n",
       "mean   ...   1320.292308   1266.958974   1191.820513   1246.394872   \n",
       "std    ...   4425.957828   3926.717747   3443.542409   3694.573544   \n",
       "min    ...      0.000000      0.000000      0.000000      0.000000   \n",
       "25%    ...     28.500000     25.000000     31.000000     31.000000   \n",
       "50%    ...    210.000000    218.000000    198.000000    205.000000   \n",
       "75%    ...    832.000000    842.000000    899.000000    934.500000   \n",
       "max    ...  42584.000000  33848.000000  28742.000000  30037.000000   \n",
       "\n",
       "               2009          2010          2011          2012          2013  \\\n",
       "count    195.000000    195.000000    195.000000    195.000000    195.000000   \n",
       "mean    1275.733333   1420.287179   1262.533333   1313.958974   1320.702564   \n",
       "std     3829.630424   4462.946328   4030.084313   4247.555161   4237.951988   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%       36.000000     40.500000     37.500000     42.500000     45.000000   \n",
       "50%      214.000000    211.000000    179.000000    233.000000    213.000000   \n",
       "75%      888.000000    932.000000    772.000000    783.000000    796.000000   \n",
       "max    29622.000000  38617.000000  36765.000000  34315.000000  34129.000000   \n",
       "\n",
       "               Total  \n",
       "count     195.000000  \n",
       "mean    32867.451282  \n",
       "std     91785.498686  \n",
       "min         1.000000  \n",
       "25%       952.000000  \n",
       "50%      5018.000000  \n",
       "75%     22239.500000  \n",
       "max    691904.000000  \n",
       "\n",
       "[8 rows x 35 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## *pandas* Intermediate: Indexing and Selection (slicing)<a id=\"3\"></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select Column\n",
    "**There are two ways to filter on a column name:**\n",
    "\n",
    "Method 1: Quick and easy, but only works if the column name does NOT have spaces or special characters.\n",
    "```python\n",
    "    df.column_name               # returns series\n",
    "```\n",
    "\n",
    "Method 2: More robust, and can filter on multiple columns.\n",
    "\n",
    "```python\n",
    "    df['column']                  # returns series\n",
    "```\n",
    "\n",
    "```python \n",
    "    df[['column 1', 'column 2']]  # returns dataframe\n",
    "```\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: Let's try filtering on the list of countries ('Country').\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         Afghanistan\n",
       "1             Albania\n",
       "2             Algeria\n",
       "3      American Samoa\n",
       "4             Andorra\n",
       "            ...      \n",
       "190          Viet Nam\n",
       "191    Western Sahara\n",
       "192             Yemen\n",
       "193            Zambia\n",
       "194          Zimbabwe\n",
       "Name: Country, Length: 195, dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.Country  # returns a series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try filtering on the list of countries ('Country') and the data for years: 1980 - 1985.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Country</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>16</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>47</td>\n",
       "      <td>71</td>\n",
       "      <td>340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Albania</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Algeria</td>\n",
       "      <td>80</td>\n",
       "      <td>67</td>\n",
       "      <td>71</td>\n",
       "      <td>69</td>\n",
       "      <td>63</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>American Samoa</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Andorra</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>190</th>\n",
       "      <td>Viet Nam</td>\n",
       "      <td>1191</td>\n",
       "      <td>1829</td>\n",
       "      <td>2162</td>\n",
       "      <td>3404</td>\n",
       "      <td>7583</td>\n",
       "      <td>5907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>191</th>\n",
       "      <td>Western Sahara</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>Yemen</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193</th>\n",
       "      <td>Zambia</td>\n",
       "      <td>11</td>\n",
       "      <td>17</td>\n",
       "      <td>11</td>\n",
       "      <td>7</td>\n",
       "      <td>16</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>194</th>\n",
       "      <td>Zimbabwe</td>\n",
       "      <td>72</td>\n",
       "      <td>114</td>\n",
       "      <td>102</td>\n",
       "      <td>44</td>\n",
       "      <td>32</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>195 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Country  1980  1981  1982  1983  1984  1985\n",
       "0       Afghanistan    16    39    39    47    71   340\n",
       "1           Albania     1     0     0     0     0     0\n",
       "2           Algeria    80    67    71    69    63    44\n",
       "3    American Samoa     0     1     0     0     0     0\n",
       "4           Andorra     0     0     0     0     0     0\n",
       "..              ...   ...   ...   ...   ...   ...   ...\n",
       "190        Viet Nam  1191  1829  2162  3404  7583  5907\n",
       "191  Western Sahara     0     0     0     0     0     0\n",
       "192           Yemen     1     2     1     6     0    18\n",
       "193          Zambia    11    17    11     7    16     9\n",
       "194        Zimbabwe    72   114   102    44    32    29\n",
       "\n",
       "[195 rows x 7 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can[['Country', 1980, 1981, 1982, 1983, 1984, 1985]] # returns a dataframe\n",
    "# notice that 'Country' is string, and the years are integers. \n",
    "# for the sake of consistency, we will convert all column names to string later on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select Row\n",
    "\n",
    "There are main 2 ways to select rows:\n",
    "\n",
    "```python\n",
    "    df.loc[label]    # filters by the labels of the index/column\n",
    "    df.iloc[index]   # filters by the positions of the index/column\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we proceed, notice that the default index of the dataset is a numeric range from 0 to 194. This makes it very difficult to do a query by a specific country. For example to search for data on Japan, we need to know the corresponding index value.\n",
    "\n",
    "This can be fixed very easily by setting the 'Country' column as the index using `set_index()` method.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_can.set_index('Country', inplace=True)\n",
    "# tip: The opposite of set is reset. So to reset the index, we can use df_can.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Country</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Afghanistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>16</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>47</td>\n",
       "      <td>71</td>\n",
       "      <td>340</td>\n",
       "      <td>496</td>\n",
       "      <td>...</td>\n",
       "      <td>3436</td>\n",
       "      <td>3009</td>\n",
       "      <td>2652</td>\n",
       "      <td>2111</td>\n",
       "      <td>1746</td>\n",
       "      <td>1758</td>\n",
       "      <td>2203</td>\n",
       "      <td>2635</td>\n",
       "      <td>2004</td>\n",
       "      <td>58639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Albania</th>\n",
       "      <td>Europe</td>\n",
       "      <td>Southern Europe</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1223</td>\n",
       "      <td>856</td>\n",
       "      <td>702</td>\n",
       "      <td>560</td>\n",
       "      <td>716</td>\n",
       "      <td>561</td>\n",
       "      <td>539</td>\n",
       "      <td>620</td>\n",
       "      <td>603</td>\n",
       "      <td>15699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Algeria</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Northern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>80</td>\n",
       "      <td>67</td>\n",
       "      <td>71</td>\n",
       "      <td>69</td>\n",
       "      <td>63</td>\n",
       "      <td>44</td>\n",
       "      <td>69</td>\n",
       "      <td>...</td>\n",
       "      <td>3626</td>\n",
       "      <td>4807</td>\n",
       "      <td>3623</td>\n",
       "      <td>4005</td>\n",
       "      <td>5393</td>\n",
       "      <td>4752</td>\n",
       "      <td>4325</td>\n",
       "      <td>3774</td>\n",
       "      <td>4331</td>\n",
       "      <td>69439</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Continent           Region             DevName  1980  1981  1982  \\\n",
       "Country                                                                        \n",
       "Afghanistan      Asia    Southern Asia  Developing regions    16    39    39   \n",
       "Albania        Europe  Southern Europe   Developed regions     1     0     0   \n",
       "Algeria        Africa  Northern Africa  Developing regions    80    67    71   \n",
       "\n",
       "             1983  1984  1985  1986  ...  2005  2006  2007  2008  2009  2010  \\\n",
       "Country                              ...                                       \n",
       "Afghanistan    47    71   340   496  ...  3436  3009  2652  2111  1746  1758   \n",
       "Albania         0     0     0     1  ...  1223   856   702   560   716   561   \n",
       "Algeria        69    63    44    69  ...  3626  4807  3623  4005  5393  4752   \n",
       "\n",
       "             2011  2012  2013  Total  \n",
       "Country                               \n",
       "Afghanistan  2203  2635  2004  58639  \n",
       "Albania       539   620   603  15699  \n",
       "Algeria      4325  3774  4331  69439  \n",
       "\n",
       "[3 rows x 38 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# optional: to remove the name of the index\n",
    "df_can.index.name = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: Let's view the number of immigrants from Japan (row 87) for the following scenarios:\n",
    "    1. The full row data (all columns)\n",
    "    2. For year 2013\n",
    "    3. For years 1980 to 1985\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Continent                 Asia\n",
       "Region            Eastern Asia\n",
       "DevName      Developed regions\n",
       "1980                       701\n",
       "1981                       756\n",
       "1982                       598\n",
       "1983                       309\n",
       "1984                       246\n",
       "1985                       198\n",
       "1986                       248\n",
       "1987                       422\n",
       "1988                       324\n",
       "1989                       494\n",
       "1990                       379\n",
       "1991                       506\n",
       "1992                       605\n",
       "1993                       907\n",
       "1994                       956\n",
       "1995                       826\n",
       "1996                       994\n",
       "1997                       924\n",
       "1998                       897\n",
       "1999                      1083\n",
       "2000                      1010\n",
       "2001                      1092\n",
       "2002                       806\n",
       "2003                       817\n",
       "2004                       973\n",
       "2005                      1067\n",
       "2006                      1212\n",
       "2007                      1250\n",
       "2008                      1284\n",
       "2009                      1194\n",
       "2010                      1168\n",
       "2011                      1265\n",
       "2012                      1214\n",
       "2013                       982\n",
       "Total                    27707\n",
       "Name: Japan, dtype: object"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1. the full row data (all columns)\n",
    "df_can.loc['Japan']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Continent                 Asia\n",
       "Region            Eastern Asia\n",
       "DevName      Developed regions\n",
       "1980                       701\n",
       "1981                       756\n",
       "1982                       598\n",
       "1983                       309\n",
       "1984                       246\n",
       "1985                       198\n",
       "1986                       248\n",
       "1987                       422\n",
       "1988                       324\n",
       "1989                       494\n",
       "1990                       379\n",
       "1991                       506\n",
       "1992                       605\n",
       "1993                       907\n",
       "1994                       956\n",
       "1995                       826\n",
       "1996                       994\n",
       "1997                       924\n",
       "1998                       897\n",
       "1999                      1083\n",
       "2000                      1010\n",
       "2001                      1092\n",
       "2002                       806\n",
       "2003                       817\n",
       "2004                       973\n",
       "2005                      1067\n",
       "2006                      1212\n",
       "2007                      1250\n",
       "2008                      1284\n",
       "2009                      1194\n",
       "2010                      1168\n",
       "2011                      1265\n",
       "2012                      1214\n",
       "2013                       982\n",
       "Total                    27707\n",
       "Name: Japan, dtype: object"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# alternate methods\n",
    "df_can.iloc[87]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Japan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>701</td>\n",
       "      <td>756</td>\n",
       "      <td>598</td>\n",
       "      <td>309</td>\n",
       "      <td>246</td>\n",
       "      <td>198</td>\n",
       "      <td>248</td>\n",
       "      <td>...</td>\n",
       "      <td>1067</td>\n",
       "      <td>1212</td>\n",
       "      <td>1250</td>\n",
       "      <td>1284</td>\n",
       "      <td>1194</td>\n",
       "      <td>1168</td>\n",
       "      <td>1265</td>\n",
       "      <td>1214</td>\n",
       "      <td>982</td>\n",
       "      <td>27707</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Continent        Region            DevName  1980  1981  1982  1983  \\\n",
       "Japan      Asia  Eastern Asia  Developed regions   701   756   598   309   \n",
       "\n",
       "       1984  1985  1986  ...  2005  2006  2007  2008  2009  2010  2011  2012  \\\n",
       "Japan   246   198   248  ...  1067  1212  1250  1284  1194  1168  1265  1214   \n",
       "\n",
       "       2013  Total  \n",
       "Japan   982  27707  \n",
       "\n",
       "[1 rows x 38 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can[df_can.index == 'Japan']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "982"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2. for year 2013\n",
    "df_can.loc['Japan', 2013]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "982"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# alternate method\n",
    "# year 2013 is the last column, with a positional index of 36\n",
    "df_can.iloc[87, 36]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1980    701\n",
       "1981    756\n",
       "1982    598\n",
       "1983    309\n",
       "1984    246\n",
       "1984    246\n",
       "Name: Japan, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 3. for years 1980 to 1985\n",
    "df_can.loc['Japan', [1980, 1981, 1982, 1983, 1984, 1984]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1980    701\n",
       "1981    756\n",
       "1982    598\n",
       "1983    309\n",
       "1984    246\n",
       "1985    198\n",
       "Name: Japan, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Alternative Method\n",
    "df_can.iloc[87, [3, 4, 5, 6, 7, 8]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Let's view the number of immigrants from **Haiti** for the following scenarios: <br>1. The full row data (all columns) <br>2. For year 2000 <br>3. For years 1990 to 1995\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1990    2379\n",
       "1991    2829\n",
       "1992    2399\n",
       "1993    3655\n",
       "1994    2100\n",
       "1995    2014\n",
       "Name: Haiti, dtype: object"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.loc['Haiti']\n",
    "df_can.loc['Haiti', 2000]\n",
    "df_can.loc['Haiti', [1990, 1991, 1992, 1993, 1994, 1995]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for a sample python solution</summary>\n",
    "\n",
    "```python\n",
    "   # 1. the full row data (all columns)\n",
    "    df_can.loc['Haiti']\n",
    "    #or\n",
    "    df_can[df_can.index == 'Haiti']\n",
    "\n",
    "    # 2. for year 2000\n",
    "    df_can.loc['Haiti', 2000]\n",
    "           \n",
    "    #  3. for years 1990 to 1995\n",
    "    df_can.loc['Haiti', [1990, 1991, 1992, 1993, 1994, 1995]]\n",
    " ```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Column names that are integers (such as the years) might introduce some confusion. For example, when we are referencing the year 2013, one might confuse that when the 2013th positional index. \n",
    "\n",
    "To avoid this ambuigity, let's convert the column names into strings: '1980' to '2013'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_can.columns = list(map(str, df_can.columns))\n",
    "# [print (type(x)) for x in df_can.columns.values] #<-- uncomment to check type of column headers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we converted the years to string, let's declare a variable that will allow us to easily call upon the full range of years:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['1980',\n",
       " '1981',\n",
       " '1982',\n",
       " '1983',\n",
       " '1984',\n",
       " '1985',\n",
       " '1986',\n",
       " '1987',\n",
       " '1988',\n",
       " '1989',\n",
       " '1990',\n",
       " '1991',\n",
       " '1992',\n",
       " '1993',\n",
       " '1994',\n",
       " '1995',\n",
       " '1996',\n",
       " '1997',\n",
       " '1998',\n",
       " '1999',\n",
       " '2000',\n",
       " '2001',\n",
       " '2002',\n",
       " '2003',\n",
       " '2004',\n",
       " '2005',\n",
       " '2006',\n",
       " '2007',\n",
       " '2008',\n",
       " '2009',\n",
       " '2010',\n",
       " '2011',\n",
       " '2012',\n",
       " '2013']"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# useful for plotting later on\n",
    "years = list(map(str, range(1980, 2014)))\n",
    "years"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Create a list named 'year' using map function for years ranging from 1990 to 2013. <br>Then extract the data series from the dataframe df_can for Haiti using year list. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "year = list(map(str, range(1990, 2014)))\n",
    "haiti = df_can.loc['Haiti', year]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for a sample python solution</summary>\n",
    "\n",
    "```python\n",
    "    #The correct answer is:\n",
    "    year = list(map(str, range(1990, 2014)))\n",
    "    haiti = df_can.loc['Haiti', year] # passing in years 1990 - 2013\n",
    "    \n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filtering based on a criteria <a id=\"4\"></a>\n",
    "To filter the dataframe based on a condition, we simply pass the condition as a boolean vector. \n",
    "\n",
    "For example, Let's filter the dataframe to show the data on Asian countries (AreaName = Asia).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Afghanistan        True\n",
      "Albania           False\n",
      "Algeria           False\n",
      "American Samoa    False\n",
      "Andorra           False\n",
      "                  ...  \n",
      "Viet Nam           True\n",
      "Western Sahara    False\n",
      "Yemen              True\n",
      "Zambia            False\n",
      "Zimbabwe          False\n",
      "Name: Continent, Length: 195, dtype: bool\n"
     ]
    }
   ],
   "source": [
    "# 1. create the condition boolean series\n",
    "condition = df_can['Continent'] == 'Asia'\n",
    "print(condition)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Afghanistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>16</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>47</td>\n",
       "      <td>71</td>\n",
       "      <td>340</td>\n",
       "      <td>496</td>\n",
       "      <td>...</td>\n",
       "      <td>3436</td>\n",
       "      <td>3009</td>\n",
       "      <td>2652</td>\n",
       "      <td>2111</td>\n",
       "      <td>1746</td>\n",
       "      <td>1758</td>\n",
       "      <td>2203</td>\n",
       "      <td>2635</td>\n",
       "      <td>2004</td>\n",
       "      <td>58639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Armenia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>224</td>\n",
       "      <td>218</td>\n",
       "      <td>198</td>\n",
       "      <td>205</td>\n",
       "      <td>267</td>\n",
       "      <td>252</td>\n",
       "      <td>236</td>\n",
       "      <td>258</td>\n",
       "      <td>207</td>\n",
       "      <td>3310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Azerbaijan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>359</td>\n",
       "      <td>236</td>\n",
       "      <td>203</td>\n",
       "      <td>125</td>\n",
       "      <td>165</td>\n",
       "      <td>209</td>\n",
       "      <td>138</td>\n",
       "      <td>161</td>\n",
       "      <td>57</td>\n",
       "      <td>2649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bahrain</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "      <td>35</td>\n",
       "      <td>28</td>\n",
       "      <td>21</td>\n",
       "      <td>39</td>\n",
       "      <td>32</td>\n",
       "      <td>475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bangladesh</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>83</td>\n",
       "      <td>84</td>\n",
       "      <td>86</td>\n",
       "      <td>81</td>\n",
       "      <td>98</td>\n",
       "      <td>92</td>\n",
       "      <td>486</td>\n",
       "      <td>...</td>\n",
       "      <td>4171</td>\n",
       "      <td>4014</td>\n",
       "      <td>2897</td>\n",
       "      <td>2939</td>\n",
       "      <td>2104</td>\n",
       "      <td>4721</td>\n",
       "      <td>2694</td>\n",
       "      <td>2640</td>\n",
       "      <td>3789</td>\n",
       "      <td>65568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bhutan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>36</td>\n",
       "      <td>865</td>\n",
       "      <td>1464</td>\n",
       "      <td>1879</td>\n",
       "      <td>1075</td>\n",
       "      <td>487</td>\n",
       "      <td>5876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Brunei Darussalam</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>79</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cambodia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>12</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>33</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>370</td>\n",
       "      <td>529</td>\n",
       "      <td>460</td>\n",
       "      <td>354</td>\n",
       "      <td>203</td>\n",
       "      <td>200</td>\n",
       "      <td>196</td>\n",
       "      <td>233</td>\n",
       "      <td>288</td>\n",
       "      <td>6538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>China</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>5123</td>\n",
       "      <td>6682</td>\n",
       "      <td>3308</td>\n",
       "      <td>1863</td>\n",
       "      <td>1527</td>\n",
       "      <td>1816</td>\n",
       "      <td>1960</td>\n",
       "      <td>...</td>\n",
       "      <td>42584</td>\n",
       "      <td>33518</td>\n",
       "      <td>27642</td>\n",
       "      <td>30037</td>\n",
       "      <td>29622</td>\n",
       "      <td>30391</td>\n",
       "      <td>28502</td>\n",
       "      <td>33024</td>\n",
       "      <td>34129</td>\n",
       "      <td>659962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>China, Hong Kong Special Administrative Region</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>729</td>\n",
       "      <td>712</td>\n",
       "      <td>674</td>\n",
       "      <td>897</td>\n",
       "      <td>657</td>\n",
       "      <td>623</td>\n",
       "      <td>591</td>\n",
       "      <td>728</td>\n",
       "      <td>774</td>\n",
       "      <td>9327</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>China, Macao Special Administrative Region</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>21</td>\n",
       "      <td>32</td>\n",
       "      <td>16</td>\n",
       "      <td>12</td>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>13</td>\n",
       "      <td>33</td>\n",
       "      <td>29</td>\n",
       "      <td>284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cyprus</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>132</td>\n",
       "      <td>128</td>\n",
       "      <td>84</td>\n",
       "      <td>46</td>\n",
       "      <td>46</td>\n",
       "      <td>43</td>\n",
       "      <td>48</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>9</td>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "      <td>6</td>\n",
       "      <td>18</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>16</td>\n",
       "      <td>1126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Democratic People's Republic of Korea</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>19</td>\n",
       "      <td>11</td>\n",
       "      <td>45</td>\n",
       "      <td>97</td>\n",
       "      <td>66</td>\n",
       "      <td>17</td>\n",
       "      <td>388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Georgia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>114</td>\n",
       "      <td>125</td>\n",
       "      <td>132</td>\n",
       "      <td>112</td>\n",
       "      <td>128</td>\n",
       "      <td>126</td>\n",
       "      <td>139</td>\n",
       "      <td>147</td>\n",
       "      <td>125</td>\n",
       "      <td>2068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>India</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>8880</td>\n",
       "      <td>8670</td>\n",
       "      <td>8147</td>\n",
       "      <td>7338</td>\n",
       "      <td>5704</td>\n",
       "      <td>4211</td>\n",
       "      <td>7150</td>\n",
       "      <td>...</td>\n",
       "      <td>36210</td>\n",
       "      <td>33848</td>\n",
       "      <td>28742</td>\n",
       "      <td>28261</td>\n",
       "      <td>29456</td>\n",
       "      <td>34235</td>\n",
       "      <td>27509</td>\n",
       "      <td>30933</td>\n",
       "      <td>33087</td>\n",
       "      <td>691904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Indonesia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>186</td>\n",
       "      <td>178</td>\n",
       "      <td>252</td>\n",
       "      <td>115</td>\n",
       "      <td>123</td>\n",
       "      <td>100</td>\n",
       "      <td>127</td>\n",
       "      <td>...</td>\n",
       "      <td>632</td>\n",
       "      <td>613</td>\n",
       "      <td>657</td>\n",
       "      <td>661</td>\n",
       "      <td>504</td>\n",
       "      <td>712</td>\n",
       "      <td>390</td>\n",
       "      <td>395</td>\n",
       "      <td>387</td>\n",
       "      <td>13150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iran (Islamic Republic of)</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1172</td>\n",
       "      <td>1429</td>\n",
       "      <td>1822</td>\n",
       "      <td>1592</td>\n",
       "      <td>1977</td>\n",
       "      <td>1648</td>\n",
       "      <td>1794</td>\n",
       "      <td>...</td>\n",
       "      <td>5837</td>\n",
       "      <td>7480</td>\n",
       "      <td>6974</td>\n",
       "      <td>6475</td>\n",
       "      <td>6580</td>\n",
       "      <td>7477</td>\n",
       "      <td>7479</td>\n",
       "      <td>7534</td>\n",
       "      <td>11291</td>\n",
       "      <td>175923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iraq</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>262</td>\n",
       "      <td>245</td>\n",
       "      <td>260</td>\n",
       "      <td>380</td>\n",
       "      <td>428</td>\n",
       "      <td>231</td>\n",
       "      <td>265</td>\n",
       "      <td>...</td>\n",
       "      <td>2226</td>\n",
       "      <td>1788</td>\n",
       "      <td>2406</td>\n",
       "      <td>3543</td>\n",
       "      <td>5450</td>\n",
       "      <td>5941</td>\n",
       "      <td>6196</td>\n",
       "      <td>4041</td>\n",
       "      <td>4918</td>\n",
       "      <td>69789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Israel</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1403</td>\n",
       "      <td>1711</td>\n",
       "      <td>1334</td>\n",
       "      <td>541</td>\n",
       "      <td>446</td>\n",
       "      <td>680</td>\n",
       "      <td>1212</td>\n",
       "      <td>...</td>\n",
       "      <td>2446</td>\n",
       "      <td>2625</td>\n",
       "      <td>2401</td>\n",
       "      <td>2562</td>\n",
       "      <td>2316</td>\n",
       "      <td>2755</td>\n",
       "      <td>1970</td>\n",
       "      <td>2134</td>\n",
       "      <td>1945</td>\n",
       "      <td>66508</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Japan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>701</td>\n",
       "      <td>756</td>\n",
       "      <td>598</td>\n",
       "      <td>309</td>\n",
       "      <td>246</td>\n",
       "      <td>198</td>\n",
       "      <td>248</td>\n",
       "      <td>...</td>\n",
       "      <td>1067</td>\n",
       "      <td>1212</td>\n",
       "      <td>1250</td>\n",
       "      <td>1284</td>\n",
       "      <td>1194</td>\n",
       "      <td>1168</td>\n",
       "      <td>1265</td>\n",
       "      <td>1214</td>\n",
       "      <td>982</td>\n",
       "      <td>27707</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jordan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>177</td>\n",
       "      <td>160</td>\n",
       "      <td>155</td>\n",
       "      <td>113</td>\n",
       "      <td>102</td>\n",
       "      <td>179</td>\n",
       "      <td>181</td>\n",
       "      <td>...</td>\n",
       "      <td>1940</td>\n",
       "      <td>1827</td>\n",
       "      <td>1421</td>\n",
       "      <td>1581</td>\n",
       "      <td>1235</td>\n",
       "      <td>1831</td>\n",
       "      <td>1635</td>\n",
       "      <td>1206</td>\n",
       "      <td>1255</td>\n",
       "      <td>35406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kazakhstan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Central Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>506</td>\n",
       "      <td>408</td>\n",
       "      <td>436</td>\n",
       "      <td>394</td>\n",
       "      <td>431</td>\n",
       "      <td>377</td>\n",
       "      <td>381</td>\n",
       "      <td>462</td>\n",
       "      <td>348</td>\n",
       "      <td>8490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kuwait</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>66</td>\n",
       "      <td>35</td>\n",
       "      <td>62</td>\n",
       "      <td>53</td>\n",
       "      <td>68</td>\n",
       "      <td>67</td>\n",
       "      <td>58</td>\n",
       "      <td>73</td>\n",
       "      <td>48</td>\n",
       "      <td>2025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kyrgyzstan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Central Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>173</td>\n",
       "      <td>161</td>\n",
       "      <td>135</td>\n",
       "      <td>168</td>\n",
       "      <td>173</td>\n",
       "      <td>157</td>\n",
       "      <td>159</td>\n",
       "      <td>278</td>\n",
       "      <td>123</td>\n",
       "      <td>2353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lao People's Democratic Republic</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>11</td>\n",
       "      <td>6</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>7</td>\n",
       "      <td>17</td>\n",
       "      <td>21</td>\n",
       "      <td>...</td>\n",
       "      <td>42</td>\n",
       "      <td>74</td>\n",
       "      <td>53</td>\n",
       "      <td>32</td>\n",
       "      <td>39</td>\n",
       "      <td>54</td>\n",
       "      <td>22</td>\n",
       "      <td>25</td>\n",
       "      <td>15</td>\n",
       "      <td>1089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lebanon</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1409</td>\n",
       "      <td>1119</td>\n",
       "      <td>1159</td>\n",
       "      <td>789</td>\n",
       "      <td>1253</td>\n",
       "      <td>1683</td>\n",
       "      <td>2576</td>\n",
       "      <td>...</td>\n",
       "      <td>3709</td>\n",
       "      <td>3802</td>\n",
       "      <td>3467</td>\n",
       "      <td>3566</td>\n",
       "      <td>3077</td>\n",
       "      <td>3432</td>\n",
       "      <td>3072</td>\n",
       "      <td>1614</td>\n",
       "      <td>2172</td>\n",
       "      <td>115359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Malaysia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>786</td>\n",
       "      <td>816</td>\n",
       "      <td>813</td>\n",
       "      <td>448</td>\n",
       "      <td>384</td>\n",
       "      <td>374</td>\n",
       "      <td>425</td>\n",
       "      <td>...</td>\n",
       "      <td>593</td>\n",
       "      <td>580</td>\n",
       "      <td>600</td>\n",
       "      <td>658</td>\n",
       "      <td>640</td>\n",
       "      <td>802</td>\n",
       "      <td>409</td>\n",
       "      <td>358</td>\n",
       "      <td>204</td>\n",
       "      <td>24417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Maldives</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mongolia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>59</td>\n",
       "      <td>64</td>\n",
       "      <td>82</td>\n",
       "      <td>59</td>\n",
       "      <td>118</td>\n",
       "      <td>169</td>\n",
       "      <td>103</td>\n",
       "      <td>68</td>\n",
       "      <td>99</td>\n",
       "      <td>952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Myanmar</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>80</td>\n",
       "      <td>62</td>\n",
       "      <td>46</td>\n",
       "      <td>31</td>\n",
       "      <td>41</td>\n",
       "      <td>23</td>\n",
       "      <td>18</td>\n",
       "      <td>...</td>\n",
       "      <td>210</td>\n",
       "      <td>953</td>\n",
       "      <td>1887</td>\n",
       "      <td>975</td>\n",
       "      <td>1153</td>\n",
       "      <td>556</td>\n",
       "      <td>368</td>\n",
       "      <td>193</td>\n",
       "      <td>262</td>\n",
       "      <td>9245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nepal</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>607</td>\n",
       "      <td>540</td>\n",
       "      <td>511</td>\n",
       "      <td>581</td>\n",
       "      <td>561</td>\n",
       "      <td>1392</td>\n",
       "      <td>1129</td>\n",
       "      <td>1185</td>\n",
       "      <td>1308</td>\n",
       "      <td>10222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Oman</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>14</td>\n",
       "      <td>18</td>\n",
       "      <td>16</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>13</td>\n",
       "      <td>11</td>\n",
       "      <td>224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pakistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>978</td>\n",
       "      <td>972</td>\n",
       "      <td>1201</td>\n",
       "      <td>900</td>\n",
       "      <td>668</td>\n",
       "      <td>514</td>\n",
       "      <td>691</td>\n",
       "      <td>...</td>\n",
       "      <td>14314</td>\n",
       "      <td>13127</td>\n",
       "      <td>10124</td>\n",
       "      <td>8994</td>\n",
       "      <td>7217</td>\n",
       "      <td>6811</td>\n",
       "      <td>7468</td>\n",
       "      <td>11227</td>\n",
       "      <td>12603</td>\n",
       "      <td>241600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippines</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>6051</td>\n",
       "      <td>5921</td>\n",
       "      <td>5249</td>\n",
       "      <td>4562</td>\n",
       "      <td>3801</td>\n",
       "      <td>3150</td>\n",
       "      <td>4166</td>\n",
       "      <td>...</td>\n",
       "      <td>18139</td>\n",
       "      <td>18400</td>\n",
       "      <td>19837</td>\n",
       "      <td>24887</td>\n",
       "      <td>28573</td>\n",
       "      <td>38617</td>\n",
       "      <td>36765</td>\n",
       "      <td>34315</td>\n",
       "      <td>29544</td>\n",
       "      <td>511391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Qatar</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>18</td>\n",
       "      <td>3</td>\n",
       "      <td>14</td>\n",
       "      <td>6</td>\n",
       "      <td>157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Republic of Korea</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1011</td>\n",
       "      <td>1456</td>\n",
       "      <td>1572</td>\n",
       "      <td>1081</td>\n",
       "      <td>847</td>\n",
       "      <td>962</td>\n",
       "      <td>1208</td>\n",
       "      <td>...</td>\n",
       "      <td>5832</td>\n",
       "      <td>6215</td>\n",
       "      <td>5920</td>\n",
       "      <td>7294</td>\n",
       "      <td>5874</td>\n",
       "      <td>5537</td>\n",
       "      <td>4588</td>\n",
       "      <td>5316</td>\n",
       "      <td>4509</td>\n",
       "      <td>142581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Saudi Arabia</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>198</td>\n",
       "      <td>252</td>\n",
       "      <td>188</td>\n",
       "      <td>249</td>\n",
       "      <td>246</td>\n",
       "      <td>330</td>\n",
       "      <td>278</td>\n",
       "      <td>286</td>\n",
       "      <td>267</td>\n",
       "      <td>3425</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Singapore</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>241</td>\n",
       "      <td>301</td>\n",
       "      <td>337</td>\n",
       "      <td>169</td>\n",
       "      <td>128</td>\n",
       "      <td>139</td>\n",
       "      <td>205</td>\n",
       "      <td>...</td>\n",
       "      <td>392</td>\n",
       "      <td>298</td>\n",
       "      <td>690</td>\n",
       "      <td>734</td>\n",
       "      <td>366</td>\n",
       "      <td>805</td>\n",
       "      <td>219</td>\n",
       "      <td>146</td>\n",
       "      <td>141</td>\n",
       "      <td>14579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sri Lanka</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>185</td>\n",
       "      <td>371</td>\n",
       "      <td>290</td>\n",
       "      <td>197</td>\n",
       "      <td>1086</td>\n",
       "      <td>845</td>\n",
       "      <td>1838</td>\n",
       "      <td>...</td>\n",
       "      <td>4930</td>\n",
       "      <td>4714</td>\n",
       "      <td>4123</td>\n",
       "      <td>4756</td>\n",
       "      <td>4547</td>\n",
       "      <td>4422</td>\n",
       "      <td>3309</td>\n",
       "      <td>3338</td>\n",
       "      <td>2394</td>\n",
       "      <td>148358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>State of Palestine</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>453</td>\n",
       "      <td>627</td>\n",
       "      <td>441</td>\n",
       "      <td>481</td>\n",
       "      <td>400</td>\n",
       "      <td>654</td>\n",
       "      <td>555</td>\n",
       "      <td>533</td>\n",
       "      <td>462</td>\n",
       "      <td>6512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Syrian Arab Republic</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>315</td>\n",
       "      <td>419</td>\n",
       "      <td>409</td>\n",
       "      <td>269</td>\n",
       "      <td>264</td>\n",
       "      <td>385</td>\n",
       "      <td>493</td>\n",
       "      <td>...</td>\n",
       "      <td>1458</td>\n",
       "      <td>1145</td>\n",
       "      <td>1056</td>\n",
       "      <td>919</td>\n",
       "      <td>917</td>\n",
       "      <td>1039</td>\n",
       "      <td>1005</td>\n",
       "      <td>650</td>\n",
       "      <td>1009</td>\n",
       "      <td>31485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tajikistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Central Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>85</td>\n",
       "      <td>46</td>\n",
       "      <td>44</td>\n",
       "      <td>15</td>\n",
       "      <td>50</td>\n",
       "      <td>52</td>\n",
       "      <td>47</td>\n",
       "      <td>34</td>\n",
       "      <td>39</td>\n",
       "      <td>503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thailand</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>56</td>\n",
       "      <td>53</td>\n",
       "      <td>113</td>\n",
       "      <td>65</td>\n",
       "      <td>82</td>\n",
       "      <td>66</td>\n",
       "      <td>78</td>\n",
       "      <td>...</td>\n",
       "      <td>575</td>\n",
       "      <td>500</td>\n",
       "      <td>487</td>\n",
       "      <td>519</td>\n",
       "      <td>512</td>\n",
       "      <td>499</td>\n",
       "      <td>396</td>\n",
       "      <td>296</td>\n",
       "      <td>400</td>\n",
       "      <td>9174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turkey</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>481</td>\n",
       "      <td>874</td>\n",
       "      <td>706</td>\n",
       "      <td>280</td>\n",
       "      <td>338</td>\n",
       "      <td>202</td>\n",
       "      <td>257</td>\n",
       "      <td>...</td>\n",
       "      <td>2065</td>\n",
       "      <td>1638</td>\n",
       "      <td>1463</td>\n",
       "      <td>1122</td>\n",
       "      <td>1238</td>\n",
       "      <td>1492</td>\n",
       "      <td>1257</td>\n",
       "      <td>1068</td>\n",
       "      <td>729</td>\n",
       "      <td>31781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turkmenistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Central Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>40</td>\n",
       "      <td>26</td>\n",
       "      <td>37</td>\n",
       "      <td>13</td>\n",
       "      <td>20</td>\n",
       "      <td>30</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>14</td>\n",
       "      <td>310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>United Arab Emirates</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>31</td>\n",
       "      <td>42</td>\n",
       "      <td>37</td>\n",
       "      <td>33</td>\n",
       "      <td>37</td>\n",
       "      <td>86</td>\n",
       "      <td>60</td>\n",
       "      <td>54</td>\n",
       "      <td>46</td>\n",
       "      <td>836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Uzbekistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Central Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>330</td>\n",
       "      <td>262</td>\n",
       "      <td>284</td>\n",
       "      <td>215</td>\n",
       "      <td>288</td>\n",
       "      <td>289</td>\n",
       "      <td>162</td>\n",
       "      <td>235</td>\n",
       "      <td>167</td>\n",
       "      <td>3368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Viet Nam</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1191</td>\n",
       "      <td>1829</td>\n",
       "      <td>2162</td>\n",
       "      <td>3404</td>\n",
       "      <td>7583</td>\n",
       "      <td>5907</td>\n",
       "      <td>2741</td>\n",
       "      <td>...</td>\n",
       "      <td>1852</td>\n",
       "      <td>3153</td>\n",
       "      <td>2574</td>\n",
       "      <td>1784</td>\n",
       "      <td>2171</td>\n",
       "      <td>1942</td>\n",
       "      <td>1723</td>\n",
       "      <td>1731</td>\n",
       "      <td>2112</td>\n",
       "      <td>97146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Yemen</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Western Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>161</td>\n",
       "      <td>140</td>\n",
       "      <td>122</td>\n",
       "      <td>133</td>\n",
       "      <td>128</td>\n",
       "      <td>211</td>\n",
       "      <td>160</td>\n",
       "      <td>174</td>\n",
       "      <td>217</td>\n",
       "      <td>2985</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>49 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Continent              Region  \\\n",
       "Afghanistan                                         Asia       Southern Asia   \n",
       "Armenia                                             Asia        Western Asia   \n",
       "Azerbaijan                                          Asia        Western Asia   \n",
       "Bahrain                                             Asia        Western Asia   \n",
       "Bangladesh                                          Asia       Southern Asia   \n",
       "Bhutan                                              Asia       Southern Asia   \n",
       "Brunei Darussalam                                   Asia  South-Eastern Asia   \n",
       "Cambodia                                            Asia  South-Eastern Asia   \n",
       "China                                               Asia        Eastern Asia   \n",
       "China, Hong Kong Special Administrative Region      Asia        Eastern Asia   \n",
       "China, Macao Special Administrative Region          Asia        Eastern Asia   \n",
       "Cyprus                                              Asia        Western Asia   \n",
       "Democratic People's Republic of Korea               Asia        Eastern Asia   \n",
       "Georgia                                             Asia        Western Asia   \n",
       "India                                               Asia       Southern Asia   \n",
       "Indonesia                                           Asia  South-Eastern Asia   \n",
       "Iran (Islamic Republic of)                          Asia       Southern Asia   \n",
       "Iraq                                                Asia        Western Asia   \n",
       "Israel                                              Asia        Western Asia   \n",
       "Japan                                               Asia        Eastern Asia   \n",
       "Jordan                                              Asia        Western Asia   \n",
       "Kazakhstan                                          Asia        Central Asia   \n",
       "Kuwait                                              Asia        Western Asia   \n",
       "Kyrgyzstan                                          Asia        Central Asia   \n",
       "Lao People's Democratic Republic                    Asia  South-Eastern Asia   \n",
       "Lebanon                                             Asia        Western Asia   \n",
       "Malaysia                                            Asia  South-Eastern Asia   \n",
       "Maldives                                            Asia       Southern Asia   \n",
       "Mongolia                                            Asia        Eastern Asia   \n",
       "Myanmar                                             Asia  South-Eastern Asia   \n",
       "Nepal                                               Asia       Southern Asia   \n",
       "Oman                                                Asia        Western Asia   \n",
       "Pakistan                                            Asia       Southern Asia   \n",
       "Philippines                                         Asia  South-Eastern Asia   \n",
       "Qatar                                               Asia        Western Asia   \n",
       "Republic of Korea                                   Asia        Eastern Asia   \n",
       "Saudi Arabia                                        Asia        Western Asia   \n",
       "Singapore                                           Asia  South-Eastern Asia   \n",
       "Sri Lanka                                           Asia       Southern Asia   \n",
       "State of Palestine                                  Asia        Western Asia   \n",
       "Syrian Arab Republic                                Asia        Western Asia   \n",
       "Tajikistan                                          Asia        Central Asia   \n",
       "Thailand                                            Asia  South-Eastern Asia   \n",
       "Turkey                                              Asia        Western Asia   \n",
       "Turkmenistan                                        Asia        Central Asia   \n",
       "United Arab Emirates                                Asia        Western Asia   \n",
       "Uzbekistan                                          Asia        Central Asia   \n",
       "Viet Nam                                            Asia  South-Eastern Asia   \n",
       "Yemen                                               Asia        Western Asia   \n",
       "\n",
       "                                                           DevName  1980  \\\n",
       "Afghanistan                                     Developing regions    16   \n",
       "Armenia                                         Developing regions     0   \n",
       "Azerbaijan                                      Developing regions     0   \n",
       "Bahrain                                         Developing regions     0   \n",
       "Bangladesh                                      Developing regions    83   \n",
       "Bhutan                                          Developing regions     0   \n",
       "Brunei Darussalam                               Developing regions    79   \n",
       "Cambodia                                        Developing regions    12   \n",
       "China                                           Developing regions  5123   \n",
       "China, Hong Kong Special Administrative Region  Developing regions     0   \n",
       "China, Macao Special Administrative Region      Developing regions     0   \n",
       "Cyprus                                          Developing regions   132   \n",
       "Democratic People's Republic of Korea           Developing regions     1   \n",
       "Georgia                                         Developing regions     0   \n",
       "India                                           Developing regions  8880   \n",
       "Indonesia                                       Developing regions   186   \n",
       "Iran (Islamic Republic of)                      Developing regions  1172   \n",
       "Iraq                                            Developing regions   262   \n",
       "Israel                                          Developing regions  1403   \n",
       "Japan                                            Developed regions   701   \n",
       "Jordan                                          Developing regions   177   \n",
       "Kazakhstan                                      Developing regions     0   \n",
       "Kuwait                                          Developing regions     1   \n",
       "Kyrgyzstan                                      Developing regions     0   \n",
       "Lao People's Democratic Republic                Developing regions    11   \n",
       "Lebanon                                         Developing regions  1409   \n",
       "Malaysia                                        Developing regions   786   \n",
       "Maldives                                        Developing regions     0   \n",
       "Mongolia                                        Developing regions     0   \n",
       "Myanmar                                         Developing regions    80   \n",
       "Nepal                                           Developing regions     1   \n",
       "Oman                                            Developing regions     0   \n",
       "Pakistan                                        Developing regions   978   \n",
       "Philippines                                     Developing regions  6051   \n",
       "Qatar                                           Developing regions     0   \n",
       "Republic of Korea                               Developing regions  1011   \n",
       "Saudi Arabia                                    Developing regions     0   \n",
       "Singapore                                       Developing regions   241   \n",
       "Sri Lanka                                       Developing regions   185   \n",
       "State of Palestine                              Developing regions     0   \n",
       "Syrian Arab Republic                            Developing regions   315   \n",
       "Tajikistan                                      Developing regions     0   \n",
       "Thailand                                        Developing regions    56   \n",
       "Turkey                                          Developing regions   481   \n",
       "Turkmenistan                                    Developing regions     0   \n",
       "United Arab Emirates                            Developing regions     0   \n",
       "Uzbekistan                                      Developing regions     0   \n",
       "Viet Nam                                        Developing regions  1191   \n",
       "Yemen                                           Developing regions     1   \n",
       "\n",
       "                                                1981  1982  1983  1984  1985  \\\n",
       "Afghanistan                                       39    39    47    71   340   \n",
       "Armenia                                            0     0     0     0     0   \n",
       "Azerbaijan                                         0     0     0     0     0   \n",
       "Bahrain                                            2     1     1     1     3   \n",
       "Bangladesh                                        84    86    81    98    92   \n",
       "Bhutan                                             0     0     0     1     0   \n",
       "Brunei Darussalam                                  6     8     2     2     4   \n",
       "Cambodia                                          19    26    33    10     7   \n",
       "China                                           6682  3308  1863  1527  1816   \n",
       "China, Hong Kong Special Administrative Region     0     0     0     0     0   \n",
       "China, Macao Special Administrative Region         0     0     0     0     0   \n",
       "Cyprus                                           128    84    46    46    43   \n",
       "Democratic People's Republic of Korea              1     3     1     4     3   \n",
       "Georgia                                            0     0     0     0     0   \n",
       "India                                           8670  8147  7338  5704  4211   \n",
       "Indonesia                                        178   252   115   123   100   \n",
       "Iran (Islamic Republic of)                      1429  1822  1592  1977  1648   \n",
       "Iraq                                             245   260   380   428   231   \n",
       "Israel                                          1711  1334   541   446   680   \n",
       "Japan                                            756   598   309   246   198   \n",
       "Jordan                                           160   155   113   102   179   \n",
       "Kazakhstan                                         0     0     0     0     0   \n",
       "Kuwait                                             0     8     2     1     4   \n",
       "Kyrgyzstan                                         0     0     0     0     0   \n",
       "Lao People's Democratic Republic                   6    16    16     7    17   \n",
       "Lebanon                                         1119  1159   789  1253  1683   \n",
       "Malaysia                                         816   813   448   384   374   \n",
       "Maldives                                           0     0     1     0     0   \n",
       "Mongolia                                           0     0     0     0     0   \n",
       "Myanmar                                           62    46    31    41    23   \n",
       "Nepal                                              1     6     1     2     4   \n",
       "Oman                                               0     0     8     0     0   \n",
       "Pakistan                                         972  1201   900   668   514   \n",
       "Philippines                                     5921  5249  4562  3801  3150   \n",
       "Qatar                                              0     0     0     0     0   \n",
       "Republic of Korea                               1456  1572  1081   847   962   \n",
       "Saudi Arabia                                       0     1     4     1     2   \n",
       "Singapore                                        301   337   169   128   139   \n",
       "Sri Lanka                                        371   290   197  1086   845   \n",
       "State of Palestine                                 0     0     0     0     0   \n",
       "Syrian Arab Republic                             419   409   269   264   385   \n",
       "Tajikistan                                         0     0     0     0     0   \n",
       "Thailand                                          53   113    65    82    66   \n",
       "Turkey                                           874   706   280   338   202   \n",
       "Turkmenistan                                       0     0     0     0     0   \n",
       "United Arab Emirates                               2     2     1     2     0   \n",
       "Uzbekistan                                         0     0     0     0     0   \n",
       "Viet Nam                                        1829  2162  3404  7583  5907   \n",
       "Yemen                                              2     1     6     0    18   \n",
       "\n",
       "                                                1986  ...   2005   2006  \\\n",
       "Afghanistan                                      496  ...   3436   3009   \n",
       "Armenia                                            0  ...    224    218   \n",
       "Azerbaijan                                         0  ...    359    236   \n",
       "Bahrain                                            0  ...     12     12   \n",
       "Bangladesh                                       486  ...   4171   4014   \n",
       "Bhutan                                             0  ...      5     10   \n",
       "Brunei Darussalam                                 12  ...      4      5   \n",
       "Cambodia                                           8  ...    370    529   \n",
       "China                                           1960  ...  42584  33518   \n",
       "China, Hong Kong Special Administrative Region     0  ...    729    712   \n",
       "China, Macao Special Administrative Region         0  ...     21     32   \n",
       "Cyprus                                            48  ...      7      9   \n",
       "Democratic People's Republic of Korea              0  ...     14     10   \n",
       "Georgia                                            0  ...    114    125   \n",
       "India                                           7150  ...  36210  33848   \n",
       "Indonesia                                        127  ...    632    613   \n",
       "Iran (Islamic Republic of)                      1794  ...   5837   7480   \n",
       "Iraq                                             265  ...   2226   1788   \n",
       "Israel                                          1212  ...   2446   2625   \n",
       "Japan                                            248  ...   1067   1212   \n",
       "Jordan                                           181  ...   1940   1827   \n",
       "Kazakhstan                                         0  ...    506    408   \n",
       "Kuwait                                             4  ...     66     35   \n",
       "Kyrgyzstan                                         0  ...    173    161   \n",
       "Lao People's Democratic Republic                  21  ...     42     74   \n",
       "Lebanon                                         2576  ...   3709   3802   \n",
       "Malaysia                                         425  ...    593    580   \n",
       "Maldives                                           0  ...      0      0   \n",
       "Mongolia                                           0  ...     59     64   \n",
       "Myanmar                                           18  ...    210    953   \n",
       "Nepal                                             13  ...    607    540   \n",
       "Oman                                               0  ...     14     18   \n",
       "Pakistan                                         691  ...  14314  13127   \n",
       "Philippines                                     4166  ...  18139  18400   \n",
       "Qatar                                              1  ...     11      2   \n",
       "Republic of Korea                               1208  ...   5832   6215   \n",
       "Saudi Arabia                                       5  ...    198    252   \n",
       "Singapore                                        205  ...    392    298   \n",
       "Sri Lanka                                       1838  ...   4930   4714   \n",
       "State of Palestine                                 0  ...    453    627   \n",
       "Syrian Arab Republic                             493  ...   1458   1145   \n",
       "Tajikistan                                         0  ...     85     46   \n",
       "Thailand                                          78  ...    575    500   \n",
       "Turkey                                           257  ...   2065   1638   \n",
       "Turkmenistan                                       0  ...     40     26   \n",
       "United Arab Emirates                               5  ...     31     42   \n",
       "Uzbekistan                                         0  ...    330    262   \n",
       "Viet Nam                                        2741  ...   1852   3153   \n",
       "Yemen                                              7  ...    161    140   \n",
       "\n",
       "                                                 2007   2008   2009   2010  \\\n",
       "Afghanistan                                      2652   2111   1746   1758   \n",
       "Armenia                                           198    205    267    252   \n",
       "Azerbaijan                                        203    125    165    209   \n",
       "Bahrain                                            22      9     35     28   \n",
       "Bangladesh                                       2897   2939   2104   4721   \n",
       "Bhutan                                              7     36    865   1464   \n",
       "Brunei Darussalam                                  11     10      5     12   \n",
       "Cambodia                                          460    354    203    200   \n",
       "China                                           27642  30037  29622  30391   \n",
       "China, Hong Kong Special Administrative Region    674    897    657    623   \n",
       "China, Macao Special Administrative Region         16     12     21     21   \n",
       "Cyprus                                              4      7      6     18   \n",
       "Democratic People's Republic of Korea               7     19     11     45   \n",
       "Georgia                                           132    112    128    126   \n",
       "India                                           28742  28261  29456  34235   \n",
       "Indonesia                                         657    661    504    712   \n",
       "Iran (Islamic Republic of)                       6974   6475   6580   7477   \n",
       "Iraq                                             2406   3543   5450   5941   \n",
       "Israel                                           2401   2562   2316   2755   \n",
       "Japan                                            1250   1284   1194   1168   \n",
       "Jordan                                           1421   1581   1235   1831   \n",
       "Kazakhstan                                        436    394    431    377   \n",
       "Kuwait                                             62     53     68     67   \n",
       "Kyrgyzstan                                        135    168    173    157   \n",
       "Lao People's Democratic Republic                   53     32     39     54   \n",
       "Lebanon                                          3467   3566   3077   3432   \n",
       "Malaysia                                          600    658    640    802   \n",
       "Maldives                                            2      1      7      4   \n",
       "Mongolia                                           82     59    118    169   \n",
       "Myanmar                                          1887    975   1153    556   \n",
       "Nepal                                             511    581    561   1392   \n",
       "Oman                                               16     10      7     14   \n",
       "Pakistan                                        10124   8994   7217   6811   \n",
       "Philippines                                     19837  24887  28573  38617   \n",
       "Qatar                                               5      9      6     18   \n",
       "Republic of Korea                                5920   7294   5874   5537   \n",
       "Saudi Arabia                                      188    249    246    330   \n",
       "Singapore                                         690    734    366    805   \n",
       "Sri Lanka                                        4123   4756   4547   4422   \n",
       "State of Palestine                                441    481    400    654   \n",
       "Syrian Arab Republic                             1056    919    917   1039   \n",
       "Tajikistan                                         44     15     50     52   \n",
       "Thailand                                          487    519    512    499   \n",
       "Turkey                                           1463   1122   1238   1492   \n",
       "Turkmenistan                                       37     13     20     30   \n",
       "United Arab Emirates                               37     33     37     86   \n",
       "Uzbekistan                                        284    215    288    289   \n",
       "Viet Nam                                         2574   1784   2171   1942   \n",
       "Yemen                                             122    133    128    211   \n",
       "\n",
       "                                                 2011   2012   2013   Total  \n",
       "Afghanistan                                      2203   2635   2004   58639  \n",
       "Armenia                                           236    258    207    3310  \n",
       "Azerbaijan                                        138    161     57    2649  \n",
       "Bahrain                                            21     39     32     475  \n",
       "Bangladesh                                       2694   2640   3789   65568  \n",
       "Bhutan                                           1879   1075    487    5876  \n",
       "Brunei Darussalam                                   6      3      6     600  \n",
       "Cambodia                                          196    233    288    6538  \n",
       "China                                           28502  33024  34129  659962  \n",
       "China, Hong Kong Special Administrative Region    591    728    774    9327  \n",
       "China, Macao Special Administrative Region         13     33     29     284  \n",
       "Cyprus                                              6     12     16    1126  \n",
       "Democratic People's Republic of Korea              97     66     17     388  \n",
       "Georgia                                           139    147    125    2068  \n",
       "India                                           27509  30933  33087  691904  \n",
       "Indonesia                                         390    395    387   13150  \n",
       "Iran (Islamic Republic of)                       7479   7534  11291  175923  \n",
       "Iraq                                             6196   4041   4918   69789  \n",
       "Israel                                           1970   2134   1945   66508  \n",
       "Japan                                            1265   1214    982   27707  \n",
       "Jordan                                           1635   1206   1255   35406  \n",
       "Kazakhstan                                        381    462    348    8490  \n",
       "Kuwait                                             58     73     48    2025  \n",
       "Kyrgyzstan                                        159    278    123    2353  \n",
       "Lao People's Democratic Republic                   22     25     15    1089  \n",
       "Lebanon                                          3072   1614   2172  115359  \n",
       "Malaysia                                          409    358    204   24417  \n",
       "Maldives                                            3      1      1      30  \n",
       "Mongolia                                          103     68     99     952  \n",
       "Myanmar                                           368    193    262    9245  \n",
       "Nepal                                            1129   1185   1308   10222  \n",
       "Oman                                               10     13     11     224  \n",
       "Pakistan                                         7468  11227  12603  241600  \n",
       "Philippines                                     36765  34315  29544  511391  \n",
       "Qatar                                               3     14      6     157  \n",
       "Republic of Korea                                4588   5316   4509  142581  \n",
       "Saudi Arabia                                      278    286    267    3425  \n",
       "Singapore                                         219    146    141   14579  \n",
       "Sri Lanka                                        3309   3338   2394  148358  \n",
       "State of Palestine                                555    533    462    6512  \n",
       "Syrian Arab Republic                             1005    650   1009   31485  \n",
       "Tajikistan                                         47     34     39     503  \n",
       "Thailand                                          396    296    400    9174  \n",
       "Turkey                                           1257   1068    729   31781  \n",
       "Turkmenistan                                       20     20     14     310  \n",
       "United Arab Emirates                               60     54     46     836  \n",
       "Uzbekistan                                        162    235    167    3368  \n",
       "Viet Nam                                         1723   1731   2112   97146  \n",
       "Yemen                                             160    174    217    2985  \n",
       "\n",
       "[49 rows x 38 columns]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2. pass this condition into the dataFrame\n",
    "df_can[condition]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Afghanistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>16</td>\n",
       "      <td>39</td>\n",
       "      <td>39</td>\n",
       "      <td>47</td>\n",
       "      <td>71</td>\n",
       "      <td>340</td>\n",
       "      <td>496</td>\n",
       "      <td>...</td>\n",
       "      <td>3436</td>\n",
       "      <td>3009</td>\n",
       "      <td>2652</td>\n",
       "      <td>2111</td>\n",
       "      <td>1746</td>\n",
       "      <td>1758</td>\n",
       "      <td>2203</td>\n",
       "      <td>2635</td>\n",
       "      <td>2004</td>\n",
       "      <td>58639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bangladesh</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>83</td>\n",
       "      <td>84</td>\n",
       "      <td>86</td>\n",
       "      <td>81</td>\n",
       "      <td>98</td>\n",
       "      <td>92</td>\n",
       "      <td>486</td>\n",
       "      <td>...</td>\n",
       "      <td>4171</td>\n",
       "      <td>4014</td>\n",
       "      <td>2897</td>\n",
       "      <td>2939</td>\n",
       "      <td>2104</td>\n",
       "      <td>4721</td>\n",
       "      <td>2694</td>\n",
       "      <td>2640</td>\n",
       "      <td>3789</td>\n",
       "      <td>65568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bhutan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>36</td>\n",
       "      <td>865</td>\n",
       "      <td>1464</td>\n",
       "      <td>1879</td>\n",
       "      <td>1075</td>\n",
       "      <td>487</td>\n",
       "      <td>5876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>India</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>8880</td>\n",
       "      <td>8670</td>\n",
       "      <td>8147</td>\n",
       "      <td>7338</td>\n",
       "      <td>5704</td>\n",
       "      <td>4211</td>\n",
       "      <td>7150</td>\n",
       "      <td>...</td>\n",
       "      <td>36210</td>\n",
       "      <td>33848</td>\n",
       "      <td>28742</td>\n",
       "      <td>28261</td>\n",
       "      <td>29456</td>\n",
       "      <td>34235</td>\n",
       "      <td>27509</td>\n",
       "      <td>30933</td>\n",
       "      <td>33087</td>\n",
       "      <td>691904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iran (Islamic Republic of)</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1172</td>\n",
       "      <td>1429</td>\n",
       "      <td>1822</td>\n",
       "      <td>1592</td>\n",
       "      <td>1977</td>\n",
       "      <td>1648</td>\n",
       "      <td>1794</td>\n",
       "      <td>...</td>\n",
       "      <td>5837</td>\n",
       "      <td>7480</td>\n",
       "      <td>6974</td>\n",
       "      <td>6475</td>\n",
       "      <td>6580</td>\n",
       "      <td>7477</td>\n",
       "      <td>7479</td>\n",
       "      <td>7534</td>\n",
       "      <td>11291</td>\n",
       "      <td>175923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Maldives</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nepal</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>607</td>\n",
       "      <td>540</td>\n",
       "      <td>511</td>\n",
       "      <td>581</td>\n",
       "      <td>561</td>\n",
       "      <td>1392</td>\n",
       "      <td>1129</td>\n",
       "      <td>1185</td>\n",
       "      <td>1308</td>\n",
       "      <td>10222</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pakistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>978</td>\n",
       "      <td>972</td>\n",
       "      <td>1201</td>\n",
       "      <td>900</td>\n",
       "      <td>668</td>\n",
       "      <td>514</td>\n",
       "      <td>691</td>\n",
       "      <td>...</td>\n",
       "      <td>14314</td>\n",
       "      <td>13127</td>\n",
       "      <td>10124</td>\n",
       "      <td>8994</td>\n",
       "      <td>7217</td>\n",
       "      <td>6811</td>\n",
       "      <td>7468</td>\n",
       "      <td>11227</td>\n",
       "      <td>12603</td>\n",
       "      <td>241600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sri Lanka</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>185</td>\n",
       "      <td>371</td>\n",
       "      <td>290</td>\n",
       "      <td>197</td>\n",
       "      <td>1086</td>\n",
       "      <td>845</td>\n",
       "      <td>1838</td>\n",
       "      <td>...</td>\n",
       "      <td>4930</td>\n",
       "      <td>4714</td>\n",
       "      <td>4123</td>\n",
       "      <td>4756</td>\n",
       "      <td>4547</td>\n",
       "      <td>4422</td>\n",
       "      <td>3309</td>\n",
       "      <td>3338</td>\n",
       "      <td>2394</td>\n",
       "      <td>148358</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Continent         Region             DevName  1980  \\\n",
       "Afghanistan                     Asia  Southern Asia  Developing regions    16   \n",
       "Bangladesh                      Asia  Southern Asia  Developing regions    83   \n",
       "Bhutan                          Asia  Southern Asia  Developing regions     0   \n",
       "India                           Asia  Southern Asia  Developing regions  8880   \n",
       "Iran (Islamic Republic of)      Asia  Southern Asia  Developing regions  1172   \n",
       "Maldives                        Asia  Southern Asia  Developing regions     0   \n",
       "Nepal                           Asia  Southern Asia  Developing regions     1   \n",
       "Pakistan                        Asia  Southern Asia  Developing regions   978   \n",
       "Sri Lanka                       Asia  Southern Asia  Developing regions   185   \n",
       "\n",
       "                            1981  1982  1983  1984  1985  1986  ...   2005  \\\n",
       "Afghanistan                   39    39    47    71   340   496  ...   3436   \n",
       "Bangladesh                    84    86    81    98    92   486  ...   4171   \n",
       "Bhutan                         0     0     0     1     0     0  ...      5   \n",
       "India                       8670  8147  7338  5704  4211  7150  ...  36210   \n",
       "Iran (Islamic Republic of)  1429  1822  1592  1977  1648  1794  ...   5837   \n",
       "Maldives                       0     0     1     0     0     0  ...      0   \n",
       "Nepal                          1     6     1     2     4    13  ...    607   \n",
       "Pakistan                     972  1201   900   668   514   691  ...  14314   \n",
       "Sri Lanka                    371   290   197  1086   845  1838  ...   4930   \n",
       "\n",
       "                             2006   2007   2008   2009   2010   2011   2012  \\\n",
       "Afghanistan                  3009   2652   2111   1746   1758   2203   2635   \n",
       "Bangladesh                   4014   2897   2939   2104   4721   2694   2640   \n",
       "Bhutan                         10      7     36    865   1464   1879   1075   \n",
       "India                       33848  28742  28261  29456  34235  27509  30933   \n",
       "Iran (Islamic Republic of)   7480   6974   6475   6580   7477   7479   7534   \n",
       "Maldives                        0      2      1      7      4      3      1   \n",
       "Nepal                         540    511    581    561   1392   1129   1185   \n",
       "Pakistan                    13127  10124   8994   7217   6811   7468  11227   \n",
       "Sri Lanka                    4714   4123   4756   4547   4422   3309   3338   \n",
       "\n",
       "                             2013   Total  \n",
       "Afghanistan                  2004   58639  \n",
       "Bangladesh                   3789   65568  \n",
       "Bhutan                        487    5876  \n",
       "India                       33087  691904  \n",
       "Iran (Islamic Republic of)  11291  175923  \n",
       "Maldives                        1      30  \n",
       "Nepal                        1308   10222  \n",
       "Pakistan                    12603  241600  \n",
       "Sri Lanka                    2394  148358  \n",
       "\n",
       "[9 rows x 38 columns]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can pass multiple criteria in the same line.\n",
    "# let's filter for AreaNAme = Asia and RegName = Southern Asia\n",
    "\n",
    "df_can[(df_can['Continent']=='Asia') & (df_can['Region']=='Southern Asia')]\n",
    "\n",
    "# note: When using 'and' and 'or' operators, pandas requires we use '&' and '|' instead of 'and' and 'or'\n",
    "# don't forget to enclose the two conditions in parentheses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Fetch the data where AreaName is 'Africa' and RegName is 'Southern Africa'. <br>Display the dataframe and find out how many instances are there?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Botswana</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Southern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>11</td>\n",
       "      <td>8</td>\n",
       "      <td>28</td>\n",
       "      <td>15</td>\n",
       "      <td>42</td>\n",
       "      <td>53</td>\n",
       "      <td>64</td>\n",
       "      <td>76</td>\n",
       "      <td>396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lesotho</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Southern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Namibia</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Southern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>19</td>\n",
       "      <td>13</td>\n",
       "      <td>26</td>\n",
       "      <td>14</td>\n",
       "      <td>16</td>\n",
       "      <td>23</td>\n",
       "      <td>24</td>\n",
       "      <td>83</td>\n",
       "      <td>320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South Africa</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Southern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>1026</td>\n",
       "      <td>1118</td>\n",
       "      <td>781</td>\n",
       "      <td>379</td>\n",
       "      <td>271</td>\n",
       "      <td>310</td>\n",
       "      <td>718</td>\n",
       "      <td>...</td>\n",
       "      <td>988</td>\n",
       "      <td>1111</td>\n",
       "      <td>1200</td>\n",
       "      <td>1123</td>\n",
       "      <td>1188</td>\n",
       "      <td>1238</td>\n",
       "      <td>959</td>\n",
       "      <td>1243</td>\n",
       "      <td>1240</td>\n",
       "      <td>40568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Swaziland</th>\n",
       "      <td>Africa</td>\n",
       "      <td>Southern Africa</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "      <td>17</td>\n",
       "      <td>39</td>\n",
       "      <td>188</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             Continent           Region             DevName  1980  1981  1982  \\\n",
       "Botswana        Africa  Southern Africa  Developing regions    10     1     3   \n",
       "Lesotho         Africa  Southern Africa  Developing regions     1     1     1   \n",
       "Namibia         Africa  Southern Africa  Developing regions     0     5     5   \n",
       "South Africa    Africa  Southern Africa  Developing regions  1026  1118   781   \n",
       "Swaziland       Africa  Southern Africa  Developing regions     4     1     1   \n",
       "\n",
       "              1983  1984  1985  1986  ...  2005  2006  2007  2008  2009  2010  \\\n",
       "Botswana         3     7     4     2  ...     7    11     8    28    15    42   \n",
       "Lesotho          2     7     5     3  ...     4     0     4     1     8     7   \n",
       "Namibia          3     2     1     1  ...     6    19    13    26    14    16   \n",
       "South Africa   379   271   310   718  ...   988  1111  1200  1123  1188  1238   \n",
       "Swaziland        0    10     7     1  ...     7     7     5     6    10     3   \n",
       "\n",
       "              2011  2012  2013  Total  \n",
       "Botswana        53    64    76    396  \n",
       "Lesotho          1     0     6    107  \n",
       "Namibia         23    24    83    320  \n",
       "South Africa   959  1243  1240  40568  \n",
       "Swaziland       13    17    39    188  \n",
       "\n",
       "[5 rows x 38 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can[(df_can['Continent']=='Africa') & (df_can['Region']=='Southern Africa')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for a sample python solution</summary>\n",
    "\n",
    "```python\n",
    "    df_can[(df_can['Continent']=='Africa') & (df_can['Region']=='Southern Africa')]\n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sorting Values of a Dataframe or Series <a id=\"5\"></a><br>\n",
    "You can use the `sort_values()` function is used to sort a DataFrame or a Series based on one or more columns. <br>You to specify the column(s) by which you want to sort and the order (ascending or descending). Below is the syntax to use it:-<br><br>\n",
    "```df.sort_values(col_name, axis=0, ascending=True, inplace=False, ignore_index=False)```<br><br>\n",
    "col_nam - the column(s) to sort by. <br>\n",
    "axis - axis along which to sort. 0 for sorting by rows (default) and 1 for sorting by columns.<br>\n",
    "ascending - to sort in ascending order (True, default) or descending order (False).<br>\n",
    "inplace - to perform the sorting operation in-place (True) or return a sorted copy (False, default).<br>\n",
    "ignore_index - to reset the index after sorting (True) or keep the original index values (False, default).<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's sort out dataframe df_can on 'Total' column, in descending order to find out the top 5 countries that contributed the most to immigration to Canada. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Continent</th>\n",
       "      <th>Region</th>\n",
       "      <th>DevName</th>\n",
       "      <th>1980</th>\n",
       "      <th>1981</th>\n",
       "      <th>1982</th>\n",
       "      <th>1983</th>\n",
       "      <th>1984</th>\n",
       "      <th>1985</th>\n",
       "      <th>1986</th>\n",
       "      <th>...</th>\n",
       "      <th>2005</th>\n",
       "      <th>2006</th>\n",
       "      <th>2007</th>\n",
       "      <th>2008</th>\n",
       "      <th>2009</th>\n",
       "      <th>2010</th>\n",
       "      <th>2011</th>\n",
       "      <th>2012</th>\n",
       "      <th>2013</th>\n",
       "      <th>Total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>India</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>8880</td>\n",
       "      <td>8670</td>\n",
       "      <td>8147</td>\n",
       "      <td>7338</td>\n",
       "      <td>5704</td>\n",
       "      <td>4211</td>\n",
       "      <td>7150</td>\n",
       "      <td>...</td>\n",
       "      <td>36210</td>\n",
       "      <td>33848</td>\n",
       "      <td>28742</td>\n",
       "      <td>28261</td>\n",
       "      <td>29456</td>\n",
       "      <td>34235</td>\n",
       "      <td>27509</td>\n",
       "      <td>30933</td>\n",
       "      <td>33087</td>\n",
       "      <td>691904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>China</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>5123</td>\n",
       "      <td>6682</td>\n",
       "      <td>3308</td>\n",
       "      <td>1863</td>\n",
       "      <td>1527</td>\n",
       "      <td>1816</td>\n",
       "      <td>1960</td>\n",
       "      <td>...</td>\n",
       "      <td>42584</td>\n",
       "      <td>33518</td>\n",
       "      <td>27642</td>\n",
       "      <td>30037</td>\n",
       "      <td>29622</td>\n",
       "      <td>30391</td>\n",
       "      <td>28502</td>\n",
       "      <td>33024</td>\n",
       "      <td>34129</td>\n",
       "      <td>659962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>United Kingdom of Great Britain and Northern Ireland</th>\n",
       "      <td>Europe</td>\n",
       "      <td>Northern Europe</td>\n",
       "      <td>Developed regions</td>\n",
       "      <td>22045</td>\n",
       "      <td>24796</td>\n",
       "      <td>20620</td>\n",
       "      <td>10015</td>\n",
       "      <td>10170</td>\n",
       "      <td>9564</td>\n",
       "      <td>9470</td>\n",
       "      <td>...</td>\n",
       "      <td>7258</td>\n",
       "      <td>7140</td>\n",
       "      <td>8216</td>\n",
       "      <td>8979</td>\n",
       "      <td>8876</td>\n",
       "      <td>8724</td>\n",
       "      <td>6204</td>\n",
       "      <td>6195</td>\n",
       "      <td>5827</td>\n",
       "      <td>551500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippines</th>\n",
       "      <td>Asia</td>\n",
       "      <td>South-Eastern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>6051</td>\n",
       "      <td>5921</td>\n",
       "      <td>5249</td>\n",
       "      <td>4562</td>\n",
       "      <td>3801</td>\n",
       "      <td>3150</td>\n",
       "      <td>4166</td>\n",
       "      <td>...</td>\n",
       "      <td>18139</td>\n",
       "      <td>18400</td>\n",
       "      <td>19837</td>\n",
       "      <td>24887</td>\n",
       "      <td>28573</td>\n",
       "      <td>38617</td>\n",
       "      <td>36765</td>\n",
       "      <td>34315</td>\n",
       "      <td>29544</td>\n",
       "      <td>511391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pakistan</th>\n",
       "      <td>Asia</td>\n",
       "      <td>Southern Asia</td>\n",
       "      <td>Developing regions</td>\n",
       "      <td>978</td>\n",
       "      <td>972</td>\n",
       "      <td>1201</td>\n",
       "      <td>900</td>\n",
       "      <td>668</td>\n",
       "      <td>514</td>\n",
       "      <td>691</td>\n",
       "      <td>...</td>\n",
       "      <td>14314</td>\n",
       "      <td>13127</td>\n",
       "      <td>10124</td>\n",
       "      <td>8994</td>\n",
       "      <td>7217</td>\n",
       "      <td>6811</td>\n",
       "      <td>7468</td>\n",
       "      <td>11227</td>\n",
       "      <td>12603</td>\n",
       "      <td>241600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   Continent  \\\n",
       "India                                                   Asia   \n",
       "China                                                   Asia   \n",
       "United Kingdom of Great Britain and Northern Ir...    Europe   \n",
       "Philippines                                             Asia   \n",
       "Pakistan                                                Asia   \n",
       "\n",
       "                                                                Region  \\\n",
       "India                                                    Southern Asia   \n",
       "China                                                     Eastern Asia   \n",
       "United Kingdom of Great Britain and Northern Ir...     Northern Europe   \n",
       "Philippines                                         South-Eastern Asia   \n",
       "Pakistan                                                 Southern Asia   \n",
       "\n",
       "                                                               DevName   1980  \\\n",
       "India                                               Developing regions   8880   \n",
       "China                                               Developing regions   5123   \n",
       "United Kingdom of Great Britain and Northern Ir...   Developed regions  22045   \n",
       "Philippines                                         Developing regions   6051   \n",
       "Pakistan                                            Developing regions    978   \n",
       "\n",
       "                                                     1981   1982   1983  \\\n",
       "India                                                8670   8147   7338   \n",
       "China                                                6682   3308   1863   \n",
       "United Kingdom of Great Britain and Northern Ir...  24796  20620  10015   \n",
       "Philippines                                          5921   5249   4562   \n",
       "Pakistan                                              972   1201    900   \n",
       "\n",
       "                                                     1984  1985  1986  ...  \\\n",
       "India                                                5704  4211  7150  ...   \n",
       "China                                                1527  1816  1960  ...   \n",
       "United Kingdom of Great Britain and Northern Ir...  10170  9564  9470  ...   \n",
       "Philippines                                          3801  3150  4166  ...   \n",
       "Pakistan                                              668   514   691  ...   \n",
       "\n",
       "                                                     2005   2006   2007  \\\n",
       "India                                               36210  33848  28742   \n",
       "China                                               42584  33518  27642   \n",
       "United Kingdom of Great Britain and Northern Ir...   7258   7140   8216   \n",
       "Philippines                                         18139  18400  19837   \n",
       "Pakistan                                            14314  13127  10124   \n",
       "\n",
       "                                                     2008   2009   2010  \\\n",
       "India                                               28261  29456  34235   \n",
       "China                                               30037  29622  30391   \n",
       "United Kingdom of Great Britain and Northern Ir...   8979   8876   8724   \n",
       "Philippines                                         24887  28573  38617   \n",
       "Pakistan                                             8994   7217   6811   \n",
       "\n",
       "                                                     2011   2012   2013  \\\n",
       "India                                               27509  30933  33087   \n",
       "China                                               28502  33024  34129   \n",
       "United Kingdom of Great Britain and Northern Ir...   6204   6195   5827   \n",
       "Philippines                                         36765  34315  29544   \n",
       "Pakistan                                             7468  11227  12603   \n",
       "\n",
       "                                                     Total  \n",
       "India                                               691904  \n",
       "China                                               659962  \n",
       "United Kingdom of Great Britain and Northern Ir...  551500  \n",
       "Philippines                                         511391  \n",
       "Pakistan                                            241600  \n",
       "\n",
       "[5 rows x 38 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.sort_values(by='Total', ascending=False, axis=0, inplace=True)\n",
    "top_5 = df_can.head(5)\n",
    "top_5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** Find out top 3 countries that contributes the most to immigration to Canda in the year 2010. <br> Display the country names with the immigrant count in this year\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Philippines    38617\n",
       "India          34235\n",
       "China          30391\n",
       "Name: 2010, dtype: int64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_can.sort_values(by='2010', ascending=False, axis=0, inplace=True)\n",
    "top3_2010 = df_can['2010'].head(3)\n",
    "top3_2010"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary>Click here for a sample python solution</summary>\n",
    "\n",
    "```python\n",
    "    df_can.sort_values(by='2010', ascending=False, axis=0, inplace=True)\n",
    "    top3_2010 = df_can['2010'].head(3)\n",
    "    top3_2010\n",
    "```\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations! you have learned how to wrangle data with Pandas. You will be using alot of these commands to preprocess the data before its can be used for data visualization.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Thank you for completing this lab!\n",
    "\n",
    "\n",
    "## Author\n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/aklson/\" target=\"_blank\">Alex Aklson</a>\n",
    "\n",
    "\n",
    "### Other Contributors\n",
    "[Jay Rajasekharan](https://www.linkedin.com/in/jayrajasekharan),\n",
    "[Ehsan M. Kermani](https://www.linkedin.com/in/ehsanmkermani),\n",
    "[Slobodan Markovic](https://www.linkedin.com/in/slobodan-markovic),\n",
    "[Weiqing Wang](https://www.linkedin.com/in/weiqing-wang-641640133/),\n",
    "[Dr. Pooja](https://www.linkedin.com/in/p-b28802262/)\n",
    "\n",
    "\n",
    "## Change Log\n",
    "\n",
    "\n",
    "|  Date (YYYY-MM-DD) | Version | Changed By    |  Change Description                   |\n",
    "|--------------------|---------|---------------|---------------------------------------|\n",
    "| 2023-06-08         | 2.5     | Dr. Pooja         |  Separated from original lab        |\n",
    "| 2021-05-29         | 2.4     | Weiqing Wang  |  Fixed typos and code smells.         |\n",
    "| 2021-01-20         | 2.3     | Lakshmi Holla |  Changed TOC cell markdown            |\n",
    "| 2020-11-20         | 2.2     | Lakshmi Holla |  Changed IBM box URL                  |\n",
    "| 2020-11-03         | 2.1     | Lakshmi Holla |  Changed URL and info method          |\n",
    "| 2020-08-27         | 2.0     | Lavanya       |  Moved Lab to course repo in GitLab   |\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}