Apache_PySpark_by_Example.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "qJoeN3e8_Gzk",
        "Dd6t0uFzuR4X",
        "8qf-TmnsBgXd",
        "3UJoYm5zwOWp",
        "mI_zYOVpf2yK",
        "NiU1nhA3y0j4",
        "9sDnyyzozVxA",
        "D7Dw2UpQ08-9",
        "hiKFyArf14B0",
        "9lAyzg282V_4",
        "-aQQ2gBF3Azz",
        "oSOpIOhJ4etN"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Apache PySpark by Example"
      ],
      "metadata": {
        "id": "ZZPtipYTrbgk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**[June 2023 update]**\n",
        "\n",
        "I've consolidated all the notebooks for this course into a single notebook. (The course videos will still show individual notebooks)\n"
      ],
      "metadata": {
        "id": "dTZDpUrdtXZf"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJoeN3e8_Gzk"
      },
      "source": [
        "## Introduction to Google Colab"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_N5-lspH_N8B"
      },
      "source": [
        "### Jupyter notebook basics"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Ul54hAYyHyd"
      },
      "source": [
        "#### Code cells"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j38beRUTCI5c"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_Jewe_e9CIYa"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "g8Y7w6_CCIIT"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VOqLNkRKyUIS"
      },
      "source": [
        "#### Text cells"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "neY8vxXzyXDq"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X6zdrH15_CCW"
      },
      "source": [
        "### Access to the shell"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zdO9sjSdEVnr"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0785Cp2qD0VX"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QF9e3lDDEX3I"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sfzbZVIwEeaF"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ip_onRMlEi_p"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DN5HQbosEm1k"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IX947X96Gdpj"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dd6t0uFzuR4X"
      },
      "source": [
        "## Install Spark"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Google colab recently made some changes which breaks the Spark installation.\n",
        "- Please use the code below where we install from the pyspark package instead"
      ],
      "metadata": {
        "id": "qu34sKRJvhq_"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "We2yp-o7Hgu9"
      },
      "outputs": [],
      "source": [
        "!pip install pyspark==3.4.0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QdrFO3yrwptm"
      },
      "outputs": [],
      "source": [
        "from pyspark.sql import SparkSession\n",
        "spark = SparkSession.builder.master(\"local[*]\").getOrCreate()\n",
        "spark"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "XO081lQtutJ-"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8qf-TmnsBgXd"
      },
      "source": [
        "## (02-02) Download Chicago's Reported Crime Data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ysUlfFrN5OD"
      },
      "source": [
        "### Downloading and preprocessing Chicago's Reported Crime Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FDtw5Hy3N-pV"
      },
      "outputs": [],
      "source": [
        "!wget https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD\n",
        "!ls -l"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v4P5mMONYyVd"
      },
      "outputs": [],
      "source": [
        "!mv rows.csv\\?accessType\\=DOWNLOAD reported-crimes.csv\n",
        "!ls -l"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9-1cK0nPNS95"
      },
      "outputs": [],
      "source": [
        "from pyspark.sql.functions import to_timestamp,col,lit\n",
        "rc = spark.read.csv('reported-crimes.csv',header=True).withColumn('Date',to_timestamp(col('Date'),'MM/dd/yyyy hh:mm:ss a')).filter(col('Date') <= lit('2018-11-11'))\n",
        "rc.show(5)"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "YDoMDzk0kWLF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "6jBgmCFKkWCw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3UJoYm5zwOWp"
      },
      "source": [
        "## (03-03) Schemas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PeeOFy5cgDRq"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qLS_NEpEgDBJ"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E7NuMVpygC2h"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ugRDxNwNgHNa"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T6w6-fN9gG_7"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JkUF4yJFgG25"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o3IB6VxLgGYi"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mI_zYOVpf2yK"
      },
      "source": [
        "## (03-04) Working with columns"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WvOfJmzhf0kG"
      },
      "source": [
        "**Display only the first 5 rows of the column name IUCR**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "808f1HzwgDyh"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LZNgF-PFyJl0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_G1rHnJbyJl0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uP_fcMxGyJl0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A4u5vp73gEtB"
      },
      "source": [
        "  **Display only the first 4 rows of the column names Case Number, Date and Arrest**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vWhJLf41yJl1"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tV73d_TkyJl1"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sltirSelyJl1"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fEyRgddhyJl1"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GKnmLXJWr35c"
      },
      "source": [
        "**Add a column with name One, with entries all 1s**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_OhKvPoItNtK"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dSA_eLl6uH3p"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LsweBKZNuHzA"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BKV2ymA2uI2h"
      },
      "source": [
        "**Remove the column IUCR**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3wr2Q9djtXBV"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3HBJev2Ctdbo"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "O738q3tQyyRR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NiU1nhA3y0j4"
      },
      "source": [
        "## (03-05) Working with rows"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hgnCWSfNZsLV"
      },
      "source": [
        "**Add the reported crimes for an additional day, 12-Nov-2018, to our dataset.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "52EiVt15y0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lg5XEVXiy0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cZY70_Dzy0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0AWxi1tfy0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1fTTDzg-y0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PyPd5oppy0j4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rzaD3gLyy0j4"
      },
      "source": [
        "**What are the top 10 number of reported crimes by Primary type, in descending order of occurence?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vl3SHZdty0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CwuOpT8Uy0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lxr5psb0y0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "U7PBx7_7y0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EphdHjTCy0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nPD0MXeMy0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wpBN1d2uy0j5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9sDnyyzozVxA"
      },
      "source": [
        "## (03-06) Challenge"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OdV2UOUczVxA"
      },
      "source": [
        "**What percentage of reported crimes resulted in an arrest?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-3yQ3XA3zVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rxCxcHFUzVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_XguGvUFzVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7zazwi5jzVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6etuZFwczVxB"
      },
      "source": [
        "  **What are the top 3 locations for reported crimes?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "05kJeui5zVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tXar05-8zVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ScxfJKtMzVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xDukHOJkzVxB"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "2hvwNPI907EQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D7Dw2UpQ08-9"
      },
      "source": [
        "## (04-01) Built-in functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8b9jy_g708--"
      },
      "outputs": [],
      "source": [
        "from pyspark.sql import functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z7m60N3_08--"
      },
      "outputs": [],
      "source": [
        "print(dir(functions))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1yL-b-swunTM"
      },
      "source": [
        "### String functions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rHp9A7wyp1L4"
      },
      "source": [
        "**Display the Primary Type column in lower and upper characters, and the first 4 characters of the column**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4HNqGQKCqE9K"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BgwSt2ta08--"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZYDsLbbrf6dK"
      },
      "source": [
        "### Numeric functions\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H5CZwvAwhpCx"
      },
      "source": [
        "**Show the oldest date and the most recent date**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oP8NWAiUuSJC"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "juNQDv2108-_"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4BJvThINk7o-"
      },
      "source": [
        "### Date"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qjh1t1XamJ-E"
      },
      "source": [
        "**What is 3 days earlier that the oldest date and 3 days later than the most recent date?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NMFcA4rhlOs3"
      },
      "outputs": [],
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6_12orCSlXPe"
      },
      "outputs": [],
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b7gl5IbB08-_"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gC1spS8LhBtJ"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "vpb4-TQA13M6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiKFyArf14B0"
      },
      "source": [
        "## (04-02) Working with dates"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9npwvNTx1SCj"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T72w9V0mK0K2"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xdelci_zKzgX"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XPRbC6XnKzaY"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eGGXo0tVQBc4"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WkUMZnY5yc6q"
      },
      "source": [
        "  **2019-12-25 13:30:00**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vvFFnqmQ14B0"
      },
      "outputs": [],
      "source": [
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "87RJNjRty-W7"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YbSNtcMSQAiH"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r57RvNLFy1qr"
      },
      "source": [
        "**25/Dec/2019 13:30:00**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qWSM7QZIy1O7"
      },
      "outputs": [],
      "source": [
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_Bth-66Zy_na"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oX5JKpxiy_dT"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4DcjnoaKzBQr"
      },
      "source": [
        "**12/25/2019 01:30:00 PM**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WldJB29T80O2"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SqOg9qzQRAbi"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YupfXKVpRhg6"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "6SwZtjvL2Ugj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9lAyzg282V_4"
      },
      "source": [
        "## (04-03) Joins"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5klVM9jJZBDl"
      },
      "source": [
        "**Download police station data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c7CFhY3K2V_5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "R47fcwV82V_5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3-igLcB-wyWG"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Wz8D-NF2wyQb"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CdBqB9Oiw0gV"
      },
      "source": [
        "**The reported crimes dataset has only the district number. Add the district name by joining with the police station dataset**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S84o4-yZ2V_5"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GP90Bm-lc52-"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zbdoVA6Gc6mG"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V06UpJ4Tc7oJ"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S_YiRfSgc7em"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nWQMpWQec7SO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n4eBu-vMc6bN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8CR9XUeCc6Tn"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PxL3DKpPc6KG"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "WtFqwWp73ARQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-aQQ2gBF3Azz"
      },
      "source": [
        "## (04-05) Challenge questions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AopcTx5q3Az0"
      },
      "source": [
        "**What is the most frequently reported non-criminal activity?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PEw5vyF63Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "p1k8sxcY3Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cODrDxdb3Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Vf5DPbQ43Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5uC0WTu8KI59"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AHOdjsiBKI0W"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e5J1WQkBKIuN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Sx76PSOIKImG"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff7cLpWh3Az0"
      },
      "source": [
        "**Using a bar chart, plot which day of the week has the most number of reported crime.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mo8JV7083Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w0UWY8A13Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3OMXnCYr3Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kEbySVAC3Az0"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bSxKI4ePKJ5j"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j_iQqpL1KJ2d"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GbcSuGRpKJzq"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h95xRmYbKJxE"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "xRxHZgpG4d5p"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oSOpIOhJ4etN"
      },
      "source": [
        "## (05-01) RDDs setup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xcOe8kAS4etN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uxMRo2C14etN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eYDxom404etN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OT4nbSiJ4etN"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uiOTLr8N4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FWNY2gxZ4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nyAtd_Rl4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fAqt2hEX4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eG-5DmgE4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LpAhPr-0RW0E"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ShYgMK6jPsSt"
      },
      "source": [
        "**How many police stations are there?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2FbZPk84Pwxc"
      },
      "outputs": [],
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Iv-dVhhhPwrO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cQ1u4_cpGZqs"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8M0_0ZW1GZCO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WKYiypY0MbJ1"
      },
      "source": [
        "**Display the District ID, District name, Address and Zip for the police station with District ID 7**\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iHxLAHHq4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Z6dW5knty7pT"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6rMBWoqm4etO"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wHgN7nVW4etP"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1mWxQg2mN4W7"
      },
      "source": [
        "**Police stations 10 and 11 are geographically close to each other. Display the District ID, District name, address and zip code**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9DiIzEr-Gadn"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CcdG-eOk4etP"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KnghPtPt4etP"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Bvd3ldUs4etP"
      },
      "outputs": [],
      "source": []
    }
  ]
}