Notebook_1_DataCleansing_FeatureEngineering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook #1\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we show how to import, clean and create features relevent for predictive maintenance data using PySpark. This notebook uses Spark **2.0.2** and Python Python **2.7.5**. The API documentation for that version can be found [here](https://spark.apache.org/docs/2.0.2/api/python/index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import subprocess\n",
    "import sys\n",
    "import os\n",
    "import re\n",
    "import time\n",
    "import atexit\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from pyspark import SparkConf\n",
    "from pyspark import SparkContext\n",
    "from pyspark import SQLContext\n",
    "import pyspark.sql.functions as F\n",
    "from pyspark.sql.functions import concat, col, udf, lag, date_add, explode, lit, unix_timestamp\n",
    "from pyspark.sql.functions import month, weekofyear, dayofmonth\n",
    "from pyspark.sql.types import *\n",
    "from pyspark.sql.types import DateType\n",
    "from pyspark.sql.dataframe import *\n",
    "from pyspark.sql.window import Window\n",
    "from pyspark.sql import Row\n",
    "from pyspark.ml.classification import *\n",
    "from pyspark.ml.feature import StringIndexer, OneHotEncoder, VectorAssembler, VectorIndexer\n",
    "from pyspark.ml.feature import StandardScaler, PCA, RFormula\n",
    "\n",
    "start_time = time.time()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We initially encountered some issue while reading the data\n",
    "-  For Spark 2.0 and above, we can use the sqlContext.read.csv to import data directly into the Spark context. The data import seems to work fine and you can also perform some data transformation without any problem. However, when we tried to show the top n rows of the entire data (e.g. data.show(3)) or do some data manipulation on certain columns downstream, we encountered error of “Null Pointer” or “NumberFormatException: null”.\n",
    "-  In our case, it was because for some numeric columns with missing records containing \"\", Spark still recognizes those column as numeric but somehow cannot parse them correctly.\n",
    "-  We fixed the problem by reformating the data before loading into Spark context as csv format.\n",
    "```bash\n",
    " cat data.csv | tr -d \"\\\"\\\"\" > sampledata.csv \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input data is hosted on a publicly accessible Azure Blob Storage container and can be downloaded from [here](https://pysparksampledata.blob.core.windows.net/sampledata/sampledata.csv).\n",
    "\n",
    "To learn how to grant read-only access to Azure storage containers or blobs without sharing your account key and without requiring a shared access signature (SAS), please follow the instructions [here](https://docs.microsoft.com/en-us/azure/storage/storage-manage-access-to-resources).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Import data from Azure Blob Storage\n",
    "dataFile = \"wasb://sampledata@pysparksampledata.blob.core.windows.net/sampledata.csv\"\n",
    "dataFileSep = ','\n",
    "df = sqlContext.read.csv(dataFile, header=True, sep=dataFileSep, inferSchema=True, nanValue=\"\", mode='PERMISSIVE')\n",
    "\n",
    "# Import data from the home directory on your machine \n",
    "# dataFile = '/home/katlin/Desktop/PysparkExample/sampledata.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration & Cleansing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's look at the dataset dimension and data schema."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2085102, 172)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the dimensions of the data\n",
    "df.count(), len(df.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------------------+-------------+-------------+-------------+-------------+--------------+-------------+-------------+--------------+--------------+--------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------+\n",
      "|DeviceID|                Date|Categorical_1|Categorical_2|Categorical_3|Categorical_4|Problem_Type_1|Usage_Count_1|Usage_Count_2|Problem_Type_2|Problem_Type_3|Problem_Type_4|Warning_1.2|Warning_1.3|Warning_1.4|Warning_1.5|Warning_1.6|Warning_1.7|Warning_1.8|Warning_1.9|Warning_1.10|Warning_1.11|Warning_1.12|Warning_1.13|Warning_1.14|Warning_1.15|Warning_1.16|Warning_1.17|Warning_1.18|Warning_1.19|Warning_1.20|Warning_1.21|Warning_1.22|Warning_1.23|Warning_1.24|Warning_1.25|Warning_1.26|Warning_1.27|Warning_1.28|Warning_1.29|Warning_1.30|Warning_1.31|Warning_1.32|Warning_1.33|Warning_1.36|Warning_1.37|Warning_1.38|Warning_1.39|Warning_1.40|Warning_1.41|Warning_1.42|Warning_1.43|Warning_1.44|Warning_1.45|Warning_1.46|Warning_2.1|Warning_2.2|Warning_2.3|Warning_2.4|Warning_2.5|Warning_2.6|Warning_2.7|Warning_2.8|Warning_2.9|Warning_2.10|Warning_2.11|Warning_2.12|Warning_2.13|Warning_2.14|Warning_2.15|Warning_2.16|Warning_2.17|Warning_2.18|Warning_2.19|Warning_2.20|Warning_2.21|Warning_2.22|Warning_2.23|Warning_2.24|Warning_2.25|Warning_2.26|Warning_2.27|Warning_2.28|Warning_2.29|Warning_2.30|Warning_2.31|Warning_2.32|Warning_2.33|Warning_2.34|Warning_2.35|Warning_2.36|Warning_2.37|Warning_2.38|Warning_2.39|Warning_2.40|Warning_2.41|Warning_2.42|Warning_2.43|Warning_2.44|Warning_2.45|Warning_2.46|Warning_2.47|Warning_2.48|Warning_2.49|Warning_2.50|Warning_2.51|Warning_2.52|Warning_2.53|Warning_2.54|Warning_2.55|Warning_2.56|Warning_2.57|Warning_2.58|Warning_2.59|Warning_2.60|Warning_2.61|Warning_2.62|Warning_2.63|Warning_2.64|Warning_2.65|Warning_2.66|Warning_2.67|Warning_2.68|Warning_2.69|Warning_2.70|Warning_2.71|Warning_2.72|Warning_2.73|Warning_2.74|Warning_2.75|Warning_2.76|Warning_2.77|Warning_2.78|Warning_2.79|Warning_2.80|Warning_2.81|Warning_2.82|Warning_2.83|Warning_2.84|Warning_2.88|Warning_2.89|Warning_2.90|Warning_2.91|Warning_2.92|Warning_2.93|Warning_2.94|Warning_2.95|Warning_2.96|Warning_2.97|Warning_2.98|Warning_2.99|Warning_2.100|Warning_2.101|Warning_2.102|Warning_2.103|Warning_2.104|Warning_2.105|Warning_2.106|Warning_2.107|Error_Count_1|Error_Count_2|Error_Count_3|Error_Count_4|Error_Count_5|Error_Count_6|Error_Count_7|Error_Count_8|Fault_Code_Type_1|Fault_Code_Type_2|Fault_Code_Type_3|Fault_Code_Type_4|ProblemReported|\n",
      "+--------+--------------------+-------------+-------------+-------------+-------------+--------------+-------------+-------------+--------------+--------------+--------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------+\n",
      "|   N0001|2012-12-08 00:00:...|            8|         1500|          100|         7106|             0|            0|            0|             0|             0|             0|        -10|        -10|        -10|        -10|        -10|        -10|        -10|        -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|        -10|        -10|        -10|        -10|        -10|        -10|        -10|        -10|        -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|         -10|          -10|          -10|          -10|          -10|          -10|          -10|          -10|          -10|            0|            0|            0|            0|            0|            0|            0|            0|             null|             null|             null|             null|              0|\n",
      "+--------+--------------------+-------------+-------------+-------------+-------------+--------------+-------------+-------------+--------------+--------------+--------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+-----------------+-----------------+-----------------+-----------------+---------------+\n",
      "only showing top 1 row\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# check whether the issue of df.show() is fixed\n",
    "df.show(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('DeviceID', 'string'),\n",
       " ('Date', 'timestamp'),\n",
       " ('Categorical_1', 'int'),\n",
       " ('Categorical_2', 'int'),\n",
       " ('Categorical_3', 'int'),\n",
       " ('Categorical_4', 'int'),\n",
       " ('Problem_Type_1', 'int'),\n",
       " ('Usage_Count_1', 'int'),\n",
       " ('Usage_Count_2', 'int'),\n",
       " ('Problem_Type_2', 'int'),\n",
       " ('Problem_Type_3', 'int'),\n",
       " ('Problem_Type_4', 'int'),\n",
       " ('Warning_1.2', 'int'),\n",
       " ('Warning_1.3', 'int'),\n",
       " ('Warning_1.4', 'int'),\n",
       " ('Warning_1.5', 'int'),\n",
       " ('Warning_1.6', 'int'),\n",
       " ('Warning_1.7', 'int'),\n",
       " ('Warning_1.8', 'int'),\n",
       " ('Warning_1.9', 'int'),\n",
       " ('Warning_1.10', 'int'),\n",
       " ('Warning_1.11', 'int'),\n",
       " ('Warning_1.12', 'int'),\n",
       " ('Warning_1.13', 'int'),\n",
       " ('Warning_1.14', 'int'),\n",
       " ('Warning_1.15', 'int'),\n",
       " ('Warning_1.16', 'int'),\n",
       " ('Warning_1.17', 'int'),\n",
       " ('Warning_1.18', 'int'),\n",
       " ('Warning_1.19', 'int'),\n",
       " ('Warning_1.20', 'int'),\n",
       " ('Warning_1.21', 'int'),\n",
       " ('Warning_1.22', 'int'),\n",
       " ('Warning_1.23', 'int'),\n",
       " ('Warning_1.24', 'int'),\n",
       " ('Warning_1.25', 'int'),\n",
       " ('Warning_1.26', 'int'),\n",
       " ('Warning_1.27', 'int'),\n",
       " ('Warning_1.28', 'int'),\n",
       " ('Warning_1.29', 'int'),\n",
       " ('Warning_1.30', 'int'),\n",
       " ('Warning_1.31', 'int'),\n",
       " ('Warning_1.32', 'int'),\n",
       " ('Warning_1.33', 'int'),\n",
       " ('Warning_1.36', 'int'),\n",
       " ('Warning_1.37', 'int'),\n",
       " ('Warning_1.38', 'int'),\n",
       " ('Warning_1.39', 'int'),\n",
       " ('Warning_1.40', 'int'),\n",
       " ('Warning_1.41', 'int'),\n",
       " ('Warning_1.42', 'int'),\n",
       " ('Warning_1.43', 'int'),\n",
       " ('Warning_1.44', 'int'),\n",
       " ('Warning_1.45', 'int'),\n",
       " ('Warning_1.46', 'int'),\n",
       " ('Warning_2.1', 'int'),\n",
       " ('Warning_2.2', 'int'),\n",
       " ('Warning_2.3', 'int'),\n",
       " ('Warning_2.4', 'int'),\n",
       " ('Warning_2.5', 'int'),\n",
       " ('Warning_2.6', 'int'),\n",
       " ('Warning_2.7', 'int'),\n",
       " ('Warning_2.8', 'int'),\n",
       " ('Warning_2.9', 'int'),\n",
       " ('Warning_2.10', 'int'),\n",
       " ('Warning_2.11', 'int'),\n",
       " ('Warning_2.12', 'int'),\n",
       " ('Warning_2.13', 'int'),\n",
       " ('Warning_2.14', 'int'),\n",
       " ('Warning_2.15', 'int'),\n",
       " ('Warning_2.16', 'int'),\n",
       " ('Warning_2.17', 'int'),\n",
       " ('Warning_2.18', 'int'),\n",
       " ('Warning_2.19', 'int'),\n",
       " ('Warning_2.20', 'int'),\n",
       " ('Warning_2.21', 'int'),\n",
       " ('Warning_2.22', 'int'),\n",
       " ('Warning_2.23', 'int'),\n",
       " ('Warning_2.24', 'int'),\n",
       " ('Warning_2.25', 'int'),\n",
       " ('Warning_2.26', 'int'),\n",
       " ('Warning_2.27', 'int'),\n",
       " ('Warning_2.28', 'int'),\n",
       " ('Warning_2.29', 'int'),\n",
       " ('Warning_2.30', 'int'),\n",
       " ('Warning_2.31', 'int'),\n",
       " ('Warning_2.32', 'int'),\n",
       " ('Warning_2.33', 'int'),\n",
       " ('Warning_2.34', 'int'),\n",
       " ('Warning_2.35', 'int'),\n",
       " ('Warning_2.36', 'int'),\n",
       " ('Warning_2.37', 'int'),\n",
       " ('Warning_2.38', 'int'),\n",
       " ('Warning_2.39', 'int'),\n",
       " ('Warning_2.40', 'int'),\n",
       " ('Warning_2.41', 'int'),\n",
       " ('Warning_2.42', 'int'),\n",
       " ('Warning_2.43', 'int'),\n",
       " ('Warning_2.44', 'int'),\n",
       " ('Warning_2.45', 'int'),\n",
       " ('Warning_2.46', 'int'),\n",
       " ('Warning_2.47', 'int'),\n",
       " ('Warning_2.48', 'int'),\n",
       " ('Warning_2.49', 'int'),\n",
       " ('Warning_2.50', 'int'),\n",
       " ('Warning_2.51', 'int'),\n",
       " ('Warning_2.52', 'int'),\n",
       " ('Warning_2.53', 'int'),\n",
       " ('Warning_2.54', 'int'),\n",
       " ('Warning_2.55', 'int'),\n",
       " ('Warning_2.56', 'int'),\n",
       " ('Warning_2.57', 'int'),\n",
       " ('Warning_2.58', 'int'),\n",
       " ('Warning_2.59', 'int'),\n",
       " ('Warning_2.60', 'int'),\n",
       " ('Warning_2.61', 'int'),\n",
       " ('Warning_2.62', 'int'),\n",
       " ('Warning_2.63', 'int'),\n",
       " ('Warning_2.64', 'int'),\n",
       " ('Warning_2.65', 'int'),\n",
       " ('Warning_2.66', 'int'),\n",
       " ('Warning_2.67', 'int'),\n",
       " ('Warning_2.68', 'int'),\n",
       " ('Warning_2.69', 'int'),\n",
       " ('Warning_2.70', 'int'),\n",
       " ('Warning_2.71', 'int'),\n",
       " ('Warning_2.72', 'int'),\n",
       " ('Warning_2.73', 'int'),\n",
       " ('Warning_2.74', 'int'),\n",
       " ('Warning_2.75', 'int'),\n",
       " ('Warning_2.76', 'int'),\n",
       " ('Warning_2.77', 'int'),\n",
       " ('Warning_2.78', 'int'),\n",
       " ('Warning_2.79', 'int'),\n",
       " ('Warning_2.80', 'int'),\n",
       " ('Warning_2.81', 'int'),\n",
       " ('Warning_2.82', 'int'),\n",
       " ('Warning_2.83', 'int'),\n",
       " ('Warning_2.84', 'int'),\n",
       " ('Warning_2.88', 'int'),\n",
       " ('Warning_2.89', 'int'),\n",
       " ('Warning_2.90', 'int'),\n",
       " ('Warning_2.91', 'int'),\n",
       " ('Warning_2.92', 'int'),\n",
       " ('Warning_2.93', 'int'),\n",
       " ('Warning_2.94', 'int'),\n",
       " ('Warning_2.95', 'int'),\n",
       " ('Warning_2.96', 'int'),\n",
       " ('Warning_2.97', 'int'),\n",
       " ('Warning_2.98', 'int'),\n",
       " ('Warning_2.99', 'int'),\n",
       " ('Warning_2.100', 'int'),\n",
       " ('Warning_2.101', 'int'),\n",
       " ('Warning_2.102', 'int'),\n",
       " ('Warning_2.103', 'int'),\n",
       " ('Warning_2.104', 'int'),\n",
       " ('Warning_2.105', 'int'),\n",
       " ('Warning_2.106', 'int'),\n",
       " ('Warning_2.107', 'int'),\n",
       " ('Error_Count_1', 'int'),\n",
       " ('Error_Count_2', 'int'),\n",
       " ('Error_Count_3', 'int'),\n",
       " ('Error_Count_4', 'int'),\n",
       " ('Error_Count_5', 'int'),\n",
       " ('Error_Count_6', 'int'),\n",
       " ('Error_Count_7', 'int'),\n",
       " ('Error_Count_8', 'int'),\n",
       " ('Fault_Code_Type_1', 'string'),\n",
       " ('Fault_Code_Type_2', 'string'),\n",
       " ('Fault_Code_Type_3', 'string'),\n",
       " ('Fault_Code_Type_4', 'string'),\n",
       " ('ProblemReported', 'int')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check data schema\n",
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explanations on the data schema:\n",
    "* ***DeviceID***: machine identifier\n",
    "* ***Date***: the day when that row of data was collected for that machine\n",
    "* ***Categorical_1 to 4***: some categorical features about the machine\n",
    "* ***Problem_Type_1 to 4***: the total number of times Problem type 1 (2, 3, 4) occured on that day for that machine\n",
    "* ***Usage_Count_1 (2)***: the total number of times that machine had been used on that day for purpose type 1 or 2\n",
    "* ***Warning_xxx***: the total number of Warning type_xxx occured for that machine on that day\n",
    "* ***Error_Count_1 to 8***: the total number of times Error type 1 (to 8) occured on that day for that machine\n",
    "* ***Fault_Code_Type_1 to 4***: fault code type 1 (2, 3, 4) occured on that day for that machine\n",
    "* ***Problemreported***: prediction target column whether or not there is a machine problem on that day"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### As part of the data cleansing process, we standardized all the column names to lower case and replaced all the symbols with underscore. We also removed any duplicated records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+--------------------+\n",
      "|deviceid|                date|\n",
      "+--------+--------------------+\n",
      "|   N0001|2013-05-02 00:00:...|\n",
      "|   N0001|2014-04-27 00:00:...|\n",
      "|   N0001|2014-09-17 00:00:...|\n",
      "+--------+--------------------+\n",
      "only showing top 3 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#--------------------------------------- initial data cleansing ---------------------------------------------#\n",
    "\n",
    "# standardize the column names\n",
    "def StandardizeNames(df):\n",
    "    l = df.columns\n",
    "    cols = [c.replace(' ','_').\n",
    "              replace('[.]','_').\n",
    "              replace('.','_').\n",
    "              replace('[[:punct:]]','_').\n",
    "              lower() for c in l]\n",
    "    return df.toDF(*cols)\n",
    "df = StandardizeNames(df)\n",
    "\n",
    "# remove duplicated rows based on deviceid and date\n",
    "df = df.dropDuplicates(['deviceid', 'date'])\n",
    "\n",
    "# remove rows with missing deviceid, date\n",
    "df = df.dropna(how='any', subset=['deviceid', 'date'])\n",
    "\n",
    "df.select('deviceid','date').show(3)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Define groups of features -- date, categorical, numeric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "#------------------------------------------- Define groups of features -----------------------------------------#\n",
    "\n",
    "features_datetime = ['date']\n",
    "\n",
    "features_categorical = ['deviceid','Categorical_1','Categorical_2','Categorical_3','Categorical_4',\n",
    "                        'fault_code_type_1','fault_code_type_2',\n",
    "                        'fault_code_type_3','fault_code_type_4',\n",
    "                        'problemreported']\n",
    "\n",
    "features_numeric = list(set(df.columns) -set(features_datetime)-set(features_categorical))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Handling missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Row(fault_code_type_3=None), Row(fault_code_type_3=None), Row(fault_code_type_3=None)]\n"
     ]
    }
   ],
   "source": [
    "print(df['fault_code_type_3',].head(3))\n",
    "# there are some missing values, we need to handle in the subsequent steps\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Row(fault_code_type_3=u'Unknown'), Row(fault_code_type_3=u'Unknown'), Row(fault_code_type_3=u'Unknown')]\n"
     ]
    }
   ],
   "source": [
    "# handle missing values\n",
    "df = df.fillna(0, subset=features_numeric)\n",
    "df = df.fillna(\"Unknown\", subset=features_categorical)\n",
    "\n",
    "# check the results\n",
    "print(df['fault_code_type_3',].head(3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### For data exploration part, people usually would like to visualize the distribution of certain columns or the interation among columns. Here, we hand picked some columns to demonstrate how to do some basic visualizations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEVCAYAAAAy15htAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuclVX5///XcCpBQsAhkPwopr3NMC3yFPgNVEoU85ua\nmudTJmZJpWZlnkrpqz9NzTTPmmUfLbM84CFUAi0JK0VFrlJSSzQmBAQ1TjO/P9bauhnmtJm594zw\nfj4e82Dvtde97uu+Z9jXvtda+141DQ0NmJmZFaFbZwdgZmbrLicZMzMrjJOMmZkVxknGzMwK4yRj\nZmaFcZIxM7PCOMnYOkPSC5JGNVE+WtJzVYrhi9XYT6N91kg6VdKKpo6/mW0+IKkhPz5J0vdaqb+T\npI8289rnJF2fH0+VdFiF8b9H0hH58VBJT1eyvXVtPTo7ALN1haTBwGnANVXe9ZVAd2D+2mwcEZe3\nodrRwCPArCa2vwO4Y232nX0MOAL4aUS8DAxvR1vWxTjJWKeTNBq4DPgdMB7oBXwhIh6TdDYwFNgO\nuCXX+x6wf978MeDLEfFGfr6bpB8BGwM3RcQZjfb1HuBCYM+8n6sj4vz82gvARaQ31KHABGD3XLcO\nGBcRC1s4lD8AH5A0B7gX6BkRJ+W2+wMvA/+T2zoZOAbYBDgzIn6S6x0PfB14L/BH4JiIeKuVU3hT\nRPwxx98sSccAZwGvAz8vKz8b+EBEHCfp87lOd2AF8FVga1IS+KykQcBrwGeBfsCfgdnAYRGxR25y\nW0l/AoYA9wEnAJsCz0VEj7zPzYHnSOf5DuB9kqYDh5fqSepGM79rSVOBO4H9gGHANOCQiPC3y7sY\nd5dZV7EN8KeIEHAe6dN5yV7AXhFxCXAgMA4YAXwE2Aj4WlndEcAn8r8nStqu0X5Oy/vaNm9/gKTx\nZa8Pj4iPk97cbgZ+CWxJ+r+yXyvHcAzwUkRsnbf9vKTSB7nxwLSI+E9+vlVEbA/sClwiaaCkXfN+\nd4uIzYHF+XmLIuKPrdXJSe4yYM+I2JaU3JpyBbB3RHwYOBH4bE6AfwJOi4iLc71PAydExGlNtDEG\nGA0I+BTp2JuL/d/At4A/RsSujV5u7Xe9DzAW+BCwG/DJ5vZjncdJxrqKpcBt+fHtwPaSeufnM8re\nnPcmfXJ/IyJWATeQ3vBKfh4RqyJiPvB7YJdG+9kHuCIiluWrn5+yevL4Tf73KeCtiJiaPx0/Q/Nv\nzGuIiL8Ai0hXQgCfA24tq3J9rhdAADvm2G6NiHm5zk9oPbG11U7A3yPi2fz8pmbqzQdOkLRZRDwS\nEV9vpt7fIuLvzbz2q4h4MyLeBO5hzd9BW7X2u/5VRLyVf49/I10lWhfj7jLrKhaWdXUsyv9ulP99\nraxeLVDeZbUQGFT2vK7s8WKgf6P9bAT8UNL5+fl7SJ/SS5bkf1eREh9lz7u3cgyN/QI4RNI00if7\nY8peKz+mhTnOjYDPSSq9kXYjdel1hAGk81G+z6Z8FjgD+LOkfwITI+L3TdR7rYmyksa/gyGVBFqm\ntd91+fGsze/HqsBJxrqKgWWPS4mhqTeyfzeqOzCXlQxo1E7jNuYB/19E3L2WcVbiF8AM0vjMoxGx\nqOy1jYEX8+MBpDjnkT65n1JALAtJYygltU1ViojngaPzeMgRpHGwoRXuq6nfwSqgm6Sa/GGicfJv\nSmu/a3sXcHeZdRW9Jf3f/PgA4PGI+G8T9e4GDpPUO493HEvqkik5WFK3PEC9KzC90fa/BY6T1D1P\n/T1D0p4ddAwrgA1L4zC5K+x54Aes3lUG8AUASR8GtiIlozuB/STV5tf2lfTNDort8dSktsrPj2xc\nQVKtpN9Jel9E1JMG2ktXlyt458qyNftJeq+kPqQxlenAf0iJZttc54iy+itIA/81jdpp7Xdt7wJO\nMtZVvACMkvQ34NukQeem/AqYTJrV9DTwT9KAdslMUvfX48API2J2o+1/TLqCeAaYA3yYNDW3I8wi\nfWp/VVJpfOAXwPtJya3cfElPkGZFfTUiFuZxnPOBqZKeJc0ya7zdGiQ9nWe0DQV+LmmOpB3L60RE\nHfANYEr+Hko0bifXuQ+YKWk28L+kN3ZIM8D+n6SLG2/XhCnAw8Cz+fF9eYbcWcB9kh4Hniir/whp\nvGseq3d5tfa7tneBGq8nY50tT2G+NiK27OxYOpqkA4EDIuLAsrIGYNOI+FfnRWZWHb6SMStInh33\nTfzp29ZjHvg3q0D+kuH7mnl5h4hYkuuNJ33n5PqIaFd3nKTLgT2aefnLEfFge9o3K5K7y8zMrDDu\nLjMzs8K4u6yRurola31p179/bxYufLMjw+kQjqsyjqsyjqsy62JctbV9G08/f5uvZDpQjx5d8wvH\njqsyjqsyjqsy61tcTjJmZlYYJxkzMyuMk4yZmRXGScbMzArjJGNmZoVxkjEzs8I4yZiZWWGcZMzM\nrDD+xr+ZdTmDBvXtoJZSO/PnL2mlXuVeeWUeZ5zxTa677ubVys8772xGj96dkSN37dD9PfHEX9hs\ns83p339A65XX0r///Srf/vapfOxjIzjppIkd0qaTTAeqqYHSH3XXU0xcRfznNbM13XPPnXzhC4cV\nmmQmTTqXESN2oL6+vsPadJIxs/Xe5Ml3MWPGH3jjjTeoq5vPgQcews0338DOO4+kf//+jBs3nkmT\nzmXFihV069aN00//LjU1NaxcuZJzz/0u//zni2y1lTjttO+83eaqVau44ILzmDfvZVauXMlxx53A\niBE7cPjhhzN8+PbMnDmDbt26MW7c3kyefDfdunXj0kuvpHv3NW/vMnPmY0yfPpV//GMun/zkKFat\nWsUXvzgBgIkTT+Skk77GKad8ldGjd+PZZ2dTW1vLWWedx4oVyzn//HNYsmQJq1atYuLEU9lyy63W\naL/k/PMvZOrUh5g79/kOO7cekzEzA/7xj7n84AcXc+mlP+Gaa65k+fLl7LzzJznyyGO59tqfMH78\nvlx++dV87nMHcP31VwPwwgtz+dKXvszVV9/E3/4WPP/8c2+397vf3cfAgRvzox9dxaRJF3HZZRe9\n/drAgRtz5ZXXUV+/itdff50rrriW+vp65s59bo24AHbYYWe23PJDfPvbZ3LAAQcxffpUAJYuXcrr\nry9myy234j//qWOPPfbkqqtuoKGhgccee5TbbvsFO+30SS699Eq+8Y3TufzyH7Z4Dnr37tPOs7gm\nX8mYmQHbb/9xevTowUYbbUTfvn2ZN+9lttnmIwBEPMsJJ5wEwMc//gluvPFaAD7wgU15//sHA7D1\n1tvw0ksvvN3e00/P4skn/8qsWU8AsGzZMlasWAHwdrsDB27MVlsJgAEDBrB06dJW43zf+/rxgQ/8\nDxFzeOmlFxgzJq1nt8EGGzB8+LYAfOQjH+Wll17kqadmsWjRQu6/f3KO4b9rf4LWkpOMmRlQX//O\nKh8NDVBTU0OPHj1zSQ2lBR5XrFhJTU3qBKqpWf0O9+XPe/ToyRFHHMPYsXuusa/yLrHyx21dRHLP\nPffm4Yen8Oqrr/ClL305x18+jtJATU0NPXv24GtfO5Xhwz/apnaLUGiSkXQBsGvezyRgJnAz0B14\nBTg8IpZJOhSYCNQDV0fEdZJ6AjcCmwGrgKMjYq6k7YArgQZgVkRMyPs6Ffh8Lj8nIiZL6gfcAvQD\nlgKHRMRrRR6zmb07PfPMLFatWsWSJUt48803eN/7+r392oc/vA1/+cvjjB27J0888We23vrDALz8\n8r/4z3/+w4ABA5gzZzb7738gjz46HYBtthnOI4/8nrFj92Thwte47bZfvJ0Q1ka3bt1YtWoVALvs\nMpJf/OKn9OmzIUOGbAKkK6U5c55l660/zNNPP8X48fuycuVKpk2byvDhH+Uf/5jLjBl/4OCDD1vr\nGNZGYUlG0hhgeETsImkg8FfgQeDHEfFLSecDx0j6KXAmsCOwHJgp6Q5gH2BRRBwq6dOkJHUQcAlw\nckTMlHSLpHHAHOBgYBdSQpku6X5S4poaERdKOh74Zv4xsy6sI2Yt1tb2pa6u7e0MHrwJ3/3u6bz8\n8j85/vgTufban7z92nHHncCkSd/jrrt+Q48ePfnWt77LypUr2XLLrbj66h/zj3/MZfjwbRk2bIu3\nt9lttz34y19mcsIJx7Bq1SqOOeb4dh3P9tt/nDPO+CaTJl3EFlt8kM02G4b04bdf79evHw88MJnL\nLruIgQM3Zscdd2b77T/GeeedzYknHkd9fT0TJ57SbPv//ve/+epXJ/Laawv473//y5w5s/nGN05f\n7ZjWRk1bL88qJak78N6IeCM/ng+8Dmydr152AU4BfgwcExGH5e2uAu4mXZX8NCKmSOoGvARsAfwt\nIjbPdb8AfAJ4BtgiIs7I5fc3avs5SUOAuyNiREtxt2dlzI6b2//u0Z43g0rfBKrFcVVmXYhr8uS7\nmDv3+Q77bkhLOuJ8LVu2jC9/+YtccskVbLjhhgDsvffu3HPPg50SV0srYxZ2JRMRq4A38tNjgcnA\nZyJiWS6bDwwBBgN1ZZuuUR4R9ZIactnCJuouaK2NsrIW9e/fu8uuXNcV1da2L7G2d/uiOK7KvNvj\n6tv3vfTu3atqx9HcfmbNmsWFF164Rvm4ceM45JBDAHjiiSc488wzOfbYYxk27J23tJqamjbFf/nl\nlzNjxow1ys8//3w23XTTth5CmxU+8C9pX1KS+TTw97KXmst8lZR3RN3VtG/t7a75H61I7flEti58\nAq4mx1WZSuLaddex7Lrr2KocR0txDRkyjIsvvqLJ10rbDB36Qa677uerlQHcffeUNsV/0EFHctBB\nR1YUV2taSm6Ffk9G0meA7wDjImIxsFTSBvnlocC8/DO4bLM1yvMkgBrSZIGBLdVtobxUZmZmVVJY\nkskzuy4ExpfN6JoC7J8f7w/cB8wAdpC0kaQNgZHAdOAB0rgMpEkAD0fECmCOpFG5fL/cxkPA3pJ6\nSdqElFBmN2qjtD8zM6uSIrvLDgI2Bm6TVCo7ErhW0peAF4GbImKFpNOB+3ln+vFiSbcCYyU9AiwD\njsptTASuypMBZkTEFABJ1wDTchsT8jjOZcDPJE0HFgHVnbtnZraeK2x22buVZ5dVxrPLqsdxVcZx\nVaao2WW+d5mZmRXGScbMzArjJGNmZoVxkjEzs8I4yZiZWWGcZMzMrDBOMmZmVhgnGTMzK4yTjJmZ\nFcZJxszMCuMkY2ZmhXGSMTOzwjjJmJlZYZxkzMysME4yZmZWGCcZMzMrTJErYyJpOPBb4IcRcbmk\nXwK1+eUBwGMRcbykFcCjZZvuTkqANwKbAauAoyNirqTtgCtJK2DOiogJeV+nkpZaLq2uOTkvAX0L\n0A9YChxSthS0mZkVrLAkI6kP8CPgwVJZRHy+7PXrgWvz08URMbrR9ocBiyLiUEmfBiaRlnS+BDg5\nImZKukXSOGAOcDCwCymhTJd0P2mp5qkRcaGk44Fv5h8zM6uCIrvLlgF7AfMavyBJwEYR8acWtt8d\nuCM/ngKMlNQLGBYRM3P5XcAewBjg3ohYHhF1wIvANo3aKNU1M7MqKexKJiJWAitTPlnDyaSrnJL3\nSrqF1DV2e0RcDAwG6nJb9ZIactnCsu3mA0OABaW6jcoHl5WXylrUv39vevTo3urxWVJb27dTty+K\n46qM46rM+hRXoWMyTclXI6Mi4sSy4lOAn5HGU6ZJmtbEpjVtLKu07moWLnyzLdWa0TX/cIpUV7dk\nrbetre3bru2L4rgq47gqsy7G1VJyqnqSAT4FrNZNFhE/KT2W9CCwLambbTDwpKSepCTxCjCwbNOh\nud48QM2UDwYWl5WZmVmVdMYU5h2AJ0tPlNwiqUZSD2Ak8AzwAGm2GMA+wMMRsQKYI2lULt8PuA94\nCNhbUi9Jm5ASyuxGbeyf65qZWZUUObtsBHARsDmwQtIBpKQwBHi+VC8iQtI/SVc39cCdEfEnSX8G\nxkp6hDSJ4Ki8yUTgKkndgBkRMSXv7xpgGqnLbUIex7kM+Jmk6cAi4LCijtfMzNZU09DQ0NkxdCl1\ndUvW+oQMGrT+jcnMn+8xmWpxXJVxXJVp55hMs2Pe/sa/mZkVxknGzMwK4yRjZmaFcZIxM7PCOMmY\nmVlhnGTMzKwwTjJmZlYYJxkzMyuMk4yZmRXGScbMzArjJGNmZoVxkjEzs8I4yZiZWWGcZMzMrDBO\nMmZmVhgnGTMzK0xhK2MCSBoO/Bb4YURcLulGYASwIFe5MCLukXQoacXLeuDqiLhOUk/gRmAzYBVw\ndETMlbQdcCVpBcxZETEh7+tU0lLLDcA5ETFZUj/gFqAfsBQ4JCJeK/KYzczsHYVdyUjqA/wIeLDR\nS9+KiNH5555c70xgD2A08DVJA4BDgEURMQo4D5iUt78EODkiRgL9JI2TNAw4GBgFjAcultSdlLim\n5jZ+DXyzqOM1M7M1FdldtgzYC5jXSr2dgJkRsTgi3gIeBUYCuwN35DpTgJGSegHDImJmLr+LlJzG\nAPdGxPKIqANeBLZp1EaprpmZVUlh3WURsRJYKanxSydJ+jowHzgJGAzUlb0+HxhSXh4R9ZIactnC\nJuouaK2NsrIW9e/fmx49urfhCA3SuuCduX1RHFdlHFdl1qe4Ch2TacLNwIKIeELS6cDZwB8a1alp\nZtumyjui7moWLnyzLdWa0TX/cIpUV7dkrbetre3bru2L4rgq47gqsy7G1VJyqurssoh4MCKeyE/v\nBLYldacNLqs2NJe9XZ4nAdQArwADW6rbQnmpzMzMqqSqSUbS7ZK2yE9HA08DM4AdJG0kaUPSeMx0\n4AHSbDGAfYCHI2IFMEfSqFy+H3Af8BCwt6RekjYhJZTZjdrYP9c1M7MqKay7TNII4CJgc2CFpANI\ns81ulfQmaUrx0RHxVu46u593ph8vlnQrMFbSI6RJBEflpicCV0nqBsyIiCl5f9cA03IbE/I4zmXA\nzyRNBxYBhxV1vGZmtqaahoaGzo6hS6mrW7LWJ2TQoPVvTGb+fI/JVIvjqozjqkw7x2SaHfP2N/7N\nzKwwTjJmZlYYJxkzMyuMk4yZmRXGScbMzArjJGNmZoVxkjEzs8I4yZiZWWGcZMzMrDBOMmZmVhgn\nGTMzK4yTjJmZFcZJxszMCuMkY2ZmhXGSMTOzwhS2aBmApOHAb4EfRsTlkjYFbgB6AiuAwyLiVUkr\ngEfLNt2dlABvBDYDVpEWOJsraTvgStLiZLMiYkLe16mkVTBLC59NltQPuAXoR1ok7ZCIeK3IYzYz\ns3cUdiUjqQ9pJcwHy4q/D1wdEZ8C7gC+nssXR8Tosp9VwCHAoogYBZwHTMp1LwFOjoiRQD9J4yQN\nAw4GRgHjgYsldSetojk1t/Fr4JtFHa+Zma2pyO6yZcBewLyyshOB2/PjOmBgC9vvTkpEAFOAkZJ6\nAcMiYmYuvwvYAxgD3BsRyyOiDngR2KZRG6W6ZmZWJYV1l0XESmClpPKyNwDyVcaXgXPzS++VdAup\na+z2iLgYGExKREREvaSGXLawbDfzgSHAglLdRuWDy8pLZWZmViVtSjKSegLvj4h/SfoosB0pGbxZ\n6Q5zgrkZeCgiSl1ppwA/I42nTJM0rYlNm1pDurl1pSupu5r+/XvTo0f3tlQ10rrgnbl9URxXZRxX\nZdanuNp6JXMT8GtJfySNbfwa2Ac4cC32eQPw94g4p1QQET8pPZb0ILAtqZttMPBkTnI1wCus3sU2\nNNebB6iZ8sHA4rKyFi1cWHHeLNM1/3CKVFe3ZK23ra3t267ti+K4KuO4KrMuxtVScmrrmMzQiPgV\ncBBwRUScBgyoNBBJhwLLI+KssjJJukVSjaQewEjgGeAB0mwxSAnt4YhYAcyRNCqX7wfcBzwE7C2p\nl6RNSAlldqM29s91zcysStp6JfMeSTXA54Bjc1mLH9sljQAuAjYHVkg6ABgE/FfS1FxtdkScKOmf\nwJ+AeuDOiPiTpD8DYyU9QppEcFTeZiJwlaRuwIyImJL3dw0wjdTlNiGP41wG/EzSdGARcFgbj9fM\nzDpAW5PMVFKX030R8TdJE4E5LW0QEX8GRrel8YhYY2pxnsZ8dBPls4Fdmyj/EWnKdHnZUuD/tiUG\nMzPreG1NMjcCP4iIRfn5b4HHC4nIzMzWGS0mGUkbkQbabwAOkVQah+kJXA98qNjwzMzs3ay1K5ld\ngK8B25MG10vqgfuLCsrMzNYNLSaZiLgXuFfSCeXTjM3MzNqirWMyv5F0Mmna8ttfaoyIMwuJyszM\n1glt/Z7MPaRv+deT7ohc+jEzM2tWW69klkbEMYVGYmZm65y2Xsk8JmnrQiMxM7N1TluvZPYEvi6p\nDlhJGpdpiIj/KSwyMzN712trkvlsoVGYmdk6qa1JZvdmyq/vqEDMzGzd09YkU36vsF7ATsCjOMmY\nmVkL2pRkImK1G1VK6k261YyZmVmz2jq7bDV5RcwtOzgWMzNbx7R1+eXppHVaSoYCswqJyMzM1hlt\nHZM5o+xxA/A68GTHh2NmZuuSto7J/F7SrsAOpCTzWEQ0tLIZkoaT1p75YURcLmlT4GagO/AKcHhE\nLMvLMk8k3bbm6oi4TlJP0jo2m5FuYXN0RMyVtB1wZY5jVkRMyPs6lbTUcgNwTkRMltQPuAXoBywF\nDomI19p0ZszMrN3aNCYj6VzgQmAIqavsMknfamWbPqSVKh8sKz4X+HFE7Ao8BxyT650J7EFaSfNr\ned2aQ4BFETEKOA+YlNu4BDg5IkYC/SSNkzQMOBgYBYwHLpbUnZS4puY2fg2ssQKnmZkVp60D/2OA\nT0bEqRFxCmmdmX1a2WYZsBcwr6xsNHBnfnwXKbHsBMyMiMUR8RZpavRI0ndz7sh1pwAjJfUChkXE\nzEZtjAHujYjlEVEHvAhs06iNUl0zM6uSto7JdIuI+tKTiFgpqb6lDSJiJbBSUnlxn4hYlh/PJ10Z\nDQbqyuqsUR4R9ZIactnCJuouaK2NsrIW9e/fmx49urdWzbLa2r6dun1RHFdlHFdl1qe42ppk/izp\nTtIVBcBY4PF27rumA8o7ou5qFi58sy3VmtE1/3CKVFe3ZK23ra3t267ti+K4KuO4KrMuxtVScmq1\nuyyPd0wkDaAPAzYHpkXExLWIZamkDfLjoaSutHmkKw6aK8+TAGpIkwUGtlS3hfJSmZmZVUmLSUbS\n7qQxkr4R8b8R8TXSN/0nSBqxFvubAuyfH+8P3AfMAHaQtJGkDUnjMdOBB0izxSCN/zwcESuAOZJG\n5fL9chsPAXtL6iVpE1JCmd2ojdL+zMysSlq7kjkL+HRELC4VRMRTpDf977e0oaQRkqYCRwEn58fn\nAEfmL3cOAG7Kg/2nA/eTktA5eX+3At0lPQJ8GSjNZpsITJL0KPB8REyJiJeAa4BpwO3AhDyGdBnw\niby/MaQZcmZmViU1DQ3Nf91F0vQ83bip1x6OiDGFRdZJ6uqWtPr9n+YMGrT+jcnMn+8xmWpxXJVx\nXJVp55hMs2PerV3JbNjCawNbeM3MzKzVJPO0pBMaF0o6jTSWYmZm1qzWpjCfCvxG0hHATNLtYEaS\n7l22d8GxmZnZu1yLSSYiXgV2zrPMPkK6h9htETGtGsGZmdm7W1tvkPkgq9+DzMzMrFVrtWiZmZlZ\nWzjJmJlZYZxkzMysME4yZmZWGCcZMzMrjJOMmZkVxknGzMwK4yRjZmaFcZIxM7PCOMmYmVlhnGTM\nzKwwbbp3WUeRdCxweFnRJ4BfASOABbnswoi4R9KhpFUw64GrI+I6ST2BG4HNSDfrPDoi5kraDrgS\naABmRcSEvL9TScsvN5BW3Jxc9DGamdk7qppkIuI64DoASZ8CDgT6AN+KiLtL9ST1Ac4EdgSWAzMl\n3UFa9nlRRBwq6dPAJOAg4BLg5IiYKekWSeOAOcDBwC5AP2C6pPsjYlWVDtfMbL3Xmd1lZwLfa+a1\nnYCZEbE4It4CHiWtY7M7cEeuMwUYKakXMCwiZubyu4A9gDHAvRGxPCLqgBeBbYo5FDMza0pVr2RK\nJO0A/DMiXpUEcJKkrwPzgZOAwUBd2SbzgSHl5RFRL6khly1sou6CZtp4qqXY+vfvTY8e3df+4NYz\ntbV9O3X7ojiuyjiuyqxPcXVKkgGOI42tANwMLIiIJySdDpwN/KFR/Zpm2mmqvJK6a1i48M22VGtG\n1/zDKVJd3ZK13ra2tm+7ti+K46qM46rMuhhXS8mps7rLRpMTSUQ8GBFP5PI7gW2BeaQrlJKhuezt\n8jwJoAZ4BRjYUt1G5WZmViVVTzKSNgGWRsTy/Px2SVvkl0cDTwMzgB0kbSRpQ9J4zHTgAdJsMUiT\nAB6OiBXAHEmjcvl+wH3AQ8DeknrlfQ4FZhd+gGZm9rbO6C4bQhofKbkcuFXSm8BS0rTkt3LX2f28\nM/14saRbgbGSHgGWAUflNiYCV0nqBsyIiCkAkq4BpuU2JkREffGHZ2ZmJTUNDQ2dHUOXUle3ZK1P\nyKBB69+YzPz5HpOpFsdVGcdVmXaOyTQ75u1v/JuZWWGcZMzMrDBOMmZmVhgnGTMzK4yTjJmZFcZJ\nxszMCuMkY2ZmhXGSMTOzwjjJmJlZYZxkzMysME4yZmZWGCcZMzMrjJOMmZkVxknGzMwK4yRjZmaF\ncZIxM7PCVHVlTEmjgV8Cz+Sip4ALgJuB7sArwOERsUzSoaQVL+uBqyPiOkk9gRuBzYBVpFU050ra\nDriStALmrIiYkPd3Kmm55tLqmpOrcqBmZgZ0zpXM7yNidP75CnAu8OOI2BV4DjhGUh/gTGAPYDTw\nNUkDgEOARRExCjgPmJTbvAQ4OSJGAv0kjZM0DDgYGAWMBy6W1L16h2lmZl2hu2w0cGd+fBcpsewE\nzIyIxRHxFvAoMBLYHbgj150CjJTUCxgWETMbtTEGuDcilkdEHfAisE0VjsfMzLKqdpdl20i6ExgA\nnAP0iYhl+bX5wBBgMFBXts0a5RFRL6khly1sou6CZtp4qqXg+vfvTY8evuBpq9ravp26fVEcV2Uc\nV2XWp7iqnWT+TkostwFbAA83iqGmme0qKa+0jdUsXPhmW6o1o2v+4RSprm7JWm9bW9u3XdsXxXFV\nxnFVZl3Ialh7AAAOrElEQVSMq6XkVNXusoh4OSJujYiGiHgeeBXoL2mDXGUoMC//DC7bdI3yPAmg\nhjRZYGBLdRuVm5lZlVQ1yUg6VNIp+fFg4P3ADcD+ucr+wH3ADGAHSRtJ2pA0HjMdeIA0WwxgH+Dh\niFgBzJE0Kpfvl9t4CNhbUi9Jm5CSzOyij9HMzN5R7e6yO4FbJO0L9AImAH8FfirpS6TB+ZsiYoWk\n04H7eWf68WJJtwJjJT0CLAOOyu1OBK6S1A2YERFTACRdA0zLbUyIiPpqHaiZmUFNQ0NDZ8fQpdTV\nLVnrEzJo0Po3JjN/vsdkqsVxVcZxVaadYzLNjnl3hSnMZma2jnKSMTOzwjjJmJlZYZxkzMysME4y\nZmZWGCcZMzMrjJOMmZkVxknGzMwK4yRjZmaFcZIxM7PCOMmYmVlhnGTMzKwwTjJmZlYYJxkzMyuM\nk4yZmRXGScbMzApT7ZUxkXQBsGve9yTgs8AIYEGucmFE3CPpUNKKl/XA1RFxnaSewI3AZsAq4OiI\nmCtpO+BK0gqYsyJiQt7XqaTlmkura06u0mGamRlVTjKSxgDDI2IXSQNJSy8/BHwrIu4uq9cHOBPY\nEVgOzJR0B7APsCgiDpX0aVKSOgi4BDg5ImZKukXSOGAOcDCwC9APmC7p/ohYVbUDNjNbz1W7u2wa\n6coCYBHQB+jeRL2dgJkRsTgi3gIeBUYCuwN35DpTgJGSegHDImJmLr8L2AMYA9wbEcsjog54Edim\ngGMyM7NmVPVKJl9FvJGfHgtMJnV7nSTp68B84CRgMFBXtul8YEh5eUTUS2rIZQubqLugmTaeainG\n/v1706NHU3nPmlJb27dTty+K46qM46rM+hRX1cdkACTtS0oynwY+ASyIiCcknQ6cDfyh0SY1zTTV\nVHklddewcOGbbanWjK75h1Okurola71tbW3fdm1fFMdVGcdVmXUxrpaSU2cM/H8G+A6wZ0QsBh4s\ne/lO0gD+r0hXKCVDgceAebn8yTwJoAZ4BRjYqO68/KMmys3MrEqqOiYjqR9wITA+Il7LZbdL2iJX\nGQ08DcwAdpC0kaQNSeMx04EHeGdMZx/g4YhYAcyRNCqX7wfcR5pQsLekXpI2ISWZ2UUfo5mZvaPa\nVzIHARsDt0lvX2TcANwq6U1gKWla8lu56+x+3pl+vFjSrcBYSY8Ay4CjchsTgaskdQNmRMQUAEnX\nkCYbNAATIqK+GgdpZmZJTUNDQ2fH0KXU1S1Z6xMyaND6NyYzf77HZKrFcVXGcVWmnWMyzY55+xv/\nZmZWGCcZMzMrjJOMmZkVxknGzMwK4yRjZmaFcZIxM7PCOMmYmVlhnGTMzKwwTjJmZlYYJxkzMyuM\nk4yZmRXGScbMzArjJGNmZoVxkjEzs8I4yZiZWWGcZMzMrDDVXhmz6iT9ENiZtDrmyRExs5NDMjNb\nb6zTVzKSPgVsFRG7AMcCl3VySGZm65V1OskAuwO/AYiIZ4H+kt7XuSGZma0/1vXussHAn8ue1+Wy\n15vboKW1qlvT0LC2W76b9W3X1rW17du+KI6rMo6rMutTXOv6lUxja51AzMyscut6kplHunIp2QR4\npZNiMTNb76zrSeYB4AAASR8H5kXEks4Nycxs/VHTsI4PJEj6AfB/gHrgyxHxZCeHZGa23ljnk4yZ\nmXWedb27zMzMOpGTjJmZFWZd/55MVXS1W9dIGg78FvhhRFwuaVPgZqA7aXbd4RGxrBPiugDYlfR3\nNwmY2dlxSeoN3Ai8H3gv8D3gyc6Oqyy+DYCnc1wPdnZckkYDvwSeyUVPARd0dlw5tkOB04CVwJnA\nrM6OS9KxwOFlRZ8APtwF4toQ+CnQH3gPcA4wu4i4fCXTTl3t1jWS+gA/Ir0hlZwL/DgidgWeA47p\nhLjGAMPzedoTuKQrxAXsAzweEZ8CDgQu7iJxlZwBvJYfd5W4fh8Ro/PPV7pCXJIGAmcBo4DxwL5d\nIa6IuK50rnJ8N3WFuICjUngxhjQD99Ki4nKSab+uduuaZcBepO8IlYwG7syP7wL2qHJMANOAz+fH\ni4A+dIG4IuLWiLggP90U+FdXiAtA0tbANsA9uWg0XSCuJoym8+PaA5gSEUsi4pWIOL6LxFXuTNIV\n6Wg6P67/AAPz4/75+WgKiMvdZe1X8a1rihQRK4GVksqL+5Rd9s4HhnRCXKuAN/LTY4HJwGc6O64S\nSX8APkD6FDyli8R1EXAScGR+3um/x2wbSXcCA0jdLF0hrs2B3jmu/sDZXSQuACTtAPwzIl6V1Olx\nRcT/SjpK0nOk87U3cGcRcflKpuN19VvXdGp8kvYlJZmTGr3UqXFFxCeBzwI/axRLp8Ql6QjgjxHx\nj2aqdNb5+jspsexLSn7XsfqH1c6Kq4b0yXw/UlfQDXSB32OZ40hjf4111t/XYcBLEbElsBtweaMq\nHRaXk0z7vRtuXbM0DyADDGX1rrSqkfQZ4DvAuIhY3BXikjQiT4wgIp4gvWEu6ey4SJ8s95X0GOkN\n6rt0gfMVES/nLsaGiHgeeJXURdzZ5+vfwB8iYmWOawld4/dYMhr4Q37c6b9HYCRwP0D+gvomwBtF\nxOUk037vhlvXTAH2z4/3B+6rdgCS+gEXAuMjojSQ3elxke4G8Q0ASe8HNuwKcUXEQRGxQ0TsDFxL\n6svv9LgkHSrplPx4MGlW3g2dHRfp/+FukrrlSQBd4vcIIGkTYGlELM9FXSGu54CdACRtBiwFfldE\nXP7GfwfoSreukTSC1Je/ObACeBk4lHSp/l7gReDoiFhR5biOJ/WT/62s+EjSG2hnxrUBqctnU2AD\nUlfQ46TpnZ0WV6MYzwZeIH3y7NS4JPUFbgE2AnqRztdfOzuuHNuXSF2xAN8nTZHvCnGNAL4fEePy\n8yGdHVeewnw96UNCD9KV8rNFxOUkY2ZmhXF3mZmZFcZJxszMCuMkY2ZmhXGSMTOzwjjJmJlZYZxk\n7F1D0lRJhd7nSdKWkv4u6YoObvdsSd9vovwoST/ryH01sY/Dimy/0b62kvRE0cfUzL5/Jumoau/X\nWuYkY7a6XYC/RMSJnR1IR5A0FDihSvvqQ/o+1uRq7M/eHXyDTOtwec2R00l3NP4I6UuhewKDgEci\n4gO53tlAj4g4Q9JS0hfo9iF9ye984IuAgAkR8UBufh9Jp5Fue/G9fKO//sBPgFqgH3BRRNyS2x8G\nbAZ8IyLevpGppA/lbbqR/h+cTrpFyndIt0m5ojzR5La2ADYm3TjwoYj4Rv7kPJ50k8GLSV/kvI70\njfP3ABdExB25mQ9KujvH/nBEfL3Refso6Yu0PfPPSRHxV0lTSXex3gnYCphI+iLrcOCnEXFeC7+O\nW4BtJf0U+CDwnYiYmvd3L2lZiNOAv+T2hgDnR8QvmjuvLexrGTCWtGTC/zRVIf9tfJ/0Zb9hpDty\nHxwRr0s6EPgK6b5ZdcBxEbFA0uukc9o9Ir5a1la3XL5tbq9P2Wvnku6QDunv8DDSl4FXRMTZuc43\ngYERcVoLx2Tt5CsZK8ouwLfz+jGrgM+0Ur8PaV2XkaS7Ne8TEXuRbqdSflXRIyI+TbpB46X5jeb7\nwH0RsRvpzgvnSqrN9YcBY8oTTPYj4Mq8zscE0pv1c8APgN81cyUznHQTzZ1I9xX7aC7fHtgrIu4h\nrcnx+9zuvsCV+VvyAFsDnyvbfnij9n8OnJC3PZF0N4SSmoj4DOkb2f8P+ALpnJ7aRJzlzgKeiogj\ngKtIN49E0gBSAi/dOqRnPq+fAy5pw3ldQ75v2JutxAMwAjgt35R0AXBUvn/cd4A9ImIUMBX4dq6/\nITC5PMFke5DO6Q6khcG2y8fWA3gT2DX/PW1EOlfXAIdJKt388fOkJGUFcpKxojwbEfPz4xdJt4Vv\nzSP533/xzs0E/0X6FF3yO4CcECB9yh4DTMif+O8hXTkNy68/FhFN3dZip7K2ngLeJ2njVuJ7KL+R\nLiddsWyTy/9Sdov08nbn5/hL6y78PiJWlG3/kVLDkgbletfl47g0x1T6P/po2fn4c26j8blpzW2k\n+3ttSEomP4+I+vxa6WaJz5FWeB1Ey+e1PZ6JiJfz40dJ53EX0lXU/Xl/B/POreZreOf4y21Luilm\nQ05uM/IxrCR9sJku6fekDwEbR8QLpLtIf0rSB4E3IyI64HisBe4us6KsbPS8hvTmVa4X6X5vTW1T\n/rj8tuP1jcobSN00J0bE4+WNS9oLWE7TGsfSVHyNlX8oK69fvo+W2m0q9pJlwLJ8FbOavDZQc+em\nzSLiv5J+TUowB7D6FWJTx9bkee0Aze3rTxExvpltmvo91rD6Oe0OIGkkaVXHT0TEG5J+VVbnKtJV\nz3P4KqYqfCVj1fQ6MEBSb0ndSV0wldod3h5TWUnqu3+ENA6ApA0kXZG7TFryGLkLT9LHgAURsaCV\nbf6PpO6S3kPqopnVSrubkD6Nlz4tf0pSD0m9SGu9P1XaKC998EJOjEj6kKQzW4mnLepJ4zslV5OS\nS02jtWp2K+2XdBWwtue1LbbON4mEtFzyLNLNLHfMd3ZG0ufz2kMtmQ3sLKkmd0nulMvfD7yQE8xm\nwM6k8TGAu4EdSd2ev+yAY7FWOMlY1UTEQtLso8eBO0h3763USkm/zdt/NXeFnQ1sJekR0gD5X3OX\nSUu+AnxR0sOk8ZnD27DvuaQ3pseA/83LbTd2FjAqd/n8Gjg+Ipbm154BbgX+BPyyie2PAL4laRpp\nLfjftSGm1jwDvF9SqQtvNukT/42N6vXM5/V24Cu5G+1sKjivknbMx306MDZPOW/qvD4DTMrt9iWN\nh80DTgbuzsd/LOk8t+R+4CVSN9n1wB9z+QOkrsZHSOM6ZwPfkfShHP+9wJNtHD+ydvJdmM3aoHwm\nXGfH0h6SNidNMd6udBv3nBi+HxFTqrD/0Xlfo4reVzP770W6QjsqJ1wrmMdkzN7lJH2OdBWwhvIx\nHknfBg4Cvri264S0dV9dkaRxpJl5VzvBVI+vZMzMrDAekzEzs8I4yZiZWWGcZMzMrDBOMmZmVhgn\nGTMzK8z/D+GOZwesmThtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f597a0d9750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5997043cd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEVCAYAAAAy15htAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8lVXd9/HPYVJBYvIQaN6KaV8jTO/M1MA7zCFxyMch\nNTXnTMwSy6kyM0spfRwy0xwzLb21wXLAIVQCLBE1xYlfGandYg8nBQT1Zjic54+1Nm4O+wwM1zln\nb77v1+u82Htd61rXWvsc9m+vYV+rrqmpCTMzsyJ06+wKmJlZ7XKQMTOzwjjImJlZYRxkzMysMA4y\nZmZWGAcZMzMrjIOM1QxJL0saVSF9tKSXOqgOX+yI6zS75khJ0yS9KOlJSf/VjnM+IKkpPz5F0vfa\nyL+jpI+2cOwASTfmx5MkHbmK9V9P0lH58SaSnluV861r69HZFTCrFZKGAGcC13XgNdcDfg98LiIe\nkbQ3cBuwSXvLiIgr25HtWGAqMKPC+XcCd7b3ehX8J3AUcHNEvAaMWIOyrItxkLFOJ2k0cAXwB2Bf\noBfw+Yh4TNJ5pDfMbYFbc77vAQfl0x8DvhwRb+fnn5b0Y2Aj4OcRcU6za60HXAzsla9zbURcmI+9\nDFxCekPdBBgL7JbzNgBjImJuK035E/ABSTOB+4CeEXFKLnsA8BrwH7msU4HjgI2BcyPipznficDX\ngPWBPwPHRcS7rVyzJ3BiRDySn08FNpbUPyLmNWv7ccB3gLeAX5alnwd8ICJOkPS5nKc7sAT4KrA1\nKQh8VtJg4E3gs0A/4EngBeDIiNg9F7mNpMeBocD9wEnApsBLEdEjX3Nz4CXS63wn8D5JU4AvlPJJ\n6kYLv2tJk4C7gAOBYcBk4PCI8LfLuxgPl1lXMRx4PCIEXABcXXZsb2DviLgcOAQYA2wPfAToD5xW\nlnd74OP535MlbdvsOmfma22Tzz9Y0r5lx0dExMdIb263AL8CtiT9XzmwjTYcB7waEVvncz8nqfRB\nbl9gckT8Oz/fKiK2A3YBLpc0SNIu+bqfjojNgfn5eYsiYmFE/LYsaQzw1woBZgApQO8VEduQglsl\nVwH7RMSHgZOBz+YA+DhwZkRcmvPtCZwUEWdWKGNXYDQg4FO57S3V//8B3wD+HBG7NDvc1u96P2AP\n4EPAp4FPtnQd6zwOMtZVLATuyI9/A2wnqXd+Pq3szXkfUg/l7YhoBH5GesMr+WVENEbEHOCPwM7N\nrrMfcFVELMq9n5tZMXj8Lv/7LPBuREzKn46fp+U35pVExFPAPFJPCOAA4PayLDfmfAEE8Ilct9sj\nYnbO81PaDmzL5TmTy4AvVTi8I/C3iHgxP/95C8XMAU6StFlETI2Ir7WQ768R8bcWjv06It6JiHeA\ne1n5d9Bebf2ufx0R7+bf419JvUTrYjxcZl3F3LKhjtKn8P753zfL8tUD5UNWc4HBZc8byh7PBwY0\nu05/4DJJF+bn65E+pZcsyP82kgIfZc+7t9GG5m4DDpc0mfTJ/riyY+Vtmpvr2R84QFLpjbQbaUiv\nTZI+SQrSJ0TEpApZBpJej/JrVvJZ4BzgSUn/BMZFxB8r5HuzQlpJ89/B0Fbytqat33V5e1bn92Md\nwEHGuopBZY9LgaHSG9n/a5Z3UE4rGdisnOZlzAb+b0Tcs5r1XBW3AdNI8zOPNhvC2gh4JT8eSKrn\nbNIn99NX5SK5B/Mr4LCImNJCtrmkOZSS+kqZIuLvwLF5PuQo0jxYuxcRZJV+B41AN0l1+cNE8+Bf\nSVu/a6sCHi6zrqK3pP+THx8MPBER/1sh3z3AkZJ65/mO40lDMiWHSeqWJ6h3AZq/6f4eOEFSd0l1\nks6RtNdaasMSYMPSPEweCvs78ANWHCoD+DyApA8DW5GC0V3AgZLq87H9JZ3V2gUl1ZGGvk5uJcAA\nPJGya6v8/OgKZdVL+oOk90XEMtJEe6l3uYT3epZtOVDS+pL6kOZUpgD/JgWabXKeo8ryLyFN/Nc1\nK6et37VVAQcZ6ypeBkZJ+ivwTdKkcyW/BiaQVjU9B/yTNKFdMp00/PUEcFlEvNDs/J+QehDPAzOB\nD5NWZK0NM0if2v8lqTQ/cBvwflJwKzdH0tOkVVFfjYi5eR7nQmCSpBdJq8yan9fcTsBHgR9Kmln2\n87HyTBHRAHwdmJi/hxLNC8p57gemS3oB+G/SGzukFWA/lHRp8/MqmAg8AryYH9+fV8h9B7hf0hPA\n02X5p5Lmu2az4pBXW79rqwJ13k/GOltewnx9RGzZ2XVZ2yQdAhwcEYeUpTUBm0bE/3Rezcw6hnsy\nZgXJq+POwp++bR3miX+zVZC/ZPi+Fg7vEBELcr59Sd85uTEi1mg4TtKVwO4tHP5yRDy0JuWbFcnD\nZWZmVhgPl5mZWWE8XNZMQ8OC1e7aDRjQm7lz31mb1ekS3K7qUYttArerq6uv79t8+fly7smsRT16\n1OYXjt2u6lGLbQK3q5o5yJiZWWEcZMzMrDAOMmZmVhgHGTMzK4yDjJmZFcZBxszMCuMgY2ZmhXGQ\nMTOzwvgb/2bW5Qwe3LdCaqW09pkzZ0HbmVbR66/P5pxzzuKGG25ZIf2CC85j9OjdGDlyl7V6vaef\nforNNtucAQMGtp15Nd1xx2384Q/30dQEe++9Hwce+Lk1LtNBZi2qq4M1+Y/QtVVuVxH/ec1sZffe\nexef//yRhQWZ1177HyZMuJvrr7+ZpqYmPv/5A9lzzzFsuOGGa1Sug4yZrfMmTLibadP+xNtvv01D\nwxwOOeRwbrnlZ+y000gGDBjAmDH7Mn78+SxZsoRu3bpx9tnfpq6ujqVLl3L++d/mn/98ha22Emee\n+a3lZTY2NnLRRRcwe/ZrLF26lBNOOIntt9+BU045kY997ONMnz6N9dbrye6778WECffQrVs3fvSj\nq+nefeVbzUyf/hhTpkziH/+YxSc/OYrGxka++MWxAIwbdzKnnHIap5/+VUaP/jQvvvgC9fX1fOc7\nF7BkyWIuvPC7LFiwgMbGRsaNO4Mtt9xqpfIBhg7dmKuuup4ePVJYWH/99Xn77YVrHGQ8J2NmBvzj\nH7P4wQ8u5Uc/+inXXXc1ixcvZqedPsnRRx/P9df/lH333Z8rr7yWAw44mBtvvBaAl1+exZe+9GWu\nvfbn/PWvwd///tLy8v7wh/sZNGgjfvzjaxg//hKuuOKS5ccGDdqIq6++gcbGRt566y2uuup6li1b\nxqxZL61UL4AddtiJLbf8EN/85rkcfPChTJkyCYCFCxfy1lvz2XLLrfj3vxvYffe9uOaan9HU1MRj\njz3KHXfcxo47fpIf/ehqvv71s7nyystabH+3bt3o3bs3AI8//hj9+vXn/e8fsqYvq3syZmYA2233\nMXr06EH//v3p27cvs2e/xvDhHwEg4kVOOukUAD72sY9z003XA/CBD2y6/I14662H8+qrLy8v77nn\nZvDMM39hxoynAVi0aBFLliwBWF7u4MGD2WorATBw4EAWLlzYZj3f975+fOAD/0HETF599WV23TXt\nZ7fBBhswYsQ2AHzkIx/l1Vdf4dlnZzBv3lweeGBCrsP/tln+c889y09+cjkXXXR5m3nbw0HGzAxY\ntuy9XT6amqCuro4ePXrmlDpKGzwuWbKUuro0CFRXt+Id7suf9+jRk6OOOo499thrpWuVD4mVP27v\nJpJ77bUPjzwykX/963W+9KUv5/ovK8vRRF1dHT179uC0085gxIiPtqvcv/3tr/zwh9/joosuXyu9\nGCg4yEi6CNglX2c8MB24BegOvA58ISIWSToCGAcsA66NiBsk9QRuAjYDGoFjI2KWpG2Bq4EmYEZE\njM3XOgP4XE7/bkRMkNQPuBXoBywEDo+IN4tss5lVp+efn0FjYyMLFizgnXfe5n3v67f82Ic/PJyn\nnnqCPfbYi6effpKtt/4wkCbL//3vfzNw4EBmznyBgw46hEcfnQLA8OEjmDr1j+yxx17Mnfsmd9xx\n2/KAsDq6detGY2MjADvvPJLbbruZPn02ZOjQjYHUU5o580W23vrDPPfcs+y77/4sXbqUyZMnMWLE\nR/nHP2YxbdqfOOywIyuW39jYyPjx53PBBRctL3NtKCzISNoVGBERO0saBPwFeAj4SUT8StKFwHGS\nbgbOBT4BLAamS7oT2A+YFxFHSNqTFKQOBS4HTo2I6ZJulTQGmAkcBuxMCihTJD1AClyTIuJiSScC\nZ+UfM+vCmq9arK/vS0NDsSsZhwzZmG9/+2xee+2fnHjiyVx//U+XHzvhhJMYP/573H337+jRoyff\n+Ma3Wbp0KVtuuRXXXvsT/vGPWYwYsQ3Dhm2x/JxPf3p3nnpqOieddByNjY0cd9yJa1S/7bb7GOec\ncxbjx1/CFlt8kM02G4b04eXH+/Xrx4MPTuCKKy5h0KCN+MQndmK77f6TCy44j5NPPoFly5Yxbtzp\nLZb/5JPTef312Vx00YXL004++asMHz5ijepd197u2aqS1B1YPyLezo/nAG8BW+fey87A6cBPgOMi\n4sh83jXAPaReyc0RMVFSN+BVYAvgrxGxec77eeDjwPPAFhFxTk5/oFnZL0kaCtwTEdu3Vu812Rmz\n8tr+2lbNS5g74o2ro9Vim6D4dk2YcDezZv2dU04ZV9g1Klnddi1atIgvf/mLXH75VctXf+2zz27c\ne+9Da7uK7dLazpiF9WQiohF4Oz89HpgAfCYiFuW0OcBQYAjQUHbqSukRsUxSU06bWyHvG22VUZbW\nqgEDeq8Tu9WtLfX11R1Yq73+ldRim6DYdvXtuz69e/fqlNeu/JozZszg4osvXinPmDFjOPzwwwF4\n+umnOffcczn++OMZNuy9t7S6urp21f/KK69k2rRpK6VfeOGFbLrppqvThFYVPvEvaX9SkNkT+FvZ\noZYi36qkr428K1iz/bZr8z93a6r5U3MtfuqvxTZB8e3aZZc92GWXPTr8tWverqFDh3HppVdVzFvK\nt8kmH+SGG365QhrAPfdMbFf9Dz30aA499OhWr7GqWgtuhX5PRtJngG8BYyJiPrBQ0gb58CbA7PxT\nvoxhpfS8CKCOtFhgUGt5W0kvpZmZWQcpLMjklV0XA/uWreiaCByUHx8E3A9MA3aQ1F/ShsBIYArw\nIGleBtIigEciYgkwU9KonH5gLuNhYB9JvSRtTAooLzQro3Q9MzPrIEUOlx0KbATcIamUdjRwvaQv\nAa8AP4+IJZLOBh7gveXH8yXdDuwhaSqwCDgmlzEOuCYvBpgWERMBJF0HTM5ljM3zOFcAv5A0BZgH\nVF67Z2ZmhShsdVm18uqyVePVZV1LLbYJ3K6urrXVZb53mZmZFcZBxszMCuMgY2ZmhXGQMTOzwjjI\nmJlZYRxkzMysMA4yZmZWGAcZMzMrjIOMmZkVxkHGzMwK4yBjZmaFcZAxM7PCOMiYmVlhHGTMzKww\nDjJmZlYYBxkzMytMkTtjImkE8Hvgsoi4UtKvgPp8eCDwWEScKGkJ8GjZqbuRAuBNwGZAI3BsRMyS\ntC1wNWkHzBkRMTZf6wzSVsul3TUn5C2gbwX6AQuBw8u2gjYzs4IVFmQk9QF+DDxUSouIz5UdvxG4\nPj+dHxGjm51/JDAvIo6QtCcwnrSl8+XAqRExXdKtksYAM4HDgJ1JAWWKpAdIWzVPioiLJZ0InJV/\nzMysAxQ5XLYI2BuY3fyAJAH9I+LxVs7fDbgzP54IjJTUCxgWEdNz+t3A7sCuwH0RsTgiGoBXgOHN\nyijlNTOzDlJYTyYilgJLUzxZyamkXk7J+pJuJQ2N/SYiLgWGAA25rGWSmnLa3LLz5gBDgTdKeZul\nDylLL6W1asCA3vTo0b3N9llSX9+3s6uwRqq9/pXUYpvA7apWhc7JVJJ7I6Mi4uSy5NOBX5DmUyZL\nmlzh1Lp2pq1q3hXMnftOe7K1oLb/WCppaFjQ2VVYbfX1fau6/pXUYpvA7erqWguUHR5kgE8BKwyT\nRcRPS48lPQRsQxpmGwI8I6knKUi8DgwqO3WTnG82oBbShwDzy9LMzKyDdMYS5h2AZ0pPlNwqqU5S\nD2Ak8DzwIGm1GMB+wCMRsQSYKWlUTj8QuB94GNhHUi9JG5MCygvNyjgo5zUzsw5S5Oqy7YFLgM2B\nJZIOJgWFocDfS/kiIiT9k9S7WQbcFRGPS3oS2EPSVNIigmPyKeOAayR1A6ZFxMR8veuAyaQht7F5\nHucK4BeSpgDzgCOLaq+Zma2srqmpqbPr0KU0NCxY7Rdk8OB1b05mzpzqHU+ulfHwcrXYJnC7urr6\n+r4tznn7G/9mZlYYBxkzMyuMg4yZmRXGQcbMzArjIGNmZoVxkDEzs8I4yJiZWWEcZMzMrDAOMmZm\nVhgHGTMzK4yDjJmZFcZBxszMCuMgY2ZmhXGQMTOzwjjImJlZYRxkzMysMIXtjAkgaQTwe+CyiLhS\n0k3A9sAbOcvFEXGvpCNIO14uA66NiBsk9QRuAjYDGoFjI2KWpG2Bq0k7YM6IiLH5WmeQtlpuAr4b\nERMk9QNuBfoBC4HDI+LNIttsZmbvKawnI6kP8GPgoWaHvhERo/PPvTnfucDuwGjgNEkDgcOBeREx\nCrgAGJ/Pvxw4NSJGAv0kjZE0DDgMGAXsC1wqqTspcE3KZfwWOKuo9pqZ2cqKHC5bBOwNzG4j347A\n9IiYHxHvAo8CI4HdgDtznonASEm9gGERMT2n300KTrsC90XE4ohoAF4Bhjcro5TXzMw6SGHDZRGx\nFFgqqfmhUyR9DZgDnAIMARrKjs8BhpanR8QySU05bW6FvG+0VUZZWqsGDOhNjx7d29FCg7RHeTWr\n9vpXUottArerWhU6J1PBLcAbEfG0pLOB84A/NctT18K5ldLXRt4VzJ37TnuytaC2/1gqaWhY0NlV\nWG319X2ruv6V1GKbwO3q6loLlB26uiwiHoqIp/PTu4BtSMNpQ8qybZLTlqfnRQB1wOvAoNbytpJe\nSjMzsw7SoUFG0m8kbZGfjgaeA6YBO0jqL2lD0nzMFOBB0moxgP2ARyJiCTBT0qicfiBwP/AwsI+k\nXpI2JgWUF5qVcVDOa2ZmHaSw4TJJ2wOXAJsDSyQdTFptdrukd0hLio+NiHfz0NkDvLf8eL6k24E9\nJE0lLSI4Jhc9DrhGUjdgWkRMzNe7Dpicyxib53GuAH4haQowDziyqPaamdnK6pqamjq7Dl1KQ8OC\n1X5BBg9e9+Zk5syp3vHkWhkPL1eLbQK3q6urr+/b4py3v/FvZmaFcZAxM7PCOMiYmVlhHGTMzKww\nDjJmZlYYBxkzMyuMg4yZmRXGQcbMzArjIGNmZoVxkDEzs8I4yJiZWWEcZMzMrDAOMmZmVhgHGTMz\nK4yDjJmZFaawTcsAJI0Afg9cFhFXStoU+BnQE1gCHBkR/5K0BHi07NTdSAHwJmAzoJG0wdksSdsC\nV5M2J5sREWPztc4g7YJZ2vhsgqR+wK1AP9ImaYdHxJtFttnMzN5TWE9GUh/STpgPlSV/H7g2Ij4F\n3Al8LafPj4jRZT+NwOHAvIgYBVwAjM95LwdOjYiRQD9JYyQNAw4DRgH7ApdK6k7aRXNSLuO3wFlF\ntdfMzFZW5HDZImBvYHZZ2snAb/LjBmBQK+fvRgpEABOBkZJ6AcMiYnpOvxvYHdgVuC8iFkdEA/AK\nMLxZGaW8ZmbWQQobLouIpcBSSeVpbwPkXsaXgfPzofUl3UoaGvtNRFwKDCEFIiJimaSmnDa37DJz\ngKHAG6W8zdKHlKWX0szMrIO0K8hI6gm8PyL+R9JHgW1JweCdVb1gDjC3AA9HRGko7XTgF6T5lMmS\nJlc4tdIe0i3tK70qeVcwYEBvevTo3p6sRtqjvJpVe/0rqcU2gdtVrdrbk/k58FtJfybNbfwW2A84\nZDWu+TPgbxHx3VJCRPy09FjSQ8A2pGG2IcAzOcjVAa+z4hDbJjnfbEAtpA8B5peltWru3FWOm2Vq\n+4+lkoaGBZ1dhdVWX9+3qutfSS22Cdyurq61QNneOZlNIuLXwKHAVRFxJjBwVSsi6QhgcUR8pyxN\nkm6VVCepBzASeB54kLRaDFJAeyQilgAzJY3K6QcC9wMPA/tI6iVpY1JAeaFZGQflvGZm1kHa25NZ\nT1IdcABwfE5r9WO7pO2BS4DNgSWSDgYGA/8raVLO9kJEnCzpn8DjwDLgroh4XNKTwB6SppIWERyT\nzxkHXCOpGzAtIibm610HTCYNuY3N8zhXAL+QNAWYBxzZzvaamdlaUNfU1NRmJkk/IK0Muz8iDpE0\nDvjPiDi66Ap2tIaGBW2/IC0YPHjdGy6bM6d6u/q1MlRRrhbbBG5XV1df37fFOe/29mRuAn4QEfPy\n898DT6xhvczMrMa1GmQk9SdNtP8MOFxSaR6mJ3Aj8KFiq2dmZtWsrZ7MzsBpwHakyfWSZcADRVXK\nzMxqQ6tBJiLuA+6TdFL5MmMzM7P2aO+czO8knUpatrx8gicizi2kVmZmVhPa+z2Ze0nf8l9GuiNy\n6cfMzKxF7e3JLIyI4wqtiZmZ1Zz29mQek7R1oTUxM7Oa096ezF7A1yQ1AEtJ8zJNEfEfhdXMzMyq\nXnuDzGcLrYWZmdWk9gaZ3VpIv3FtVcTMzGpPe4PMLmWPewE7Ao/iIGNmZq1oV5CJiGPLn0vqTbrV\njJmZWYvau7psBXlHzC3Xcl3MzKzGtHf75SmkfVpKNgFmFFIjMzOrGe2dkzmn7HET8BbwzNqvjpmZ\n1ZL2zsn8UdIuwA6kIPNYRLS5uZekEaS9Zy6LiCslbQrcAnQHXge+EBGL8rbM40i3rbk2Im6Q1JO0\nj81mpFvYHBsRsyRtC1yd6zEjIsbma51B2mq5CfhuREyQ1A+4FegHLAQOj4g32/XKmJnZGmvXnIyk\n84GLgaGkobIrJH2jjXP6AD8GHipLPh/4SUTsArwEHJfznQvsDowGTsv71hwOzIuIUcAFwPhcxuXA\nqRExEugnaYykYcBhwChgX+BSSd1JgWtSLuO3wFntaa+Zma0d7Z343xX4ZEScERGnk/aZ2a+NcxYB\newOzy9JGA3flx3eTAsuOwPSImB8R75KWRo8kfTfnzpx3IjBSUi9gWERMb1bGrsB9EbE4IhqAV4Dh\nzcoo5TUzsw7S3jmZbhGxrPQkIpZKWtbaCRGxFFgqqTy5T0Qsyo/nkHpGQ4CGsjwrpUfEMklNOW1u\nhbxvtFVGWVqrBgzoTY8e3dvKZll9fd/OrsIaqfb6V1KLbQK3q1q1N8g8KekuUo8CYA/giTW8dt1a\nSF8beVcwd+477cnWgtr+Y6mkoWFBZ1dhtdXX963q+ldSi20Ct6uray1Qtjlcluc7xpEm0IcBmwOT\nI2LcatRloaQN8uNNSENps0k9DlpKz4sA6kiLBQa1lreV9FKamZl1kFaDjKTdSHMkfSPivyPiNNI3\n/cdK2n41rjcROCg/Pgi4H5gG7CCpv6QNSfMxU4AHSavFIM3/PBIRS4CZkkbl9ANzGQ8D+0jqJWlj\nUkB5oVkZpeuZmVkHaasn8x1gz4iYX0qIiGdJb/rfb+1ESdtLmgQcA5yaH38XODp/uXMg8PM82X82\n8AApCH03X+92oLukqcCXgdJqtnHAeEmPAn+PiIkR8SpwHTAZ+A0wNs8hXQF8PF9vV9IKOTMz6yB1\nTU0tf91F0pS83LjSsUciYtfCatZJGhoWtPn9n5YMHrzuzcnMmVO948m1Mh5erhbbBG5XV1df37fF\nOe+2ejIbtnJsUCvHzMzM2gwyz0k6qXmipDNJcylmZmYtamsJ8xnA7yQdBUwn3Q5mJOneZfsUXDcz\nM6tyrQaZiPgXsFNeZfYR0j3E7oiIyR1ROTMzq27tvUHmQ6x4DzIzM7M2rdamZWZmZu3hIGNmZoVx\nkDEzs8I4yJiZWWEcZMzMrDAOMmZmVhgHGTMzK4yDjJmZFcZBxszMCuMgY2ZmhXGQMTOzwrTr3mVr\ni6TjgS+UJX0c+DWwPfBGTrs4Iu6VdARpF8xlwLURcYOknsBNwGakm3UeGxGzJG0LXA00ATMiYmy+\n3hmk7ZebSDtuTii6jWZm9p4ODTIRcQNwA4CkTwGHAH2Ab0TEPaV8kvoA5wKfABYD0yXdSdr2eV5E\nHCFpT2A8cChwOXBqREyXdKukMcBM4DBgZ6AfMEXSAxHR2EHNNTNb53XmcNm5wPdaOLYjMD0i5kfE\nu8CjpH1sdgPuzHkmAiMl9QKGRcT0nH43sDuwK3BfRCyOiAbgFWB4MU0xM7NKOrQnUyJpB+CfEfEv\nSQCnSPoaMAc4BRgCNJSdMgcYWp4eEcskNeW0uRXyvtFCGc+2VrcBA3rTo0f31W/cOqa+vm9nV2GN\nVHv9K6nFNoHbVa06JcgAJ5DmVgBuAd6IiKclnQ2cB/ypWf66FsqplL4qeVcyd+477cnWgtr+Y6mk\noWFBZ1dhtdXX963q+ldSi20Ct6uray1QdtZw2WhyIImIhyLi6Zx+F7ANMJvUQynZJKctT8+LAOqA\n14FBreVtlm5mZh2kw4OMpI2BhRGxOD//jaQt8uHRwHPANGAHSf0lbUiaj5kCPEhaLQZpEcAjEbEE\nmClpVE4/ELgfeBjYR1KvfM1NgBcKb6CZmS3XGcNlQ0nzIyVXArdLegdYSFqW/G4eOnuA95Yfz5d0\nO7CHpKnAIuCYXMY44BpJ3YBpETERQNJ1wORcxtiIWFZ888zMrKSuqamps+vQpTQ0LFjtF2Tw4HVv\nTmbOnOodT66V8fBytdgmcLu6uvr6vi3Oefsb/2ZmVhgHGTMzK4yDjJmZFcZBxszMCuMgY2ZmhXGQ\nMTOzwjjImJlZYRxkzMysMA4yZmZWGAcZMzMrjIOMmZkVxkHGzMwK4yBjZmaFcZAxM7PCOMiYmVlh\nHGTMzKwwHbozpqTRwK+A53PSs8BFwC1Ad+B14AsRsUjSEaQdL5cB10bEDZJ6AjcBmwGNpF00Z0na\nFriatAPmjIgYm693Bmm75tLumhM6pKFmZgZ0Tk/mjxExOv98BTgf+ElE7AK8BBwnqQ9wLrA7MBo4\nTdJA4HBgXkSMAi4AxucyLwdOjYiRQD9JYyQNAw4DRgH7ApdK6t5xzTQzs64wXDYauCs/vpsUWHYE\npkfE/IgFLLfZAAAPBElEQVR4F3gUGAnsBtyZ804ERkrqBQyLiOnNytgVuC8iFkdEA/AKMLwD2mNm\nZlmHDpdlwyXdBQwEvgv0iYhF+dgcYCgwBGgoO2el9IhYJqkpp82tkPeNFsp4trXKDRjQmx493OFp\nr/r6vp1dhTVS7fWvpBbbBG5XteroIPM3UmC5A9gCeKRZHepaOG9V0le1jBXMnftOe7K1oLb/WCpp\naFjQ2VVYbfX1fau6/pXUYpvA7erqWguUHTpcFhGvRcTtEdEUEX8H/gUMkLRBzrIJMDv/DCk7daX0\nvAigjrRYYFBreZulm5lZB+nQICPpCEmn58dDgPcDPwMOylkOAu4HpgE7SOovaUPSfMwU4EHSajGA\n/YBHImIJMFPSqJx+YC7jYWAfSb0kbUwKMi8U3UYzM3tPRw+X3QXcKml/oBcwFvgLcLOkL5Em538e\nEUsknQ08wHvLj+dLuh3YQ9JUYBFwTC53HHCNpG7AtIiYCCDpOmByLmNsRCzrqIaamRnUNTU1dXYd\nupSGhgWr/YIMHrzuzcnMmVO948m1Mh5erhbbBG5XV1df37fFOe+usITZzMxqlIOMmZkVxkHGzMwK\n4yBjZmaFcZAxM7PCOMiYmVlhHGTMzKwwDjJmZlYYBxkzMyuMg4yZmRXGQcbMzArjIGNmZoVxkDEz\ns8I4yJiZWWEcZMzMrDAOMmZmVpiO3hkTSRcBu+Rrjwc+C2wPvJGzXBwR90o6grTj5TLg2oi4QVJP\n4CZgM6ARODYiZknaFriatAPmjIgYm691Bmm75tLumhM6qJlmZkYHBxlJuwIjImJnSYNIWy8/DHwj\nIu4py9cHOBf4BLAYmC7pTmA/YF5EHCFpT1KQOhS4HDg1IqZLulXSGGAmcBiwM9APmCLpgYho7LAG\nm5mt4zp6uGwyqWcBMA/oA3SvkG9HYHpEzI+Id4FHgZHAbsCdOc9EYKSkXsCwiJie0+8Gdgd2Be6L\niMUR0QC8AgwvoE1mZtaCDu3J5F7E2/np8cAE0rDXKZK+BswBTgGGAA1lp84BhpanR8QySU05bW6F\nvG+0UMazrdVxwIDe9OhRKe5ZJfX1fTu7Cmuk2utfSS22CdyuatXhczIAkvYnBZk9gY8Db0TE05LO\nBs4D/tTslLoWiqqUvip5VzJ37jvtydaC2v5jqaShYUFnV2G11df3rer6V1KLbQK3q6trLVB2xsT/\nZ4BvAXtFxHzgobLDd5Em8H9N6qGUbAI8BszO6c/kRQB1wOvAoGZ5Z+cfVUg3M7MO0qFzMpL6ARcD\n+0bEmzntN5K2yFlGA88B04AdJPWXtCFpPmYK8CDvzensBzwSEUuAmZJG5fQDgftJCwr2kdRL0sak\nIPNC0W00M7P3dHRP5lBgI+AOaXkn42fA7ZLeARaSliW/m4fOHuC95cfzJd0O7CFpKrAIOCaXMQ64\nRlI3YFpETASQdB1psUETMDYilnVEI83MLKlramrq7Dp0KQ0NC1b7BRk8eN2bk5kzp3rHk2tlPLxc\nLbYJ3K6urr6+b4tz3v7Gv5mZFcZBxszMCuMgY2ZmhXGQMTOzwjjImJlZYRxkzMysMA4yZmZWGAcZ\nMzMrjIOMmZkVxkHGzMwK4yBjZmaFcZAxM7PCOMiYmVlhHGTMzKwwDjJmZlYYBxkzMytMR++M2eEk\nXQbsRNod89SImN7JVTIzW2fUdE9G0qeArSJiZ+B44IpOrpKZ2TqlpoMMsBvwO4CIeBEYIOl9nVsl\nM7N1R60Plw0Bnix73pDT3mrphNb2qm5LU9PqnlnN+nZ2BdZIfX1117+SWmwTuF3VqtZ7Ms2tdgAx\nM7NVV+tBZjap51KyMfB6J9XFzGydU+tB5kHgYABJHwNmR8SCzq2Smdm6o66pxicSJP0A+C9gGfDl\niHimk6tkZrbOqPkgY2ZmnafWh8vMzKwTOciYmVlhav17Mh2iWm9dI2kE8Hvgsoi4UtKmwC1Ad9Iq\nvC9ExCJJRwDjSPNa10bEDZJ6AjcBmwGNwLERMasz2tGcpIuAXUh/3+OB6VR5uyT1JtXr/cD6wPeA\nZ6jydgFI2gB4jtSmh6jyNkkaDfwKeD4nPQtcRJW3a3W5J7OGqvXWNZL6AD8m/acuOR/4SUTsArwE\nHJfznQvsDowGTpM0EDgcmBcRo4ALSG/mnU7SrsCI/PvYC7icGmgXsB/wRER8CjgEuJTaaBfAOcCb\n+XGttOmPETE6/3yF2mnXKnOQWXPVeuuaRcDepO8SlYwG7sqP7yb98e8ITI+I+RHxLvAoMJLU7jtz\n3ok5rSuYDHwuP54H9KEG2hURt0fERfnppsD/UAPtkrQ1MBy4NyeNpsrb1ILR1Ga72uQgs+aGkG5X\nU1K6dU2XFhFL8x92uT4RsSg/ngMMZeX2rZQeEcuAJkm9iq112yKiMSLezk+PByZQA+0qkfQn4FbS\nEEsttOsS4Gtlz2uhTQDDJd0laaqkPaiddq0yB5m1r1ZuXdNSO1Y1vVNI2p8UZE5pdqiq2xURnwQ+\nC/yCFetWde2SdBTw54j4RwtZqq5N2d+A7wL7A0cDN7Di/He1tmu1OMisuVq6dc3CPAkLsAmpbc3b\nt1J6nqisi4jFHVjXFkn6DPAtYExEzKcG2iVp+7wwg4h4mvSmtaDK27UPsL+kx4ATgG9TA7+riHgt\nD282RcTfgX+RhtGrul2ry0FmzdXSrWsmAgflxwcB9wPTgB0k9Ze0IWl8eAqp3aW5j/2ARzq4rhVJ\n6gdcDOwbEaXJ5KpvF+muFV8HkPR+YEOqvF0RcWhE7BAROwHXk1aXVXWbACQdIen0/HgIaUXgz6jy\ndq0uf+N/LajGW9dI2p40Hr45sAR4DTiCtHRyfeAV0tLJJZIOBs4gLdH+cUT8UlJ30hvDVqRFBMdE\nxD87uh3NSToROA/4a1ny0aS6VnO7NiANu2wKbEAajnkCuJkqbleJpPOAl4EHqPI2SepLmjfrD/Qi\n/a7+QpW3a3U5yJiZWWE8XGZmZoVxkDEzs8I4yJiZWWEcZMzMrDAOMmZmVhgHGasakiZJ2r3ga2wp\n6W+SrlrL5Z4n6fsV0o+R9Iu1ea0K1ziyyPLLrtNT0vWSHpX0lKQzOuK6Zdf/haRjOvKa1jYHGbMV\n7Qw8FREnd3ZF1gZJmwAnddDlTgTWi4iRpC8WflXS5h10beuivJ+MrXV5P42zSXcK/gjpy557AYOB\nqRHxgZzvPKBHRJwjaSHwfdI3nHsBFwJfBASMjYgHc/H7STqTdAuO70XEf0saAPwUqAf6AZdExK25\n/GGkfTm+HhFPltXxQ/mcbqT/B2eTbv/xLdItQK4qDzS5rC2AjUg3MXw4Ir6ePznvCwwg3X7/CdKX\nJjcE1gMuiojSHXU/KOmeXPdHIqL8xpBI+ijpC7I9888pEfEXSZNId5fekfQFvXGkL5iOAG6OiAta\n+XXcCmwj6Wbgg8C3ImJSvt59pO0ezgSeyuUNBS6MiNtael1budZ1wI0AEfGupLeBQaQvWZbaOJr0\ne36F9LuZBxwWEW9JOgT4CuleXQ3ACRHxhqS38mvaPSK+WlZWt5y+TS6vT9mx80l3M4b0d3gk6Uu6\nSyLivJznLGBQRJzZSptsDbknY0XZGfhm3telEfhMG/n7kPZLGQm8DewXEXuTbjVS3qvoERF7km4+\n+KP8RvN94P6I+DTpzgvnS6rP+YcBu5YHmOzHwNURMRoYS3qzfgn4AfCHFnoyI0g3p9yRdM+tj+b0\n7YC9I+Je0r4hf8zl7g9cnb8BDrA1cEDZ+SOalf9L4KR87smkb32X1EXEZ0jfGv8h8HnSa9rWkNR3\ngGcj4ijgGuAYgLxviUi3NwHomV/XA4DL2/G6riQiFpfu7C3pQOAd0jfdm9seODPf7PMN4Jh8X7Zv\nAbvnfVQmAd/M+TcEJpQHmGx30mu6A/AFYNt87R752rvkv6f+pNfqOuBISaUbTn6OFKSsQA4yVpQX\nI2JOfvwKMLAd50zN//4P8Keyx/3K8vwBIAcESJ+ydwXG5k/895J6TsPy8cciotJtLXYsK+tZ4H2S\nNmqjfg/nLRIWk3osw3P6U2W3cS8vd06uv/KxP0bEkrLzP1IqWNLgnO+G3I4f5TqV/o8+WvZ6PJnL\naP7atOUO4NP5PlkHAL/Mt5KHdDuX0uvaROp1tva6tijfKmU8cGBZ+eWej4jXyto1nPShZCjwQL7e\nYfk5pJ7No80LIfVg/pRvRPkO6V5gRMRS0gebKZL+SPoQsFFEvEy6Q/KnJH0QeCcioq322JrxcJkV\nZWmz53WkN69yvUj3e6t0Tvnj8ludL2uW3kS6v9PJEfFEeeGS9gZauntt87pUql9z5R/KyvOXX6O1\ncivVvWQRsCj3YlYgCVp+bdotIv5X0m9JAeZgVuwhVmpbxde1NZI+D5wOjI6Ilu5G3tK1Ho+IfVs4\np9LvsY4VX9PuuQ4jgeOAj0fE25J+XZbnGlKv5yXci+kQ7slYR3oLGCipd74J4H+tRhm7wfI5laWk\nsfuppC2JkbSBpKvykElrHiMP4Un6T+CNiHijjXP+S1J3SeuRhmhmtFHuxqRP46VPy5+S1ENpA6qP\nk/Z+ByBvSfByDoxI+pCkc9uoT3ssI83vlFxLCi51zfZx+XTpuqRewCq/rvncb5KGvFrb7mJrSaVe\nyijS6zgd+ES+azGSPqe0J1BrXgB2klSXhyR3zOnvB17OAWYzYCfS/BjAPcAnSMOev2qjfFsLHGSs\nw0TEXNJdnp8gbS9baby+LUsl/T6f/9U8FHYesJWkqaQJ8r/kIZPWfAX4oqRHSPMzX2jHtWeR3pge\nA/470nbbzX0HGJWHfH4LnBgRC/Ox54HbgceBX1U4/yjgG5ImAz8nD7utoeeB90sqDeG9QPrEf1Oz\nfD3z6/ob4Ct5mOs8Vu11PRXoC9yZl5tPkrRPC3Uan8vtS5oPm53Pvye3/3jS69yaB4BXScNkNwJ/\nzukPkoYap5KC3nnAtyR9KNf/PuCZPMRmBfNdmM3aoXwlXGfXZU3kJcUTgG0jYklOmwR8PyImdsD1\nR+drjSr6Wi1cvxeph3ZMDrhWMM/JmFU5SQeQegErKZ/jkfRN4FDgi6UAU9S1uiJJY0gr8651gOk4\n7smYmVlhPCdjZmaFcZAxM7PCOMiYmVlhHGTMzKwwDjJmZlaY/w9GOLBgnrbZKQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5979c5a610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f59823d0910>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f596c27d050>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAE3CAYAAADsVZWiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYZGV59/FvdzML+44IRBCkf4oogij7JuprFI2IBCXG\ngLgTA8ZoUAERBY2JlwpRg4KiGNBIXjcCEdmXGWBUxKh4R8XBBV/ZBGQRmOl6/3ieYmqanu5TVedU\nnVPz+1zXuaaWU3fd3TNTTz3nPOe+x1qtFmZmZoM2PuwEzMxs9eQByMzMhsIDkJmZDYUHIDMzGwoP\nQGZmNhRrDDuBUfbmsW1KXWJ4+u+vKDMcAMvX2bT0mKUb8/ckq5exZQ+XHnPBuhuM9Rujm8+cf2st\n7fv9+uX/2WZmNhSeAZmZjYiJoc9puuMByMxsRMwfb9YI5AHIzGxETIx5ADIzsyFo2iG4yhYhSNpG\n0ndnePxsSQdV8H77Stqs7LjT3uPPJH1X0r9U+T5mZr2YGBsrvNXBKK2Cex1Q6QAEfA64tOL3MDPr\nycRY8a0OCh2Ck3QE8CJgPWAr4GPAe4ALgduBL5A+nOcDU8BRQAuYJ+lLwCRwY0S8qSPmBPAZYFtg\nHnBiRFwm6QrgcuAFOdYXgCOA5cCBEbF8hvxeALwceLqkC4A1IuKE/Nx3gHcAFwH/CTwH+C1wOLAA\n+DywYf5dvC0ifjjLr+IVwCHAjkV+b2Zmg1SXmU1R3cyAng68DHge8EHSh/dFEXEKcDJwVkTsD3wK\nOCm/Zgfg3cBuwC6SntER73DgdxFxAGnw+HjHc7+LiL2BCWCjiNgn3+58/WMi4jvAD4AjgdNzPCSt\nD2ycB5UtgHMjYg9gDPhz4FjgvyPiQOAtwEdn+wVExB9ne97MbJjmjY0V3uqgm0UIV0bEMuBOSX8g\nzVxuyM/tShpoIM1eTsy3fx4RvwaQtARQR7w9gX0k7Z3vrylpfr7djvs74MZ8+/fA+nMlGRF3S/qZ\npF3y+301P/VARFyXby/Oz+0JbCrpNfnxteaKb2ZWV3U5tFZUNwNQ52xpjHSI7ZF8v5UfgxWH4dqP\nd+q8/whwSkSc17mDJIBlHQ913i766/0icCiwNelQ4Wz5vy0iFheMa2ZWW6N8CG4PSROSNgHWBe7q\neG4JcEC+vR/QXv22naQnShonnXu5ueM11wN/ASBpM0mn9vIDdJhixYB6IbAvsEFELM2PrSnp2e2f\nBfhJzqF9uG4HSX/fZw5mZkPTtEUI3QxAS0mHsy4D3suKWQ6kQ26vlXQZacHA+/LjNwGnkA55LY6I\nn3S85j+A+yUtAr4FXN1D/p2uBM6X9PSIeIQ02H2r4/m7gNdIupo0q/o26XzRU/JjZwJXrSq4pC3z\nAonjgMMkXSFphz5zNjMrTdOWYY+1WnMXT82r4HaMiH+oPKMSSFpIGtCeHxH35sfujIhNBpmHq2GX\nxNWwrWbqWg370xs8tfBnzlvu+enQR6FGVUKQ9FzgIzM89ZWI+HTeZ3fgDOCf24NPl+9xImml33RH\nRsQvu41nZjYodTm0VlShGZD1xjOgkngGZDVT1xnQWRsVnwEddbdnQGZmVpK6nNspygNQhcqesbzt\nCfuXGg/gYw/9tPSYY1OPK1bRn9bU3Pt0a3WdVZX9u1xNf4+tiXnDTmFGTTsE5wHIzGxElDkDkvQx\nYHfSNZPHRMSSjueOBl5DKpH23Yg4tpf3WD2/vpiZjaD542OFt9lI2g/YPpcuOwo4reO59YB3Avvk\nkmk75MVfXfMAZGY2Ikq8EPVA4OsAEXEzsGEeeCBVkHkEWEfSGqQSZnf3kq/7AXX3HsdIukHSEklv\nrfK9zMy6VeKFqJsDd3TcvyM/RkT8CXg/cAtwK3B9RPxvL/mO0gyo0n5AkrYlVdveE9gLeFeutm1m\nVgvjY2OFty499oI8E3oPqc3Ok4HdJO3US77uB1S8H9BSYO9cERxJD+bfR9cXu5qZVWGsvGVwt5Fn\nPNkWpO4EAE8DbomIOwFyKbNnk0qvdcX9gAr2A4qIqYi4P8d9IXBnu9WEmVkdTMyfKLzN4WLglQC5\ntc1tHf3QlgJPk7Rmvr8r8LNe8nU/oC77AeXVHv8CvGSufc3MBqmsGVBELJL0vVwsego4Oh8Juzci\nvibpn4HLJS0DFkVET8Wk3Q+oi35A+TjnmcBBnv2YWd2Ml3glakQcN+2hmzqeO4NUc7Mv7gdUsB9Q\nPmf1OeCQjphmZrUxNj5eeKsD9wMq2A+ItC7+ycAZuRfQFbk6t5lZLYxPjBXe6sD9gCr06O1LS62G\nvdrWgqvCalrDzLXgSlJBfcKFa63d96hw+c67Ff7MOeDG64c+CjWqFpz7AZmZrVqB1W214n5AFfIM\nqMb8zb0c/j2WpowZ0NV77FX4M2efxdd6BmRmZuUYn2jWFwIPQBUqu9toFbOVt6/51NJjfvzBm+fe\nyYZjdZ2xlK2mv8cSKyEMhAcgM7MR4QHIzMyGwofgzMxsKCbmeQAyM7MhGPMMKJG0DXB+ROw67fGz\n8+MXlPx++wI/jYjby4zbEX8c+FfgmaT2EZ+JiLOqeC8zs17UpcJBUc0aLmdXaUM6UuXsR3ObiAOB\nD+VBycysFsYmxgpvdeCGdAUb0kXENcA1+e5mwN0RUf7VaGZmPWraITg3pCvYkK5N0leBa4Gj59rX\nzGyQJuaNF97qoJssroyIZbkN6x+ATVi5Id0V+fblwM759s8j4tcR0SK1bJjekO7lecZzPiU2pAPa\nDelezNwN6d6cc/hUwfiHArsDn5S07lz7m5kNyvjEeOGtDtyQrmBDOklPBcYi4uaIuFXSLaTe6DfM\n8VIzs4Goy7mdotyQrmBDOtJgc2redy3SDMrVsc2sNsYmxgtvdeCGdMUb0n0d+E3O92rgwxFxR585\nm5mVpmkdUd2QrkJ/evCBUtsxtMbL7/XhYqRm9bBwzTX7Pn72s7e8svBnzvafPn/ox+saVQnBDenM\nzFZtfH6jPtLdkK5KngGZWVFlzIBuefvhhT9ztv3YuZ4BmZlZOcYmmtWS2wNQg1TR6rqK2cqxaz2t\n1Hir64yqNVb+F9QxH/EoRwUtuctQl9VtRXkAMjMbEeM1Wd1WlAcgM7MR4RmQmZkNxfi8Zn2kNytb\nMzNbpabNgCrLVtI2kr47w+NnSzqogvfbV1KV/YDa7zMm6VpJJ1X9XmZm3RjlUjx1V3VDurbXkwqu\nmpnVykhWw3ZDusfeZ5P8ujPy78HMrDbqUuOtKDek664h3UdIhViXzbGfmdnAjc9fo/BWB25IV7Ah\nnaR9geURsWiuHMzMhqFp1bDdkK5gQzpS76JdJV0HbAoskPSLiDinYE5mZpUab1gpHjekK9iQLiLe\nERE7R8TuwAeAMz34mFmdlLkKTtLHJC2WtEjSc1axz4fyEaSeuCFd8YZ0Zma1VtYAJGk/YPt8zvwo\n4LQZ9tmB9EW/93zdkK46ZbdjqMRY+ceCXYy0HC5GWmMVFCNduNbaff+F3/f5Ewv/Ba935MmrfD9J\nJwO/iogz8/2fAs+NiPs69rkI+CfgpLwArWv1WApRkBvSmZmtWomr2zYHvtdx/4782H3w2KTkStKR\nsZ4VyjYizu7nTcoSETcA+8+xz3XATjM8Xmj2ExEnk5aVm5k1SoWr2x6bLUnaiHTJy/OBLfsJWo+1\neGZm1rex8YnC2xxuI8142rYgXRYD6QjRpqTTHF8jXeP5sV7ybdQhuMYp+/xKTZtgTVf2OZuyzykB\n7LjeglLj7bD1nJeodW339x5ceswFO/d1zvhxptbeqNR4AK2JcitdjVXw/2b5epvPvdMwzD2wFHUx\n8H7gjHxN5W0R8UeAiDifdO0mkrYBzo6It/fyJh6AzMxGRUmH4CJikaTv5VXKU8DR+bzPvRHxtVLe\nBA9AZmYjY6zEC1Ej4rhpD900wz5LmeO8/Gw8AJmZjYo1mlWo34sQMkmHzPH8uKQPS7pjUDmZmXWj\nabXg6pHFkOUTaa+eY7fjgF9RvB6dmdlgjU8U32qgtENwM/X3yduP8i535ueeTDpm+CFgr5zDv0bE\nObmm0I8AIuJvV/E+LwBOJfUH+nJEfFzS/vmxR4HfkJrTvZpcvUHSOsCPImIbST/PeR5EainxfOCT\nwHMlnZivA5rJ6RHxx3yFsJlZ/dRkYCmqzBnQqvr7/KhjMJmfe/vsRRoc9iKtKT9J0roz7L8SSWOk\ntgkvzjGeL2lN4N+AwyJiP1KriMNnyXMN4OaI2Bf4JXAg8M+kdhOrHFzaSxDNzOqqaYfgylyEsCew\nj6S98/01Sa0ZbujYp7N/0JUAEfGApJ8A20/bZyabAn+KiPZ5mIPyVbmtiPh1fuxyUkXu788Sp134\n9DekHkD3zPaDmZk1QsMWIZQ5AD2uv08+pPbItH1g5f5BsHIPoc79p1vO42dtq4rVWZRv3rTX9NJj\nyMys1spchj0IZc7Duunvs4S8djyfn9kO+NlcbxARdwETkraUNCbpAtJA05L0pLxbux/RfcAT82N7\nPz7aSjp7CZmZNdP4ePGtBsrMonB/n4i4BviepKuA7wDHRcQDBd/nraQyEIuASyPiHuANwLl5xjUP\n+DJwKaD82FNZuX/RdDczRz0jSafnWOtLumK25nVmZkPRsFVwhfoBWW/+9NBD5f5yq6gFV0E/oLK5\nFlx5FroWXCmqqAW3YO11+z4d8Oj1Xy/8mTNvt5cP/fRDLQ87Fen7M0rva2ZWipocWiuqlgNQkb4/\no/S+ZmZlGFuNV8GZmdkweQZklWnA+ZoqlH2+BuBH9z1caryDD9m51HgAyx58qPSYrXlrlhyw/ucl\nWxX8v6kiZhmatgzbA5CZ2aioyeq2ojwAmZmNCg9AZmY2DGNrTC/6Um8egMzMRkVNz02tigegTNIh\nEfGfszz/TFLbhilyxe2IeHBQ+ZmZzalhA1Czsq1IwYZ0pwPvyC0ffgYcUXFaZmZdaY2NF97qwA3p\nijeke2lE3Jdv3wFs3NUvyMysajUZWIpyQ7riDenuyzmsDbyWVBDVzKw+xsaKbzXghnRdNKTLg883\ngX+JiJuLvMbMbFBaE806re+GdAUb0klaA/gGcG5EnF3kNWZmA7UaH4Ib9YZ0/whcERFnzZWnmdlQ\njI0X32rADemSORvSAUcDL87N6K6QdGLBfM3MBqNhA5Ab0lWo9IZ0q6kzN39W6THLLkZ6wkn/p9R4\nAOs+6Qmlx1x7/5eXGq81r/xCsa15a5Ues2zL192s9JgL11q7/4Z0/+8XxRvSbb7d0Fci1PKMlRvS\nmZn1oCYzm6JqOQC5IZ2ZWQ9cjNTMzIahLhUOivIAZLW3w9brlx6z7AZyHzjp26XGAzjxA39eesy1\nS66WXMX5mrGpZXPvZDNzR1QzMxsKz4DMzGwoPACZmdkwtMab9ZHerGzNzGzVPANqpgIN6V4GvJtU\nq+524K8j4k+Dys/MbE4lVrnOlWF2J5U7OyYilnQ893xWtMW5MCI+0Mt7NGu4rEjBhnTHAC/KLR/u\nB15RdV5mZl0pqRSPpP2A7SNiD+Ao4LRpu5wGHEJqi/NCSTv0kq4b0hVsSBcRB+b3XwPYHPht978l\nM7PqlHgd0IHA1wEi4mZJG0paLyLuk7QtcHe7BY6kC/P+P+n2TdyQrmBDuvz+RwC3AL+IiCtn29fM\nbODKK0a6Oanzc9sd+bGZnrudFZ0HulLmALQn8PI8izmfLhrSkUbOrhrSRcTyiDgov8/0hnRzXWU4\nvSFdIbkP0LbAhpJmG+TMzAZuirHCW5dme0HPJ57KHIDaDen2z9v2+bHGN6STtFDSiwAiYhmpMd1c\nPYbMzAZqqtUqvM3hNlbMeAC2AH63iue2zI91zQ3pkrka0i0DPitpi3x/NyDmytfMbJBaXWxzuBh4\nJYCkXYDbIuKPABGxFFhP0jb5nPhBef+uuSFdMmtDujzreSPwdUlXA1sDny2Yr5nZQEy1im+ziYhF\npM/oRaQVb0dLOkLSwXmXtwDnkT7nvxIR/9tLvm5IVyE3pCvHot32Kz2mVtNipE/4m7eWGm9q/jql\nxoNmFCNdtv4Wc+/UpTIa0t1z/4OFP3M2WGctN6SbiRvSmZl1b66ZTd3UcgByQzozs+4t9wBkZmbD\n0LRTKj4HVKHV9RxQq8R6VACtb8y4NqQvyx58qNR49//2jrl36tLJJ1xUeszT7lpUbsCp2db29Kbs\nc0BVVIieWnvj0mMuXHPNvv/j/L97Hyj8mbP5+v2fc+qXZ0BmZiOiafMJD0BmZiPCixDMzGwoljds\nCuQByMxsRDRs/HE/oDZJhxTc702SllabjZlZ90qsBTcQHoAo3JAOSZvhRnRmVlMl1oIbCDekK9iQ\nLvtI/pm+0tUvx8xsAJq2CMEN6Qo2pMuD3EMRcf1svwQzs2FptYpvdVDmIoQ9gX0ktVsfFG5IJ6nr\nhnT5/kGSNuLxDen2A74/S5zpDenume0HkzQfOJncbsLMrI5W51Vw7YZ057UfyIfUGt+QjtRh9QnA\nRZIAnijpyxHxqgKvNTMbiKYdgitzAGo3pDsvn6w/dpZ9lwDHAx/utiGdpAlJ7Q583wJeQ25IFxG/\nIs1+rgEeoKSGdPmwm9r3JS314GNmddOwCZAb0mWzNqQzM2uCKVqFtzpwMdIKuRhpSfFcjLQ0LkZa\njroWI/3hbfcW/sx55hbruxjpTNyQzsyse3W5wLSoWg5AbkhnZta9RxvWka6WA5CZmXWvacuwfQ6o\nQqWfA2qVf7ydsfpXY5r/69ku6epNa96a5QZcY/pK//4tr+A8w99tvGep8T74iUIlFLuy9pO2KjXe\nsvvuLTUewPyDZ1vk25sF627Q9zmZRUvvKvyZs+c2G/sckJmZlWN5Bd9Rq+QByMxsRHgRgpmZDcWj\nDSuF4AEok3RIRPznLM9fAaxNqrAA8I6I+N4gcjMzK2K5B6Dm6egHtMoBKDsyIn40xz5mZkOx2h6C\nW036AZmZ1VbDLgNyPyAK9gPKTpZ0laQz8vuamdXG6tySe0/g5XkWcz5d9AMCuu4HFBHLI+Kg/D7T\n+wHtPEeu0/sBFfEJ4J154JoCji74OjOzgVg+1Sq81YH7ARXrB0REfK3j7reAw4q8zsxsUJq2Cq7M\nGVC7HxCSNpN06iz7LiHXXOu2HxAwIWlLSWOSLiANNC1JT8q77Qd8F7iPkvoB5fe6RNIG+aH9WXFu\ny8ysFlbnQ3Aj2w8oIlqkhQuX5pz/jLRwwcysNqamWoW3OnAtuAq5Flw5XAuuPK4FV4661oI75/u/\nKfyZ89e7bOVacDNxPyAzs+7V5dBaUbUcgNwPyMyse49WWI1U0jzgbGBr0oKwIyPillXsex7wcEQc\nMVvM+h9/MTOzQpa3im89OBy4JyL2Bk4hFRN4nFwsYLsiAWs5A7JVaMD5mipMrb1R+UFLPp/WmrdW\nqfEAmCr/22zZ52yOP2au6lXde8NB28+9UxfW3HBhqfEAtju49JClqPgQ3IHAF/PtS4DPTd9B0gLg\neOCDwCvmCrh6fqKZmY2g5a1W4a0HmwN3AETEFOnyl/nT9nk38GnSZTBz8gzIzGxElFXhQNLrgddP\ne3i3afdXWkUnaXtg14g4KdfnnJMHIDOzEVHWABQRZwJndj4m6WzSLOimvCBhLCI6K9e8BHiSpOuA\n9YBNJb0rImZaWQx4ADIzGxmPLKu0J/fFwKHAt4GXkupuPiYiPk4uQp1nQEfMNviAB6DHFGhItz6p\nwsJGwG+BV0fEw4PKz8xsLhUXGf0K8AJJ1wAPA0cASDqO1E1gcbcBPQBRuCHde4GLI+Jjkk4EdmL2\nyt1mZgNV5QAUEcuBI2d4/MMzPHYFcMVcMd2QrnhDupeSCp3ipnVmVkd1abNQlBvSFW9ItznwZklX\n54Z0C2bZ18xs4JrWD8gN6Yo3pFsIfCcPoOM8fomimdlQNW0AckO6gg3pgF93nGS7GDig4OvMzAbi\n4WpXwZXODemSWRvSZZdJag86zwZirnzNzAapaTMgN6RLZm1Il50AvFvS1cBTmHaRlpnZsDVtAHJD\nugqV3pBuNbXG3UvLD9qAYqStifKb3N137mzfsbq32hYj/eQ5pccsoyHdW86/qfBnzqdfuZMb0s3E\nDenMzLpXl5lNUbUcgNyQzsysexWX4ildLQcgMzPr3vIKekhVyQOQ1V5rYnrLkRKU3NxvbGrZ3Dt1\nHbT8Q/RrP2mrUuOVfb4G4LMXzLkgtitHPv/JpcYDKvm7KYMPwZmZ2VB4ADIzs6FY5gHIzMyGwTMg\nMzMbCq+Ca6jZGtLlVhOXdjy0BXB2RMxWbsjMbKA8A2qguRrS5UZM+3fsfxFQ/qXQZmZ9WG0HoNWg\nIV37/Z8P/G9H+wczs1poNWwAckO64g3p2o4BTiuwn5nZQE1NtQpvdVDmIbg9gX0ktVsfFG5IJ6nr\nhnT5/kGSNuLxDen2A74/S5zpDenume0Ha5O0JbB2RPyiyP5mZoPUtOLSbkhXvCEdpJnXZV3sb2Y2\nMMsbtgrODemSIg3pAJ4D3FRgPzOzgWtNFd/qwA3pkiIN6SANaLcXzNPMbKBarVbhrQ7ckK5CbkhX\njol7bys/aAOKkVbRkG7Zoq+VGu8nZ11YajxoRjHSZ33t66XHXLDO+n1XON3j1EsLf+Ysfs+BQ6+o\nWsvrgNyQzsyse01bhl3LAcgN6czMurd8eU1O7hRUywHIzMy65xmQWcnGKliy0yr5HFAVWuPl//dc\ndt+9pcZbc8OFpcaD8s/ZfP6SX5YaD+ATNT137gHIzMyGoi4VDoryAGRmNiKatqrZA5CZ2YioywWm\nRXkAMjMbEU0rxeMBKJutIV37eeAfSLXqfgscERGz1a0zMxuopi1CqP9SoAHoaEg3m9OAF+WWD/cD\nr6g6LzOzbky1WoW3OnBDuuIN6e4GNgDuzX/e2eWvyMysUqvzDGjUG9K9DbhR0i3ARERcMsu+ZmYD\n15pqFd66JWmepH+XdI2kKyVtO8M+p0i6VtJiSe+aK2aZA9CewMvzLOZ8umhIB3TdkC4ilkfEQfl9\npjek23mOXKc3pJuVpHHSIbjnkFpHLJf0srleZ2Y2SBV3RD0cuCci9gZOIR3FeoykHYED8sRiL+BI\nSZvPFtAN6Yo1pNsUGGt3QpV0KWkQ/WaB15qZDcRUtbXgDgS+mG9fAnxu2vP3AgslLQAmSJ/DD84W\n0A3pkrka0t0JbChp03z/OUXyNTMbpIpnQJsDdwBExBTpc3d++8l8FOqrwK15+7eIuG+2gG5Il8za\nkC4ilgNHA9+SdCVpsPpywXzNzAaiNbW88DYbSa+XdF3nBrxg2m5j016zLXAwabHZU4A3S9pstvdx\nQ7oKuSFdOda45zelxyy70GcVDemm5q1ZesyHLzyz1Hi/vuR7pcYD+OPv7i81XiXFSO/7QekxF6y7\nQd8N4rZ+3bmFP3Nu/dzhXb2fpLOB8yLi25LmAUsjYsuO5w8D9o6It+X75wGfjYjLVhWzlheiuiGd\nmVn35prZ9Oli4FDg28BLSQu+Ov0cODYv2poAngHcMlvAWg5AbkhnZta9qUcrLc7yFeAFkq4BHgaO\nAJB0HOkylsWSLgauyfufGRFLZwtYywHIzMy6V+UMKJ8LP3KGxz/ccft9wPuKxvQAVKGxZQ+XGq81\nMX01eQmqaMxWckne5evNeilBT5rQkK6Kv5v5Bx9barztDi41XDLW96mQlVTRPO6Y9Z5Vesx/ay3t\nO0bFh+BK5wHIzGxEeAAyM7Oh8ABkZmZDMdWwAaiSA+GStpH03RkeP1vSQVW8Zxlyz5+i+/6LpCMq\nTMfMrCtTyx4pvNVBA87EDkbBnkBmZrXVWr688FYHcx6Cy9/yXwSsB2wFfAx4D3AhcDvwBVJRunYR\n0KNI9dnmSfoSMAncGBFv6oj5uN5BEXFZLptzOankw1SOfQSpCOmBwAms3FPoZGAf0kVP/xoR5+Wr\nde8nld/ZBDgyIm6UdAzwqpzC1yPin/K+jwAbAwvJPYHyz/h5YMP8O3pbRPxQ0muAfyRV0X6IFb2O\nzMyGrmnngIrOgJ4OvIzUu+eDpEZuF0XEKaRB4KyI2J/Uq+ek/JodgHcDu5HqrD2jI96qegeRH9+b\nNKhslPsHta+qhRU9hfYEts59fZ4HHJ97AwGsERHPJw1YJ0p6Mmkg2ydvh0naLu97d0Qcwso9gY4F\n/jsiDgTeAnw09yI6lTQQvoxU68jMrDbKqgU3KEUXIVwZEcuAOyX9gTQL6ezt8+58+3JSF1SAn7d7\n9EhaAqgj3p7APpLaVarX7Kiq2o77O+DGfPv3rOjb035+T2D3PGuCNJi2q1+3m8UtBv6J1B/ouvwz\nIOlaYKdp8TrtCWyaZzwAa5FmSX+MiNs7YpiZ1UZdBpaiig5AnTOlMdIhtpl6+3T29Zl+9Vfn/cf1\nDgKQBCv36pmpb88jHX+eFRHTmyJ15tvOtdv+Q4+QDrst7oi7KStX1Pb5MzOrldZUpf2ASlf0Q3QP\nSROSNgHWBe7qeG4JcEC+3e7FA7CdpCfmwnTPIbU8aOumd9CqXA+8VNK4pIWSTu94bp923qRuqzfm\nn2ENSWuQDgveuHK4lXoCXU86NIikHST9ff6Z15e0Qa4Eu1cPOZuZVWZUV8EtJTUaugx4LyvPBE4E\nXivpMtJ5lnYdoJtIbVsXA4sj4icdryncO2hVImIR6ZDfYuAqoLMu/MLcrO4DwMm5IN5nSG3AryYV\nybt1WsjOnkCnA0+RdDVwJnBVbsB0Uo5xPl6AYGY1MzW1vPBWB3P2A8qr4HaMiH8YSEZ9yivbzo+I\nC4ady8N/vKfUIlSray24sZLjwepbC67s+oSVKLkWHM2pBdf3D77WHn9X+Id9cPFpJf+iu+dKCGZm\nI2LkFiFExNkDyKM0EXHEsHMwMxuGkRuAzMysGeqyuKCoOc8BmZmZVaEBZ2LNzGwUeQAyM7Oh8ABk\nZmZD4QHIzMyGwgOQmZkNhQcgMzMbCg9AZmY2FB6AzMxsKDwA1Yikp/bx2nUlbTnD47v2lxVIWlvS\nrpI27jfWtLh/VWY8M2sWl+Kpl0+R2ot3RdJbgHcBD0q6Azg8Im7LT3+k25iSXgd8iNQD6eic1y3A\npKQPRcT22VJ8AAARWklEQVTnesjxxGkPjQFHtVuj51bopZB0bER8fO49H/e6haSWIhuTKqpHx3PH\nR8QHu4w3H3gF8NuIuFrSq4C9gQA+GxF/6jbHVbzPuRFxeB+v3zkibsy3F5D+zp9BajnyyW7zzP2y\ndoqI7+bbrweeTvq5z4yIh3rI8TjgnIj4bbevnSXmGqS/nzsj4jJJB5E6PP8M+HJEdFVYTdJrZ3s+\nIr7Yc7IjygPQgEn6yCqeGgO26zHsEcBkRDwq6YXAhZJemlui91Jy/Q2ktutPIPVP2iMifiVpLVI/\npK4HIOCFwDzgDKD9H/sRYHpfpjK8DOh6AALOBX4O3AGcL+kjEXFOfu55QFcDEHAO8ACwuaRDgY2A\nb5AaIn4R+MtuE5T0S1Z0F27/3W7efjwitu02JvBRVnxJ+XiO/x/A/sBZQLcz1fOAH5CaU54GTAAX\nA88GvkAPPzdwJLCPpP8FTouIX/YQY7qzSX8/G+QvcVPApaSf+0DgdV3Ge0b+c1vgKcC1pKNMewH/\nQ/o7tw4egAbvAOAS4KczPPeSPuIuA4iIiyXdB/yXpEN4fGv0QrEi4gHgFknXRsSvcuwHJfVU7TAi\n9pb0BuA1wHsi4jpJb4qIL/QST9Ltq3hqDFivl5jAhhHxrhz/U8A3JE3kivC9DOSbRcQB+Zv2L4An\n58aGX5V0ZY85fhg4GDghIpbkXBdHxB49xoOVf7YdImK/fPsiSVf0EG+riHhlvv30iNg33z6/j5/7\ntoh4iaQDgU/kL0OXkRpf3t7+XXRpy4g4AEDSzyPiKfnxz0i6vNtgEfHOHOu/gGdHxLJ8fx5pQLdp\nfA5o8F5B+iZ4fkR8oXMDft9jzHOB7+X/lETEdcBfk76JPrOHeD/MnWGJiL+EdH5K0v9lRcv1rkXE\nZ4FDgTdIOgNY0Gss0izsxIjYbNq2KbCox5gTkp6dc32A1Db+ryS9hzR769YCSevkD6IT8+CDpM2B\nhb0kGBFnkP5u3yrpk5LWp7cvGZ3WkvQ0STsAd0h6cs5zfWCdHuLdLenvJG0CXCzpuTne/kCvhx1b\nABFxaUS8DHgtcDtptnt8jzEXSFpH0pOADSVtk/PcmB7/frI/A9bvuL8m8OQ+4o0sD0ADFhG/jogX\n5g+46f4KQNLWXcb8BHBIRDzY8dhNwHOBV/cQ829Jbcc7tYAvRsQxveTYkdddEXEU8CVSS/XHdBnz\n3aQPjbVneO5/esmN9HN/VNI6OdcHgBeRZgi9fIB8GPhajvUFAEn/B7iB1Mq+JxFxR0QcSfpW/U1g\nw15jZQ+SzvN9knT+q/2l5XzSz9CtVwFbAlcBfw9cIemnpENvvS48WWkGGhG3RcSZEfGmiPiLHmN+\nlHQk4lukL4bfkHQTcCNwao8xIZ13/b6k/5H0Q+CH9HZIePS1Wi1vNdsmJycvq3vMhuT4pgpy7Cvm\n5OTkgsnJyYkS482bnJx8bt1/7n7jTU5Ojg/g72ZscnJys8736ifm5OTkxpOTk5tMTk6Olf33MSqb\nZ0D1VEWv9rJjNiHHw0qO13fMiHh42uqqfuM9GhE3THu4dj93v/Hahy/LjDnDe7Qi4vZp79V1TEk7\nSroYuCAi7gSOkbRLP7mNKg9A9VRFl8CyYzYhxyYMkk3IsYqYTcix15inA8ew4nzXxaTVgDaNByAb\nZU0YJJuQYxUxm5BjrzGXRcTN7TsR8RPSEm+bxsuw66ku3+QGGa+qmGaDdk++mHttSbuRls2v6rKB\n1ZpnQPV0WQNiNiHHJgySTcixiphNyLHXmEcCWwB3klZr3gv8TZlJjYxhr4JYXbfJyckdJycnL56c\nnFyc7799cnJylzrFbEiOx8/w2Efzn8+pQ8wm5OifezA/t7eVt7FWq4rDpjaXfKX1W4FP5avldwA+\nExF71yVmnXOU9ArSNU77ksoDtc0Ddo6IbXrIrdSYTcixiphNyLGKmFXkOOp8Dmh4lkXEzZKAdKJS\nUr8nKsuOWdscI+L/Svo+8K+kCyjbpoCbZ37VYGM2IccqYjYhxypiVpHjqPMMaEgkfRW4CHgz8DbS\nicqndNTQGnrMhuS4DqmQ5vp0HK/vp/Jw2TGbkGMVMZuQYxUxJX01Ig7tNZ/ViWdAw3MkcCzpROVx\nwPWkqtZ1itmEHL8D/BLoLNPf77eqsmM2IccqYjYhxypi3i3pVFLJpceK90bEhX3EHEkegIbnT6Qa\nUQ+T/rH/mFSTq04xm5DjI9FHL5wBxWxCjlXEbEKOVcScDzyRVMy2rQV4AJrGA9DwnEua7l+X/3w9\naanmq2oUswk5XiDpxcA15JYUkFpH9JFj2TGbkGMVMZuQY+kxI+JISdsCO5F6X90YqTeXTeMBaHi2\niog9Ox+QdFXNYjYhxzfy+H/HLVJTsLrEbEKOVcRsQo6lx5T0TlINuWtJLUdOkvTZiPh0HzmOJA9A\nw7NE0nNiRVOxnYFemmpVGbP2OUbE9n3mU3nMJuRYRcwm5FhRzJcDu7WLzuaGhFcCHoCm8Sq4IZH0\nG9LV0g+QKlKsCdyVn25FxGbDjtmQHDtbVLct7+dDpeyYTcixiphNyLGKmJIWAXtFRCvfHweu6uf6\nuVHlGdDw/HlE9No4bVAxm5Djjh235wH7AKpZzCbkWEXMJuRYRcyvkDoULyZ9ydod+Ewf8UaWZ0BD\nkvuFbAR8FTgvIn5Vt5hNyHEV73FZRDyvzjGbkGMVMZuQYxkxc3vvZ5FmVj+IiFvLym2UeAY0JBHx\nQknrAS8GTpW0Aak18LkR8cc6xGxCjpL+mZUPn2wBrNtLblXFbEKOVcRsQo5VxJS0E/A+0iyqBfxY\n0skR8eN+8hxFroY9RBFxH6kn/a+ADYBdgUskvbouMRuQ449I1xL9ON8+n3RVez/KjtmEHKuI2YQc\nq4j5eeBzpJpw+wP/DnypvxRHkwegIZF0sqQfAseTrok5ICLeAOwH/GMdYjYhR+A80vVEu5AOeaxF\nWuDQj7JjNiHHKmI2IccqYt4VERdExF0RcWdEfBP4TZ85jiQfghueR4H9IuIP7QckPS0X6nxTTWI2\nIcezgD8AV5CuQN8POAB4Q4/5VRGzCTlWEbMJOVYR86eSPgVcQvqSvw9wW77Y1SV5OngAGjBJmwBP\nAA4Czpf0xPzUPNKJ+cmIuH6YMZuQY4etIuKvO+5/WVK/je3KjtmEHKuI2YQcq4i5Tv7zpdMePxSX\n5FmJB6DBexrwOmCSVLK9XX13it6PE5cdswk5ts2XtEVE3AYgaSvSoNaPsmM2IccqYjYhx9Jj5lI8\nC4AnRsTSPnMbaR6ABiwirgaulvTvEXHJTPtIelNEnDGsmE3IscN7gUuVegqNkwa0N3YZo+qYTcix\niphNyLH0mJIOA07Id3eUdBqwJCLO6TPPkePrgGqojtc1VB2vn5iSFpIqKrRI1RTuLSGXUmM2Iccq\nYjYhx7JjSroaOBD4dqSuvwuBKyJi937zHDVeBVdPY3PvMvSYtchR0jHAf0TEHyLiHuBLkv6unyTK\njtmEHKuI2YQcK4q5PCIeYcW1RQ/3k98o8wBUT1VMS8uOWZccDyMVf2x7WX6sH2XHbEKOVcRsQo5V\nxLxG0jnAVpL+Ebia1PTOpvE5IGu6NUgXs96d729O/7OzsmM2IccqYjYhxypingDsBfwPqSPqOyNi\ncV8ZjigPQPVUi8NbA47Xa8z3AtdJegiYIM3q39pnHmXHbEKOVcRsQo5VxLwiIvYjNbizWXgRwpBI\nGgOeAaxPxwdvRFyljn45w4zZhBw74m5KOvZ+d8djvayqqyxmE3KsImYTciwzpqQvkJZx30CaAQEQ\nEZ/qNbdR5RnQ8FxK+v3/vuOxFqlvSK8N2sqO2YQcAYiIO2Z4+DCg5w+ksmM2IccqYjYhx5Jj3pL/\nXL/jMX/Tn4EHoOFZIyL2rXnMJuQ4m7ocJhxkvKbEbEKOvcY8mRlm+fZ4HoCG52xJ7wBuBJa1H4yI\nq2oUswk5zqYuK/UGGa8pMZuQY68xLyWdS7p9Wpwq/o03mgeg4fkb0j/SzovT+v1HWnbMJuRoVjeD\nnOU3mgeg4RmP8nvElx2zCTnOpi6HZAYZrykxm5BjrzEHOctvNA9Aw/MdSa8nrZTp/Ef6kxrFrH2O\nkk6LiFVdtf6uOsRsQo5VxGxCjhXF9Cy/IC/DHhJJl8/wcKuf+mplx2xIjqeTLvibvuS150Gy7JhN\nyLGKmE3IsYqYkq6OiH16zWd14hnQkETEAQCS5kXEo3WM2YQcgR3z1tnOu0V/LZXLjtmEHKuI2YQc\nq4hZxZGDkeQZ0JBI2h/4BLAgIp4q6RTgyoi4uC4xm5BjjrkOsD2wHPhZRDzUa6yqYjYhxypiNiHH\nsmNWceRgVLkY6fCcTPqG9bt8/xPA+2sWs/Y5Svor0snek4CPADdJOrifBMuO2YQcq4jZhByriBkR\nB+SZ/gvbtz34zMyH4Ibn0Yi4S1ILICJuV2qIVaeYTcjxb4GdIuJBeOyb7LeBr9UoZhNyrCJmE3Is\nPWbnLB8oZZY/qjwDGp5fSjoZ2ETSYZLOA/o9Rlx2zCbkuLz9wQEQEffTcdy9JjGbkGMVMZuQYxUx\nqzhyMJI8AxqeNwKHkyrm7gF8E/iPmsVsQo6LJF0AXEm6ZmN/+l/uWnbMJuRYRcwm5FhFzCqOHIwk\nL0IYMEkvnu35iLhw2DGbkGNH3DFgb2BX0sqlJRFxbS+xqorZhByriNmEHKuIKelM4DZSk7tT8p/3\nR8Qb+slzFHkGNHiHTrvf/gYwlm/38kFcdswm5NjW7r1ydY+vH0TMJuRYRcwm5FhFzCqOHIwkz4CG\nSNKOgIAp4OaI+GndYtY9xyp6r5Qdswk5VhGzCTmWGbOqWf4o8wxoSCR9GtgFWEKaBbxb0rUR8fa6\nxGxCjszce6VfZcdsQo5VxGxCjmXGrGqWP7I8AA3PLhGxW/uOpHFgUc1iNiHHjWep41WXmE3IsYqY\nTcixtJgRcWT7dhVHDkaRB6DhCUlbRMRt+f6mwI9qFrMJOY5JeiMl1garIGYTcqwiZhNyLD1mFUcO\nRpUHoAGTtIQ0HZ8PLJX0s/zUdsAP6hCzCTl22BF4OvBa4FFWHPbotzZYmTGbkGMVMZuQYxUxqzhy\nMJI8AA3eKxsQswk5tp0MfBxYGBHKV533e11I2TGbkGMVMZuQYxUxqzhyMJJcCWHAIuLWiLg1330/\n8A1SyY/30jH9H2bMJuTY4f2kb6rt/+yfINX06kfZMZuQYxUxm5BjaTElLZF0A/BM0iz/x5J+DNxK\nmmXZNB6Ahucs4Fukf/gvBi7Lj9UpZhNyfDQi7iIfNomI20knfvtRdswm5FhFzCbkWGbMV5JWwv0F\nqbL2i/Mm4LA+cxxJPgQ3PBMR8Z8d978sqd8rpcuO2YQcV6otR7rq/Md9ZVh+zCbkWEXMJuRYWsz2\nDF/S1qRZ1bNIA9l3gff1meNI8gA0PI9IOhS4grRS5nnAwzWL2YQcm1Cvrgk5VhGzCTlWEfMs4NPA\n35MW3eyfH5v1QtXVkSshDImkLUknP3clfUtaArwvIn436wsHGLMJOZrVjaTLI3f+7Xjs0og4cFg5\n1ZVnQMPz2og4quYxm5CjWd1UceRgJHkAGp7NJL2ANAPovPjtwVW/ZOAxm5CjWd28jjTLP54Vs3x/\n6ZqBB6DheQlwMLAJafXNXaR/rNvWKGYTcjSrG8/yC/Iy7OE5lfQF4Jek6wTGgBNqFrMJOZrVzWaS\nXiBpA0lrtbdhJ1VHngENz7GkPvR3AUjaBLgE+PcaxWxCjmZ141l+QZ4BDc9vgbs77t8F/KJmMZuQ\no1ndeJZfkJdhD4mk84AdSH3ox0nXHywlfxhHxLuGHbMJOZrVjaQfAAdOn+VHxLOGm1n9+BDc8Px3\n3tqW1DBmE3I0qxvP8gvyDMjMrESe5RfnGZCZWbk8yy/IMyAzMxsKr4IzM7Oh8ABkZmZD4QHIzMyG\nwgOQmZkNxf8HoIJ8apkwGdgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f596c208510>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#------------------------------------ data exploration and visualization ------------------------------------#\n",
    "\n",
    "# Register dataframe as a temp table in SQL context\n",
    "df.createOrReplaceTempView(\"df1\")\n",
    "\n",
    "sqlStatement = \"\"\"\n",
    "    SELECT problem_type_1, problem_type_2, problem_type_3, problem_type_4,\n",
    "    error_count_1, error_count_2, error_count_3, error_count_4, \n",
    "    error_count_5, error_count_6, error_count_7, error_count_8, problemreported\n",
    "    FROM df1\n",
    "\"\"\"\n",
    "plotdata = spark.sql(sqlStatement).toPandas();\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# show histogram distribution of some features\n",
    "ax1 = plotdata[['problem_type_1']].plot(kind='hist', bins=5, facecolor='blue')\n",
    "ax1.set_title('problem_type_1 distribution')\n",
    "ax1.set_xlabel('number of problem_type_1 per day'); ax1.set_ylabel('Counts');\n",
    "plt.figure(figsize=(4,4)); plt.suptitle(''); plt.show()\n",
    "\n",
    "ax1 = plotdata[['problem_type_2']].plot(kind='hist', bins=5, facecolor='blue')\n",
    "ax1.set_title('problem_type_2 distribution')\n",
    "ax1.set_xlabel('number of problem_type_2 per day'); ax1.set_ylabel('Counts');\n",
    "plt.figure(figsize=(4,4)); plt.suptitle(''); plt.show()\n",
    "\n",
    "\n",
    "# show correlation matrix heatmap to explore some potential interesting patterns\n",
    "corr = plotdata.corr()\n",
    "sns.heatmap(corr, \n",
    "            xticklabels=corr.columns.values,\n",
    "            yticklabels=corr.columns.values)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering\n",
    "In the remaining part of the Notebook #1, we will demonstrate how to generate new features for this kind of use case. It is definitely not meant to be a comprehensive list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following cell, we created some time features, calculated the total number of warning_type1 (type2) occured for a macine on a particular day. We also identified some data quality issue that some event counts had negative values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Row(warn_type1_total=-430), Row(warn_type1_total=-430), Row(warn_type1_total=-419)]\n",
      "[Row(warn_type2_total=-1040), Row(warn_type2_total=-1040), Row(warn_type2_total=-1040)]\n"
     ]
    }
   ],
   "source": [
    "# Extract some time features from \"date\" column\n",
    "df = df.withColumn('month', month(df['date']))\n",
    "df = df.withColumn('weekofyear', weekofyear(df['date']))\n",
    "df = df.withColumn('dayofmonth', dayofmonth(df['date']))\n",
    "\n",
    "\n",
    "# warning related raw features\n",
    "warning_type1_features = list(s for s in df.columns if \"warning_1_\" in s) \n",
    "                            \n",
    "warning_type2_features = list(s for s in df.columns if \"warning_2_\" in s)\n",
    "\n",
    "warning_all = warning_type1_features + warning_type2_features\n",
    "\n",
    "# total count of all type1 warnings each day each device\n",
    "df = df.withColumn('warn_type1_total', sum(df[col_n] for col_n in warning_type1_features))\n",
    "# total count of all type2 warnings each day each device\n",
    "df = df.withColumn('warn_type2_total', sum(df[col_n] for col_n in warning_type2_features))\n",
    "\n",
    "print(df['warn_type1_total',].head(3))\n",
    "print(df['warn_type2_total',].head(3))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Row(warn_type1_total=0), Row(warn_type1_total=0), Row(warn_type1_total=1)]\n",
      "[Row(warn_type2_total=0), Row(warn_type2_total=0), Row(warn_type2_total=0)]\n"
     ]
    }
   ],
   "source": [
    "# We realized that the warning counts have negative values\n",
    "# Replace all the negative values with 0\n",
    "\n",
    "def negative_replace(num):\n",
    "    if num < 0: return 0\n",
    "    else: return num\n",
    "    \n",
    "negative_replace_Udf = udf(negative_replace, IntegerType())\n",
    "\n",
    "m = warning_type1_features + warning_type2_features\n",
    "for col_n in m:\n",
    "    df = df.withColumn(col_n, negative_replace_Udf(df[col_n]))\n",
    "\n",
    "# Then we have to re-calculate the total warnings again \n",
    "df = df.withColumn('warn_type1_total', sum(df[col_n] for col_n in warning_type1_features))\n",
    "df = df.withColumn('warn_type2_total', sum(df[col_n] for col_n in warning_type2_features))\n",
    "\n",
    "print(df['warn_type1_total',].head(3))\n",
    "print(df['warn_type2_total',].head(3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Variables \"categorical_1 to 4\" are integer type but in fact they are categorical features. In the following cell, we binned those variables and created four new columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'11-20', u'21-30', u'0-10', u'31-40']\n",
      "[u'2000-3000', u'morethan6000', u'5000-6000', u'3000-4000', u'0-2000', u'4000-5000']\n",
      "[u'600-800', u'200-400', u'400-600', u'0-200']\n",
      "[u'5000-10000', u'morethan20000', u'0-5000', u'15000-20000', u'10000-15000']\n"
     ]
    }
   ],
   "source": [
    "# Note: we can also use SparkSQL for this binning task\n",
    "\n",
    "def Cat1(num):\n",
    "    if num <= 10: return '0-10'\n",
    "    elif 10 < num and num <= 20: return '11-20'\n",
    "    elif 20 < num and num <= 30: return '21-30'\n",
    "    elif 30 < num and num <= 40: return '31-40'\n",
    "    else: return 'morethan40'\n",
    "cat1Udf = udf(Cat1, StringType())\n",
    "df = df.withColumn(\"cat1\", cat1Udf('categorical_1'))\n",
    "\n",
    "\n",
    "def Cat2(num):\n",
    "    if num <= 2000: return '0-2000'\n",
    "    elif 2000 < num and num <= 3000: return '2000-3000'\n",
    "    elif 3000 < num and num <= 4000: return '3000-4000'\n",
    "    elif 4000 < num and num <= 5000: return '4000-5000'\n",
    "    elif 5000 < num and num <= 6000: return '5000-6000'\n",
    "    else: return 'morethan6000'\n",
    "cat2Udf = udf(Cat2, StringType())\n",
    "df = df.withColumn(\"cat2\", cat2Udf('categorical_2'))\n",
    "\n",
    "\n",
    "def Cat3(num):\n",
    "    if num <= 200: return '0-200'\n",
    "    elif 200 < num and num <= 400: return '200-400'\n",
    "    elif 400 < num and num <= 600: return '400-600'\n",
    "    elif 600 < num and num <= 800: return '600-800'\n",
    "    else: return 'morethan800'\n",
    "cat3Udf = udf(Cat3, StringType())\n",
    "df = df.withColumn(\"cat3\", cat3Udf('categorical_3'))\n",
    "\n",
    "\n",
    "def Cat4(num):\n",
    "    if num <= 5000: return '0-5000'\n",
    "    elif 5000 < num and num <= 10000: return '5000-10000'\n",
    "    elif 10000 < num and num <= 15000: return '10000-15000'\n",
    "    elif 15000 < num and num <= 20000: return '15000-20000'\n",
    "    else: return 'morethan20000'\n",
    "cat4Udf = udf(Cat4, StringType())\n",
    "df = df.withColumn(\"cat4\", cat4Udf('categorical_4'))\n",
    "\n",
    "\n",
    "print(df.select('cat1').distinct().rdd.map(lambda r: r[0]).collect())\n",
    "print(df.select('cat2').distinct().rdd.map(lambda r: r[0]).collect())\n",
    "print(df.select('cat3').distinct().rdd.map(lambda r: r[0]).collect())\n",
    "print(df.select('cat4').distinct().rdd.map(lambda r: r[0]).collect())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### For variables \"fault_code_type_1 to 4\", if it is \"Unknown\" that means there is \"0\" fault code reported on that day for that machine, otherwise the count of fault code type 1 (2, 3, or 4) is 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-----------------------+-------+\n",
      "|fault_code_type_1_count|  count|\n",
      "+-----------------------+-------+\n",
      "|                      1|  50636|\n",
      "|                      0|2034466|\n",
      "+-----------------------+-------+\n",
      "\n",
      "+-----------------------+-------+\n",
      "|fault_code_type_2_count|  count|\n",
      "+-----------------------+-------+\n",
      "|                      1|  65676|\n",
      "|                      0|2019426|\n",
      "+-----------------------+-------+\n",
      "\n",
      "+-----------------------+-------+\n",
      "|fault_code_type_3_count|  count|\n",
      "+-----------------------+-------+\n",
      "|                      1|  86409|\n",
      "|                      0|1998693|\n",
      "+-----------------------+-------+\n",
      "\n",
      "+-----------------------+-------+\n",
      "|fault_code_type_4_count|  count|\n",
      "+-----------------------+-------+\n",
      "|                      1|  39593|\n",
      "|                      0|2045509|\n",
      "+-----------------------+-------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df = df.withColumn(\"fault_code_type_1_count\",F.when(df.fault_code_type_1!= \"Unknown\", 1).otherwise(0))\\\n",
    "       .withColumn(\"fault_code_type_2_count\",F.when(df.fault_code_type_2!= \"Unknown\", 1).otherwise(0))\\\n",
    "       .withColumn(\"fault_code_type_3_count\",F.when(df.fault_code_type_3!= \"Unknown\", 1).otherwise(0))\\\n",
    "       .withColumn(\"fault_code_type_4_count\",F.when(df.fault_code_type_4!= \"Unknown\", 1).otherwise(0))\n",
    "\n",
    "df.groupby('fault_code_type_1_count').count().show()\n",
    "df.groupby('fault_code_type_2_count').count().show()\n",
    "df.groupby('fault_code_type_3_count').count().show()\n",
    "df.groupby('fault_code_type_4_count').count().show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Feature engineering performance related features\n",
    "We first select 8 raw performance features to be normalized and then select 2 normalizers.  \n",
    "The idea behind this normalization is that device with more problem/error/fault reported might simply because it is used more frequently. Therefore, we need to normalize the problem counts by the corresponding usage counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# First, select the 8 raw performance features to be normalized\n",
    "performance_normal_raw = ['problem_type_1','problem_type_2','problem_type_3','problem_type_4',\n",
    "                          'fault_code_type_1_count','fault_code_type_2_count',\n",
    "                          'fault_code_type_3_count', 'fault_code_type_4_count']\n",
    "\n",
    "# Then, select 2 normalizers\n",
    "performance_normalizer = ['usage_count_1','usage_count_2']\n",
    "\n",
    "# Normalize performance_normal_raw by \"usage_count_1\"\n",
    "df = df.withColumn(\"problem_type_1_per_usage1\", F.when(df.usage_count_1==0,0).otherwise(df.problem_type_1/df.usage_count_1))\\\n",
    "       .withColumn(\"problem_type_2_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.problem_type_2/df.usage_count_1))\\\n",
    "       .withColumn(\"problem_type_3_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.problem_type_3/df.usage_count_1))\\\n",
    "       .withColumn(\"problem_type_4_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.problem_type_4/df.usage_count_1))\\\n",
    "       .withColumn(\"fault_code_type_1_count_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.fault_code_type_1_count/df.usage_count_1))\\\n",
    "       .withColumn(\"fault_code_type_2_count_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.fault_code_type_2_count/df.usage_count_1))\\\n",
    "       .withColumn(\"fault_code_type_3_count_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.fault_code_type_3_count/df.usage_count_1))\\\n",
    "       .withColumn(\"fault_code_type_4_count_per_usage1\",F.when(df.usage_count_1==0,0).otherwise(df.fault_code_type_4_count/df.usage_count_1))\n",
    "\n",
    "# Normalize performance_normal_raw by \"usage_count_2\"\n",
    "df = df.withColumn(\"problem_type_1_per_usage2\", F.when(df.usage_count_2==0,0).otherwise(df.problem_type_1/df.usage_count_2))\\\n",
    "       .withColumn(\"problem_type_2_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.problem_type_2/df.usage_count_2))\\\n",
    "       .withColumn(\"problem_type_3_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.problem_type_3/df.usage_count_2))\\\n",
    "       .withColumn(\"problem_type_4_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.problem_type_4/df.usage_count_2))\\\n",
    "       .withColumn(\"fault_code_type_1_count_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.fault_code_type_1_count/df.usage_count_2))\\\n",
    "       .withColumn(\"fault_code_type_2_count_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.fault_code_type_2_count/df.usage_count_2))\\\n",
    "       .withColumn(\"fault_code_type_3_count_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.fault_code_type_3_count/df.usage_count_2))\\\n",
    "       .withColumn(\"fault_code_type_4_count_per_usage2\",F.when(df.usage_count_2==0,0).otherwise(df.fault_code_type_4_count/df.usage_count_2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Similar to what we did for \"categorical_1 to 4\", in the following cell we binned performance related features and created new categorical features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'0', u'>1', u'1']\n",
      "[u'>10', u'0', u'6-10', u'1-5']\n"
     ]
    }
   ],
   "source": [
    "# Define the list of performance related features which we would like to perform binning\n",
    "c_names = ['problem_type_1', 'problem_type_3', 'problem_type_4',\n",
    "           'problem_type_1_per_usage1','problem_type_2_per_usage1','problem_type_3_per_usage1','problem_type_4_per_usage1',\n",
    "           'problem_type_1_per_usage2','problem_type_2_per_usage2','problem_type_3_per_usage2','problem_type_4_per_usage2',\n",
    "           'fault_code_type_1_count', 'fault_code_type_2_count', 'fault_code_type_3_count', 'fault_code_type_4_count',                          \n",
    "           'fault_code_type_1_count_per_usage1','fault_code_type_2_count_per_usage1',\n",
    "           'fault_code_type_3_count_per_usage1', 'fault_code_type_4_count_per_usage1',\n",
    "           'fault_code_type_1_count_per_usage2','fault_code_type_2_count_per_usage2',\n",
    "           'fault_code_type_3_count_per_usage2', 'fault_code_type_4_count_per_usage2']\n",
    "\n",
    "# Bin size ('0','1','>1') for most of the performance features because majority of the values fall into the range of 0 to slightly more than 1.\n",
    "def performanceCat(num):\n",
    "    if num == 0: return '0'\n",
    "    elif num ==1: return '1'\n",
    "    else: return '>1'\n",
    "    \n",
    "performanceCatUdf = udf(performanceCat, StringType())\n",
    "for col_n in c_names:\n",
    "    df = df.withColumn(col_n+'_category',performanceCatUdf(df[col_n]))\n",
    "\n",
    "# Use different bin for \"problem_type_2\" because we saw a larger spread of the values\n",
    "def problem_type_2_Cat(num):\n",
    "    if num == 0: return '0'\n",
    "    elif 0 < num and num <= 5: return '1-5'\n",
    "    elif 5 < num and num <= 10: return '6-10'\n",
    "    else: return '>10'\n",
    "\n",
    "problem_type_2_CatUdf = udf(problem_type_2_Cat, StringType())\n",
    "df = df.withColumn('problem_type_2_category',problem_type_2_CatUdf(df['problem_type_2']))\n",
    "\n",
    "\n",
    "print(df.select('problem_type_1_category').distinct().rdd.map(lambda r: r[0]).collect())\n",
    "print(df.select('problem_type_2_category').distinct().rdd.map(lambda r: r[0]).collect())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### One hot encode some categotical features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Row(problem_type_1_category_encoded=SparseVector(2, {0: 1.0})), Row(problem_type_1_category_encoded=SparseVector(2, {0: 1.0})), Row(problem_type_1_category_encoded=SparseVector(2, {0: 1.0}))]\n",
      "CPU times: user 747 ms, sys: 199 ms, total: 946 ms\n",
      "Wall time: 13min 22s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# Define the list of categorical features\n",
    "\n",
    "catVarNames = ['problem_type_1_category', 'problem_type_2_category',\n",
    "               'problem_type_3_category', 'problem_type_4_category',\n",
    "               'problem_type_1_per_usage1_category', 'problem_type_2_per_usage1_category',\n",
    "               'problem_type_3_per_usage1_category', 'problem_type_4_per_usage1_category',\n",
    "               'problem_type_1_per_usage2_category', 'problem_type_2_per_usage2_category',\n",
    "               'problem_type_3_per_usage2_category', 'problem_type_4_per_usage2_category',\n",
    "               'fault_code_type_1_count_category', 'fault_code_type_2_count_category',\n",
    "               'fault_code_type_3_count_category', 'fault_code_type_4_count_category',\n",
    "               'fault_code_type_1_count_per_usage1_category', 'fault_code_type_2_count_per_usage1_category',\n",
    "               'fault_code_type_3_count_per_usage1_category', 'fault_code_type_4_count_per_usage1_category',\n",
    "               'fault_code_type_1_count_per_usage2_category', 'fault_code_type_2_count_per_usage2_category',\n",
    "               'fault_code_type_3_count_per_usage2_category', 'fault_code_type_4_count_per_usage2_category',\n",
    "               'cat1','cat2','cat3','cat4']\n",
    "    \n",
    "    \n",
    "sIndexers = [StringIndexer(inputCol=x, outputCol=x + '_indexed') for x in catVarNames]\n",
    "\n",
    "df_cat = Pipeline(stages=sIndexers).fit(df).transform(df)\n",
    "\n",
    "# Remove columns with only 1 level (compute variances of columns)\n",
    "catColVariance = df_cat.select(\n",
    "    *(F.variance(df_cat[c]).alias(c + '_sd') for c in [cv + '_indexed' for cv in catVarNames]))\n",
    "catColVariance = catColVariance.rdd.flatMap(lambda x: x).collect()\n",
    "catVarNames = [catVarNames[k] for k in [i for i, v in enumerate(catColVariance) if v != 0]]\n",
    "\n",
    "# Encode\n",
    "ohEncoders = [OneHotEncoder(inputCol=x + '_indexed', outputCol=x + '_encoded')\n",
    "              for x in catVarNames]\n",
    "ohPipelineModel = Pipeline(stages=ohEncoders).fit(df_cat)\n",
    "df_cat = ohPipelineModel.transform(df_cat)\n",
    "\n",
    "drop_list = [col_n for col_n in df_cat.columns if 'indexed' in col_n]\n",
    "df = df_cat.select([column for column in df_cat.columns if column not in drop_list])\n",
    "\n",
    "print(df['problem_type_1_category_encoded',].head(3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Use PCA to reduce number of features\n",
    "In Notebook #2, we will perform a series of rolling computation for various features, time windows and aggregated statistics. This process is very computational expensive and therefore we need to first reduce the feature list.  \n",
    "In the dataset, there are many warning related features and most of them have value of 0 so quite sparse. We can group or find correlations among those warning features, reduce the feature space for downstream work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "147"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## check the number of warning related features\n",
    "len([col_n for col_n in df.columns if 'warning' in col_n])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.0483107423499,0.0278314776592,0.0254509402103,0.0236091224214,0.0200247085239,0.0188475892993,0.0183962875162,0.0177510201652,0.0167993077022,0.0163456905185,0.016006950633,0.0155092512672,0.0144804538583,0.0140982528803,0.0139322569416,0.0137500740086,0.0136491126177,0.0129200916821,0.0122476113985,0.0115038758841]\n",
      "[Row(pca_1_warn=0.0), Row(pca_1_warn=0.0), Row(pca_1_warn=0.0)]\n",
      "CPU times: user 191 ms, sys: 39.2 ms, total: 230 ms\n",
      "Wall time: 2min 43s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "#----------------------------- PCA feature grouping on warning related features --------------------------#\n",
    "\n",
    "df = df.withColumn(\"key\", concat(df.deviceid,lit(\"_\"),df.date))\n",
    "\n",
    "# step 1\n",
    "# Use RFormula to create the feature vector\n",
    "formula = RFormula(formula = \"~\" + \"+\".join(warning_all))\n",
    "output = formula.fit(df).transform(df).select(\"key\",\"features\") \n",
    "\n",
    "\n",
    "# step 2\n",
    "# Before PCA, we need to standardize the features, it is very important...\n",
    "# Note that StandardScaler does not work for sparse vector unless withMean=false\n",
    "# OR we can convert sparse vector to dense vector first using toArray\n",
    "scaler = StandardScaler(inputCol=\"features\", outputCol=\"scaledFeatures\",\n",
    "                        withStd=True, withMean=False)\n",
    "\n",
    "# Compute summary statistics by fitting the StandardScaler\n",
    "scalerModel = scaler.fit(output)\n",
    "\n",
    "# Normalize each feature to have unit standard deviation.\n",
    "scaledData = scalerModel.transform(output)\n",
    "\n",
    "\n",
    "# step 3\n",
    "pca = PCA(k=20, inputCol=\"scaledFeatures\", outputCol=\"pcaFeatures\")\n",
    "model = pca.fit(scaledData)\n",
    "result = model.transform(scaledData).select(\"key\",\"pcaFeatures\")\n",
    "\n",
    "# to check how much variance explained by each component\n",
    "print(model.explainedVariance)  \n",
    "\n",
    "\n",
    "# step 4\n",
    "# convert pca result, a vector column, to mulitple columns\n",
    "# The reason why we did this was because later on we need to use those columns to generate more features (rolling compute) \n",
    "def extract(row):\n",
    "    return (row.key, ) + tuple(float(x) for x in row.pcaFeatures.values)\n",
    "\n",
    "pca_outcome = result.rdd.map(extract).toDF([\"key\"])\n",
    "\n",
    "# rename columns of pca_outcome\n",
    "oldColumns = pca_outcome.schema.names\n",
    "\n",
    "newColumns = [\"key\", \n",
    "              \"pca_1_warn\",\"pca_2_warn\",\"pca_3_warn\",\"pca_4_warn\",\"pca_5_warn\",\n",
    "              \"pca_6_warn\",\"pca_7_warn\",\"pca_8_warn\",\"pca_9_warn\",\"pca_10_warn\",\n",
    "              \"pca_11_warn\",\"pca_12_warn\",\"pca_13_warn\",\"pca_14_warn\",\"pca_15_warn\",\n",
    "              \"pca_16_warn\",\"pca_17_warn\",\"pca_18_warn\",\"pca_19_warn\",\"pca_20_warn\",\n",
    "             ]\n",
    "\n",
    "pca_result = reduce(lambda pca_outcome, idx: pca_outcome.withColumnRenamed(oldColumns[idx], newColumns[idx]), \\\n",
    "                                        xrange(len(oldColumns)), pca_outcome)\n",
    "\n",
    "df = df.join(pca_result, 'key', 'inner')\n",
    "\n",
    "print(df['pca_1_warn',].head(3))\n",
    "\n",
    "warning_drop_list = [col_n for col_n in df.columns if 'warning_' in col_n]\n",
    "df = df.select([column for column in df.columns if column not in warning_drop_list])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f5976291f50>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEqCAYAAADQ2b3cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYXGWV/z/dTUICigFBVhWQ1AlgBBGQGBRQHAWFUVF5\nBBUQFxRGQMEfiCOLzLA4yjoIDCouoMgwKERR1gSBsI5EQDggkgEBMYCEJQkJ6f798b5FV4qq6tu3\nqm+9dfv7eZ77dNWte+45VXX7nnqX8337hoaGEEIIIdqhv9sBCCGE6H2UTIQQQrSNkokQQoi2UTIR\nQgjRNkomQggh2malbgfQKxzQt+Gop72duujesQilYwz19XU7hJ6nL8dsyDyfex4/RfsqI3n/RyZP\nmtT2P9do7jlnD83v+j+zWiZCCCHaRi0TIYRIkIGutzVGR1LJxMy+DHwHWN3dn+92PEII0S0m9vdW\nNkkmmZjZp4G1gce6HYsQQnSbgR4b08yVTMxsX+D9wGrABsApwF+BfweWAz9391PNbG/gX+K+e9z9\n8y1Oe6m7PxdtWvleBZjr7luY2XrAI8A67r7AzOYB2wI/iHGtChzj7rPMbDZwdzzNk8AUwICNgUPc\n/YrRfg5CCDFW9Fo3VzsD8JsDuwPvBo4HzgJ2BWYCO5vZZMLN/P3uPhOYZmbTm53M3Z/L4tTdFwHP\nmtmU6Ot6YDszW4uQJF4DXOnuOwAfB46tMb/b3Q+Kjzdw912Ag4EvZHzPQghRCAN9fZm3FGinm2uO\nu78EPGlmC4E+d18QX/sggJk9DfzKzAA2BV7bTrA1/B54OyGZnAbMICTGOcA/gG3M7PPAYJ3PW2se\n3xD//pWQgIQQIhnGU8uk1raPcON+GTObCPwnsGdsJdzShq96ZgPbAVOBywitpJnAdcBewBrAO4EP\n19ktrXn8Us3jHvvahBBlp9daJu0kkxlmNmBmaxK6syaY2fpm1mdms4BXAy+5+9/M7PXA1sDEDsQM\ncBOwPbDE3QeBIWArQsJaE3go7v9IB30KIURhTOjry7ylQDvJZD5wMXAtcBRwAPDfhBv9Ne7+FHCV\nmd0GHA2cDJxiZhManczMjoqD5OsAV5jZyc0cx2nDqwK3xV13A4PuvhS4BNjNzK4BXgD+ambfbON9\nCiFE4Qz0Zd9SoC/P4lhxNteb3f2wjkeUKJJTEY2QnEq56aacyrdfVcn8RRz+/P1d/2cutM7EzN4A\n/LjBS3Pc/egGx/8PYfyjloXu/s9jEV8r8iSGQ1bZdNQ2pyy+b9Q2eSnyplFk4tLNcJiyfsd5yPNZ\ndPNaSqXFkZVcycTdz89p9zCw4yiO/0geP0II0eukMrCelZQq4F8P/BCYACwDPunuf+tuVEII0R16\nTU4lJdXg44Fz4zTiS4GvdDkeIYToGr02AJ+SnMqXgCXx8QLCVN9GviWnIoQoPakkiaykJKfygrsv\nN7MB4EDgwibHSU5FCFF6eq1oMSk5lZhIfgJc6+7XtDhUcipCiFLTay2TdpJJVjmVLWIV/KwM5/wh\n8IC7HzvCcbMJCWQqYWxlP8J7uZwV5VTWAG6vsZOcihCiJ+hki8PMTiFIUA0BB7v7bTWvHQh8kjAc\ncbu7H5LHRzJyKnF8ZWmjepMGSE5FCFFqJvb3Zd5aYWY7AFPdfQawP3B6zWurAYcD73T37YHNzGy7\nPPEmI6dCGCfZysxmx+2sZo4lpyKEKDsdnM31HuCXAO5+L7B6TCIQemuWAq8ys5WAVYCn88QrOZWM\nLFm8eNQflCrgh1EF/DBllTgpYwV8XiZNntz2h3HZum/OHPDuj9/d1J+ZnQv82t1/FZ//Htjf3e+P\nz/cGzgAWE2bifjVPvJJTGUPyJIZDJ0/L5SuP3Evef/6iZClSvzkViT6L9um1JN4/dt/5yyeOLZSv\nAxXgWeBaM9vC3eeN9qTJyKmY2Qzg24Tq9xeBT0lORQgxXunr3HSuxwhq7FXWAx6PjzcF/uLuT8LL\nrZa3AaNOJilVwH8F+LS77wTMBT7X5XiEEKJrDEwcyLyNwJXARwHMbCvgsZpl0ucDm8a6QAgTpR7I\nE28yFfDu/rF47j5gfYbrQOp9qwJeCFF6OtUycfebzOwOM7uJUMJxYLyHL3T3S83s28B1ZvYScJO7\n/z6Pn3bGTDYH3kq4Kc8DFgHvIMwE+JWZncNwBfwzZna9mU1397uandDM3k+YtnYv8NNGx7j7IjNr\nVAF/MytWwP/IzDYmzDir1rjc7e5nm9kxxAr46PMAQMlECJEM/R2sWnT3I+p2zat57RzgnHZ9JFUB\n7+6/tXDwicARhJZOI1QBL4QoNX39KY1CjEw70WatgN8z6mTd0upkZvZhAHcfItSKbN/i8NmEas6p\nwGWEVtJM4DpWrID/cJ2dKuCFED1B/0Bf5i0FkqmAB44xsy3j47cD3uJYVcALIUpN30Bf5i0FUqqA\n3x84y8yuJ3STndDMsSrghRBlp4OzuQpBFfAZyVMBn6dIqqxFi3lIPb68qACxd8h7LXWiAv73M2Zm\ndv7OuTd2/aJSBXxi5EkKkE+6Ja+voiqJJfcyTC8koKLeV1mvi3r6B3prAD6ZCvgmx6sCXggxLkll\nLCQryaU+M3ufmaXdTyGEEGNMrw3AF9rNNRJmNgk4kmHdGCGEGJeMi26usZBTiXydUJvy7Ra+Jaci\nhCg9AxN6K5m0E+3mwO7Au4HjgbOAXQnFgztH4bCqnMpMYJqZTW92MjOrEJb4vbiVU3dfBDSSU1mL\nFeVUdgA+DtQuAXy3ux8UH2/g7rsABwNfGN1bF0KIsaVvoD/zlgIpyamcAnw5o2/JqQghSk0qle1Z\nSUJOxczWB6YBF0TBxnXNbE4L37ORnIoQosSMpwH4GWY2AKxO6M5aEpPCY8DlwD5klFNx90eBN1Wf\nm9n8mICacRNwOPCsuw/G2V9bAd8gjJk8FPdLTkUI0ZOk0n2VlZTkVDIjORUhRNkZmNCfeUsByalk\npCg5lbzVvWWsgC8SVcC3jyrgh5k8aVLbX9ifPr1b5je62Y8v7/oFIjmVjBR10ee9ePMkhjwJKK+v\n1G8aeUn9Bpr65576d9zN+FIZC8mK5FSEECJBem3MJJkKeDM7H3gb8FTc9W13/3X3IhJCiO7Rayst\nJpNMIke6+6yRDxNCiHIjOZX25FSy+JacihCi9PRPTO23fmuSkVOJHGRm15rZz+NywK9AcipCiPFA\nX39/5i0FUpJT+QnwlLvfaWZHAMcABzU5VnIqQohS0zeQxnK8WWknmWSVU9kiVsG3HAtx92tqnl4G\nfK/F4bMJCWQq8BVgP8J7uZwV5VTWAG6vsZOcihCiJ+i12VztRDvDzAZid9SqwAQzW9/M+mLieDUZ\n5VQAzOwSM9s4Pt2R4fGNRtwEbA8scfdBoCqncguwJlFOBZCcihCiJ+nv78+8pUBKcipnAhdFgccP\nsOJYxwpITkUIUXZ6TYJecioZWbxkyag/qCIr4PP4UgX8MEV+7qqALz+TJk9uu+v80WO/kPmLWP/o\nc7reVS85lcQo8qaRV5urKB2wIm/wRX7uRZL6D5rUk103v99UWhxZkZyKEEIkyLhIJmNBHEv5EbAJ\n8BzwUXf/R3ejEkKI7tBrFfApRfs5YIG7bwtcRJjaK4QQ45JxUbQ4RnIquxFmfeHu57bwLTkVIUTp\nkZxKfjmVDYFdzGx2lFOpH3gHJKcihBgfjIuWSaTTcip9gLv7sWb2DeBIwjrvjZCcihCi1PRLTiUw\nWjkV4AlCMgD4HS2KFpGcihCi5HRyNpeZnQJsR1ALOdjdb2twzAnADHffMY+PZORUgCsI4zAQFsny\nFsdKTkUIUWo6VQFvZjsAU919BrA/cHqDYzYD3tVOvCnJqZwO7GpmNwAfAk5s5lhyKkKIstPBMZP3\nAL8EcPd7gdXNbLW6Y75DuI/npp1urgcbyKnMqH3i7vvWvf7dZieLA+sfy+rc3d9R8/gbNY/nA2+p\nOfSC+Pe4mmOOqXl8N6MopBRCiCLo4GyudYA7ap4viPuehZdn584hNBByIzmVMaRIqYgiZTPySKMU\nJcFSNEVJe/SCtEzqpC59U88YztJ6+YOIs2b3A3YG1m/npJJTEUKIBOnr79hsrscILZEq6wGPx8fv\nBtYizJBdGXiTmZ3i7oeO1kkyVTFmdjHhTUFojdzciTXjhRCiJ+lcMrmSMDv2HDPbCnjM3Z8DcPf/\nJox1Y2YbAufnSSSQUDJx95fHS8zsB8B5XQxHCCG6S4e6udz9JjO7w8xuIpRwHBjHSRa6+6UdcUJa\ncirVcxswxd1vbfK65FSEEKWnk2vAu/sRdbvmNThmPm1MRkpJTqXKwcAZzV6UnIoQYlyw0sTsWwKk\nJKdSrZrf3t2/NIJvyakIIUpNKppbWWkn2qxyKnvGVsItGc65Ayve8JsxmyANMBW4jNBKmglcx4py\nKh+us5OcihCiN+gfyL4lQEpyKgDb0KAvrwGSUxFClJtxlEzm01k5FYB1gb+P5FhyKkKIstNrEvR9\nQzmqXeNsrjc3kFMpLYuXLCmkLLjI6uMiq6rLWgFfFEV+V0VSpEpEkRXwkydNatvZi7MvyPxGV95x\n76531UtOJTF6QTYjT4xFSbAAnLL4vlx2RZH6d5WX1BNXkZJDnaCTU4OLQHIqQgiRIol0X2UlmQp4\nM3sXoehxGWGs41Pu/o/uRiWEEF0ikYH1rKSU+r4L7O/uOxEG8VVIKIQYt/T1D2TeUiAlOZUnGS4w\nXJ0mKy1KTkUIMS7osW6ulORUDgV+aWZOKDg8v9FBklMRQowH+laamHlLgZTkVM4APuzuN5rZfwBf\nosFaxRHJqQghys04apl0Wk7lLe5+Y3x8FaFivhmzkZyKEKLE9A0MZN5SICU5lb+Z2Wbx8TbAAy2O\nlZyKEKLcSE4lt5zKAcB/xYHyrWgtQy85FSFEuemxZCI5lYwUJaeSl9RlWIqSYIFyVsDnJfUK+CLl\nVPKQ9/PrhJzK4J9vzvxG+zfZrutd9ZJTGUNSl5fIS1HvK29SOHTytFHb5NUBK+pmXVZtrtTj6yp9\nvTUAn6tlMh7J0zLRP0p75L2Bpp5MitSI0jXYHl1tmfzl9uwtk423Hl8tk1aY2TTgXMJg+v3AF+PU\nYyGEGHcM9VjLJKVoTwJOiNOIHyYUHAohxPikrz/7lgApyalMZbio8HeEosULG/iWnIoQovx0Uf4+\nDynJqdwFfCA+fh+wdqODJKcihBgPDA2slHlLgZTkVA4DvhdbPXNoXZUuORUhRLlJpPsqK+0kk6xy\nKlvEKvhZrU7m7o8wnITeR1gPvhmzCQlkKvAVYD/Ce7mcFeVU1gBur7GTnIoQojfosWSSjJyKmR1r\nZtVurv0IiaEZklMRQpSbHhuAT0lO5ULg6Hj8Y+7+62aOJacihCg7Q339mbcUkJxKRlS0WDwqWmzP\nT15fYphuFi0ue+KhzF/ehLU36npXveRUSkLqGkxFkicxFKkDVqQeVRm1r4r8LLpKIgKOWcmVTNz9\n/Jx2DwM7ApjZl4HvAKvHbitiXcohhMH8c939I3n8CCFEr5NK91VWuhKtmX2aUEfyWM2+VYFvAjsT\nEs6hZlbfKhFCiPFBf3/2LQFGbJmMUbX7pe7+XLSp8nbgNndfGP3eSKgjecWsLjP7LPBadz/JzL4O\nzHD33cxsBvB54FTCtORlhFbOx2L8PwWeB86Mx5xLmI68MrCzuz830uchhBCFUNKWSUer3ZvctNcB\nFtQ8/zvNa03mEJbtBXgbw3Ui1aV7Xwf8i7vvBNwIVJPWW4G93X0WIZHe6+7vAh4C3tMsXiGEKJwe\nmxqcdcyk09XuWWg6WubuD5jZ682sD5gA3GdmFUIyOZigu3VS1PFaD7ggmj4YpyxX+X38qyp4IURS\nDPWnIZOSlawpLWu1+55RE+uWHLE8RmidVFmfmjGVBtwP7ALcB9wMvIMg+PgwQWLltBjLOTU2S+vO\noSp4IUSalLRlMsPMBoDVCd1ZS8yserO/HNiHUVS7N+EW4Lwo4PgSoZVxSIvj5xCkVP6TULz4A+Ce\n+NqawINmtjKhO+7mUcYihBDdpYNTmc3sFMLQwBBwsLvfVvPazgyPgf/G3b+Vx0fWlDafDla7m9lR\nURJ+HeAKMzvZ3RcDRxDk568Gjq0OxjdhDmEMZ667/xWYRtDsAjgD+GWM+QxCslM3lhCid+hQy8TM\ndgCmuvsMYH/g9LpDTgf2IPyA/ycz2yxXuCNVwI/HavdGpF4BX8aixSKrvossWsyDihaH6YWixU5U\nwC9Z9ELmNzpplVWb+jOz44CH3f28+Pw+YFt3f9bMNgZ+7O7bx9eOBJ539zNGG++YjfCMttq9yTm2\nJbRy6rnI3b/XTnxCCJE0nRsLWQe4o+b5grjvWRrPon1THicjJpNOVLvnxd1vbfccnSJ1CZE86Nfu\nMHlbGHl0wIpqzfQC0ilrzuDYzQlqdeLcTrs296yJnMrqwM8IzayPdis2IYToNoOdS371M2XXAx5v\n8tpIs2ibkoycSuRshldAFEKIccvQKLYRuBL4KICZbUVY4uM5AHefD6xmZhua2UqEusEr88SbkpwK\nwGcJFe1bjhCT5FSEEKVmsEMNE3e/yczuMLObCPfDA+N9faG7Xwp8kdAjBGE8+v48frJ2c21OkCKZ\nAswDFhGKBKtV7+cwLKfyjJldb2bT3f2uJm+u4U07Jpgs8cxheGC+lZzKH+JMhr0J9TBvBd7g7k+Z\n2ZkEOZWTzeznBDmVX2ZxLoQQY02etaaa4e5H1O2aV/Pa9YRl0NsiZTmVpkhORQhRdjrVMimKlORU\nRovkVIQQpWX5UPYtBVKSUxktklMRQpSWTnZzFUHWZDKfIE2yCUFO5VGCnArAL+IYRFVOZR7Dcipb\nuvuy+pOZ2VHAexmWU5kLHAlcQ+iiWj/KrRzn7tc2iWkO8D3gkzGJTWO4SLIqp/JgfHwmcFHG9yqE\nEF1ncORDkkJyKhlZsnhx0j8TUpelKLI4rUiKKlpM/bsqkl4oWpw0eXLbF+/jz2SXU1l3SnM5laKQ\nnEpGUr+xlfFmnXrVPORLDHkS0KmL7h21DaRfYV7kNZj69V5Prw3Aj9gyEYE8Qo9Fkvo/cll/7eah\nyGSSh7ImkyLphNDjw08/n/kDfcMar+r6B5manMqewFcJ3YXXuPtR3YpPCCG6SeK/v15BMnIqsSbk\nJELx4AzC2vK5dPWFEKLXGRwayrylQDJyKu6+KFbNPxf9PkWTwkfJqQghyk4aKSI7WVsmmwO7E1Y2\nPB44i1C/MZPQgpjMsJzKTGCamU1vdrJWcioA0XZDmteHzCEsQQmt5VR2Am4kyKlAkFPZ291nERLp\nve7+LuAhQotICCGSYHAo+5YCycmpmNlU4EJgr0Y1KiA5FSFE+Umk9yozScmpmNkGhGLDfdz9zhEO\nl5yKEKK0LB8ayrylQGpyKt8Hvuju/5vhWMmpCCFKSyrdV1lJSU7lPOCdwHE1MvTfdffLmsQkORUh\nRGlJpMGRGcmpZERFi+2hosVhVLTYvq/U6UTR4l2PL8z8gU5f9zVd/yAlp5KRom6Gef+5UtdgKqN2\nWF7yJIZDVtk0l68idcCKosjropsk/jW8AsmpZKQooceythaUTNqjrMkk9esiL51omdz56DOZ3+iW\n60/peqZMTU7lm4QZWn3ALHc/vlvxCSFEN1mWyqpXGUlJTmVDYLq7zyDUi+xjZut1Iz4hhOg2pZsa\nXKCcynyC7AmEKciDwLNNYpKcihCi1KSiuZWVpORUAMzsNEK9yLeq3V8NkJyKEKLULB/MvqVA1mQy\nx91fcvcngYWEAsUF7r7c3T/o7ouBqpzKHNqQU3H3g4FpwOFmtlGTYx4AmsmpzAaeAP49xvKJmlgk\npyKE6Al6TTU4GTmVqLW1NYC7/4PQotimhYnkVIQQpWXZ4FDmLQVSklNZC/heHPcYInRfndvieMmp\nCCFKy/JEkkRWsrZM5hPkVK4lyKkcQJBTuYmwIuJTQFVO5WiG5VQmNDqZmR1lZrMZllM5Oepx/Q+h\nRTIX+M0IYo9zCGM4c939r4SusdnxtaqcysXx8T6oG0sI0UP0WjeX5FQyoqLF9ki9OE1Fi8Poumif\nThQtzrr3icxv9IObrt31bnrJqWSkjDeoIjWYitRtSl0jKk98eZICpK8DVsbvqlOk0uLIyojJxN3P\nz3PiOBC+Yx7bmnPc2u45hBCiF+m1MZOk5FRqXvsZ8KK779uN2IQQotukMksrK8nIqdS89l7gTYUH\nJYQQCdFrA/DJyKlEXysD3yBU2X+kRUySUxFClJrBkrZMipJTOZKwemJDTa4aJKcihCg1y4eybymQ\ndcxkjru/BDxpZguBPndfEF/7IICZVeVUIIeciplNBbZ292PMbMdWx7r7A7FivpGcysHAFOAkM1sF\nWA+4IJpKTkUI0ROk0n2VlazJJKucyhaxCn5Wjlg+ALzBzG4mdEmtZWZfc/dGU4NhRTmVW6mRUzGz\nHwEnuftvzeww4FXRRnIqQoieYNkYKjjGgvLzgTcShib2c/e/NDk204SoZORU3P1UwjgGsWWyb4tE\nApJTEUKUmDHuvtoLeMbd9zazfwJOAPasP6hmQtSfRjphMnIqGeOoRXIqQojSMsazud4DXBofX00Y\nIliBuglRIyI5lYwsXrJk1N9Y6tW9eSmqKlgV8MPkjS/1Cvg8pP5dAUyaPLntIM+c+1Bm5wfN2GhU\n/szsSuBwd58Xnz8CvMndl9YccwxhGOFvhJ6ifVudU3IqGUldGiUP0r4apqx6VHkSQ14dsDy+euHH\nU7foVAV8LKX4bN3ut9c9X+GLGM2EqCqSUxFCiATpVDJx9/OA82r3mdn5hGGGeXE4oq+2VcLoJ0Sl\nJadiZssIdSFV3uPuy7sRnxBCdJOlL43perxXEoq5fwfsRqjPe5kcE6K6k0xayKksdPcdi49ICCHS\nYoyFHi8C3mtmNwAvAvsCmNkRhKGIuaM9YVJyKlmRnIoQouyMZTKJPT77Ndh/YoN9sxmeKduU1ORU\nJpnZhWZ2o5l9pUU8klMRQpSa5YNDmbcUSEZOJXIYofUwBFxvZte7++31B0lORQhRdlJJElnJ2jLJ\nKqeyp7vvANySJxh3P9vdn3f3F4BrgKatG1aUU7mZGjkV4DTgtBjLOTU2klMRQvQEZW2ZjLmcioUm\nzdGELqkBQivjv1uYSE5FCFFaXhzb2VwdJxk5FXd34BGCaOONwG9inUkzJKcihCgtvdYykZxKRpYs\nXlzIN9YLUhFFVXAXKadSJKkrCKgCfphuyqkcdMkfMzs/c4+3dP2DlJzKGNILiSFlX6knhbyk/r7y\nanPlSUJF6oClnrjqWZ74dVKP5FSEECJBUum+ykpqcipbAN+Ph/zK3b/VrfiEEKKbjLGcSsfJOgDf\nUVrIqZxLqGDfFtgs1okIIcS4Y/ngYOYtBZKRUzGztYFXufv/xl2faBGT5FSEEKWm17q5UpJT2RB4\n2szOj3Iqh7SIR3IqQohS02tTg1OSU+kDNgI+BCwG5prZVe5+T/2BklMRQpSdlxJJEllJSU7lCUL3\n2FPuvgi4gdAiaobkVIQQpaWsLZMxl1Nx94fM7NVmtgbwDLAlYUyjGZJTEUKUlrLO5prPGMupxJcO\nBa6I572quth9EySnIoQoLb3WMpGcSkbyyKmUtQJeiEaUsQK+m3IqO595Q2bnVx+0fde76SWnUhKK\n/EdJXZYidR2w1H9k5I0vT2IoMgGl/l29wnciLY6sSE5FCCESZLBsyWSsqJdTMbO3xedVNgM+5O43\ndSVAIYToIiMNQaRGV5JJIzkVd7+D2AoxsynAr9AsLCHEOGV5j83mSkZOpY7DgFPdveGnKTkVIUTZ\nGeqtXJKUnAoA8VzvI7RMmiE5FSFEqRkaGsq8pUBKcipVPgT8ulmrBCSnIoQoP702AJ+SnEqVDwJX\nZzhOcipCiNIyNDiUeUuBZORUatiGUGE/EpJTEUKUluXLe2vQJDU5FYApGQfCJacihCgtvdYykZxK\nRlKXU8mDKuCHUQX8MEV+FmWUYAGYPGlS21/y2/71t5k/0Du+9f6u/1NKTqUkFKnNVUYdsNQTa+rx\n5aUoCZa8vrpJrw3Ad01Opb4CPu77t2jTT6hFObnVOYQQoqykMuU3K1nHTDpKowp4M3szsFOsU5kJ\n7Gdm63QjPiGE6DZDg9m3FEipAn4hMCnOwBogTD9e1CQmVcALIUpNr8mpJFMB7+6PEGZf/V/cznb3\nZ5ucQhXwQohS02uzuZKpgDezjYEPAxsTqtpvMrOL3P3v9ceqAl4IUXYGSzpmUkQF/DbALe6+yN0X\nAn8E3tzieFXACyFKS1lbJkVUwP8ZOMTM+gljJtOBv7Q4XhXwQojSMpZJIhaUnw+8kTDOvZ+7/6Xu\nmEaza5uSTAV8XM/kSuAGQqI4z93nt4hJFfBCiNIyODiUecvBXsAz7r498G/ACbUv5pldqwr4jKRe\nAV/GQsJeQEWLwxR1DRZZtNjNCvg3HXBJ5g/0wbP3GJU/M/sx8GN3vzr2Bj3s7hvUvP56wjIgMwg9\nRbcRZs02mxSlCnghhEiRMa6AXwdYAODug2Y2ZGYT3X1p3PeImVVn1w4Ax7VKJNDFCviM57i13XN0\nk9R1m4okz/sqsrWVujZXL1wXRX3HeWVRUtcBq2docHlHzhPr8j5bt/vtdc9X+PJGM7u2SlfWgIem\ncipfILzppcB33f2SbsUnhBDdpFPJxN3PA86r3Wdm5xNaJ/Pi2HZftVUSeXl2bTy+Orv22mZ+UpJT\neR1h7fd3EgoIvxqLIYUQYtwxNLg885aDKwnKIAC7EYq9a/kzsLWZ9cdkM9Ls2qTkVDYE7nP3JdHv\nnYSm2OwGMUlORQhRagaX1ZfFdZSLgPea2Q3Ai8C+AGZ2BGFce66ZVWfXwsizazN3c21OkCKZAswj\naGa9A6hWvZ/DsJzKM2Z2vZlNd/e7Gp2syU37z8B0M1sTWBLPP6dJPHMYHphvJafyBzM7jiCncnl8\nD29w96fM7EyCnMrJZvZzQmvolxk/DyGEGFM61c3VCHdfDuzXYP+JNY+PJpR6ZCIZORV3f9rMDgcu\nAx4nFCB2zNTHAAARjElEQVQ2HNGTnIoQouyMZTIZC1KSU8HdL3b3d7j7HtHn/BaHS05FCFFaxnjM\npOMkI6diZisBVxPGZ6YAWwK3tzCRnIoQorSkkiSykpKcykvRx1zgN8BBcV8zJKcihCgtg4PLM28p\nIDmVjOSRU8lDkcVzRaKixfZ9lZEiv+MiixYnTZ7c9pe85m4nZv5wnrz8iK5fVJJTyUhRN8MiNZhS\n13vSTbd9yvgjIy95EkNeHbCzh+bnsqtlaHkaLY6sSE5FCCESpNfGTLoipxIH6X9ImNa7DPhkHLzf\nGziEMFvsXHf/fjfiE0KIbtNryaQrciqEdeTPjVN3LwW+YmarAt8Edia0Rg41szW6FJ8QQnSV0k0N\nHiM5lS8RqtwhyCBvRZBOuS0u2YuZ3UgoQry8QUySUxFClJqhwcGRD0qIrC2TzYHdCVNxjwfOItRv\nzAR2joKMVTmVmcA0M5ve7GTu/oK7L4+1KwcCF1Kjrx/5O7Buk1PMAbaLj1vJqewE3EiQU4Egp7K3\nu88iJNJ73f1dwEMEORUhhEiCwZeWZt5SoGtyKjGR/AS41t2vMbO96g5pOs1DcipCiLKTSv1IVrop\np/JD4AF3PzY+f4zQOqlSrbBvhuRUhBClZWj58sxbCnRFTiWOryytqze5BTjPzKYQbvIzCTO7miE5\nFSFEaUllYD0rWZPJfII0ySYEOZVHCXIqAL+Iku5VOZV5DMupbOnuyxqc70BgUpRUAfiTu38paun/\nDhgCjq0OxjdhDvA9hqcVT2O4SLIqp/JgfHwmQb9fCCF6gl5LJpJTycjiJUtGXUqcuhxI6hXwvUDq\nciqqgB8mz2fRRgV8229s4ls/kzngpX/4Qdf/KSWnkpHJkyZ1/csSQhRLJ2RR8pJCghgNI7ZMhBBC\niJHoVgW8EEKIEqFkIoQQom2UTIQQQrSNkokQQoi2UTIRQgjRNkomQggh2kbJRAghRNsomQghhGib\nrizbWwbM7H3AAYRFt16uVHX3d3fbRvF1xNeqhDVuXlNn00jVIbdNWX2lHl/RvsYDSib5OZWgavzX\nBG2K9JV6fHntriYInNbajCQXkcemrL5Sj69oX6VHySQ/D7n77xK1KdJX6vHltVvq7p8owKasvlKP\nr2hfpUfJJD9uZr8AbqBmkS13PysBG8XXvt0sM9u1gc2iDtuU1Vfq8RXtq/QomeTnmbitnqBNkb5S\njy+v3ed55f/HELBxh23K6iv1+Ir2VX6Ghoa05dgqlcrpqdoovo74+kARNmX1lXp8RfsaD5taJvnp\nM7PPA7dSs7a8u/8pARvF177dgWZ2o7s/M8K527Upq6/U4yvaV+lRMsnPm+NWOxg3BLSaplqUjeJr\n32414BEze5CQgPqAIXfftsM2ZfWVenxF+yo9SiY5cfed6veZ2b+mYKP4OmK3d4N9q42BTVl9pR5f\n0b5Kj5JJTuKMjuOANeKuiYS559/qto3i64jdQsKN47U1NvsAr++wTVl9pR5f0b5Kj+RU8nMM8DHC\nTWkbws3qtERsFF/7dhcDryPcOF4AZgAHjYFNWX2lHl/RvkqPkkl+XnD3h4B+d3/K3c8FPpOIjeJr\n367f3Y8GHnf37wC7AvuNgU1ZfaUeX9G+So+6ufLzqJl9CviDmf0UeIjwiyUFG8XXvt1EM9sCWGRm\n7wX+AmwyBjZl9ZV6fEX7Kj1KJvnZjyD29jNgL2BNYPdEbBRf+3YHAmsB/4/QJfZaRu4ay2NTVl+p\nx1e0r9KjZJKf24AFwBzgOuBCd3+ptUlhNoqvfbtDgGsB9xFUidu0Kauv1OMr2lfp6RsakuBlXsxs\nTeAdcdsKGHT396dgo/jaszOz7YCZcVsbuAe4zt1/1kmbsvpKPb6ifY0HlExyYmavBbYj3Jw2JxTB\n3e3uTesXirJRfJ2xi7ZTo91egLn7hmNhU1ZfqcdXtK8yo26u/DxB6DY5zd2PSsxG8bVpZ2aXxYf3\nATcDn3H3RzttU1ZfqcdXtK/xgJJJft5A+GWyk5l9DlgC3Oru307ARvG1b3czoTvMgEFguZktdfcF\nHbYpq6/U4yvaV+lRN1cbmNlqhO6T7YDtgVXcffsUbBRfZ+yi7S7AV4Cd3D3TD7A8NmX1lXp8Rfsq\nK+P+A8iLmd1JkFb4PTAb+I67v5CCjeLriK+vAW8nyGTcT6h8blnpnMemrL5Sj69oX+OCbmvg9+pW\nqVQmt3jt0m7aKL6O+Nq/Uqls0uS1L3TKpqy+Uo+vaF/jYet6AGXcKpXKdanaKL6O+Lq2CJuy+ko9\nvqJ9lWWTNtfYkGcgqiibIn2lHl9eu76CbMrqK/X4ivZVCpRMhBg9qSe71H2lHl/RvkqBkokQQoi2\nUTIZG/6RsE2RvlKPL69dWbtb1M3VHV+lQFODO4iZnejuR7j7HnX7Xwd8lbDS34Xufl3Na2e6+0EN\nbF4LfA74q7v/1MyOJOgBOXCCuz9Zb9Miruu8ZunaBr52cfcr4uM1gGMJa6bfDRzbyJeZvQZ4p7vP\nMrMpwNeBzWJ8J7r7gkbxmdnZwHnufnuzeJv4+gLwJHA+Qbn1bcADwBnu/mwTX/3Ax4H3ESTn+4D5\nwOXV99vEbgrhs1437noM+L27Pxeff61Z7A3OtaO7z25mY2bTCMqzN7v78pr9H3T3WY3szGwd4Fl3\nX2RmbyQs+HW/u/9xNPGZ2WHu/h81u75W93qfuw/VPN8KmA7cU/P9NYpvkrsviY83JUjYuLvf1cJm\nO3e/eYSQG9m9Dljk7s9b0F97M/AXd3+4mU20W5kwxXdthq+L2919sIWvjl0XZURFi6PEzFZp8fIV\n7r5DA5srgV8SlGsPBK5x92/F1671BuqjZvYbQrXt+oQboQOXEP4BPuDuuzSJb5BwkS9l+JfSusDj\nwJC7b9zA5uUYzOxCYB5wGbAjsKu779bA5irgInc/L9rcA/yOcJP/Z3fftUl884DbCZLwZ7j7nEbH\n1dlcBswFVicUHN4AXE24ib7V3T/axO4c4GHgSkJC6QNuIUjSP+ruhzWw+QxwaPSxINqsT6igP8bd\nfz5SvHXna/j9xteOB3YGngLeCHyiesNtcV0cRVgmdjlh2eGvATcSPvdZ7n58E18/qNvVR1jY6dcA\n7v6KxcLqrotDCasLXkO4oc5y9xMb2HwD2NTd9zazgwmf9Q3AlsCV7n5ck/ieAP4EnOruv2p0TAOb\nw+P5lwNnA18E/khIeOe6+xlN7D5C+HF3J+F7vRsYALYADozJv96mo9dFGVHLZPQ8A9Rr8QwRLq61\nm9hMdPezAMzsEuAnZvbN+I/VrGk8yd2PM7M+4D53/3Dcf5uZNbx5RnYBjgDOdPdLos+57j5jxHcW\nWNvdT4qP7zWzjzc5bjV3Py8+Xtfd94qPbzezT7Y4/9Puvr+ZVYCDzew04FZCAvu7u1/cwOZV7n5C\nfC/3unv1F+CVZnZtC18Vd/9CfHybmV0dk/iVZnZLE5vPAdtUf1lXMbNXEZLSK24aZvaLJufqI/wq\nb8a73X27eI7pwAVm9il3n0fz6+IDwDRCK/cuYJq7LzSzAUJSaZhMgMnARvH15+L5twV+1CK+2hg+\nArwrtoZWIhSAviKZALu7+7bx8UeBGe6+OLYSbyAsmdyIPwEfAg4zs68TktzVwLwWBaYfIny+kwkL\nnk1192fNbAJhqYGGyYSQFN7t7i/G7/WH7v6x2OKbBWzdwGbU18V4Q2Mmo+cw4AJ336hm29jdNyL8\n6m3EMjPbI3YbDAKfAt5kZucCr25iM8HM3hi7Gb5c3WlmbwEmNAvO3X8HvB94i5ldamYbMfIskzXN\nbFcz2xV4Mfog2q7axObPZnaKmW0NXGdmHzOztc1sX0IrqBlDMc773f1AQuviF8BqhFZHIyaY2SZm\nNgNYw4IMeLWLaGILX/1m9k9mtnpMcIujXcNWU2SAxj+y+mn+//Jqwq/c/2ywtRIBHKi2dGOLZA/g\np2a2PS2+M3cfdPcnCS3DhXF3y+/Y3T8B/Cvh+l07/vpe6O5zWrQOa8/5fzXneonwOTWiz8JKhAB/\nBlaOj1ej9f1myN0XelBy3oFQXf4l4H/N7G8tbIaAlwg6WS/G+JbRevxi5Xg8hOtnvfj4Hy1izHNd\njCvUMhkl7n66mX3KzFZt8IvpqiZm+xF+Ef4GWBwTyj5mtjfNf7l+DTgZ2DMmCMzsQ8AxjLB+ubu/\nCBxtQSb7DMLKcK24A/hYfPwEoQ8f4Ns07wfeh/Br7ThgQ8I/79+AK+JrzXiyLtZlhF+gV7ew+Tph\nlcQFhBvNGTHhPQ4c0MLu88BJwFRC90f12G0JCb0RpxFaV7dGfxC6CbcmtPga8QlCN8tp9deEmS1s\nbALAd4G7zWy6u7/g7g9Y0Hs6L8bYiN+a2UXuvqe7HxJ9vA34HqEbtCnufpWZzQGONLPLgVZdtgDv\nNLO/E77bScD1wLlmdgFhgahG7B+PWZVwc/6jmd1DSLhfbmIDNTf/+Ov/54z8a3+2md1ISA6nAXNi\ni3Nrmv8vAnwfuMfM7iV0iR0e9/8W+K8mNnmui/FFt6smy7hVKpWjU7KpVCrrpRxfar4qlcoqlUpl\np0ql8om47VCpVCblPH//CK83lHqpVCpTW9i8se75BpVK5a2jjGuTSqVycM73VMlwzOsqlcq2lUrl\n7ZVKZcMMx2+VM5bNK5XKBvHxRpVK5eNZzlWpVNaqVCrbVCqV1Wv2DYxg07Hrooxb1wMo4yZZit6I\nb7R2lUrlxBznH7VNWX2lHl8ru5gcT6xUKv9VqVR2qnvtzDy+yrapm2ts0Hz9/DZd9TXCbL2Gkxjy\n2JTVV+rxtWH3U8KMzDsIXcgza2bObdbifOMGJZOxQbIU+W267SvPbL08NmX1lXp8ee3yzMgcVyiZ\nCLEihwGvc/dv1L9gZtc1OD6vTVl9pR5fXrtlZrYH8D/uPmhmnwJ+OMKMzHGFprR1kDjXH0bxS6Uo\nmyJ9pR5fKzt3Px3wOBupnoYzhPLYlNVX6vG1YbcfsBthVlt1evY+wBzilOTxjirg28DMNmd4Gu1E\n4BR3n25mr3f3R7ppo/g6Y9fkXEe7+7FjbVNWX6nHV7SvsqBurpxY0JjalFCNfCthvvlJAC1uhIXY\nKL7O2LXgFZI5Y2RTVl+px1e0r1Kgbq78bO5Bh+teD9pV2zLyrI6ibBRfZ+yaMS5mtiVmU2ZfpUDJ\nJD8rmdlqAGa2VvyFu0UiNoqvM3bNGC8z21KyKbOvUqBkkp8zCPLmZwB3mdkjBLG6FGwUX2fshBBZ\n6XbVZC9vlUpl5fh3QqVSWSMlG8XXGbu6cwzEv9eNpU1ZfaUeX9G+yrZpAD4nFtZqeA9BcnuZmf3I\nzK6K0w67aqP4Omb3ihlgBGHAT3fSpqy+Uo+vaF9lR8kkP3uyomT67oT1GlrdoIqyUXxt2qU+2yx1\nX6nHV7Sv8YDGTPKzEjCl5vk6jDyboygbxde+XeqzzVL3lXp8RfsqPWqZ5OfrwFwzqy7MM4HWa2sU\naaP42rd7xQwwG170qZM2ZfWVenxF+yo9apnkZwOGVyEcICwdul7zwwu1UXzt26U+2yx1X6nHV7Sv\n8tPtGQC9ulUqlTsrlcpra56vWalU7kzBRvF1zC7p2Wap+0o9vqJ9lX1TyyQ/jwJP1zx/CngwERvF\n16ZdnAF2Mby8tPCPzKzVsrO5bMrqK/X4ivY1HtCYSX6eBe60sKZ2P2FRnflmdjKAuzdaO70oG8XX\nvl3Ss816wFfq8RXtq/QomeTnt3GrcltCNkX6Sj2+vHbVGWDVFs1oZpuNxqasvlKPr2hfpUfJJCfu\n/qNUbYr0lXp8bdilPtssdV+px1e0r9KjMRMhGpP6bLPUfaUeX9G+So9aJkI05hBgC3d/CsDM1gSu\nBi7ssE1ZfaUeX9G+So9aJkI0JunZZj3gK/X4ivZVerRsrxANMLOfEWQyVpgBRrxxNJoBlsemrL5S\nj69oX+MBdXMJ0ZjUZ5ul7iv1+Ir2VXrUMhFCCNE2GjMRQgjRNkomQggh2kbJRAghRNsomQghhGib\n/w+IYWE2fnS3BAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f59762e16d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# I would like to visualize the relationship among the 20 pca components\n",
    "\n",
    "# Register dataframe as a temp table in SQL context\n",
    "df.createOrReplaceTempView(\"df2\")\n",
    "\n",
    "sqlStatement2 = \"\"\"\n",
    "    SELECT pca_1_warn, pca_2_warn, pca_3_warn, pca_4_warn, pca_5_warn, \n",
    "    pca_6_warn, pca_7_warn, pca_8_warn, pca_9_warn, pca_10_warn,\n",
    "    pca_11_warn, pca_12_warn, pca_13_warn, pca_14_warn, pca_15_warn, \n",
    "    pca_16_warn, pca_17_warn, pca_18_warn, pca_19_warn, pca_20_warn\n",
    "    FROM df2\n",
    "\"\"\"\n",
    "plotdata2 = spark.sql(sqlStatement2).toPandas();\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "# show correlation matrix heatmap to explore some potential interesting patterns\n",
    "corr = plotdata2.corr()\n",
    "sns.heatmap(corr, \n",
    "            xticklabels=corr.columns.values,\n",
    "            yticklabels=corr.columns.values)\n",
    "\n",
    "# From the plot we can see the 20 pca components do not overlap too much which is expected\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the result from Notebook #1 as input data for Notebook #2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 87.6 ms, sys: 58.6 ms, total: 146 ms\n",
      "Wall time: 2min 3s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "df.write.mode('overwrite').parquet('/mnt/resource/PysparkExample/notebook1_result.parquet')\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Spark - python",
   "language": "python",
   "name": "spark-python"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}