class_imbalance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "3c8c5c5f2f6bc7ab03bb842ddffdfb34ea87607c"
   },
   "source": [
    "# Methods for Dealing with Imbalanced Data\n",
    "Imbalanced classes are a common problem in machine learning classification where there are a disproportionate ratio of observations in each class.  Class imbalance can be found in many different areas including medical diagnosis, spam filtering, and fraud detection.\n",
    "\n",
    "In this guide, we'll look at five possible ways to handle an imbalanced class problem using credit card data.  Our objective will be to correctly classify the minority class of fraudulent transactions.\n",
    "\n",
    "Important Note:\n",
    "This guide will focus soley on addressing imbalanced classes and will not addressing other important machine learning steps including, but not limited to, feature selection or hyperparameter tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, recall_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_uuid": "9c68ba8512d7c66d58927843c103d467e18597a1"
   },
   "outputs": [],
   "source": [
    "# setting up default plotting parameters\n",
    "%matplotlib inline\n",
    "\n",
    "plt.rcParams['figure.figsize'] = [20.0, 7.0]\n",
    "plt.rcParams.update({'font.size': 22,})\n",
    "\n",
    "sns.set_palette('viridis')\n",
    "sns.set_style('white')\n",
    "sns.set_context('talk', font_scale=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(284807, 31)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>V11</th>\n",
       "      <th>V12</th>\n",
       "      <th>V13</th>\n",
       "      <th>V14</th>\n",
       "      <th>V15</th>\n",
       "      <th>V16</th>\n",
       "      <th>V17</th>\n",
       "      <th>V18</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>0.090794</td>\n",
       "      <td>-0.551600</td>\n",
       "      <td>-0.617801</td>\n",
       "      <td>-0.991390</td>\n",
       "      <td>-0.311169</td>\n",
       "      <td>1.468177</td>\n",
       "      <td>-0.470401</td>\n",
       "      <td>0.207971</td>\n",
       "      <td>0.025791</td>\n",
       "      <td>0.403993</td>\n",
       "      <td>0.251412</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.191857</td>\n",
       "      <td>0.266151</td>\n",
       "      <td>0.166480</td>\n",
       "      <td>0.448154</td>\n",
       "      <td>0.060018</td>\n",
       "      <td>-0.082361</td>\n",
       "      <td>-0.078803</td>\n",
       "      <td>0.085102</td>\n",
       "      <td>-0.255425</td>\n",
       "      <td>-0.166974</td>\n",
       "      <td>1.612727</td>\n",
       "      <td>1.065235</td>\n",
       "      <td>0.489095</td>\n",
       "      <td>-0.143772</td>\n",
       "      <td>0.635558</td>\n",
       "      <td>0.463917</td>\n",
       "      <td>-0.114805</td>\n",
       "      <td>-0.183361</td>\n",
       "      <td>-0.145783</td>\n",
       "      <td>-0.069083</td>\n",
       "      <td>-0.225775</td>\n",
       "      <td>-0.638672</td>\n",
       "      <td>0.101288</td>\n",
       "      <td>-0.339846</td>\n",
       "      <td>0.167170</td>\n",
       "      <td>0.125895</td>\n",
       "      <td>-0.008983</td>\n",
       "      <td>0.014724</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.358354</td>\n",
       "      <td>-1.340163</td>\n",
       "      <td>1.773209</td>\n",
       "      <td>0.379780</td>\n",
       "      <td>-0.503198</td>\n",
       "      <td>1.800499</td>\n",
       "      <td>0.791461</td>\n",
       "      <td>0.247676</td>\n",
       "      <td>-1.514654</td>\n",
       "      <td>0.207643</td>\n",
       "      <td>0.624501</td>\n",
       "      <td>0.066084</td>\n",
       "      <td>0.717293</td>\n",
       "      <td>-0.165946</td>\n",
       "      <td>2.345865</td>\n",
       "      <td>-2.890083</td>\n",
       "      <td>1.109969</td>\n",
       "      <td>-0.121359</td>\n",
       "      <td>-2.261857</td>\n",
       "      <td>0.524980</td>\n",
       "      <td>0.247998</td>\n",
       "      <td>0.771679</td>\n",
       "      <td>0.909412</td>\n",
       "      <td>-0.689281</td>\n",
       "      <td>-0.327642</td>\n",
       "      <td>-0.139097</td>\n",
       "      <td>-0.055353</td>\n",
       "      <td>-0.059752</td>\n",
       "      <td>378.66</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.966272</td>\n",
       "      <td>-0.185226</td>\n",
       "      <td>1.792993</td>\n",
       "      <td>-0.863291</td>\n",
       "      <td>-0.010309</td>\n",
       "      <td>1.247203</td>\n",
       "      <td>0.237609</td>\n",
       "      <td>0.377436</td>\n",
       "      <td>-1.387024</td>\n",
       "      <td>-0.054952</td>\n",
       "      <td>-0.226487</td>\n",
       "      <td>0.178228</td>\n",
       "      <td>0.507757</td>\n",
       "      <td>-0.287924</td>\n",
       "      <td>-0.631418</td>\n",
       "      <td>-1.059647</td>\n",
       "      <td>-0.684093</td>\n",
       "      <td>1.965775</td>\n",
       "      <td>-1.232622</td>\n",
       "      <td>-0.208038</td>\n",
       "      <td>-0.108300</td>\n",
       "      <td>0.005274</td>\n",
       "      <td>-0.190321</td>\n",
       "      <td>-1.175575</td>\n",
       "      <td>0.647376</td>\n",
       "      <td>-0.221929</td>\n",
       "      <td>0.062723</td>\n",
       "      <td>0.061458</td>\n",
       "      <td>123.50</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>-1.158233</td>\n",
       "      <td>0.877737</td>\n",
       "      <td>1.548718</td>\n",
       "      <td>0.403034</td>\n",
       "      <td>-0.407193</td>\n",
       "      <td>0.095921</td>\n",
       "      <td>0.592941</td>\n",
       "      <td>-0.270533</td>\n",
       "      <td>0.817739</td>\n",
       "      <td>0.753074</td>\n",
       "      <td>-0.822843</td>\n",
       "      <td>0.538196</td>\n",
       "      <td>1.345852</td>\n",
       "      <td>-1.119670</td>\n",
       "      <td>0.175121</td>\n",
       "      <td>-0.451449</td>\n",
       "      <td>-0.237033</td>\n",
       "      <td>-0.038195</td>\n",
       "      <td>0.803487</td>\n",
       "      <td>0.408542</td>\n",
       "      <td>-0.009431</td>\n",
       "      <td>0.798278</td>\n",
       "      <td>-0.137458</td>\n",
       "      <td>0.141267</td>\n",
       "      <td>-0.206010</td>\n",
       "      <td>0.502292</td>\n",
       "      <td>0.219422</td>\n",
       "      <td>0.215153</td>\n",
       "      <td>69.99</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Time        V1        V2        V3  ...         V27       V28  Amount  Class\n",
       "0   0.0 -1.359807 -0.072781  2.536347  ...    0.133558 -0.021053  149.62      0\n",
       "1   0.0  1.191857  0.266151  0.166480  ...   -0.008983  0.014724    2.69      0\n",
       "2   1.0 -1.358354 -1.340163  1.773209  ...   -0.055353 -0.059752  378.66      0\n",
       "3   1.0 -0.966272 -0.185226  1.792993  ...    0.062723  0.061458  123.50      0\n",
       "4   2.0 -1.158233  0.877737  1.548718  ...    0.219422  0.215153   69.99      0\n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read in data\n",
    "df = pd.read_csv('../input/creditcard.csv')\n",
    "\n",
    "print(df.shape)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_uuid": "c7d92ecec94ee2633bd378b3d55100c6b9362636"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    284315\n",
      "1       492\n",
      "Name: Class, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(df.Class.value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "00001b711204cdcfe0a91021db7197bb81e1b6b0"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHwAAAHDCAYAAABWLA0oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xm8l3PC//F3paLFlEQkw1iKhNBC1rEPbtlnGPsYzWQsM5Zi7rmHewhjLZGlJhn7qNtYkyVrCyrcSUamaKG0KEVOdX5/uPv+ztGuEpfn8/HweHzP9b2+1/fz3Y6+r3Ndn6tKeXl5eQAAAAAojKpregAAAAAArFqCDwAAAEDBCD4AAAAABSP4AAAAABSM4AMAAABQMIIPAAAAQMEIPgAsVtOmTdO0adOceOKJa3oo30inTp1Kj2H8+PGLXD9kyJDS9d26dVsDI1w1ivI4vi2TJk3KX/7yl/zsZz9Ly5YtS8/d4YcfvqaHxvfQzJkzS++h3/72t2t6OABQyVpregAAfHNNmzZd7PLq1aunTp06qVu3bjbeeONsu+222WGHHbL33ntn7bXX/pZHuXgL40Tjxo1z5JFHruHRfHfMnDkzd955Z5Jkm222yX777beGR1QcY8aMyS9+8Yt8+umn3+j2ffv2TefOnVfJWJ555plssskmq2RbrFq33XZb5s6dm/XXXz+/+MUv1vRwAOAbE3wACqisrCzTp0/P9OnT88EHH2Tw4MFJknXXXTft27fP2Wefnbp1667RMd50001JktatWws+FcycObP03BxxxBGCzyr017/+tRR79tlnn+y7776pX79+kqROnTprcmh8h9x2222ZNWtWmjVrJvgA8L0m+AAURPfu3UuXy8vLM2vWrMycOTOjRo3Kq6++mgkTJmTmzJnp06dPnnrqqVx77bXZZZddlri90aNHfxvDXm2uvPLKXHnllWt6GKtdmzZtvvev1behrKwsr7zySpJkiy22yC233JIqVaqs0Dbatm1b6XP2dX369MmQIUOSJCeeeGLatm27xHUbNGiwQvfNd9O6667r8wfAd5bgA1AQS9sTpLy8PC+88EKuuOKKjB07Nh999FE6dOiQe++9N1tttdW3OEpYM6ZPn565c+cmSZo1a7bCsSdJNt5442y88cZLvP7pp58uXd52223tnQUArFEmbQb4AahSpUr22muvPPTQQ9l5552TJLNmzco555yTBQsWrOHRwer35Zdfli7XqFFjDY4EAODbYQ8fgB+QOnXq5IYbbsghhxySmTNnZsyYMXn88cdz6KGHLrLuwgmhW7dunbvuumux2/v4449z33335ZVXXsm///3vzJ49O7Vq1Ur9+vWz/vrrp0WLFtl///0rHTr29Ymmhw4dutjJp/v06ZM2bdok+epMVCeddFKS5Kyzzsrvfve7vP/++7n33nvz0ksv5eOPP87s2bPTpUuX0nxAnTp1Sr9+/ZIs/wS5o0ePzt///vcMGjQokydPTq1atdKsWbMcffTRi32OFqo4mW/FMazIuuPHj8++++5bad1+/fqVHkNFFR/P4p6bJfn8889z//3355lnnsn777+fTz/9NHXr1s1mm22WvffeO8cff/xS53bq1q1baX6hha/PiBEjctddd2XYsGGZMmVK6tatm+233z4nnHBC9txzzyVua0XNmDEjd999d1544YWMGzcun332WerVq5ctt9wy++67b4499tjUrFlzkdtVfB8stLjndU1Novz000+nY8eOSZLOnTvnlFNOyejRo3PvvfeW3odz5sxJ9+7dK+0xNGLEiLzwwgsZPnx4xowZk+nTp6dq1apZb7310qJFixx88ME58MADU7Xqkv+217t373Tp0iVJStt/++23c9ddd2XIkCGZMmVKatWqle222y6/+MUvlrnH0uzZs/Pggw/m2WefzXvvvZeZM2emevXqqV+/fho0aJBtttkme+65Z/bZZ59Uq1ZtkduvisdU0bRp03L//feXfj/NmDEj1atXz0YbbZTtttuuNI/TwgC4yy67ZNasWaXbv/POO4v93VTxtZg5c2ZatWqVJNl3331z8803L3E88+fPzyOPPJL+/ftn5MiRmT59etZZZ51svPHG2W233XLCCSekcePGS7z94t4r48aNS58+ffLiiy/mo48+Ss2aNdO0adO0b98+Rx555FKfq7KysvzP//xP+vfvn9GjR2f69OmpVq1a6tevn/XWWy9bb7112rVrlwMOOGCxny0AvvsEH4AfmA022CDHHnts7rjjjiTJQw89tNSYsSQDBw7Meeedlzlz5lRaPnPmzMycOTPjxo3L66+/noceeiivvfbaKhn7Qv/zP/+T//qv/8oXX3yxSrf5n//5n5X2BJk7d24GDRqUQYMG5ZFHHknXrl2/t198RowYkd/97neZPHlypeXTpk3LtGnTMmzYsPTq1SvXXnttdt999+Xa5i233JKuXbtW2kts2rRpGThwYAYOHJiOHTvm7LPPXumxP/300+ncuXNmzpxZafmUKVMyZcqUDBo0KL169Ur37t2z7bbbrvT9rUl33313unTpkrKysiWuc8UVV5TO5PZ1EydOzMSJE9O/f//ssssu6datW9Zbb73luu+77rorV155ZebNm1da9uWXX+all17KSy+9lBNPPDF//OMfF3vbf/3rX/nVr36Vjz76qNLysrKyzJkzJxMmTMibb75ZCo5fj2ur+jH9/e9/z7XXXrvI76eysrKMGTMmY8aMycMPP5yLL744J5988lK3tSpMnDgxv/3tbzNq1KhKy7/88st8+umnGTVqVP7+97/noosuygknnLBc23zyySfTuXPnSo9x7ty5efXVV/Pqq69m4MCBufHGGxcb1z7++OP86le/yrvvvltpeVlZWSZNmpRJkyZl5MiR6devX6X4DsD3i+AD8AN02GGHlYLP8OHDU1ZWlurVqy/37T/++ONKsWfvvffObrvtlg022CDl5eWZOnVq3nnnnbzyyiuV/mKe/P/JpRf+pXqrrbbKueeeu8h9LGluoWHDhqVHjx6pWrVqjj766Oy0006pWbNm/v3vf2f99ddf7sdQ0VtvvZVbb701SXLUUUelVatWqVq1at5666089NBDmTNnTgYOHJgLLrggXbt2/Ub3sSwNGjRI9+7dM3Xq1PzpT39K8tWEzAv33vn6uivi7bffzsknn1wKZNtuu20OPfTQbLTRRvnkk0/yxBNPZNiwYZkxY0Y6dOiQnj17LvML3gMPPJBHH300G264YY444ohstdVWKSsry4svvpjHH3885eXl6d69e1q1apVdd911hcZb0fPPP5+zzz478+fPT5K0atUqBx54YBo0aJCJEyfm4YcfzrvvvpuJEyfml7/8ZR588MFsscUWpdufeOKJ2W+//Zb5vH4XJlF+8cUX8/LLL2edddbJ0UcfnR122CFrrbVW3nvvvdSrV6+03hdffJHq1atn5513zo477phNN900tWrVyvTp0/Phhx/m4YcfztSpU/Paa6/l3HPPTe/evZe5V8zjjz+exx57LA0aNMiRRx6Zpk2bZv78+Rk8eHAefvjhLFiwIHfddVdat26dAw44oNJty8rKctZZZ5Vizw477JD99tsvjRs3TtWqVfPpp5/mvffey5AhQxYJDKvjMd1www255ZZbSj+3adMme++9dxo1apSysrJ8+OGHGTJkSF577bWUl5eX1rvmmmsyb968nH/++fn888/TuHHjXHzxxYtsf/vtt1/qc/l106ZNy/HHH59JkyYlSRo3bpwjjjgiP/nJTzJ79uw8//zzefrppzN37txcdtllSbLM6DNs2LA8++yzqVGjRn75y1+mRYsWWWuttTJ8+PA88MAD+fLLLzNgwID06dMnp5566iK3P//880uvxZZbbpmDDz44P/7xj1O9evXMmjUr77//foYOHZqRI0eu0GMF4LtF8AH4Adpqq61Sq1atzJkzJ59//nnGjBmTZs2aLfftH3300VLsOf/883PGGWcsdr3y8vK8/vrrlZZ9/bCQ+vXrr9Dktq+88koaNmyY3r17Z8stt1zu2y3N888/n9q1a6dXr17ZcccdS8sPP/zw/PKXv8yJJ56YyZMnp3///unfv38OPPDAVXK/Fa2zzjrZb7/9Mn78+NKyjTfeeKUn/l2wYEEuvPDCUuw56aST0rlz50pflk866aR07949Xbt2TVlZWS666KL0799/qXszPfroo2nXrl1uuumm1KpVq7T8iCOOyPbbb186VKhXr17fOPh89tln6dy5cyn2dOrUaZEvr6ecckouvfTSPPDAA5k9e3YuvPDCPPTQQ6XrmzdvnubNm6/y53V1eOmll9K4cePceeedadKkyRLXO+KII3LeeeeVTin/deeee27+/Oc/p2/fvhkyZEieffbZZT7exx57LC1btsxtt92Wddddt7S8ffv2adWqVSl89OrVa5Hg8+qrr2bs2LFJkkMPPTTXXHPNEifFHjVqVKV4taof04svvpgePXokSWrVqpXrr78+e++99yLrnXXWWRk/fnylvcYWrrfWWl/987hu3bqr5H3yl7/8pRR79thjj3Tt2rXSZ+bYY4/NU089lfPOOy/z5s3LVVddlT322CObbrrpErfZv3//bL755vnb3/6WjTbaqLT80EMPzX777ZdTTz015eXl6d27d04++eRKn/dx48Zl6NChSb46ZLdnz55LnNdq7NixlcYKwPeLSZsBfoCqVauWDTfcsPTztGnTVuj248aNK10+9thjl7helSpVlnrq92/q0ksvXWWxZ6ELL7ywUuxZaLPNNsvll19e+rlXr16r9H5Xt+eeey7/+te/kiQ77rhjLr744sXuGdGxY8fSF95Jkybln//851K3W69evVx//fWL/TJ40kknlc5mNXjw4EqHCK2Ivn37ZurUqUmSgw8+eLF7Kqy11lr585//XJpr5X//939Lp1//PrrqqquWGnuSpGXLlksMI0lSs2bNXHrppaXDnpb1WiZfxZGuXbtWij0LHXXUUdl6662TfHVo4GeffVbp+oq/D4455pilngFtm222SZ06dRZZvqoe04033ljaa+fyyy9fbOxZaJNNNlnthwB++OGHeeKJJ5J8tRfZddddt9jPzAEHHJBf/epXSb46LKt3795L3W6VKlVyww03VIo9C+26667Za6+9kiQfffRR6fO/UMXXq3379kudxHyzzTbLBhtssNSxAPDdJfgA/ED96Ec/Kl2eMWPGCt12nXXWKV3++peJ1a1x48b56U9/ukq3+aMf/WipEy3vueeepcA0YsSITJkyZZXe/+o0YMCA0uXTTz99qV/GK+6p9dRTTy11u+3bt6/0HqqoatWqpYlsv/zyy3zwwQcrMuSSimNf0l5kyVcB8/TTTy/9vKyxf1c1bdq09LytrBo1aqR58+ZJkjfeeGOZ6x900EFL/WK/8BC/8vLyjBkzptJ1FX8fvPfee99kuMtlWY9p7Nixeeutt5J8FZZ+9rOfrbaxLK9nnnmmNMfVcccdt9igttApp5xSOrS24nt/cVq3br3UvTLbtm1burymXi8A1jyHdAH8QK3M6dh322230l+gf/e73+XMM8/MQQcdlEaNGq2i0S3ZTjvttNRo8U3svPPOyzxVd9u2bUtfjt56661VHp1WlzfffDPJV3sE7Lbbbktdd6eddiod6rfwdkuyww47LPX6inuQfX2y5eVRXl5e+vJev3790hf9Jak40fSyxv5dtSJ7w82bNy9PPfVUBgwYkLfffjuffPJJ5syZs9jP9eTJk1NeXr7Uz83i9m6rqGIM+vTTTytd16ZNm6y11lqZN29err766kyePDmHH354pbmUvo3HVPHw0e/K53PhezhJ2rVrt9R169evn+222y7Dhw/P5MmTM2nSpMXuwZMs+/Wq+Pn7+uu13Xbb5Uc/+lE+/fTT9O7dO1988UWOOuqoNG/efJX/bgVgzRJ8AH6gKn4JX9ycGkuz11575dBDD82jjz6aadOmpUuXLunSpUs222yztGzZMrvsskv22Wef1TIRbsUvMqvKj3/842WuU3E+ja+f6eq7bOHeSOuvv/5iD6WpqGrVqtl0003zzjvvZMaMGfnyyy+XGMKWdvhNkkq3mzt37gqO+qv5ez7//PMkXx1WsiwNGjRI3bp1M2vWrO/V61PR8r63x40bl44dOy733nULFizI7Nmzl/r6r8jrWfFMdkmy0UYb5dxzz80111yTuXPn5tZbb82tt96ahg0bZqeddsouu+ySvfbaa6mfs1XxmCqeIWxFY9PqUvG9uPnmmy9z/c022yzDhw9P8tVnd0nBZ2U+f+uss07+9Kc/5cILL8z8+fNzzz335J577km9evXSsmXL7Lzzztljjz1WaF43AL6bBB+AH6D58+fn448/Lv28vKdtruiaa65J27Ztc+edd5a+pI0dOzZjx45Nv379Uq1atRx88MG56KKLVukcEGuvvfYq29aKbLPivBtfP9Xzd9ns2bOTZLknXq243uzZs5cYfJZ11qeVtXDcSeVDUJamVq1amTVrVqXbfp8sbZLshebMmZNTTz01EyZMSJI0bNgwP/3pT7PFFlukQYMGqVmzZmkvjVtvvbW0t9Oy9uhb2dfzjDPOSNOmTXPrrbfm9ddfT3l5eaZMmVKa6Pzyyy9P27Zt07lz50VCwqp6TBXnFvquTDS8ou/jr3/+lmRl98Q59NBD06RJk9x00015+eWXM3/+/MyYMSPPPfdcnnvuuVxzzTVp3rx5OnXqlNatW6/UfQGw5gg+AD9A7777bmnviVq1an2jCZCrVKmSY445Jsccc0w+/PDDvP766xk2bFiGDBmSsWPHZv78+Xn00Ufz+uuv5x//+Mc3PmX6t2HhGayWpmLkWZkvkytzKN03Ubt27cycOXO5I1XF9WrXrr26hrVMFe974Xt1WRaOfU2Oe3X7xz/+UQoj+++/f6677rolRrm777772xxa9txzz+y5556l06cPHz48Q4cOzdtvv53y8vIMHjw4xx13XP7+97+nRYsWpdutqsdUcW+f70qU/fr7eFm/O77Nz98OO+yQ22+/PTNnzszrr7+eESNGZOjQoXnjjTcyf/78jBw5MieffHK6d+/+nTlEDoAVY9JmgB+gRx55pHS5ZcuWpdMQf1NNmjRJ+/btc9lll6V///7p27dv6ew3kyZNSs+ePVdq+6tbxbPWLEnFiYe/vsdSxS+nZWVlS93O9OnTV3B0K6dhw4ZJkk8++WSZe76Ul5eXHme9evWWOa/R6lSnTp3SHhHL8/pMmzYts2bNSrLo61MkFc9A9sc//nGpr9HEiRO/jSEtokGDBjnwwAPTqVOn9O3bN08//XQpGHzxxRe59tprK62/qh5TxTnEvj5R8ZpS8b248NT1S1NxnW/rfbzuuutmn332yXnnnZd77703L7zwQunsiwsWLEiXLl2+lXEAsOoJPgA/MJMnT86DDz5Y+vnoo49e5ffRvHnzXH311aWfK06mutDCQxIWnkJ5TRo2bNgyQ82QIUNKlyvunZAkdevWLV1e1vwxy5pQuOKhNaviudl+++1L2xo0aNBS1x02bFhpD4OFt1tTqlSpUnqep02bllGjRi11/Zdeeql0eU2PfXVaeJr6mjVrLnWS9A8//HC5AsO3YZNNNskNN9xQ2rvl678PVtVjqjjp9bPPPvuNx7vwM7gqPn8Vf1dUDFuLM2PGjPzv//5vkq9C7bcxCf7irL/++rnssstKc2d98MEH36szEwLw/wk+AD8gn332Wc4999zShM1bbLFFDjrooNVyX40bNy5dnjdv3iLXL/zy91049GLGjBnp16/fEq9/6aWXSvMUtWzZsrTXzEIVD4kbPHjwErfz4Ycf5rnnnlvqWCoe8rG8hzItzQEHHFC63LNnz6V+ib399tsXe7s1peIY7rjjjiWuN3/+/PTq1av084EHHrhax7UmLZxvau7cuZXm4fq6m2+++dsa0nKpWbNm6XOzYMGCSu/DVfWYfvzjH5fOHjdq1Kg8/vjj32isCz+Dq+Lzt99++5UC0v33319pnqGvu/POO0vheU2/h6tUqZKNN9649PP8+fPX4GgA+KYEH4AfgPLy8jz//PM56qijSn9dr1OnTm688cZvNFnrwok+lzYfzT333FO6vLizvWyyySZJkn//+9/LNYfO6nbVVVctdu+bDz74IBdffHHp51NPPXWRdRo3blw6K9Brr7222L0Lpk2blnPOOWeZexLVq1evtMfQqFGjVnovg7333jtbbbVVkq/24Ln66qsX+7r16NGjFKM22mij/Md//MdK3e+qcMQRR5TO9Pboo4+mT58+i6wzf/78XHbZZaU9gFq0aJFdd931Wx3nt6niHiPXX3/9Yt8fPXv2TN++fb+1MT344IN55JFHFjl7V0Uvv/xy6dC8pk2bVpp0eFU+prPPPru07UsuuSQDBw5c4roTJ07M22+/vcjyhb+bJk6cmBkzZizzPpdmk002yc9+9rMkX5116w9/+MNiQ9LTTz9dCq41a9bMySefvFL3uzQDBgzIvffeu9RDPEePHl36f8V66623SOQG4PvBpM0ABfH000+XLpeXl2f27NmZMWNG3nnnnbz66qsZP3586fpGjRrl2muvLYWAFTVkyJB069YtDRs2zO67755mzZqlYcOGWbBgQSZPnpxnn302r732WpKv5rdZXCTZddddM3r06MyZMycdOnRI+/btU79+/dKXte23336FTxf/Te2111555ZVXcvzxx6d9+/bZZZddUrVq1bz11lv5xz/+UdoL6cADD1ziX95PO+20XHLJJUm++tJ55JFHplWrVikvL8+oUaPSt2/fzJw5MwcddFCefPLJpY6nbdu2GTBgQD744IOce+65OeCAAyodNta6devlPltZ1apV89e//jU///nP88UXX6RXr14ZMmRIDjvssGy44YaZOnVqnnjiidKXu+rVq+eqq65arjNGrW516tRJly5d8pvf/Cbz58/P5ZdfngEDBuSggw5K/fr1M2nSpDz88MMZPXp0kq8mub3qqqvW8KhXr5///Ofp06dPysrK0q9fv7z//vv52c9+lg022CCTJ0/OE088kREjRqRx48bZaKONSp/D1endd99Nnz598uc//znt2rXLdtttl0aNGmWttdbKJ598kiFDhlQKL2eeeeZqe0y77757OnTokFtuuSVz5szJmWeembZt22avvfZKo0aNMm/evIwfPz6vvvpqhgwZkgsvvLA039hCbdu2zauvvpp58+blN7/5TY455pg0aNCg9LupefPmpRC5PC655JK89tpr+eijjzJw4MAccsghOfLII/OTn/wks2fPzgsvvJCnnnqqtP5FF12UTTfddLm3v6ImTJiQLl26pEuXLtl1113TokWLbLLJJqlZs2amTZuWESNG5Kmnniqdzv2MM85ItWrVVtt4AFh9BB+AgujYseMy11l33XVz+OGH5+yzz86666670vc5ZcqUpR4KVb9+/VxzzTWLDUunnXZa/vnPf2batGkZNGjQIvPL9OnTJ23atFnpMS6PFi1a5JBDDskf//jHPPjgg5XmOFpor732yl//+tclbuOoo47Ka6+9ln79+qWsrCz3339/7r///tL11atXz2WXXZZq1aotM/h07NgxL774Yr744os8+eSTi6z/zDPPlPZCWB7bbLNNevfund/97neZMmVKRo4cmZEjRy6yXr169XLNNdd8a8/78thrr73StWvXdOrUKbNmzcrQoUMzdOjQRdbbeOONc9NNN5X2tCqqTTfdNFdccUUuvvjilJWV5Y033sgbb7xRaZ0mTZrk5ptvzg033PCtjGlhCPnss89Kp2FfnJo1a+aiiy5aJJqu6sd07rnnpl69ernuuusyd+7cDB48eImHWi5uD8cTTjgh//jHPzJp0qQMGzYsw4YNq3R99+7ds99++y1zHAutt956ueeee/Lb3/4277zzTiZMmJBu3botsl6NGjXSqVOnnHDCCcu97W9i4es1d+7cDBw4cIl7QVWrVi1nnHHGYoM9AN8Pgg9AAVWvXj21a9dOnTp10rhx4zRv3jzbb7999tlnn+XeM2RpbrnllgwaNChDhw7NyJEj88EHH5QOfahXr1623HLL7Lnnnjn66KOXGJY23HDD9OvXLz179szgwYMzfvz4fP7552tsEufDDz88zZo1y1133ZXBgwdn8uTJWWedddKsWbMcffTROeyww5Z6+ypVqqRLly7ZY4898sADD2TUqFH5/PPP07Bhw7Rt2zYnn3xymjZtulyHpWyzzTbp27dv/va3v5X2DFjZ+URatmyZp556Kg888ECeeeaZvPfee5k1a1Zq166dzTffPHvvvXeOP/74VRICV7X99tsvAwYMyD333JPnn38+48aNy+zZs7Puuutmq622yr777ptjjz12lby3vw/+4z/+I1tttVXuuOOOvPrqq5k2bVrq1KmTJk2aZP/998/xxx9f6RTlq9v555+fffbZJ4MGDcpbb72VsWPHZurUqZk/f37q1KmTzTffPG3bts0xxxxTaW6v1fmYTjnllBx88MG577778vLLL+eDDz7IrFmzUqNGjTRu3DgtWrTIvvvum7333nuR29avXz8PPfRQevbsmZdffjkffvhh5syZs1K/mxo3bpy+ffvmkUceyZNPPpmRI0dm+vTpWXvttdO4ceO0a9cuJ5xwwhKfn1XpxBNPTIsWLfLKK6/kzTffzPvvv58pU6akrKwstWvXTpMmTdKqVascc8wxleYnA+D7p0r5d+H0KAAAAACsMiZtBgAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACiYtdb0AIDvlzvuuCMDBgzI+++/n/Ly8my11Vb5zW9+kz333LPSenfffXfuvvvuTJgwIXXr1k27du1ywQUXZP31119km1OmTEn79u3zySef5Pnnn0+jRo2SJJMmTcoll1ySd999NzNmzEi9evWy22675fe//31pnSS5/PLLM2LEiLz77rspKyvFSQYfAAAa9ElEQVTL22+/vch9/PSnP82ECRMqLdtpp51y7733roqnBQAA4DvFHj7AChk8eHCOOuqo9OnTJw8++GBatmyZDh065PXXXy+t88QTT+SKK67Iqaeemsceeyw33HBDRo4cmYsuumiR7S1YsCDnn39+WrRosch11apVywEHHJAePXrkqaeeyg033JCxY8emQ4cOi2zj0EMPzfHHH7/UsZ9xxhl56aWXSv/dcsst3/BZAAAA+G6zhw+wQu64445KP1944YV58cUXM2DAgOy8885JkmHDhqVp06Y55phjkiSbbLJJjjvuuHTt2nWR7d18882pXr16TjnllDz33HOVrttggw3y85//vPTzxhtvnF//+tfp2LFjZs2albp16yZJ/vM//zNJ0rdv36WOvVatWmnYsOEKPmIAAIDvH3v4ACtlwYIFmT17dtZZZ53Ssp133jljxozJkCFDUl5enilTpqR///7Za6+9Kt128ODBeeCBB3LVVVelSpUqy7yvadOm5Z///GeaN29eij0r4u67706bNm1yyCGH5C9/+UumT5++wtsAAAD4PrCHD7BSevTokZkzZ+a4444rLTvooIMya9as/PrXv868efMyb9687L333rniiitK63zyySe54IILctVVV6VBgwZ57733lngfv//97/PMM8/kiy++SMuWLRfZy2h5/PKXv0yzZs3SoEGDvP/++7nhhhvy4osv5uGHH87aa6+9wtsDAAD4LrOHD/CN3X333bn11lvTtWvXSpMov/baa7n++utz4YUX5qGHHsptt92WCRMmpHPnzqV1zj///LRv3z677bbbMu+nc+fO6devX26//fYkyXnnnZf58+ev0FhPO+207LbbbmnatGkOPvjg3HHHHRk3blwGDBiwQtsBAAD4PrCHD/CN9OzZM926dcstt9yySLS5/vrrs//+++eEE05IkjRr1iy1a9fOCSeckLPPPjs//vGPM2jQoAwdOjQ9e/ZMkpSXlyf56mxaRx99dC677LLS9ho2bJiGDRvmJz/5SZo1a5Y99tgjL7/88iJnBlsRTZo0yfrrr7/ImbsAAACKQPABVtiNN96Y3r1757bbbkvr1q0Xuf7zzz9P1aqVdyBc+PPCsPPII49Uuv6tt97KxRdfnJ49e+YnP/nJEu974e2//PLLlXoMH3/8caZOnVppzyQAAICiEHyAFXL55Zfn/vvvz3XXXZfNN988U6ZMSZKsvfbapYmU991339x2223Zfvvts8suu+Tjjz/OFVdckaZNm2bTTTdNkmy99daVtrtwAuXNN988G264YZKkf//++fzzz7Ptttumdu3aGTduXOnwsbZt25ZuO27cuMyZMycTJ05MkowaNSpJsummm6Z27doZPnx4hg0blrZt26ZevXp5//33c+2112ajjTbK/vvvvxqfLQAAgDVD8AFWSJ8+fZIkHTt2rLT8iCOOyJVXXpkk6dChQ6pVq5YePXpk0qRJWXfdddOmTZv84Q9/WGTPn6WpUaNGevXqlTFjxmTu3LnZcMMN065du1x//fWpU6dOab0//vGPGTp0aOnn9u3bl8bapk2b1KhRIwMGDMitt96aOXPmpFGjRmnXrl3OOuus1K5d+xs/FwAAAN9VVcoXHh/BYs2a80Xe+/CjNT0MAFjttmzSKHVrOWsdAEAR2MNnGd778KOcfd1da3oYALDadf39iWnZdLM1PQwAAFYBp2UHAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQcAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAIAlGDRoULbZZpvsv//+pWXz5s3L7bffngMPPDAtWrTIAQcckLvvvrvS7Z5++umcccYZadeuXXbYYYcccsghufPOO1NeXv5tPwTgB2qtNT0AAACA76IpU6akU6dOadeuXcaNG1da3q1bt9x///357//+7zRr1izDhw/Pn/70p1SvXj3HHntskmTo0KHZcccd89vf/jbrr79+Xn311Vx66aWZO3dufv3rX6+phwT8gAg+AAAAX7NgwYJccMEFOeGEEzJ37txKwadfv3457bTTSnv9NGnSJG+++WZ69OhRCj4XX3xxpe01adIkb7/9dp588knBB/hWOKQLAADga26++eZUqVIlZ5xxxiLXzZ07NzVq1Ki0bO21186ECRMyYcKEJW5z5syZWWeddVb5WAEWR/ABAACoYPDgwbnvvvty9dVXp0qVKotcv+eee+auu+7K6NGjU15enjfeeCMPPfRQkmTy5MmL3eaQIUPy2GOP5fTTT1+tYwdYSPABAAD4P9OmTcsFF1yQK664Ig0bNlzsOpdcckm22267tG/fPs2bN88555yTo48+OklSteqiX7FGjBiRjh075qyzzspPf/rT1Tp+gIXM4QMAAPB//vWvf2Xy5Mnp0KFDadmCBQtSXl6ebbfdNldddVUOO+yw3Hjjjfnyyy8zbdq0bLDBBrn33nuTfDVXT0VDhgxJhw4dcuaZZ1baJsDqJvgAAAD8nxYtWuSRRx6ptOyee+7JwIEDc9ttt2WjjTYqLa9Ro0YaNWqUJHnsscfSqlWrrLfeeqXrBw4cmHPOOSfnnHNOTjvttG/nAQD8H8EHAADg/9SqVStbb711pWUNGjRI9erVS8vffPPNTJgwIc2bN8/UqVPzt7/9LaNGjSrt5ZMkTzzxRC644IKcccYZOeywwzJlypQkSbVq1SpFIYDVRfBZhi2bNErX35+4pocBAKvdlk0arekhAHwvfPnll+nevXs++OCDVK9ePa1atcp9992Xpk2blta55557UlZWlptvvjk333xzaXnjxo3z7LPProlhAz8wVcrLy8vX9CAAAAAAWHWcpQsAAACgYAQfAAAAgIIRfAAAAAAKRvABAAAAKBjBBwAAAKBgBB8AAACAghF8AAAAAApG8AEAAAAoGMEHAAAAoGAEHwAAAICCEXwAAAAACkbwAQAAACgYwQdIknTr1i1NmzbN6aefvsh1Z599dk488cQV2t7UqVPTrVu3jB8/fpnr9u3bN02bNl3kv/3333+F7nN16N27d7bddts1PQwAYBVa+O+er/93yimnrNFxXX755d+Jf/8AxbDWmh4A8N3y0ksv5c0338z222+/UtuZOnVqbrrpprRu3TqbbLLJct3mzjvvzNprr136uWbNmis1BgCAJalbt27uuOOORZYBFIXgA5TUq1cvG2ywQXr06JGbb775W7//Fi1apHbt2su17hdffFEpDgEArIhq1aplxx13XK51/bsD+D5ySBdQyW9+85s8++yzGT169FLXGzVqVE4++eTssMMOadWqVf7whz/kk08+SZKMHz8+hx12WJLkpJNOKu0mvTL23HPP/PWvf81NN92UPfbYI61bt06SvP766+nQoUN23333tGzZMu3bt89jjz1W6bbXX3992rVrV2nZvHnz0rRp09x7772lZXPnzs1//dd/Zeedd06bNm1y5ZVXZt68eSs1bgDg+2XhvxHuvPPO/OUvf0nbtm3Tvn37JMmzzz6bU045JW3bts1OO+2U4447Lq+88kql259//vk59thjKy0bN25cmjZtmhdeeKG0bMaMGTnvvPOy4447Zvfdd89tt922+h8c8INiDx+gkoMOOig33nhjevTokeuvv36x60ybNi0nnnhitthii1x77bWZPXt2rr322px66ql56KGHssEGG+Saa67J+eefnz/96U9p3rz5ct33ggULKgWWatWqpUqVKqWfH3744Wy99da59NJLM3/+/CTJhAkTsvPOO+cXv/hFatSokddffz0XXnhhqlatmoMPPniFHvvVV1+dfv365fe//30233zz3H///Xn88cdXaBsAwPfH1/+wU61atdLl22+/PW3atMnVV1+d8vLyJF/9UWvffffNr371q1SpUiUDBw7M6aefnvvuuy877LDDCt13p06dMnz48FxyySVp0KBB7rjjjowfP94h7cAqI/gAlVStWjVnnnlmLrnkkpx99tnZfPPNF1mnV69eSZKePXumTp06SZLNNtssxx57bJ566v+1dy8hUXZxHMd/3maymiB10hpalWWBCGEXpouFdIFEXCitiqKBaCgqImtRLXJMWmg1DmS1GBCk6AKuWrxQE2XQJiW1mNoEFUZNSToiOeHMuwgffHystymD1+n7Wc38z5lzntkd/uf/nPOPKioqjIqexYsX/3S5dGlpqem7z+dTTU2N6dlaWlpks9mMWGVlpfE5kUho5cqV6uvr082bN5NK+PT39+vGjRs6cuSIcWDjunXrtG3btp8eAwAATB+fP3+2bEoFg0Gjijg/P1+NjY2m9l27dhmf4/G4Vq9erZcvX+rWrVtJJXzC4bBCoZD8fr+2bt0qSVq1apXKyspI+ACYMiR8AFhUVlYqEAjoypUramhosLR3d3dr7dq1RrJHkkpKSuRyufTkyRNVVFT80rxtbW2mRY7L5TK1u91uU7JH+rZY8/v9CoVCev/+vVH5M/G3/yUcDisWi6m8vNyIZWRkqLy8XK2trcn+FQAA8D/ncDgUDAZNsfEbXRs3brT85t27d2pqatLjx48ViUSMyp94PJ7U3D09PUpPT9emTZuM2OzZs+V2uxUOh5MaCwC+h4QPAIvMzEx5PB7V19frwIEDlvZIJKLCwkJLPC8vTwMDA78877Jly354aHNubq4lVltbq+fPn2v//v1atGiRZs2apba2Nj18+DCpuSORyKRz5OTkJDUOAACYHjIyMlRcXGyJj73mNXFNMDo6qn379mlkZESHDx/WwoULlZ2drfPnz2toaCipuSORiBwOh2Uji3UHgKlEwgfApKqrq3Xp0iVdvXrV0uZ0OvXp0ydL/OPHjz99Xs+vGH+ejyQNDw/rwYMHqqurM736NVblM8Zut+vr16+m2ODgoOm70+mU9O06+fGVS/39/VPy7AAAYHqZuO549eqVXrx4oWAwKLfbbcS/fPli6jfZumPihpjT6VQ0GlUsFjMlfVh3AJhK3NIFYFI2m0179+7V7du39eHDB1NbSUmJOjo6TLtZ3d3dxgHKkpSVlSXp281Xf8rIyIgSiYQxlyRFo1Hdv3/f1K+goEADAwPGLWKS1NHRYeqzdOlSZWVl6e7du0ZsdHRU9+7d+zMPDwAAppWxxM74BM2bN2/09OlTU7+CggK9fftWsVjMiD169MjUp7i4WPF4XKFQyIgNDQ1ZbvwCgN9BhQ+A79qxY4daWlrU1dVlHGAoSXv27NG1a9fk8Xjk8Xg0PDysxsZGLVmyRFu2bJEkLViwQDNmzFB7e7scDocyMzMnLZv+HXPnztXy5csVCAQ0c+ZMSdLly5c1Z84c027bhg0bZLfbdeLECe3evVuvX7/W9evXTWPl5uaqpqZGFy5cUHp6unFL18RdOwAA8HcqLCzUvHnzdPbsWR06dEjRaFR+v1/5+fmmfps3b1YgENDJkydVVVWl3t5etbe3m/oUFRWprKxMp0+f1uDgoHFL149ebQeAZFHhA+C7srOzjRurxsvJyVFra6tsNpuOHj2qM2fOqLS0VMFg0Nj1stvtqqur07Nnz7Rz505VV1f/kWdsamrS/PnzVVtbq4aGBm3fvt1yaHReXp4uXryovr4+eb1e3blzx3LrhiQdP35cVVVVam5u1rFjx+RyuUy3cQAAgL+X3W5XIBBQWlqaDh48qObmZnm9Xq1YscLUr6ioSD6fT52dnfJ6vers7FR9fb1lvHPnzmnNmjXy+Xw6deqU1q9fb9zYBQBTIS0xdrQ8AAAAAAAAUgIVPgAAAAAAACmGhA8AAAAAAECKIeEDAAAAAACQYkj4AAAAAAAApBgSPgAAAAAAACmGhA8AAAAAAECKIeEDAAAAAACQYkj4AAAAAAAApBgSPgAAAAAAACnmXw+RRSb+dwIHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x504 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# using seaborns countplot to show distribution of questions in dataset\n",
    "fig, ax = plt.subplots()\n",
    "g = sns.countplot(df.Class, palette='viridis')\n",
    "g.set_xticklabels(['Not Fraud', 'Fraud'])\n",
    "g.set_yticklabels([])\n",
    "\n",
    "# function to show values on bars\n",
    "def show_values_on_bars(axs):\n",
    "    def _show_on_single_plot(ax):        \n",
    "        for p in ax.patches:\n",
    "            _x = p.get_x() + p.get_width() / 2\n",
    "            _y = p.get_y() + p.get_height()\n",
    "            value = '{:.0f}'.format(p.get_height())\n",
    "            ax.text(_x, _y, value, ha=\"center\") \n",
    "\n",
    "    if isinstance(axs, np.ndarray):\n",
    "        for idx, ax in np.ndenumerate(axs):\n",
    "            _show_on_single_plot(ax)\n",
    "    else:\n",
    "        _show_on_single_plot(axs)\n",
    "show_values_on_bars(ax)\n",
    "\n",
    "sns.despine(left=True, bottom=True)\n",
    "plt.xlabel('')\n",
    "plt.ylabel('')\n",
    "plt.title('Distribution of Transactions', fontsize=30)\n",
    "plt.tick_params(axis='x', which='major', labelsize=15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_uuid": "a22d03aa7bf5b358cfbc7ca946f3367b98e9faa2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.17304750013189596"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print percentage of questions where target == 1\n",
    "(len(df.loc[df.Class==1])) / (len(df.loc[df.Class == 0])) * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "460b8fbc891a865d755de263e832474d22718577"
   },
   "source": [
    "From the plot above, we can see we have a very imbalanced class -  just 0.17% of our dataset belong to the target class!\n",
    "\n",
    "This is a problem because many machine learning models are designed to maximize overall accuracy, which especially with imbalanced classes may not be the best metric to use. Classification accuracy is defined as the number of correct predictions divided by total predictions times 100. For example, if we simply predicted all transactions are not fraud, we would get a classification acuracy score of over 99%!\n",
    "\n",
    "### Create Train and Test Sets\n",
    "\n",
    "The training set is used to build and validate the model, while the test set is reserved for testing the model on unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_uuid": "876d7f98c708ae91ce22f728d63abdbb1c573fac"
   },
   "outputs": [],
   "source": [
    "# Prepare data for modeling\n",
    "# Separate input features and target\n",
    "y = df.Class\n",
    "X = df.drop('Class', axis=1)\n",
    "\n",
    "# setting up testing and training sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=27)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "f5b635aecad2e3a59eee638e62194f02855f4f8f"
   },
   "source": [
    "## Baseline Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "537af7e533e55efc91ecda7a6f80b4c88bac39a1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique predicted labels:  [0]\n",
      "Test score:  0.9981461194910255\n"
     ]
    }
   ],
   "source": [
    "# DummyClassifier to predict only target 0\n",
    "dummy = DummyClassifier(strategy='most_frequent').fit(X_train, y_train)\n",
    "dummy_pred = dummy.predict(X_test)\n",
    "\n",
    "# checking unique labels\n",
    "print('Unique predicted labels: ', (np.unique(dummy_pred)))\n",
    "\n",
    "# checking accuracy\n",
    "print('Test score: ', accuracy_score(y_test, dummy_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "4b3d37e60424e9bc6a0ffdf36bc667a7489ce186"
   },
   "source": [
    "As predicted our accuracy score for classifying all transactions as not fraud is 99.8%!  \n",
    "\n",
    "As the Dummy Classifier predicts only Class 0, it is clearly not a good option for our objective of correctly classifying fraudulent transactions.\n",
    "\n",
    "Let's see how logistic regression performs on this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_uuid": "3326dfa795f14d1f4357881c3dde02f84b5574dd"
   },
   "outputs": [],
   "source": [
    "# Modeling the data as is\n",
    "# Train model\n",
    "lr = LogisticRegression(solver='liblinear').fit(X_train, y_train)\n",
    " \n",
    "# Predict on training set\n",
    "lr_pred = lr.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_uuid": "df5a5a9588b0d0ee1d9b80e94d1c63885718976d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9992135052386169"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking accuracy\n",
    "accuracy_score(y_test, lr_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_uuid": "64afcae23af7b0903f10fa0d2992de3a137e2c10"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    71108\n",
       "1       94\n",
       "Name: 0, dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking unique values\n",
    "predictions = pd.DataFrame(lr_pred)\n",
    "predictions[0].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "f50e9de7d0b21afebbdf41c36e30a3dd7e4204e0"
   },
   "source": [
    "Logistic Regression outperformed the Dummy Classifier!  We can see that it predicted 94 instances of class 1, so this is definitely an improvement.  But can we do better?\n",
    "\n",
    "Let's see if we can apply some techniques for dealing with class imbalance to improve these results.\n",
    "\n",
    "## 1.  Change the performance metric\n",
    "Accuracy is not the best metric to use when evaluating imbalanced datasets as it can be misleading.  Metrics that can provide better insight include:\n",
    " - **Confusion Matrix:**  a talbe showing correct predictions and types of incorrect predictions.\n",
    " - **Precision: **  the number of true positives divided by all positive predictions. Precision is also called Positive Predictive Value. It is a measure of a classifier's exactness. Low precision indicates a high number of false positives.\n",
    " - **Recall:**  the number of true positives divided by the number of positive values in the test data. Recall is also called Sensitivity or the True Positive Rate. It is a measure of a classifier's completeness. Low recall indicates a high number of false negatives.\n",
    " - **F1: Score:**  the weighted average of precision and recall.\n",
    " \n",
    "Since our main objective with the dataset is to prioritize accuraltely classifying fraud cases the recall score can be considered our main metric to use for evaluating outcomes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_uuid": "eccc36a7bfcea78426978634b710cc0133a38806"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7522123893805309"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# f1 score\n",
    "f1_score(y_test, lr_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_uuid": "e45f328fd05fff579301f2f060d6e8412b0c50c2"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>71061</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>47</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0   1\n",
       "0  71061   9\n",
       "1     47  85"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confusion matrix\n",
    "pd.DataFrame(confusion_matrix(y_test, lr_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_uuid": "0e1e8e7a188457a84e07f11f1789d7b864b5bef8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6439393939393939"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recall_score(y_test, lr_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
    "_uuid": "dffefb9a7656b70369b1392747345f1b7c5ad830"
=======
    "_uuid": "dffefb9a7656b70369b1392747345f1b7c5ad830",
    "collapsed": true
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
=======
    "_uuid": "dffefb9a7656b70369b1392747345f1b7c5ad830",
    "collapsed": true
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
=======
    "_uuid": "dffefb9a7656b70369b1392747345f1b7c5ad830",
    "collapsed": true
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
   },
   "source": [
    "We have a very high accuracy score of 0.999 but a F1 score of only 0.752.  And from the confusion matrix, we can see we are misclassifying several observations leading to a recall score of only 0.64.\n",
    "\n",
    "## 2. Change the algorithm\n",
    "While in every machine learning problem, its a good rule of thumb to try a variety of algorithms, it can be especially beneficial with imbalanced datasets.  Decision trees frequently perform well on imbalanced data.  They work by learning a hierachy of if/else questions.  This can force both classes to be addressed.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "_uuid": "aa114a52def766752d99bdbb8798926b08bfa7c0"
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_uuid": "f12b26ae5dd82aac3a909dba0346055d0e9162b1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9995646189713772"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train model\n",
    "rfc = RandomForestClassifier(n_estimators=10).fit(X_train, y_train)\n",
    "\n",
    "# predict on test set\n",
    "rfc_pred = rfc.predict(X_test)\n",
    "\n",
    "accuracy_score(y_test, rfc_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_uuid": "88a80554fe0898f9a7b4a129e72bd08e3a2b0e23"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8691983122362869"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# f1 score\n",
    "f1_score(y_test, rfc_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "_uuid": "48bbc426053a07d88363c8207254b8c60cd97fdf"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>71068</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>29</td>\n",
       "      <td>103</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1\n",
       "0  71068    2\n",
       "1     29  103"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confusion matrix\n",
    "pd.DataFrame(confusion_matrix(y_test, rfc_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "_uuid": "0d2fe2f258a7f3a2db17b963b76116f00fa5c1f1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7803030303030303"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# recall score\n",
    "recall_score(y_test, rfc_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "adead2ec74b7016ebc547ee60e0b44dc1189a2a4"
   },
   "source": [
    "# Resampling Techniques\n",
    "\n",
    "## 3. Oversampling Minority Class\n",
    "Oversampling can be defined as adding more copies of the minority class.  Oversampling can be a good choice when you don't have a ton of data to work with.  A con to consider when undersampling is that it can cause overfitting and poor generalization to your test set.\n",
    "\n",
    "We will use the resampling module from Scikit-Learn to randomly replicate samples from the minority class.\n",
    "\n",
    "### **Important Note**\n",
    "Always split into test and train sets BEFORE trying any resampling techniques!  Oversampling before splitting the data can allow the exact same observations to be present in both the test and train sets!  This can allow our model to simply memorize specific data points and cause overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "_uuid": "e48e7eb85c4b0a67c3f096726fef7b472e6b293d"
   },
   "outputs": [],
   "source": [
    "from sklearn.utils import resample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "_uuid": "8d7567a5b4ebc92b3aaff6236f2253d969488993"
   },
   "outputs": [],
   "source": [
    "# Separate input features and target\n",
    "y = df.Class\n",
    "X = df.drop('Class', axis=1)\n",
    "\n",
    "# setting up testing and training sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=27)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "_uuid": "523a5c12eb7520127ba53231e3872a21b3101ce7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>V10</th>\n",
       "      <th>V11</th>\n",
       "      <th>V12</th>\n",
       "      <th>V13</th>\n",
       "      <th>V14</th>\n",
       "      <th>V15</th>\n",
       "      <th>V16</th>\n",
       "      <th>V17</th>\n",
       "      <th>V18</th>\n",
       "      <th>V19</th>\n",
       "      <th>V20</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>264873</th>\n",
       "      <td>161634.0</td>\n",
       "      <td>-0.395578</td>\n",
       "      <td>1.489129</td>\n",
       "      <td>-0.833442</td>\n",
       "      <td>-0.224271</td>\n",
       "      <td>0.369444</td>\n",
       "      <td>-1.453886</td>\n",
       "      <td>0.796593</td>\n",
       "      <td>-0.060403</td>\n",
       "      <td>0.338270</td>\n",
       "      <td>-0.343885</td>\n",
       "      <td>-0.647990</td>\n",
       "      <td>-0.509490</td>\n",
       "      <td>-0.633729</td>\n",
       "      <td>-0.712156</td>\n",
       "      <td>0.765701</td>\n",
       "      <td>0.053830</td>\n",
       "      <td>0.526211</td>\n",
       "      <td>0.750505</td>\n",
       "      <td>-0.225921</td>\n",
       "      <td>0.054132</td>\n",
       "      <td>0.231624</td>\n",
       "      <td>0.955194</td>\n",
       "      <td>-0.172092</td>\n",
       "      <td>-0.041050</td>\n",
       "      <td>-0.313444</td>\n",
       "      <td>-0.174301</td>\n",
       "      <td>0.064657</td>\n",
       "      <td>-0.036960</td>\n",
       "      <td>2.74</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163821</th>\n",
       "      <td>116237.0</td>\n",
       "      <td>1.950487</td>\n",
       "      <td>0.002312</td>\n",
       "      <td>-1.761814</td>\n",
       "      <td>1.232470</td>\n",
       "      <td>0.523175</td>\n",
       "      <td>-0.650657</td>\n",
       "      <td>0.504231</td>\n",
       "      <td>-0.200857</td>\n",
       "      <td>0.116805</td>\n",
       "      <td>0.443750</td>\n",
       "      <td>0.257633</td>\n",
       "      <td>0.358875</td>\n",
       "      <td>-1.313311</td>\n",
       "      <td>0.922108</td>\n",
       "      <td>-1.078538</td>\n",
       "      <td>-0.395038</td>\n",
       "      <td>-0.437151</td>\n",
       "      <td>0.074223</td>\n",
       "      <td>0.187086</td>\n",
       "      <td>-0.285207</td>\n",
       "      <td>0.086306</td>\n",
       "      <td>0.326297</td>\n",
       "      <td>-0.068839</td>\n",
       "      <td>-0.416589</td>\n",
       "      <td>0.426044</td>\n",
       "      <td>-0.486299</td>\n",
       "      <td>-0.031266</td>\n",
       "      <td>-0.072543</td>\n",
       "      <td>38.44</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72083</th>\n",
       "      <td>54557.0</td>\n",
       "      <td>1.105167</td>\n",
       "      <td>-0.166253</td>\n",
       "      <td>0.569520</td>\n",
       "      <td>0.681043</td>\n",
       "      <td>-0.259189</td>\n",
       "      <td>0.642792</td>\n",
       "      <td>-0.437034</td>\n",
       "      <td>0.356746</td>\n",
       "      <td>0.441417</td>\n",
       "      <td>-0.113006</td>\n",
       "      <td>1.263824</td>\n",
       "      <td>1.017046</td>\n",
       "      <td>-0.791270</td>\n",
       "      <td>0.181033</td>\n",
       "      <td>-0.530067</td>\n",
       "      <td>-0.733662</td>\n",
       "      <td>0.436861</td>\n",
       "      <td>-0.903134</td>\n",
       "      <td>-0.242185</td>\n",
       "      <td>-0.226981</td>\n",
       "      <td>0.009073</td>\n",
       "      <td>0.293023</td>\n",
       "      <td>-0.028688</td>\n",
       "      <td>-0.242206</td>\n",
       "      <td>0.389813</td>\n",
       "      <td>0.482852</td>\n",
       "      <td>0.010705</td>\n",
       "      <td>-0.008399</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196949</th>\n",
       "      <td>131771.0</td>\n",
       "      <td>1.805238</td>\n",
       "      <td>0.961264</td>\n",
       "      <td>-1.717212</td>\n",
       "      <td>4.094625</td>\n",
       "      <td>0.938666</td>\n",
       "      <td>-0.227785</td>\n",
       "      <td>0.152911</td>\n",
       "      <td>0.066753</td>\n",
       "      <td>-1.073784</td>\n",
       "      <td>0.334537</td>\n",
       "      <td>1.016183</td>\n",
       "      <td>-0.510443</td>\n",
       "      <td>-0.748733</td>\n",
       "      <td>-2.708662</td>\n",
       "      <td>-1.014069</td>\n",
       "      <td>2.220866</td>\n",
       "      <td>1.285249</td>\n",
       "      <td>1.619087</td>\n",
       "      <td>-1.602343</td>\n",
       "      <td>-0.150818</td>\n",
       "      <td>-0.137875</td>\n",
       "      <td>-0.450959</td>\n",
       "      <td>0.098530</td>\n",
       "      <td>-0.662272</td>\n",
       "      <td>-0.150154</td>\n",
       "      <td>-0.098852</td>\n",
       "      <td>-0.000030</td>\n",
       "      <td>0.017622</td>\n",
       "      <td>37.89</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126213</th>\n",
       "      <td>77959.0</td>\n",
       "      <td>0.835421</td>\n",
       "      <td>-1.191847</td>\n",
       "      <td>0.578455</td>\n",
       "      <td>0.586101</td>\n",
       "      <td>-1.236663</td>\n",
       "      <td>0.194617</td>\n",
       "      <td>-0.532404</td>\n",
       "      <td>0.061561</td>\n",
       "      <td>-0.734344</td>\n",
       "      <td>0.754427</td>\n",
       "      <td>0.513247</td>\n",
       "      <td>0.704388</td>\n",
       "      <td>0.472193</td>\n",
       "      <td>-0.072043</td>\n",
       "      <td>-0.357247</td>\n",
       "      <td>-1.136932</td>\n",
       "      <td>-0.436168</td>\n",
       "      <td>2.047641</td>\n",
       "      <td>-0.775130</td>\n",
       "      <td>-0.045313</td>\n",
       "      <td>-0.072349</td>\n",
       "      <td>-0.109154</td>\n",
       "      <td>-0.308356</td>\n",
       "      <td>0.011968</td>\n",
       "      <td>0.461350</td>\n",
       "      <td>-0.244810</td>\n",
       "      <td>0.031845</td>\n",
       "      <td>0.060910</td>\n",
       "      <td>237.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Time        V1        V2  ...         V28  Amount  Class\n",
       "264873  161634.0 -0.395578  1.489129  ...   -0.036960    2.74      0\n",
       "163821  116237.0  1.950487  0.002312  ...   -0.072543   38.44      0\n",
       "72083    54557.0  1.105167 -0.166253  ...   -0.008399    1.00      0\n",
       "196949  131771.0  1.805238  0.961264  ...    0.017622   37.89      0\n",
       "126213   77959.0  0.835421 -1.191847  ...    0.060910  237.00      0\n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# concatenate our training data back together\n",
    "X = pd.concat([X_train, y_train], axis=1)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "_uuid": "fd10e5f5f96a9c3fe607d0fb35475fe62fa75675"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    213245\n",
       "0    213245\n",
       "Name: Class, dtype: int64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# separate minority and majority classes\n",
    "not_fraud = X[X.Class==0]\n",
    "fraud = X[X.Class==1]\n",
    "\n",
    "# upsample minority\n",
    "fraud_upsampled = resample(fraud,\n",
    "                          replace=True, # sample with replacement\n",
    "                          n_samples=len(not_fraud), # match number in majority class\n",
    "                          random_state=27) # reproducible results\n",
    "\n",
    "# combine majority and upsampled minority\n",
    "upsampled = pd.concat([not_fraud, fraud_upsampled])\n",
    "\n",
    "# check new class counts\n",
    "upsampled.Class.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "_uuid": "48e16be49fbda450115a2d169d3edc5979f633c0"
   },
   "outputs": [],
   "source": [
    "# trying logistic regression again with the balanced dataset\n",
    "y_train = upsampled.Class\n",
    "X_train = upsampled.drop('Class', axis=1)\n",
    "\n",
    "upsampled = LogisticRegression(solver='liblinear').fit(X_train, y_train)\n",
    "\n",
    "upsampled_pred = upsampled.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "_uuid": "24f541f7a98c359d213baf5e6463b6245369fc74"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9807589674447347"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking accuracy\n",
    "accuracy_score(y_test, upsampled_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "_uuid": "f3235cf726cf002344376b279db4c34b50a97948"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.14375000000000002"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# f1 score\n",
    "f1_score(y_test, upsampled_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "_uuid": "2334aa3ad1e8886a9756d01fd6010816ece03448"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69717</td>\n",
       "      <td>1353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>17</td>\n",
       "      <td>115</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0     1\n",
       "0  69717  1353\n",
       "1     17   115"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confusion matrix\n",
    "pd.DataFrame(confusion_matrix(y_test, upsampled_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "_uuid": "f7030ec30156d154644c0922c5d608ffdf620c66"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8712121212121212"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recall_score(y_test, upsampled_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "a0b5c7e2b673baac682bb89d9484ad9f7fbea16e"
   },
   "source": [
    "Our accuracy score decreased after upsampling, but the model is now predicting both classes more equally, making it an improvement over our plain logistic regression above.\n",
    "\n",
    "## 4. Undersampling Majority Class\n",
    "Undersampling can be defined as removing some observations of the majority class.  Undersampling can be a good choice when you have a ton of data -think millions of rows.  But a drawback to undersampling is that we are removing information that may be valuable.\n",
    "\n",
    "We will again use the resampling module from Scikit-Learn to randomly remove samples from the majority class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "_uuid": "15c289c9551acca877a3ca824c6b2f074daccce6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    360\n",
       "0    360\n",
       "Name: Class, dtype: int64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# still using our separated classes fraud and not_fraud from above\n",
    "\n",
    "# downsample majority\n",
    "not_fraud_downsampled = resample(not_fraud,\n",
    "                                replace = False, # sample without replacement\n",
    "                                n_samples = len(fraud), # match minority n\n",
    "                                random_state = 27) # reproducible results\n",
    "\n",
    "# combine minority and downsampled majority\n",
    "downsampled = pd.concat([not_fraud_downsampled, fraud])\n",
    "\n",
    "# checking counts\n",
    "downsampled.Class.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "_uuid": "339ff5f558175b8daf2a4536d4749978e7e7c4fd"
   },
   "outputs": [],
   "source": [
    "# trying logistic regression again with the undersampled dataset\n",
    "\n",
    "y_train = downsampled.Class\n",
    "X_train = downsampled.drop('Class', axis=1)\n",
    "\n",
    "undersampled = LogisticRegression(solver='liblinear').fit(X_train, y_train)\n",
    "\n",
    "undersampled_pred = undersampled.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "_uuid": "a4808d815c2531edfbb93ad420a40bf2aa4c85f4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9758574197354007"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking accuracy\n",
    "accuracy_score(y_test, undersampled_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "_uuid": "5f79a91effffd12bec83359230618af1c7ed23f3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.11710323574730355"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# f1 score\n",
    "f1_score(y_test, undersampled_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "_uuid": "9b8cab34cf1a0c10ca0f862909abbc83e35fbecd"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69369</td>\n",
       "      <td>1701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>114</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0     1\n",
       "0  69369  1701\n",
       "1     18   114"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confusion matrix\n",
    "pd.DataFrame(confusion_matrix(y_test, undersampled_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "_uuid": "8240af095c11b2a48da1eeba3a75e70aa902610c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9318181818181818"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recall_score(y_test, undersampled_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "0cdac5de057cf831e81f8766b539834accf46663"
   },
   "source": [
    "Downsampling produced a higher recall score than upsampling!  My concern here is the small number of total samples we used to train the model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9ce5844e2d4447485416cf927112a8c2cf3fd1d8"
   },
   "source": [
    "## 5. Generate Synthetic Samples\n",
    "SMOTE or Synthetic Minority Oversampling Technique is a popular algorithm to creates sythetic observations of the minority class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "_uuid": "5d01357038bb94f8a93c77141e462153c73c5ab8"
   },
   "outputs": [],
   "source": [
    "from imblearn.over_sampling import SMOTE\n",
    "\n",
    "# Separate input features and target\n",
    "y = df.Class\n",
    "X = df.drop('Class', axis=1)\n",
    "\n",
    "# setting up testing and training sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=27)\n",
    "\n",
    "sm = SMOTE(random_state=27, ratio=1.0)\n",
    "X_train, y_train = sm.fit_sample(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "_uuid": "669cb81d0cf57aa6c3769a586253ac5e1e5622bf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9858571388444145"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smote = LogisticRegression(solver='liblinear').fit(X_train, y_train)\n",
    "\n",
    "smote_pred = smote.predict(X_test)\n",
    "\n",
    "# Checking accuracy\n",
    "accuracy_score(y_test, smote_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "_uuid": "a53ecb2ee29d381dbc4da8fc897242c37153b75e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.18461538461538463"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# f1 score\n",
    "f1_score(y_test, smote_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "_uuid": "5be328bfd7f9661505264b65f085069e5de08b53"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>70081</td>\n",
       "      <td>989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18</td>\n",
       "      <td>114</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1\n",
       "0  70081  989\n",
       "1     18  114"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confustion matrix\n",
    "pd.DataFrame(confusion_matrix(y_test, smote_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "_uuid": "991a2802f90abfc195f6f0325b77ba45efc9e404"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8636363636363636"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recall_score(y_test, smote_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8bde53447fef057292a1d52b62978e53b7d44f8f"
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "We covered 5 different methods for dealing with imbalanced datasets:\n",
    "1.  Change the performance metric\n",
    "2.  Oversampling minority class\n",
    "3.  Undersampling majority class\n",
    "4.  Change the algorithm\n",
    "5.  Generate synthetic samples\n",
    "\n",
    "These are just some of the many possible methods to try when dealing with imbalanced datasets, and not an exhaustive list.  Some others methods to consider are collecting more data or choosing different resampling ratios - you don't have to have exactly a 1:1 ratio!  You should always try several approaches and then decide which is best for your problem."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
<<<<<<< HEAD
<<<<<<< HEAD
<<<<<<< HEAD
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
=======
=======
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
=======
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
<<<<<<< HEAD
<<<<<<< HEAD
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
=======
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
=======
>>>>>>> 07c098988adcfa3ac65bd07cefc36ca83c5e8f5a
}