3_TelecomChurnCaseStudy_Modelling_PCA_LR.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='red'>This file must be executed only after Pandarinath_TelecomChurnCaseStudy.ipynb is executed.</font>\n",
    "The complete solution is spred in multiple files:\n",
    "1. File-1: \"Pandarinath_TelecomChurnCaseStudy.ipynb\"\n",
    "        It comprises of cleaning of data and EDA\n",
    "           \n",
    "2. File-2: \"Pandarinath_TelecomChurnCaseStudy_Modelling_PCA_LR.ipynb\"\n",
    "        It comprises of execution of SOMTE to balance data, PCA for feature selection and finally LR model\n",
    "\n",
    "3. File-3: \"Pandarinath_TelecomChurnCaseStudy_Modelling_RFE_LR.ipynb\"\n",
    "\n",
    "4. File-4: \"Pandarinath_TelecomChurnCaseStudy_Modelling_LASSO_DT.ipynb\"\n",
    "\n",
    "5. File-5: \"Pandarinath_TelecomChurnCaseStudy_Modelling_LASSO_RF.ipynb\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='blue'>Telecom Churn Case Study</font>\n",
    "* Institution: IIIT, Bangalore and UpGrad\n",
    "* Course: PG Diploma in Machine Lerning and AI March 2018\n",
    "* Date: 13-Aug-2018\n",
    "* Submitted by:\n",
    "    1. Pandinath Siddineni (ID- APFE187000194)\n",
    "    2. AKNR Chandra Sekhar (ID- APFE187000315)\n",
    "    3. Brajesh Kumar       (ID- APFE187000149)\n",
    "    4. Shweta Tiwari\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color='blue'>Business Goals:</font>\n",
    "1. Retaining high profitable customers is the number one business goal.\n",
    "2. This project is based on the Indian and Southeast Asian market.\n",
    "3. In the Indian and the southeast Asian market, approximately 80% of revenue comes from the top 20% customers (called high-value customers). Thus, if we can reduce churn of the high-value customers, we will be able to reduce significant revenue leakage.\n",
    "4. The business objective is to predict the churn in the last (i.e. the ninth) month using the data (features) from the first three months. To do this task well, understanding the typical customer behaviour during churn will be helpful.\n",
    "\n",
    "### <font color='blue'>Analysis Goals:</font>\n",
    "1. Predict which customers are at high risk of churn\n",
    "2. Build predictive models to identify customers at high risk of churn and identify the main indicators of churn.\n",
    "3. Prepaid is the most common model in India and southeast Asia. Focus on prepaid customers.\n",
    "3. Curn definition used-- \"Usage-based churn: Customers who have not done any usage, either incoming or outgoing - in terms of calls, internet etc. over a period of time.\" In this project, we will use the usage-based definition to define churn.\n",
    "4. In this project, you will define high-value customers based on a certain metric (mentioned later below) and predict churn only on high-value customers.\n",
    "5. especially high-value customers go through  three phases of customer lifecycle: a. The ‘good’ phase, b. The ‘action’ phase, c. The ‘churn’ phase\n",
    "---------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color='blue'>PART 1: DATA UNDERSTANDING AND CLEANING</font>\n",
    "\n",
    "1. Understand the properties of loaded dataframe\n",
    "2. Idnetify Uniquness key\n",
    "3. Identify bad colums that has no infromation (all entries are null or same)\n",
    "4. Conver dates to meaningful number of days\n",
    "5. Remove columns with data that does not make much sense for our analysis\n",
    "6. Missing value treatment: replace with '0', mean or median; drop rows; drop columns\n",
    "7. Outlier Treatment\n",
    "8. Write data into a clean data file. This will be used to create master-df for analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import libraries and load datafile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# Until fuction: line seperator\n",
    "def print_ln():\n",
    "    print('-'*80, '\\n')\n",
    "    \n",
    "pd.options.display.float_format = '{:.2f}'.format\n",
    "\n",
    "# Load csv data file\n",
    "telecom_orig = pd.read_csv('telecom_churn_data_clean.csv', low_memory=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataframe Shape:  (28504, 144)\n",
      "-------------------------------------------------------------------------------- \n",
      "\n",
      "Dataframe Info: \n",
      "\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 28504 entries, 0 to 28503\n",
      "Columns: 144 entries, mobile_number to rech_days_left_8\n",
      "dtypes: float64(105), int64(39)\n",
      "memory usage: 31.3 MB\n",
      "-------------------------------------------------------------------------------- \n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mobile_number</th>\n",
       "      <th>arpu_6</th>\n",
       "      <th>arpu_7</th>\n",
       "      <th>arpu_8</th>\n",
       "      <th>onnet_mou_6</th>\n",
       "      <th>onnet_mou_7</th>\n",
       "      <th>onnet_mou_8</th>\n",
       "      <th>offnet_mou_6</th>\n",
       "      <th>offnet_mou_7</th>\n",
       "      <th>offnet_mou_8</th>\n",
       "      <th>...</th>\n",
       "      <th>fb7_1.0</th>\n",
       "      <th>fb8_0.0</th>\n",
       "      <th>fb8_1.0</th>\n",
       "      <th>total_rech_data_amt_6</th>\n",
       "      <th>total_rech_data_amt_7</th>\n",
       "      <th>total_rech_data_amt_8</th>\n",
       "      <th>churn</th>\n",
       "      <th>rech_days_left_6</th>\n",
       "      <th>rech_days_left_7</th>\n",
       "      <th>rech_days_left_8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7000701601</td>\n",
       "      <td>1069.18</td>\n",
       "      <td>1349.85</td>\n",
       "      <td>3171.48</td>\n",
       "      <td>57.84</td>\n",
       "      <td>54.68</td>\n",
       "      <td>52.29</td>\n",
       "      <td>453.43</td>\n",
       "      <td>567.16</td>\n",
       "      <td>325.91</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>3.00</td>\n",
       "      <td>6.00</td>\n",
       "      <td>5.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7001524846</td>\n",
       "      <td>378.72</td>\n",
       "      <td>492.22</td>\n",
       "      <td>137.36</td>\n",
       "      <td>413.69</td>\n",
       "      <td>351.03</td>\n",
       "      <td>35.08</td>\n",
       "      <td>94.66</td>\n",
       "      <td>80.63</td>\n",
       "      <td>136.48</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.00</td>\n",
       "      <td>354.00</td>\n",
       "      <td>207.00</td>\n",
       "      <td>0</td>\n",
       "      <td>5.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7002124215</td>\n",
       "      <td>514.45</td>\n",
       "      <td>597.75</td>\n",
       "      <td>637.76</td>\n",
       "      <td>102.41</td>\n",
       "      <td>132.11</td>\n",
       "      <td>85.14</td>\n",
       "      <td>757.93</td>\n",
       "      <td>896.68</td>\n",
       "      <td>983.39</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7000887461</td>\n",
       "      <td>74.35</td>\n",
       "      <td>193.90</td>\n",
       "      <td>366.97</td>\n",
       "      <td>48.96</td>\n",
       "      <td>50.66</td>\n",
       "      <td>33.58</td>\n",
       "      <td>85.41</td>\n",
       "      <td>89.36</td>\n",
       "      <td>205.89</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.00</td>\n",
       "      <td>712.00</td>\n",
       "      <td>540.00</td>\n",
       "      <td>0</td>\n",
       "      <td>12.00</td>\n",
       "      <td>24.00</td>\n",
       "      <td>7.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7000149764</td>\n",
       "      <td>977.02</td>\n",
       "      <td>2362.83</td>\n",
       "      <td>409.23</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5285.00</td>\n",
       "      <td>20424.00</td>\n",
       "      <td>455.00</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>5.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 144 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   mobile_number  arpu_6  arpu_7  arpu_8  onnet_mou_6  onnet_mou_7  \\\n",
       "0     7000701601 1069.18 1349.85 3171.48        57.84        54.68   \n",
       "1     7001524846  378.72  492.22  137.36       413.69       351.03   \n",
       "2     7002124215  514.45  597.75  637.76       102.41       132.11   \n",
       "3     7000887461   74.35  193.90  366.97        48.96        50.66   \n",
       "4     7000149764  977.02 2362.83  409.23         0.00         0.00   \n",
       "\n",
       "   onnet_mou_8  offnet_mou_6  offnet_mou_7  offnet_mou_8        ...         \\\n",
       "0        52.29        453.43        567.16        325.91        ...          \n",
       "1        35.08         94.66         80.63        136.48        ...          \n",
       "2        85.14        757.93        896.68        983.39        ...          \n",
       "3        33.58         85.41         89.36        205.89        ...          \n",
       "4         0.00          0.00          0.00          0.00        ...          \n",
       "\n",
       "   fb7_1.0  fb8_0.0  fb8_1.0  total_rech_data_amt_6  total_rech_data_amt_7  \\\n",
       "0        0        0        0                   0.00                   0.00   \n",
       "1        1        0        1                   0.00                 354.00   \n",
       "2        0        0        0                   0.00                   0.00   \n",
       "3        1        0        1                   0.00                 712.00   \n",
       "4        1        0        1                5285.00               20424.00   \n",
       "\n",
       "   total_rech_data_amt_8  churn  rech_days_left_6  rech_days_left_7  \\\n",
       "0                   0.00      1              3.00              6.00   \n",
       "1                 207.00      0              5.00              0.00   \n",
       "2                   0.00      0              0.00              0.00   \n",
       "3                 540.00      0             12.00             24.00   \n",
       "4                 455.00      0              0.00              1.00   \n",
       "\n",
       "   rech_days_left_8  \n",
       "0              5.00  \n",
       "1              1.00  \n",
       "2              0.00  \n",
       "3              7.00  \n",
       "4              5.00  \n",
       "\n",
       "[5 rows x 144 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Understand the properties of loaded dataframe\n",
    "telecom = telecom_orig.copy()\n",
    "print('Dataframe Shape: ', telecom.shape); print_ln();\n",
    "print(\"Dataframe Info: \\n\"); telecom.info(); print_ln();\n",
    "telecom.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Drop mobile number\n",
    "telecom.drop('mobile_number', axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create X, y variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = telecom.drop(['churn'],axis=1)\n",
    "y = telecom['churn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying standard scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "\n",
    "scaler = preprocessing.StandardScaler().fit(X)\n",
    "\n",
    "Xscaled = scaler.transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\bkumar5\\AppData\\Local\\Continuum\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2010: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(Xscaled,y, train_size=0.7,random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(19952, 142)\n",
      "(8552, 142)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape)\n",
    "print(X_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train Dataframe Shape (19952, 142)\n",
      "X_test Dataframe Shape (8552, 142)\n",
      "Imbalance in Train Data: 0.0613895095222896\n",
      "Imbalance in Test Data: 0.054500616522811345\n"
     ]
    }
   ],
   "source": [
    "print(\"X_train Dataframe Shape {}\".format(X_train.shape))\n",
    "print(\"X_test Dataframe Shape {}\".format(X_test.shape))\n",
    "\n",
    "y_train_imb = (y_train != 0).sum()/(y_train == 0).sum()\n",
    "y_test_imb = (y_test != 0).sum()/(y_test == 0).sum()\n",
    "print(\"Imbalance in Train Data: {}\".format(y_train_imb))\n",
    "print(\"Imbalance in Test Data: {}\".format(y_test_imb))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying smote"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(37596, 142)\n",
      "(37596,)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "18798"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from imblearn.over_sampling import SMOTE\n",
    "sm = SMOTE(kind = \"regular\")\n",
    "X_tr,y_tr = sm.fit_sample(X_train,y_train)\n",
    "print(X_tr.shape)\n",
    "print(y_tr.shape)\n",
    "np.count_nonzero(y_tr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_tr Dataframe Shape (37596, 142)\n",
      "y_tr Dataframe Shape (37596,)\n",
      "Imbalance in Train Data: 1.0\n"
     ]
    }
   ],
   "source": [
    "print(\"X_tr Dataframe Shape {}\".format(X_tr.shape))\n",
    "print(\"y_tr Dataframe Shape {}\".format(y_tr.shape))\n",
    "\n",
    "data_imbalance = (y_tr != 0).sum()/(y_tr == 0).sum()\n",
    "print(\"Imbalance in Train Data: {}\".format(data_imbalance))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PCA(copy=True, iterated_power='auto', n_components=None, random_state=100,\n",
       "  svd_solver='auto', tol=0.0, whiten=False)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Improting the PCA module\n",
    "from sklearn.decomposition import PCA\n",
    "pca = PCA(random_state=100)\n",
    "\n",
    "#Doing the PCA on the train data\n",
    "pca.fit(X_tr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(37596, 142)\n",
      "(8552, 142)\n"
     ]
    }
   ],
   "source": [
    "df_train_pca = pca.fit_transform(X_tr)\n",
    "print(df_train_pca.shape)\n",
    "\n",
    "df_test_pca = pca.transform(X_test)\n",
    "print(df_test_pca.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Apply logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's run the model using the selected variables\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn import metrics\n",
    "logsk = LogisticRegression(C=1e9)\n",
    "logsk.fit(df_train_pca, y_tr)\n",
    "\n",
    "# Predicted probabilities\n",
    "y_pred = logsk.predict(df_test_pca)\n",
    "# Converting y_pred to a dataframe which is an array\n",
    "y_pred_df = pd.DataFrame(y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[6668 1442]\n",
      " [  83  359]]\n"
     ]
    }
   ],
   "source": [
    "# Importing classification report and confusion matrix from sklearn metrics\n",
    "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score\n",
    "\n",
    "# Printing confusion matrix\n",
    "print(confusion_matrix(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression accuracy with PCA:  0.8216791393826005\n"
     ]
    }
   ],
   "source": [
    "print(\"LogisticRegression accuracy with PCA: \",accuracy_score(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.45819310e-01,  1.43660422e-01,  9.35753651e-02, ...,\n",
       "        -7.52974278e-02, -5.49848028e-02, -3.26576148e-02],\n",
       "       [ 8.85973032e-02,  1.14066753e-01,  1.55896768e-01, ...,\n",
       "         2.37892577e-02, -1.49781768e-02, -1.40445903e-02],\n",
       "       [ 6.69289529e-02,  6.59515168e-02,  7.37439499e-02, ...,\n",
       "         3.26909352e-02,  4.41484132e-02,  1.09180200e-02],\n",
       "       ...,\n",
       "       [-1.46530142e-16,  1.32537609e-16,  1.28073485e-16, ...,\n",
       "         3.77532260e-17, -1.10416445e-17, -1.56102889e-17],\n",
       "       [ 2.14615240e-16, -1.07054968e-16,  5.03222913e-18, ...,\n",
       "        -2.10671189e-18,  2.44028754e-17,  5.63878461e-17],\n",
       "       [ 0.00000000e+00, -1.92624939e-16,  7.88928410e-19, ...,\n",
       "        -3.23061727e-17, -2.03907165e-19,  3.23106414e-17]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature</th>\n",
       "      <th>PC1</th>\n",
       "      <th>PC2</th>\n",
       "      <th>PC3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>arpu_6</td>\n",
       "      <td>0.15</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>arpu_7</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.11</td>\n",
       "      <td>0.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>arpu_8</td>\n",
       "      <td>0.09</td>\n",
       "      <td>0.16</td>\n",
       "      <td>0.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>onnet_mou_6</td>\n",
       "      <td>0.17</td>\n",
       "      <td>-0.04</td>\n",
       "      <td>-0.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>onnet_mou_7</td>\n",
       "      <td>0.17</td>\n",
       "      <td>-0.02</td>\n",
       "      <td>-0.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>onnet_mou_8</td>\n",
       "      <td>0.11</td>\n",
       "      <td>0.03</td>\n",
       "      <td>-0.03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>offnet_mou_6</td>\n",
       "      <td>0.20</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>offnet_mou_7</td>\n",
       "      <td>0.20</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>offnet_mou_8</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>roam_ic_mou_6</td>\n",
       "      <td>0.02</td>\n",
       "      <td>-0.00</td>\n",
       "      <td>0.36</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Feature  PC1   PC2   PC3\n",
       "0         arpu_6 0.15  0.09  0.07\n",
       "1         arpu_7 0.14  0.11  0.07\n",
       "2         arpu_8 0.09  0.16  0.07\n",
       "3    onnet_mou_6 0.17 -0.04 -0.06\n",
       "4    onnet_mou_7 0.17 -0.02 -0.06\n",
       "5    onnet_mou_8 0.11  0.03 -0.03\n",
       "6   offnet_mou_6 0.20  0.03  0.01\n",
       "7   offnet_mou_7 0.20  0.06  0.01\n",
       "8   offnet_mou_8 0.12  0.10  0.01\n",
       "9  roam_ic_mou_6 0.02 -0.00  0.36"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colnames = list(X.columns)\n",
    "pcs_df = pd.DataFrame({'PC1':pca.components_[0],'PC2':pca.components_[1], 'PC3':pca.components_[2],'Feature':colnames})\n",
    "pcs_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAAHfCAYAAABAjl6PAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XmY3Fd95/t3dfW+SS2pJbX2zTre\n5X3BxmDABkPYBgIJW2CSMASSycSTTJJ770xCbmbunZvkZi7MQIAMkxAStgABs5jNxng3xjbWYh1J\n1r601Jt636vuH92S20ZLSepf/7qr3q/n0dO1dJU+Ooj25zn61vll8vk8kiRJks6sLO0AkiRJ0lxg\ncZYkSZIKYHGWJEmSCmBxliRJkgpgcZYkSZIKYHGWJEmSClCedoBCtbX1pnZuXlNTLV1dA2n99kXL\ndU2G65oc1zYZrmsyXNfkuLbJmE3r2tzckDnV4+44F6C8PJt2hKLkuibDdU2Oa5sM1zUZrmtyXNtk\nzIV1tThLkiRJBbA4S5IkSQWwOEuSJEkFsDhLkiRJBbA4S5IkSQWwOEuSJEkFsDhLkiRJBbA4S5Ik\nSQWwOEuSJEkFsDhLkiRJBbA4S5IkSQWwOEuSJEkFsDhLkiRJBbA4S5IkSQVItDiHEG4MIfz4FI+/\nMYTw0xDCoyGE30wygyRJkjQdEivOIYT/APwtUP2SxyuAvwbuBF4BfDCEsDSpHJIkSdJ0KE/wvZ8H\n/hXwDy95/BJgV4yxCyCE8BDwcuArCWaRJElzVC6fJ5fLM56b8nXysV94fOrz+fyp3/DcHv6FJ9v7\nRjl+fGDy4dO/6nS/vU7tWO/IyXUFmN9QxeL5NSkm+kWJFecY41dDCGtO8VQj0D3lfi8wL6kckiRp\neo3ncgyNjDM8Ms7gyDhDI2MMjYwzNDzl9sgYw6MTj42M5RgbzzF64ut4jrGxE1/zL7p/4numFmH7\nZ2nKZODjv/tyaqsr0o5yUpI7zqfTAzRMud8AHD/bi5qaaikvzyYW6myamxvO/k06Z65rMlzX5Li2\nyXBdk3Gmdc3l8vQPjdI7MELfwAtf+wZG6B08xeODI/QOjDIwOMrIWG5a8pVnM1SUl1GezVJZUUZV\nZZb62srJxzJky8ooK8uQLcuc/JotKyObzZz68ROPZTOUZSZ+ZTLnlilzhhec63ud7f10Zovm17Bq\nRdOsWsM0ivNzwEUhhAVAH3Ab8Jdne1FX18DZviUxzc0NtLX1pvb7FyvXNRmua3Jc22S4rtNneGSc\nnoERevpHyJRnOdjaQ0//CD0DE6X35O3+iXJc6ChBJgN11RXUVZfT1FxHVUWW6spyqqsmv1ZmJ35V\nZKmumnK/spyqiolSXJEtmyjE5RO3y8vLKJtFhehc+Hc2Gada1/b2vtSynMqMFecQwruA+hjjp0MI\ndwPfY+LDiZ+NMR6aqRySJM0luXye3v4RuvqG6eoZprN3mON9wxMluH+Ensld4Z6BEUZGz74TXFtV\nTkNdJYsX1NJQU0FdzUQhrquecrumgvrJolxbXUF1VXbOllxpOiVanGOMe4GbJm//05TH7wHuSfL3\nliRpthvP5V5Uhjt7Jr/2DnO8d5iu3iGO940wnjv91nB5NkNDbSUtC+poqKugsbaSxrpKWpobKMvn\naKyrpLG2kobaChomxyAknZ80RjUkSSoJo2PjdPQM0949SEf3EB09QxNfu4do7xmiq3f4tOMSZZkM\n8+orWbO0gaaGKuY3VLGgoZr5DZU01Vcxr76KxtpKaqqyp5wBdZxAmn4WZ0mSztPg8NjJEnyiGLd3\nv3C7p3/klK/LMHHU1vrl81jUWE1TYxVN9VU0NVSzoLGK+fVVzKurpKzM8QhpNrE4S5J0GmPjOTp7\nh2k7Pkj78UHajg/Rdnxw4n73EH2Do6d8XbYsw4LGKi5eNZ9F82pYOK+ahY3VE1/nVbOgoYryrCMT\n0lxjcZYklbShkTGOdg5ytGvgZCk+UZA7e4ZPeRGN8myGhfNqWLO0gUXzXijEixonSrK7xVJxsjhL\nkoreeC5He/cQRzsHaO0YoLXzhV/H+049TjGvvpJ1yxtpnldN8/wamufXsGjy9vyGKk+ZkEqQxVmS\nVBTy+Ty9A6MnC/HRKeX4WNfgKU+mWNhYxWVrmliyoJYlC2pZPKUgV1akd9EtSbOTxVmSNKeM53K0\nHR/iSHs/hzv6OXJiB7ljgIHhsV/4/tqqclYvbWBJUy1LF9aydMHEr8VNNVRZjiWdA4uzJGlWGhkd\np7VzgCMdAxzp6Ofw5NejnQOMjb949zhblmFxUw1h1fyTxXjJgomi3FBTMasu2Stp7rI4S5JS1Tc4\nyq5D3Rxpn9g9nthF7qf9+BAvHa6oqsyycnE9LQvraFlYy7KFdSxbVMei+dVkyzylQlKyLM6SpBkx\nOjbO4fYBDrb1Tf7q52BbH92n+HBeY20FG1fOp2XRCwW5ZWEtTQ1V7h5LSo3FWZI0rXL5PO3HB08W\n44Nt/Rxq6+No5+AvHO22oLGKay5ezKKGKloW1tIyuYNcX1ORUnpJOj2LsyTpvI2O5TjY1se+1l72\nHe3lwLE+DrX1Mzw6/qLvq6nKsm55Iyua61nRXHfya211hZeGljRnWJwlSQUZGR3nYFs/+1p72Nva\ny77WXg6197/omLdsWYalC2tZ2VzP8pMFuZ4FjY5YSJr7LM6SpF8wPDrOgWOTO8mtvext7eVwe/+L\nRi3Ks2WsWtLAmqUNrF7awOolDSxvrvNS0pKKlsVZkkrcyOg4+4/2sae1h/2tvew9OlGSp44jV5aX\nsXZZA2uWNLJqaT1rljbSsrDWkiyppFicJamE5PJ5jnYOsPtwz8lfB9v6XjRuUVWRZf3yeaxZMrmT\nvLSBloW1HvcmqeRZnCWpiPUMjJwsyHsOd7P7SC+DU66uV57NsGZpA2tbGlnb0sjqpQ0sXVBLWZnz\nyJL0UhZnSSoSuVyeA8f62HHgOM8f7mb34R7au4de9D1Lmmq4asNC1i2bx7pljaxcXO+4hSQVyOIs\nSXPU2HiOvUd6iQe62HGgm12HjjM4/MIxcHXV5VyxbiHrljWybtnEjrLnI0vS+bM4S9IcMTwyzvOH\nu9lx4PjkrnIPo2O5k88vWVDLdWEeG1fOZ8OKeSyeX+MRcJI0jSzOkjRLDQ6PnSzJOw4cZ29r78kP\n8WWA5c31hJXz2bhqPhtXzGNefVW6gSWpyFmcJWmWGB4dZ9ehbrbv62L7vi72HOk9eW5yWSbDmpYG\nNq6cz8YVEzvKjl1I0syyOEtSSsbGc+w+3MP2fV08t6+L5w93Mzb+QlFeu6yBS1Y3EVY2sX55I9WV\n/siWpDT5U1iSZkgul2ff0V6emyzKOw8eZ2R0YkY5A6xaMlGUL17dxEUr5lFT5Y9oSZpN/KksSQnq\n7Bliy55OtuzuYNveLgamnKG8fFEdF69u4pLVTWxcOd/RC0ma5SzOkjSNRkbHiQeOs2V3J1v2dHCk\nY+Dkc4vmVXPdxYtP7irPq6tMMakk6VxZnCXpAuTzeQ6390/sKu/pZMeB4yePiKusKOPK9Qu5fO0C\nLl+3kCVNHg8nSXOZxVmSztHg8Bhb93SyeXcHW/Z00tU7fPK5Fc31XL5uAZevXcBFK+ZTUe5V+SSp\nWFicJakAXb3DPLmznZ88fZDt+7pOnn5RX1PBDZcs5op1C7ls7QLme5ayJBUti7MknUI+n+dQWz9P\n72zjmV3t7DnSe/K5lYvrufqiRWzasIjVSxooK3P8QpJKgcVZkibl8xPHxT3x3DGe3H6M9u4hALJl\nGS5Z3cTLr17BhqX1LJpfk3JSSVIaLM6SSt6h9n4e33aUJ547yrGuQQCqK7Ncf/Firr5oEVesX0hd\ndQXNzQ20tfWe5d0kScXK4iypJHX3DfP4tqM8srWV/Uf7gIlTMG64ZDE3XrKEy9ctoKI8m3JKSdJs\nYnGWVDKGR8d5emcbj245ytY9neTyebJlGTatX8jNly9l0/pFVFValiVJp2ZxllTUcvk8cV8Xj2xt\n5WexjaGRcQDWtjRw82VLueHSJTTWeiESSdLZWZwlFaWO7iEe2nyEh549QkfPxIf8FjZW85rrVnDz\nZUtpWViXckJJ0lxjcZZUNEbHcjy9s40Hnz3Ctj2d5IGqiiy3XtHCLVcs5aKV8ynzyn2SpPNkcZY0\n5x041seDPz/Mo1tb6R8aA2DD8nm8/MoWrrt4MTVV/qiTJF04/2siaU4aGR3n8eeO8uOnD528OElj\nbQWvu2EVt17ZwrJFjmJIkqaXxVnSnHKsa4AfP32YB589TP/QGJkMXLl+IS+/chmbNiykPFuWdkRJ\nUpGyOEua9XK5PJt3d3DfU4fYsruDPNBQW8Ebbl7NK69azsJ51WlHlCSVAIuzpFmrb3CUB39+mPuf\nPnTy8tfrlzfyqmtWcF1YTEW5u8uSpJljcZY06xzp6OcHPz3Aw1taGR3LUVlexm2bWrj96hWsXtqQ\ndjxJUomyOEuaFfL5PNv2dvH9nx5g8+4OABbNq+bV167g1itbqKuuSDmhJKnUWZwlpWp0bJzHth7l\nB08e4GBbPwAbVszjzutWcs3GZsrKPHdZkjQ7WJwlpaKnf4T7nz7E/U8dpGdglLJMhhsvXcKd169k\nbUtj2vEkSfoFFmdJM+pgWx/f/+kBHtt6lLHxHLVV5dx14ypefe0KFjR6OoYkafayOEtKXD6fZ+ue\nTr73xH627u0CYHFTDXdct5JbrlhKdaU/iiRJs5//tZKUmFwuz5PxGN95bB/7j/YBcPGq+dx5/Squ\n3LCQsozzy5KkucPiLGnajY7leHRrK999bB9HuwbJZOD6ixfz+ptWe5ycJGnOsjhLmjZDI2M88Mxh\nvv/TA3T1DpMty3DbphbuunE1SxbUph1PkqQLYnGWdMH6Bkf54ZMH+NHPDtI/NEZVRZY7r1/Ja29Y\nRVNDVdrxJEmaFhZnSeetb3CU7z2xnx/97CBDI+PUVZfz5lvX8uprV1Bf4wVLJEnFxeIs6Zz1DIzw\nvSf2c99ThxgeGaextoI33bKWV169zBMyJElFy//CSSpYT/8I9z6xn/ufOsTw6Djz6ip568vX8Yqr\nllFVkU07niRJibI4Szqr7v4R7n18H/c/fYiR0Rzz6yt52yvWcdumZVRamCVJJcLiLOm0+odG+e5j\n+/nhkwcYGcvR1FDFL79yNbdtaqGi3MIsSSotFmdJv2B4ZJwfPHmA7z6+n8HhMebXV/LOl63h1iuX\nUVFelnY8SZJSYXGWdNLoWI4HnjnEtx7dR0//CHXV5bzj9g286prljmRIkkqexVkSuVyeR7e28i8P\n7qGjZ4iqyixvumUNd16/itpqf0xIkgQWZ6mk5fN5ntnZzld/spvD7f2UZzPccd1K3vCy1TTWVqYd\nT5KkWcXiLJWo5w938+X7drHzYDeZDLz8yhbedMtaFs6rTjuaJEmzksVZKjHHugb46gO7+en2YwBc\nfdEi3vaK9SxbVJdyMkmSZjeLs1QiegdGuOeRvdz/1CHGc3nWtjTyjtvXE1Y1pR1NkqQ5weIsFbmR\n0XF++LODfPvRfQwOj9E8v5q3vWI911+8mEwmk3Y8SZLmDIuzVKTy+Tw/3X6Mr9y/i46eYeqqy/nV\nV1/EK69e7lnMkiSdB4uzVIQOHuvjn364g+37j1OezXDXjat4w82rqa2uSDuaJElzlsVZKiL9Q6N8\n7WvP8u1H9pDPw1UbFvErr97A4qbatKNJkjTnWZylIpDL5Xnw2cN89YHd9A2OsmRBLe96zUVcsW5h\n2tEkSSoaFmdpjtt1qJt//MEO9rX2UlWZ5QO/dCk3X7KY8qxzzJIkTSeLszRH9fSP8JX7d/HwllYA\nbr5sCW9/5QY2rltEW1tvyukkSSo+Fmdpjsnl8jzwzCG++sBuBobHWLW4nnffuZGLVsxPO5okSUXN\n4izNIXtbe/iH70X2HOmlpirLu+/YyO1XL6eszPOYJUlKmsVZmgMGhkb5+k/2cN/TB8nn4aZLl/DO\nV21gXn1V2tEkSSoZFmdpFsvn8zy27Shfum8XPf0jLF1Qy3vv3MglaxakHU2SpJJjcZZmqSMd/fzD\n9yLb9x+nsryMt71iHXdev8qr/kmSlBKLszTLjI3n+M5j+/jWI3sZG8+zaf1C3nXHRprn16QdTZKk\nkmZxlmaRXQe7+bt7t3O4vZ/59ZW8+47AtaE57ViSJAmLszQrDA6P8c8PPM+PnzpEHrj96uW87RXr\nqa32/6KSJM0W/ldZStnTO9r4/A920NU7TMvCWt5/18WeySxJ0ixkcZZS0tU7zD/9cAc/i22UZzO8\n5da13HXTaj/8J0nSLJVYcQ4hlAGfADYBw8BvxBh3TXn+94FfBXLAf4kxfj2pLNJsks/neWjzEb74\no10MDo9x0Yp5/NrrLmbZorq0o0mSpDNIcsf5LUB1jPHmEMJNwF8BbwYIIcwH/i2wAagDngEszip6\nnT1D/N2929myu5Pqyizve23gtquWUZbxyn+SJM12SRbnW4F7AWKMj4UQrpvyXD+wj4nSXMfErrNU\ntPL5PA8+e4Qv3beTweFxLl+7gPffdTELGqvTjiZJkgqUZHFuBLqn3B8PIZTHGMcm7x8AtgFZ4P86\n25s1NdVSXp6d/pQFam5uSO33LmalsK7Hugb4719+hqd3tFFbXc7vvOMq7rhhFZkEd5lLYV3T4tom\nw3VNhuuaHNc2GbN9XZMszj3A1D992ZTSfBfQAqydvP+9EMLDMcYnTvdmXV0DyaQsQHNzA21tvan9\n/sWq2Nf1xC7zF3+0k6GRcS5ft4D3v25il7m9vS+x37fY1zVNrm0yXNdkuK7JcW2TMZvW9XQFPsni\n/DDwRuDLkzPOm6c81wUMAsMxxnwI4Tjg+VsqGh3dE7PMW/d0UlOV5QOvv5hbr2hJdJdZkiQlK8ni\n/HXgjhDCI0AG+EAI4W5gV4zxmyGE1wCPhRBywEPADxLMIs2Yx7a18g/fiwwOj3PFuoX82uuCs8yS\nJBWBxIpzjDEHfOglD2+f8vyfAH+S1O8vzbSBoTE+/4PIY1uPUlWR5f13XczLr3SXWZKkYuEFUKRp\nsOPAcT5zzzY6eoZY29LIB990KUuaatOOJUmSppHFWboAuVyeex7Zyzcf3gPAG1+2hjfesobyrFf/\nkySp2FicpfPU2TPEZ+7ZRjxwnIWNVfzmGy9j40o/4ypJUrGyOEvn4Zld7Xz228/RNzjKNRub+cDr\nL6auuiLtWJIkKUEWZ+kcjI7l+MqPd/HDJw9Sni3jvXdu5JVXL/cDgJIklQCLs1Sgo50DfPIbW9h/\ntI+WhbV86M2Xs3JxfdqxJEnSDLE4SwV4dEsrn/t+ZHhknFuvbOHdr9lIVWV6l4CXJEkzz+IsncHw\n6Dif/37k4c2tVFdm+eCbLuWmS5emHUuSJKXA4iydRmvnAJ/4+mYOtvWzZmkDH3rzZSz2bGZJkkqW\nxVk6hZ9uP8b/+s5zDI2M86prlvPOV11ERblnM0uSVMosztIUY+M5vnz/xKkZVRVZPvjGS7npMkcz\nJEmSxVk6qbNniE/+yxaeP9xDy8JaPvzWK1i+qC7tWJIkaZawOEvAlt0dfPqebfQNjnLTpUt43+sC\n1ZX+30OSJL3AZqCSlsvl+ebDe7jn4b1ksxkvaCJJkk7L4qySNTg8xmfu2cYzu9pZ2FjNh996OWtb\nGtOOJUmSZimLs0pSe/cgH/vnZznY1s+la5r40Jsvp76mIu1YkiRpFrM4q+TsOtjNf//as/QMjHL7\nNct512suIlvmUXOSJOnMLM4qKY9tbeWz33mOXA7efcdGXn3tirQjSZKkOcLirJKQz+e59/H9fOXH\nz1NTVc5vveUyLl+7MO1YkiRpDrE4q+jlcnm+8KOd/OhnB2lqqOLud2xieXN92rEkSdIcY3FWURsd\nG+cz92zjydjG8uY6fu+XN7GgsTrtWJIkaQ6yOKto9Q+N8vF/fpYdB7sJK+fzO2+7gtpqT86QJEnn\nx+KsotTZM8T/++Wfc7i9n+suXsxv/tIlVJRn044lSZLmMIuzis7BY3389Vd+TlfvMK+5bgW/8uqL\nKPNKgJIk6QJZnFVUtu/r4uNf28zg8BjvuH0Dr71hpZfPliRJ08LirKLx1I42/uYbW8jn4YNvupSb\nLl2adiRJklRELM4qCk9uP8anvrmV8mwZv/O2K7h0zYK0I0mSpCJjcdac98RzR/n0N7dRUVHG3e/Y\nxEUr5qcdSZIkFSGLs+a0x7a18pl7tlFVkeXud17FhuXz0o4kSZKKlMVZc9ajW1v5229to7qynLvf\nuYn1yyzNkiQpORZnzUmPTZbmmspy/v2vXMXalsa0I0mSpCJXlnYA6Vw9vu0on5ncabY0S5KkmVLQ\njnMIYQ1wGXAvsCrGuCfJUNLpPPHcUT59z1aqK7P8vqVZkiTNoLPuOIcQ3gncA3wMWAg8GkJ4T9LB\npJf66fZjfPqb26iunPggoKVZkiTNpEJGNf4QeBnQE2M8BlwN/HGiqaSXeHL7MT71ja1UVpRx9zuu\n8oOAkiRpxhVSnMdjjL0n7sQYjwC55CJJL/azOHFxk4oTpdkj5yRJUgoKmXHeGkL4baAihHAV8GHg\nmWRjSRMmLqM9cUXAu9+xiQ0rLM2SJCkdhew4fwRYDgwC/xPoZqI8S4l6emcbn/yXLZRny/g9rwgo\nSZJSVkhxHgIejTFeD7wWeA7oSzSVSt7WPZ184utbyGYz/LtfvpKNKy3NkiQpXYUU578F3jbl/u3A\n3yQTR4IDx/r4H1/fTCaT4d+9fRNhVVPakSRJkgqacb4uxngFQIyxHXhvCOHZZGOpVHX1DvPfvvJz\nhkbG+dCbL+Pi1ZZmSZI0OxSy41wWQmg5cSeEsBhP1VACBofH+P++8nO6eod5+yvXc8MlS9KOJEmS\ndFIhO87/GXg6hPDQ5P0bgd9NLpJK0Xguxye/sYX9x/p4xVXLuOvGVWlHkiRJepGz7jjHGP8JuAb4\nAvA54IYY49eSDqbS8k8/2MmW3Z1cvm4B77lzI5lMJu1IkiRJL3LWHecQwnzgrcACIANcFUIgxvhn\nSYdTabj/qYPc//QhVjTX81tvvpxsWSETRJIkSTOrkFGNrzBxdvMWIJ9sHJWa5/Z18Y8/2El9TQX/\n9m1XUFNVyF9JSZKkmVdIS1kaY7wj8SQqOa0d/Xzi65vJZOC3/9UVLJpfk3YkSZKk0yrk38SfDiFc\nmXgSlZTB4TH+z88+Tv/QGO+5c6MXOJEkSbNeITvOlzNRno8ycRXBDJCPMa5LNJmKVi6f5zP3bGN/\nay+vvnYFr7hqedqRJEmSzqqQ4vzWxFOopHzrkb08s6udKzcs4ldevSHtOJIkSQUppDi3Aq8H6pnY\nbc4Ca4H/lGAuFalnn+/gGw/uYWFjFf/hvdcxMjiSdiRJkqSCFFKcvwA0ARuAB4HbgYfO+ArpFNqO\nD/KZe7aSzWb48FuvYF59FW0WZ0mSNEcU8uHAK4FXAV8H/h/gFmBNgplUhEZGx/kfX9s8+WHAwNqW\nxrQjSZIknZNCivOxGGMe2A5cGWPcDVQmG0vFJJ/P8w/fi+w/1sdtm1q4bdOytCNJkiSds0JGNbaE\nED4OfBL4xxDCMiZmnaWCPPjsER7e0sqapQ28+46NaceRJEk6L4XsOP8W8OUY4zYmPhDYArwr0VQq\nGgeO9fGPP9hBXXU5H37r5VSUZ9OOJEmSdF5OW5xDCNdM3rwFyIQQbmPi0ttfBRbMQDbNcYPDY3zi\nX7YwOpbj199wKYvmeWVASZI0d51pVONDwAeBj57iuTwTHxiUTunEXPPRzgFee8NKrrpoUdqRJEmS\nLshpi3OM8YOTN78UY/ybGcqjIvHgs0d4bNtR1i9r5G2vWJ92HEmSpAtWyIzzbyeeQkVl6lzzv3nz\nZZRnC/lrJkmSNLsVcqrGgRDCfcDjwOCJB2OMf5ZYKs1ZwyPjfHJyrvm33ny5c82SJKloFFKcH5ty\n22PodEb/+MMdtHYOcMd1zjVLkqTictbiHGN80YcDQwgZYG1iiTRnPbatlYeePcLqJQ28/ZXONUuS\npOJy1uIcQvgg8JdA3ZSH9wAbkgqluedY1wCfuzdSVZnlQ2++jIpy55olSVJxKaTd/DGwCfgisB74\nHSbmnSUAxsZzfOqbWxkaGee9d25kyYLatCNJkiRNu0KK87EY4x5gM3BFjPETTBRpCYCvPbCbPUd6\nednlS3nZ5S1px5EkSUpEIcW5P4RwO/As8MYQwlLAoxIEwObdHdz7xH6WNNXwnjs3ph1HkiQpMYUU\n598B3gTcCywEtgMfTzKU5obuvmH+57e2kS3L8KE3X051ZSGHtEiSJM1NhTSdDcAfxBhzwNsSzqM5\nIpfP85lvbaNnYJRfffVFrF7akHYkSZKkRBWy4/xeYG8I4ZMhhFuSDqS54buP7WPb3i42rV/Ia65b\nkXYcSZKkxJ21OMcY3w5cAjwC/HEI4bkQglcNLGHPH+rm6z/Zw/z6Sv71Gy4hk/G6OJIkqfgVdNhu\njLEXeJiJ8jwCvCzJUJq9BobG+NQ3t5LP5/ngGy+jobYy7UiSJEkzopALoNwN/CpQBXweeEOM8WDS\nwTQ7ffG+nbR3D/FLL1vDxaub0o4jSZI0Ywr5cOBy4DdjjM8kHUaz27PPd/DQs0dYtbieN92yJu04\nkiRJM+qsxTnG+O9nIohmt4GhUf7+3u1kyzL86zdcQnnWS2pLkqTSYvtRQb74o1109Q7zxlvWsGqJ\nR89JkqTSY3HWWT37fDsPbT7C6iUNvP6m1WnHkSRJSsVpRzVCCO870wtjjJ+b/jiabQaGRvm7706M\naPy6IxqSJKmEnWnG+fbJr+uZuHrgd4Ax4HXAVsDiXAK+8KOdHO8b4a0vX8uKxfVpx5EkSUrNaYtz\njPEDACGE+4ErY4ztk/ebgH+ZmXhK0893tfPw5lZWL2ngLkc0JElSiSvk392XAZ1T7vcDLcnE0WzR\nPzTK393riIYkSdIJhZzj/G3gByGErwEZ4B3Al872ohBCGfAJYBMwDPxGjHHXlOfvAv5k8u5TwEdi\njPlzi6+kfOGHO+nuG+Gtt61zREOSJIkCdpxjjHczUYAvBjYCfxlj/I8FvPdbgOoY483AHwF/deKJ\nEEID8BfAL8UYbwL2AovOOb2ZI6uvAAAgAElEQVQS8czOdh7Z0srqpQ28/qZVaceRJEmaFQr99/dW\nJj4Q+AfA8QJfcytwL0CM8THguinPvQzYDPxVCOFB4GiMsa3A91WC+odG+fvvbac8OzGikS1zREOS\nJAkKGNUIIfwuE7vHy4EvA58KIfzPGONfnuWljUD3lPvjIYTyGOMYE7vLtwNXAX3AgyGER2OMO073\nZk1NtZSXZ88WNzHNzaVx0Y/Pf+EpuvtGeN/rL+HqS5MfZS+VdZ1prmtyXNtkuK7JcF2T49omY7av\nayEzzu8HbgQejzF2hhCuB54Azlace4Cpf/qyydIM0AH8NMbYChBC+AkTJfq0xbmra6CAqMlobm6g\nra03td9/pmzf18V9Tx5g9dIGXn75ksT/zKWyrjPNdU2Oa5sM1zUZrmtyXNtkzKZ1PV2BL+Tf4cdj\njCNT7g8B4wW87mHg9QAhhJuYGM044WfA5SGERSGEcuAmYFsB76mEjI3n+PwPdpAB3vfa4IiGJEnS\nSxSy4/xACOEvgboQwluADwI/KuB1XwfuCCE8wsRpHB8IIdwN7IoxfjOE8MfA9ya/98sxxi3nkV/T\n5IdPHuRwez+vvHo5a1sa044jSZI06xRSnP8A+E3g58D7mLiC4N+c7UUxxhzwoZc8vH3K818Evlhw\nUiWms2eIbzy0h/qaCv7VbevSjiNJkjQrnbU4xxhzIYQvMFGYM5MPLwP2JxlMM+dL9+1ieHScd73m\nIuprKtKOI0mSNCsVcqrG/8bEOcwdQJ6J8pwH3JosAlv3dvLT7cdYv7yRW670gpCSJEmnU8ioxq8D\n6z1nufiMjef4x+/vIJOB994ZKMtkzv4iSZKkElXI0Qn7gc6kg2jm/fDJg7R2DvCqq1ewasnsPjdR\nkiQpbYXsOO8EHgoh3M/EUXQAxBj/LLFUSlx3/wjffHjiA4FvuW1t2nEkSZJmvUKK86HJX/DChwM1\nx33tgecZGhnnvXeup67aDwRKkiSdTSGnanx0JoJo5uxt7eGhZ4+wormO265alnYcSZKkOeG0xTmE\n8FSM8ZoQQo6JUzROyAD5GGM28XSadvl8nn/64U7ywK+++iKvEChJklSg0xbnGOM1k19tVkXkp9uP\nsetgN9dubOaSNQvSjiNJkjRnFHKOczPwHqCeid3mLLA2xvi+hLNpmg2PjvPl+3dRns3wy6/akHYc\nSZKkOaWQ3eQvAVcxUZ7rgLcDuSRDKRnf/+kBOnuGee0Nq1g8vybtOJIkSXNKIcV5WYzx14B7gK8B\ntwFXJ5pK025weIzvP7GfuupyXn/T6rTjSJIkzTmFFOeuya8R2BRj7EgwjxJy31MH6R8a484bVlFT\nVcgphJIkSZqqkAZ1XwjhK8DvA98PIVwDDCYbS9NpaGSM7z1xgNqqcl5z7Yq040iSJM1JZ91xjjH+\n78AfxRj3Ab/KxM7zW5MOpulz/1OH6Bsc5c7rV7rbLEmSdJ7OdI7z+15y/5bJmx3AHcDnEsylaTI8\nMs69T+ynpqqc11znbrMkSdL5OtP24+1neC6PxXlOuP/pQ/QOjPKmW9ZQ66W1JUmSztuZLoDygRO3\nQwjlwJXAGLA5xpg/3es0ewyPTuw2V1dmec11K9OOI0mSNKeddcY5hPAaYD/waeDvgd0hhOuTDqYL\n98Azh+npH+E1162gvsbdZkmSpAtRyCfF/htwV4zx5wAhhOuAvwGuSzKYLszo2DjffXwfVRVZ7rx+\nVdpxJEmS5rxCznEePlGaAWKMTzJx6W3NYg9vbqW7b4Tbr1nubrMkSdI0KGTH+SchhL8FPsPEjPOv\nAHtDCLcBxBh/kmA+nYfxXI7vPLaP8mwZd17vbLMkSdJ0KKQ4XzX59f9+yeMfZeJ0jVdNayJdsCe2\nHaO9e4jbr1nO/PqqtONIkiQVhUKK8xtijANTHwghrJ68IIpmmVw+z7cf20dZJsNdNzjbLEmSNF0K\nmXF+OoRw04k7IYTfAh5NLpIuxNM72jnc3s/Nly1h0fyatONIkiQVjUJ2nH8d+F8hhG8C1wCDwE1n\nfonSkM/n+faje8kAr795ddpxJEmSispZd5xjjA8BHwM+AlwGfDTGuD/pYDp32/Z2sbe1l2tDMy0L\n69KOI0mSVFQKuQDKj4G3A5cD7wK+EEL4q4Rz6Tx865G9ALzh5jWp5pAkSSpGhYxqfDXG+PHJ23tD\nCNcA/zXBTDoPzx/qJh44zuXrFrB6aUPacSRJkorOWYtzjPHjIYRbgCuAzwLXxBg/kngynZN7H5+Y\nnnn9jc42S5IkJaGQUY3fBf4cuBtoAD4VQvj9pIOpcEe7BnhqRxtrljYQVs1PO44kSVJRKuQ4uvcD\nrwX6Y4wdwPXAv04ylM7N9584QB543Y2ryGS8GrokSVISCinO4zHGkSn3h4DxhPLoHPUMjPDQ5iMs\nmlfNtaE57TiSJElFq5Di/EAI4S+BuhDCW4BvAj9KNpYK9eOnDjE6luOO61eSLSvkf05JkiSdj0Ka\n1h8AO4GfA+8DvgM44zwLjIyO86OnDlJXXc7Lr2xJO44kSVJRK+RUjRzwqclfmkUe2dJK78Aob7h5\nNdWVhZwsKEmSpPPlv+3PUblcnu89sZ/ybIZXX7si7TiSJElFz+I8Rz2zq52jXYPcdNlS5tdXpR1H\nkiSp6BX07/shhDXAZcC9wKoY454kQ+nsfvSzgwC89vqVKSeRJEkqDYVcAOWdwD3Ax4CFwKMhhPck\nHUynd7i9n+f2dXHxqvksb65PO44kSVJJKGRU4w+BlwE9McZjwNXAHyeaSmd0/1OHAHjVNc42S5Ik\nzZRCL4DSe+JOjPEIkEsuks5kcHiMh7ccoamhiqs3Lko7jiRJUskoZMZ5awjht4GKEMJVwIeBZ5KN\npdN5dGsrQyPjvO7GVV7wRJIkaQYV0rw+AiwHBoHPAj1MlGfNsHw+z31PHSJbluEVm5alHUeSJKmk\nFLLj/BvAX8cYnWtOWdx/nMPt/dxwyWLmeQSdJEnSjCqkOK8EHg8hbAc+D3w9xjiQbCydyn1PTRxB\n54cCJUmSZt5ZRzVijL8fY1wL/BfgZuDpEMLnEk+mF+nqHeapHe2saK7nohXz0o4jSZJUcgr6dFkI\nIQNUAJVAHhhJMpR+0QPPHCKXz/Oqa5eTyWTSjiNJklRyzjqqEUL4GPBWJk7S+Dzwb2OMQ0kH0wty\nuTwPPnuEmqosN1+6NO04kiRJJamQGeedwNUxxvakw+jUtu3rpKt3mFdctYyqymzacSRJkkrSaYtz\nCOGDMcZPAwuAD4cQXvR8jPHPEs6mSQ9vbgXglitaUk4iSZJUus6045w5zW2YmHPWDBgYGuWpHW0s\nWVDL+mWNaceRJEkqWactzjHGT03e3Btj/Pupz4UQPpJoKp30xPZjjI7luPWKpX4oUJIkKUVnGtX4\nd0Aj8KEQwuqXvObdwP9IOJuARza3kgFuvswPBUqSJKXpTMfR7WRiROOlv4aB9yeeTLR2DrDrUDeX\nrl3AgsbqtONIkiSVtDONanwb+HYI4csxxuemPhdCqEk8mXh48xEAbrnc3WZJkqS0FXIc3YYQwpeA\nOiZ2nLNALdCcZLBSl8vleWRLKzVVWa7e6FJLkiSlrZArB/418LvAc0zMNn8R+FKSoQTP7euiq3eY\n6y9eQlWFZzdLkiSlrZDifDzGeD/wGDAvxviHwKuSjaUTYxq3enazJEnSrFBIcR4MIWxkYsf5lSGE\nSqAy2VilbXB4bOLs5qYa1i/37GZJkqTZoJDi/H8Afw58C3g1cBT4RpKhSt1z+7oYGctx/SVLPLtZ\nkiRpljjrhwNjjA8AD0zevT6E0BRj7Eo2Vml79vkOAK5cvzDlJJIkSTrhTBdAuZ/TXFo7hECM0Tnn\nBOTzeTbv7qCuupx1LY5pSJIkzRZn2nH+05kKoRccauunq3eYGy9dQlmZYxqSJEmzxZkugPIAQAjh\ntpmLo827J8c01jmmIUmSNJsUcgGUj065XQFcCTwI/CSRRCXu2ec7yACXrVuQdhRJkiRNUciHA2+f\nej+EsJaJi6Jomg0Oj7HrUDdrWhporPXEP0mSpNmkkOPoXiTGuAe4OIEsJW/b3k7Gc3mucExDkiRp\n1jnrjnMI4X/xwukaGeASYEuSoUrViWPorvAYOkmSpFmnkBnnH0+5nQe+AvwwkTQl7MQxdPU1Faxd\n6jF0kiRJs00hM85/H0JoAJqmPLwU2J9YqhJ04Fgfx/tGuOkyj6GTJEmajQoZ1fgL4INAx+RDGSZ2\nntclmKvkeAydJEnS7FbIqMZbgOUxxr6kw5Syzbs7J46hW+sxdJIkSbNRIadqPAtUJR2klA0MjbLr\nYDfrljXS4DF0kiRJs1IhO87/AOwKIWwGxk48GGN8VWKpSsy2vV3k8h5DJ0mSNJsVUpz/M/C7wL6E\ns5Ssbfu6AMc0JEmSZrNCinN3jPFziScpYdv3dVFVmWX10oa0o0iSJOk0CinOT4UQvgp8Fxg58aBl\nenoc7xumtXOAK9YtpDx7zhdylCRJ0gwppDjXAT3ALS953OI8DbbvnxjTuHj1/JSTSJIk6UwKuQDK\nB2YiSKnavu84ABevajrLd0qSJClNhVwAZQ8TFzx5kRjjGS+AEkIoAz4BbAKGgd+IMe46xfd8G/hG\njPFvziF30di+v4uaqiyrltSnHUWSJElnUMioxiun3K4A3kph5zq/BaiOMd4cQrgJ+CvgzS/5nj8H\nSvYoic6eIY51DbJp/UKyZc43S5IkzWaFjGq89Bi6vwghPMlE6T2TW4F7J9/jsRDCdVOfDCG8Hcgx\n8aHDkhT3T45prHZMQ5IkabYrZFTjtil3M8BlQE0B790IdE+5Px5CKI8xjoUQLgfeBbwd+E+FBG1q\nqqW8PFvItyaiuXn6j4rbe2xicuXmTcsTef+5oFT/3ElzXZPj2ibDdU2G65oc1zYZs31dCxnV+OiU\n23mgHfi1Al7XA0z905fFGE9cefB9wHLgPmANMBJC2BtjvPd0b9bVNVDAb5mM5uYG2tp6p/19n47H\nqK0qp76iLJH3n+2SWtdS57omx7VNhuuaDNc1Oa5tMmbTup6uwBcyqnF7CGFxjPFYCKEWWPbSD/md\nxsPAG4EvT844b57ynv/hxO0Qwp8CrWcqzcWovXuQ9u4hrr5oEWVlmbTjSJIk6SzO+om0EMLvMDmr\nDDQD94QQPljAe38dGAohPAL8NfB7IYS7QwhvOu+0RcRj6CRJkuaWQkY1/g1wI0x8UDCEcC3wOPDp\nM70oxpgDPvSSh7ef4vv+tKCkRSaevPCJxVmSJGkuKOQMtAomzmE+YYRTnOuswuXzebbv76K+poLl\nzXVpx5EkSVIBCtlx/hfgvhDCl5kozG8DvpFoqiLX1j1ER88w125spizjfLMkSdJccNYd5xjjHwIf\nAwKwHvhYjPE/Jh2smG3f55iGJEnSXFPIjjMxxn8G/jnhLCXj5HzzqvkpJ5EkSVKhvM5zCuKB49TX\nVLBskfPNkiRJc4XFeYa1dw/S2TPMxpXzyTjfLEmSNGdYnGfYzgMTVyHfuGJeykkkSZJ0LizOM2zH\nwYkLn1y00vlmSZKkucTiPMN2HDhOVWWWVUvq044iSZKkc2BxnkE9AyMc6Rhgw7JGsmUuvSRJ0lxi\ne5tBJ+ebHdOQJEmacyzOM2jn5HyzxVmSJGnusTjPoB0HjpMty7C2pTHtKJIkSTpHFucZMjg8xv6j\nfaxtaaSyIpt2HEmSJJ0ji/MMef5wN7l8notWen6zJEnSXGRxniE7Jj8YGJxvliRJmpMszjNk54Hj\nZIANy91xliRJmosszjNgdCzH7iM9rFhcT211RdpxJEmSdB4szjNgX2svo2M5Nq5wTEOSJGmusjjP\ngB2T5zf7wUBJkqS5y+I8A3Yc8MInkiRJc53FOWG5fJ6dB7tZ3FTD/PqqtONIkiTpPFmcE3a4vZ/B\n4TEu8jQNSZKkOc3inLBdhybOb96wwuIsSZI0l1mcE/b8wcni7I6zJEnSnGZxTtiuQ93UVJXTsqgu\n7SiSJEm6ABbnBPUMjHC0a5D1yxspy2TSjiNJkqQLYHFO0POHHNOQJEkqFhbnBO2yOEuSJBUNi3OC\nnj/YTSYDa1sa044iSZKkC2RxTsjYeI49rb2sbK6npqo87TiSJEm6QBbnhOw/2sfoWI71nt8sSZJU\nFCzOCXG+WZIkqbhYnBNicZYkSSouFucE5PN5dh08zry6ShbNq047jiRJkqaBxTkBnT3DHO8bYcPy\neWS88IkkSVJRsDgn4MSYxnrHNCRJkoqGxTkBJ+ebPVFDkiSpaFicE7DrUDfl2TJWL2lIO4okSZKm\nicV5mg2PjHPgaB9rWhqoKHd5JUmSioXNbprtOdJDLp/3GDpJkqQiY3GeZs8fnvxg4LLGlJNIkiRp\nOlmcp9nuwz0ArFvmjrMkSVIxsThPo3w+z+7DPTQ1VNHUUJV2HEmSJE0ji/M06uodprt/hHUtjmlI\nkiQVG4vzNHphTMPiLEmSVGwsztPoRHFe646zJElS0bE4T6Pdh7vJZGBNixc+kSRJKjYW52kynsux\n92gvyxfVUV1ZnnYcSZIkTTOL8zQ51NbPyGjO+WZJkqQiZXGeJruPON8sSZJUzCzO08QLn0iSJBU3\ni/M02XO4h6qKLMsX1aUdRZIkSQmwOE+DweExDrf3s2ZpA2VlmbTjSJIkKQEW52mwt7WXPLDWDwZK\nkiQVLYvzNNh9uBvAS21LkiQVMYvzNPBS25IkScXP4nyB8vk8u4/0ML++kgWN1WnHkSRJUkIszheo\nq3eY7r4Rz2+WJEkqchbnC+SYhiRJUmmwOF+gE1cM9MInkiRJxc3ifIH2H+0FYM3ShpSTSJIkKUkW\n5wt0pGOApoYqaqrK044iSZKkBFmcL8Dg8BhdvcO0LKxNO4okSZISZnG+AEe7BgBoWVCXchJJkiQl\nzeJ8AY50TBTnpe44S5IkFT2L8wU4UZwd1ZAkSSp+FucL0NrRD0DLQkc1JEmSip3F+QIc6RygqjLL\n/PrKtKNIkiQpYRbn85TL5TnaOUDLgloymUzacSRJkpQwi/N5au8eZGw873yzJElSibA4n6cXTtRw\nvlmSJKkUWJzPU2vniTOc3XGWJEkqBRbn8+RRdJIkSaXF4nyeWjv6yWRgcZPFWZIkqRRYnM/Tkc4B\nmufXUFHuEkqSJJUCW9956BscpXdg1PlmSZKkEmJxPg+tJ+ebPVFDkiSpVFicz8ORyUttL/WDgZIk\nSSXD4nwejkweRbfUUQ1JkqSSYXE+D60eRSdJklRyypN64xBCGfAJYBMwDPxGjHHXlOd/D/iVybvf\niTF+NKks0+1IRz/1NRU01FamHUWSJEkzJMkd57cA1THGm4E/Av7qxBMhhHXAu4GXATcDd4YQrkww\ny7QZHcvRdnzI+WZJkqQSk2RxvhW4FyDG+Bhw3ZTnDgCvizGOxxhzQAUwlGCWaXPs+CC5fN6j6CRJ\nkkpMYqMaQCPQPeX+eAihPMY4FmMcBdpDCBngL4CnY4w7zvRmTU21lJdnE4x7Zs3NDQDsPNILwIZV\nC04+pvPnGibDdU2Oa5sM1zUZrmtyXNtkzPZ1TbI49wBT//RlMcaxE3dCCNXAZ4Fe4MNne7OuroFp\nD1io5uYG2tomCnPc0wFAQ3X25GM6P1PXVdPHdU2Oa5sM1zUZrmtyXNtkzKZ1PV2BT3JU42Hg9QAh\nhJuAzSeemNxp/gbw8xjjv4kxjieYY1od8UQNSZKkkpTkjvPXgTtCCI8AGeADIYS7gV1AFngFUBVC\nuGvy+/84xvhognmmRWtnP+XZDIvmVacdRZIkSTMoseI8+aG/D73k4e1Tbs+55pnP5znSMcCSplqy\nZR6BLUmSVEpsf+egb3CUoZFxmufXpB1FkiRJM8zifA46e4YBWOiYhiRJUsmxOJ+D9u6Jo6YXNlqc\nJUmSSo3F+Rx09EwUZz8YKEmSVHoszuegc7I4L3DHWZIkqeRYnM9Bx4lRDXecJUmSSo7F+Ry09wxR\nni2jsbYi7SiSJEmaYRbnc9DZM8TCxioymUzaUSRJkjTDLM4FGh4dp3dg1DENSZKkEmVxLtCJDwZ6\nFJ0kSVJpsjgXqMPiLEmSVNIszgXyRA1JkqTSZnEukDvOkiRJpc3iXKCO7mEAFrjjLEmSVJIszgXq\n6BkiAyxoqEo7iiRJklJgcS5QR/cQ8xuqKM+6ZJIkSaXIFliA8Vyert5hFjS62yxJklSqLM4F6Owe\nIpfP+8FASZKkEmZxLsCxrgHAo+gkSZJKmcW5AG3HBwGPopMkSSplFucCtJ3YcbY4S5IklSyLcwGO\ndU3uODuqIUmSVLIszgVwx1mSJEkW5wIc6xqktqqcmqrytKNIkiQpJRbns8jn87R1DTimIUmSVOIs\nzmfRPzTG0Mi4YxqSJEklzuJ8Fh3dQ4DzzZIkSaXO4nwWHT2TxdlRDUmSpJJmcT6LE8V5QWNVykkk\nSZKUJovzWZwc1XDHWZIkqaRZnM/ixI7zImecJUmSSprF+Sw6e4Yoz5bRUFeZdhRJkiSlyOJ8Fh3d\nQzQ31VCWyaQdRZIkSSmyOJ/ByOg4PQOjLG6qSTuKJEmSUmZxPoPO3mEAmufXppxEkiRJabM4n8GJ\nDwa64yxJkiSL8xm0LKhl44p5XH/Z0rSjSJIkKWUW5zNY0FjNH73nWjasmJ92FEmSJKXM4ixJkiQV\nwOIsSZIkFcDiLEmSJBXA4ixJkiQVwOIsSZIkFcDiLEmSJBXA4ixJkiQVwOIsSZIkFcDiLEmSJBXA\n4ixJkiQVwOIsSZIkFcDiLEmSJBXA4ixJkiQVwOIsSZIkFcDiLEmSJBXA4ixJkiQVwOIsSZIkFcDi\nLEmSJBUgk8/n084gSZIkzXruOEuSJEkFsDhLkiRJBbA4S5IkSQWwOEuSJEkFsDhLkiRJBbA4S5Ik\nSQUoTzvAbBVCKAM+AWwChoHfiDHuSjfV3BVCqAA+C6wBqoA/B7YBfwfkgS3AR2KMuZQizmkhhMXA\nz4A7gDFc1wsWQvhj4E1AJRM/Cx7Adb1gkz8L/p6JnwXjwG/i39kLEkK4EfivMcZXhhA2cIq1DCH8\nCfz/7d17jFxlHcbx77YFqlCQOyKgIcAj1NRCa8HSUkRIUaJcVCBIAAVSCBdLNKDl7iWAioVyF2kU\nRFAEjJC0EKkF5GK5U0QfBQwEocpdKpReqH+874Zhu9sOu9tOhz6ff3bO6ZlzfufZ6exv3nlnDntR\nsp5oe1bLCm4TXXIdDlxAecy+BRxi+9+SjgQmUHL9vu2bW1dx+2jMtmHdQcBxtj9dl1fKbDPi3LN9\ngMH1F/ht4NwW19PuDgZesj0W+BxwIfAT4JS6rgPYu4X1ta3aiFwGvFlXJdc+krQrMBrYGRgHbE5y\n7S+fBwbZHg18F/gBybbXJJ0I/AwYXFctkaWkHSiP4x2BA4GLWlFrO+km1/MpTd2uwA3ASZI2AY6n\nPE+MB86StEYLym0r3WRLfWFyOOUxy8qcbRrnno0BpgPYvhcY2dpy2t51wKkNywuBEZRRPIBpwO4r\nuqj3iR8DlwLP1eXk2nfjgdnAjcBNwM0k1/7yd2BQfVdvbWABybYvngT2a1juLssxwK22F9t+hpL/\nhiu2zLbTNdcDbT9cbw8C5gGjgLtsv2X7NeAJYNiKLbMtvStbSesDZwMTG7ZZabNN49yztYHXGpYX\nScrUll6yPdf265KGAL8FTgE6bHdeuvJ1YJ2WFdimJB0GvGD7lobVybXvNqC8WP4KcBRwNTAgufaL\nuZRpGn8DLgemkMdsr9m+nvLio1N3WXb9e5aMl6FrrrafB5A0GjgWmExy7ZXGbCUNBK4ATqDk12ml\nzTaNc8/+CwxpWB5ge2Grink/kLQ58EfgKtu/AhrnMA4BXm1JYe3t68AekmYCw4ErgY0a/j259s5L\nwC2259s2ZXSp8Uk7ufbeCZRst6F8huQXlHnknZJt33T3vNr171ky7gVJB1De3dvL9gsk1/4wAtga\nuAS4FthO0nmsxNmmce7ZXZS5eEjaifK2bfSSpI2BW4GTbE+tqx+qc0mhzHu+sxW1tTPbu9geV+fd\nPQwcAkxLrn32J2BPSR2SNgXWBG5Lrv3iFd4ZSXoZWI08F/Sn7rK8CxgvaYCkLSgDQS+2qsB2JOlg\nykjzrrafqqtnAWMlDZa0DrAt5QOZ0STbs2wPrX/DDgQetz2RlTjbTD3o2Y2Ukby7KZPVv9bietrd\nJGBd4FRJnXOdvwFMkbQ68FfKFI7ou28ClyfX3rN9s6RdKE/eA4BjgH+SXPvDZGCqpDspI82TgPtJ\ntv1lif//thfVvO/hncdzNKlOJ5gCPAPcIAngdtunS5pCeXEyADjZ9rzWVfr+YXvOypptx+LFi5e9\nVURERETEKi5TNSIiIiIimpDGOSIiIiKiCWmcIyIiIiKakMY5IiIiIqIJaZwjIiIiIpqQxjkiop9J\nmtnwXbrL6xhrS7pf0mOStlmex2olSWdKGtvqOiIiIN/jHBHRroYD822PbHUhy9k4yhVHIyJaLt/j\nHBGrrDoqPAl4g3JlqtnAQcCmwEzbH6vbnQFg+wxJc4DfATsCc4CpwPHAZsBhtm+vl0B/ru4T4ATb\nMyWtBVwEfAIYCJxj+xpJhwGHAhsAN9me1FDjxsAVwBbAwlrvg8DdwCbADNtfbNh+cD3GGGAB8D3b\nv65XQD0fGAy8CEyw/USt9cG6/WDgJMrFibYDJtueXM//o/V8NgAus/0jSQOA84DPAouBq2yf01Ou\ntudLOgSYSHnH8wHgGNvzJD1PufDJmHqe+wNjgYtrzvsCe9Sc3gZm2Z7Q8283IqL/ZapGRKzqRlMu\npbstpTkdv4ztNwam2d6e0mjua3sscAalIew0t25zKPBLSWsApwAP2B4B7AKcLGnLuv1mwPaNTXN1\nAaU5HgZ8mdKodwBHAPc3Ns3VccBa9Xx2B06rV5G7FjjW9ieBS4FrGu7TYXsUcH093n6UpvW0hm1G\n1P2NACZI2gE4CtgcGNglEKsAAALGSURBVAaMAr4kaa+6/RK5ShoKHAmMtj0c+A/wrbr9JsBtNbM7\naq1XUq4qeATwOPAdYGStYXVJHyEiYgVK4xwRq7rHbD9r+23KJYrXa+I+0+rPp4EZDbfXbdjmCgDb\nj1IaxI9TGs+jJD1MaQ7XBIbW7R+0vbCbY+3WsK+ngD9TRrt7Mg642vbbtufYHgpsA7xi+766n+uA\nrSSt08353Gv7DdtPAx9q2O81tufafg34fa1rN+DnthfZfgO4mjL6DN3n+hlga+DemsHeNZdO0zvv\nS5ffg+1FlFH2+4DTgXNt/2spOURE9Ls0zhGxqpvXcHsxZTS382en1RrvYHt+w2J3zW7X9QMo0yYG\nAgfbHl5HXHfinWbxzR720/V5uoOlfz5lAaV+ACRt1c0+OvczsN7uzfksXEZt3eU6EPhNw/mPooxK\nA2B7Xpftu9oHOLr+23RJ43qoNSJiuUjjHBGxpFeB9SRtWKdY7NmLfXwVQNJIYAjwD8ro9NF1/YeB\nRynTGJZmBnB4vc+WwM7APUvZ/g7gAEkdkjYCbqeMJK8v6VN1P/sDT9t++T2cz76S1pC0LvAF4NZa\n26GSBkr6YD3npX2Qb2bdz0aSOoBLePf0lu4sBAZJ2pAyXWO27dPq8Ye9h/ojIvosjXNERBd1OsIP\nKdMC/gDM6sVu1pL0EGU+8UG2FwBnAh+Q9Bil6TzR9pPL2M/xwG6SZlM+lHiE7eeXsv3FwP+AR2rt\nx9XzOQC4sB772Lr8XrwJ3Elp2s+y/ThwGfBsPdZDlA823tjTDmw/QslgBvAXygj02cs47nRKhlsD\nPwXuk/QAZX751Pd4DhERfZJv1YiIiKVq/FaR1lYSEdFaGXGOiIiIiGhCRpwjIiIiIpqQEeeIiIiI\niCakcY6IiIiIaEIa54iIiIiIJqRxjoiIiIhoQhrniIiIiIgmpHGOiIiIiGjC/wFlxK2jC/aX+gAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Making the screeplot - plotting the cumulative variance against the number of components\n",
    "%matplotlib inline\n",
    "fig = plt.figure(figsize = (12,8))\n",
    "plt.plot(np.cumsum(pca.explained_variance_ratio_))\n",
    "plt.xlabel('number of components')\n",
    "plt.ylabel('cumulative explained variance')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([10.73, 18.87, 25.12, 29.67, 33.97, 38.05, 41.9 , 45.3 , 47.9 ,\n",
       "       50.23, 52.37, 54.41, 56.38, 58.07, 59.68, 61.16, 62.6 , 63.95,\n",
       "       65.26, 66.56, 67.8 , 68.99, 70.11, 71.1 , 72.07, 73.03, 73.92,\n",
       "       74.77, 75.59, 76.35, 77.08, 77.8 , 78.51, 79.2 , 79.85, 80.49,\n",
       "       81.12, 81.73, 82.33, 82.92, 83.5 , 84.07, 84.61, 85.14, 85.66,\n",
       "       86.17, 86.66, 87.14, 87.61, 88.06, 88.5 , 88.93, 89.35, 89.76,\n",
       "       90.16, 90.55, 90.92, 91.28, 91.62, 91.95, 92.26, 92.55, 92.83,\n",
       "       93.1 , 93.37, 93.63, 93.88, 94.13, 94.37, 94.6 , 94.83, 95.04,\n",
       "       95.25, 95.45, 95.65, 95.85, 96.04, 96.23, 96.41, 96.58, 96.75,\n",
       "       96.91, 97.07, 97.22, 97.37, 97.52, 97.67, 97.82, 97.96, 98.09,\n",
       "       98.22, 98.34, 98.46, 98.58, 98.7 , 98.81, 98.92, 99.03, 99.13,\n",
       "       99.22, 99.31, 99.39, 99.47, 99.54, 99.61, 99.67, 99.73, 99.78,\n",
       "       99.82, 99.86, 99.89, 99.91, 99.93, 99.94, 99.95, 99.96, 99.96,\n",
       "       99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96,\n",
       "       99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96,\n",
       "       99.96, 99.96, 99.96, 99.96, 99.96, 99.96, 99.96])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.cumsum(np.round(pca.explained_variance_ratio_, decimals=4)*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Apply logistic regression with 48 columns (90% explained variance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pca48 = PCA(n_components=48)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(37596, 48)\n",
      "(8552, 48)\n"
     ]
    }
   ],
   "source": [
    "df_train_pca48 = pca48.fit_transform(X_tr)\n",
    "print(df_train_pca48.shape)\n",
    "df_test_pca48 = pca48.transform(X_test)\n",
    "print(df_test_pca48.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's run the model using the selected variables\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn import metrics\n",
    "logsk1 = LogisticRegression(C=1e9)\n",
    "logsk1.fit(df_train_pca48, y_tr)\n",
    "\n",
    "# Predicted probabilities\n",
    "y_pred48 = logsk1.predict(df_test_pca48)\n",
    "# Converting y_pred to a dataframe which is an array\n",
    "y_pred_df = pd.DataFrame(y_pred48)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[6632 1478]\n",
      " [  80  362]]\n"
     ]
    }
   ],
   "source": [
    "# Printing confusion matrix\n",
    "print(confusion_matrix(y_test,y_pred48))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LogisticRegression accuracy with PCA:  0.817820392890552\n"
     ]
    }
   ],
   "source": [
    "print(\"LogisticRegression accuracy with PCA: \",accuracy_score(y_test,y_pred48))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='blue'>SUMMARY PART 3: PCA with Logistic Regression</font>\n",
    "\n",
    "OBSERVATIONS\n",
    "1. Data is higly imbalanced only 5.6% data is churn & remaining is non_churn\n",
    "2. SMOTE is applied to balance data\n",
    "3. 82% accuracy with default PCA\n",
    "4. 90% variance can be explained with 48 columns and maintain 82% accuracy\n",
    "5. Key features are arpu_6, arpu_7, arpu_8, onnet_mou_6, onnet_mou_7, onnet_mou_8, offnet_mou_6, offnet_mou_7, offnet_mou_8, roam_ic_mou_6\n",
    "\n",
    "NEXT STEPS:\n",
    "1. Run other models with LASSO and RFE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}