 Naive Bayes from scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Naive Bayes From Scratch\n",
    "Nemshan \n",
    "\n",
    "\n",
    "- 1- prepare the data for Naïve Bayes algorithm \n",
    "    - a.\tcreated a function named “preprocess” to take care of missing.\n",
    "    - b.\tcreated a function named “removeUcol” to drop the columns with only unique values. e.g. index numbers\n",
    "    - c.\tcreated a function named “label” to drop the response column.\n",
    "    - d.\tcreated two functions named “splitter_cat” to split the categorical variables.\n",
    "    - e.\tcreated a function named “normalize” to normalize  the features.\n",
    "- \t2- Columns with only unique values were removed from categorical dataset\n",
    "- \t3- Splitting Train and Test function \n",
    "-\t4- Takes the size of the dataset and splits it 80-20 \n",
    "\t5- Returns train and test dataset \n",
    "\t6- I Created a probability function \n",
    "\t7- I Created prediction function\n",
    "\t8- I Created an accuracy function to determine the accuracy \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "import numpy as np \n",
    "from Madefunctions import *\n",
    "import math \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass                                             name     sex   age  \\\n",
       "0       1                    Allen, Miss. Elisabeth Walton  female  29.0   \n",
       "1       1                   Allison, Master. Hudson Trevor    male   NaN   \n",
       "2       1                     Allison, Miss. Helen Loraine  female   2.0   \n",
       "3       1             Allison, Mr. Hudson Joshua Creighton    male  30.0   \n",
       "4       1  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female  25.0   \n",
       "\n",
       "   sibsp  parch  ticket      fare    cabin embarked boat   body  \\\n",
       "0      0      0   24160  211.3375       B5        S    2    NaN   \n",
       "1      1      2  113781  151.5500  C22 C26        S   11    NaN   \n",
       "2      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "3      1      2  113781  151.5500  C22 C26        S  NaN  135.0   \n",
       "4      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "\n",
       "                         home.dest  survived  \n",
       "0                     St Louis, MO         1  \n",
       "1  Montreal, PQ / Chesterville, ON         1  \n",
       "2  Montreal, PQ / Chesterville, ON         0  \n",
       "3  Montreal, PQ / Chesterville, ON         0  \n",
       "4  Montreal, PQ / Chesterville, ON         0  "
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Titanic Dataset \n",
    "titanic = pd.read_csv(\"titanic.csv\")\n",
    "titanic.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "file dimensions:  (1309, 14)\n",
      "\n",
      "Columns in dataframe:  ['pclass', 'name', 'sex', 'age', 'sibsp', 'parch', 'ticket', 'fare', 'cabin', 'embarked', 'boat', 'body', 'home.dest', 'survived']\n",
      "Number of NA in  pclass :  0\n",
      "Number of NA in  name :  0\n",
      "Number of NA in  sex :  0\n",
      "Number of NA in  age :  264\n",
      "Number of NA in  sibsp :  0\n",
      "Number of NA in  parch :  0\n",
      "Number of NA in  ticket :  0\n",
      "Number of NA in  fare :  1\n",
      "Number of NA in  cabin :  1014\n",
      "Number of NA in  embarked :  2\n",
      "Number of NA in  boat :  823\n",
      "Number of NA in  body :  1188\n",
      "Number of NA in  home.dest :  564\n",
      "Number of NA in  survived :  0\n",
      "\n",
      "DROP COLUMNS WITH MORE THAN 30 PERCENT NA\n",
      "4  columns were dropped\n",
      "\n",
      " ['pclass', 'name', 'sex', 'age', 'sibsp', 'parch', 'ticket', 'fare', 'embarked', 'survived']\n",
      "\n",
      "The new dimension:  (1309, 10)\n",
      "\n",
      "DROP ROWS WITH NA\n",
      "\n",
      "Clean dataframe dimension  1042  rows and 10  columns\n"
     ]
    }
   ],
   "source": [
    "newdf = preprocess(titanic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>Anderson, Mr. Harry</td>\n",
       "      <td>male</td>\n",
       "      <td>48.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19952</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass                                             name     sex   age  \\\n",
       "0       1                    Allen, Miss. Elisabeth Walton  female  29.0   \n",
       "2       1                     Allison, Miss. Helen Loraine  female   2.0   \n",
       "3       1             Allison, Mr. Hudson Joshua Creighton    male  30.0   \n",
       "4       1  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female  25.0   \n",
       "5       1                              Anderson, Mr. Harry    male  48.0   \n",
       "\n",
       "   sibsp  parch  ticket      fare embarked  survived  \n",
       "0      0      0   24160  211.3375        S         1  \n",
       "2      1      2  113781  151.5500        S         0  \n",
       "3      1      2  113781  151.5500        S         0  \n",
       "4      1      2  113781  151.5500        S         0  \n",
       "5      0      0   19952   26.5500        S         1  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "newdf.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1</td>\n",
       "      <td>Anderson, Mr. Harry</td>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19952</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass                                             name     sex  sibsp  \\\n",
       "0       1                    Allen, Miss. Elisabeth Walton  female      0   \n",
       "2       1                     Allison, Miss. Helen Loraine  female      1   \n",
       "3       1             Allison, Mr. Hudson Joshua Creighton    male      1   \n",
       "4       1  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female      1   \n",
       "5       1                              Anderson, Mr. Harry    male      0   \n",
       "\n",
       "   parch  ticket embarked  survived  \n",
       "0      0   24160        S         1  \n",
       "2      2  113781        S         0  \n",
       "3      2  113781        S         0  \n",
       "4      2  113781        S         0  \n",
       "5      0   19952        S         1  "
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical = splitter_cat(newdf)\n",
    "categorical.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "categorical = removeUcol(categorical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train, test = traintestsplit(categorical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Cardeza, Mrs. James Warburton Martinez (Charlo...</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>PC 17755</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>Pickard, Mr. Berk (Berk Trembisky)</td>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>SOTON/O.Q. 392078</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Bazzani, Miss. Albina</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11813</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Andersson, Miss. Erna Alexandra</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3101281</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Fortune, Miss. Alice Elizabeth</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>19950</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass                                               name     sex  sibsp  \\\n",
       "0       1  Cardeza, Mrs. James Warburton Martinez (Charlo...  female      0   \n",
       "1       3                 Pickard, Mr. Berk (Berk Trembisky)    male      0   \n",
       "2       1                              Bazzani, Miss. Albina  female      0   \n",
       "3       3                    Andersson, Miss. Erna Alexandra  female      4   \n",
       "4       1                     Fortune, Miss. Alice Elizabeth  female      3   \n",
       "\n",
       "   parch             ticket embarked  survived  \n",
       "0      1           PC 17755        C         1  \n",
       "1      0  SOTON/O.Q. 392078        S         1  \n",
       "2      0              11813        C         1  \n",
       "3      2            3101281        S         1  \n",
       "4      2              19950        S         1  "
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train2 = train.drop(\"name\", axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train3 = train2.drop(\"ticket\",axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass     sex  sibsp  parch embarked  survived\n",
       "0       1  female      0      1        C         1\n",
       "1       3    male      0      0        S         1\n",
       "2       1  female      0      0        C         1\n",
       "3       3  female      4      2        S         1\n",
       "4       1  female      3      2        S         1"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>Bateman, Rev. Robert James</td>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S.O.P. 1166</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>Panula, Master. Eino Viljami</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3101295</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Hirvonen, Miss. Hildur E</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3101298</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Lindell, Mr. Edvard Bengtsson</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>349910</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>Klasen, Mrs. (Hulda Kristina Eugenia Lofqvist)</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>350405</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass                                            name     sex  sibsp  \\\n",
       "0       2                      Bateman, Rev. Robert James    male      0   \n",
       "1       3                    Panula, Master. Eino Viljami    male      4   \n",
       "2       3                        Hirvonen, Miss. Hildur E  female      0   \n",
       "3       3                   Lindell, Mr. Edvard Bengtsson    male      1   \n",
       "4       3  Klasen, Mrs. (Hulda Kristina Eugenia Lofqvist)  female      0   \n",
       "\n",
       "   parch       ticket embarked  survived  \n",
       "0      0  S.O.P. 1166        S         0  \n",
       "1      1      3101295        S         0  \n",
       "2      1      3101298        S         1  \n",
       "3      0       349910        S         0  \n",
       "4      2       350405        S         0  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test2 = test.drop(\"name\", axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test3 = test2.drop(\"ticket\", axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass     sex  sibsp  parch embarked  survived\n",
       "0       2    male      0      0        S         0\n",
       "1       3    male      4      1        S         0\n",
       "2       3  female      0      1        S         1\n",
       "3       3    male      1      0        S         0\n",
       "4       3  female      0      2        S         0"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter Response Variable: survived\n"
     ]
    }
   ],
   "source": [
    "prob, col, L = probdict(train3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: {'embarked': {'C': 0.13877999999999999,\n",
       "   'Q': 0.065310000000000007,\n",
       "   'S': 0.79591999999999996},\n",
       "  'parch': {0: 0.81223999999999996,\n",
       "   1: 0.10000000000000001,\n",
       "   2: 0.065310000000000007,\n",
       "   3: 0.0061199999999999996,\n",
       "   4: 0.0061199999999999996,\n",
       "   5: 0.0081600000000000006,\n",
       "   6: 0.0020400000000000001},\n",
       "  'pclass': {1: 0.16531000000000001,\n",
       "   2: 0.23061000000000001,\n",
       "   3: 0.60407999999999995},\n",
       "  'sex': {'female': 0.15714, 'male': 0.84286000000000005},\n",
       "  'sibsp': {0: 0.68776000000000004,\n",
       "   1: 0.21632999999999999,\n",
       "   2: 0.038780000000000002,\n",
       "   3: 0.020410000000000001,\n",
       "   4: 0.026530000000000001,\n",
       "   5: 0.0081600000000000006,\n",
       "   8: 0.0020400000000000001}},\n",
       " 1: {'embarked': {'C': 0.31195000000000001,\n",
       "   'Q': 0.03499,\n",
       "   'S': 0.65305999999999997},\n",
       "  'parch': {0: 0.63848000000000005,\n",
       "   1: 0.21282999999999999,\n",
       "   2: 0.13120000000000001,\n",
       "   3: 0.014579999999999999,\n",
       "   5: 0.0029199999999999999},\n",
       "  'pclass': {1: 0.42565999999999998, 2: 0.26822000000000001, 3: 0.30612},\n",
       "  'sex': {'female': 0.67930000000000001, 'male': 0.32069999999999999},\n",
       "  'sibsp': {0: 0.60641,\n",
       "   1: 0.32362000000000002,\n",
       "   2: 0.043729999999999998,\n",
       "   3: 0.017489999999999999,\n",
       "   4: 0.0087500000000000008}}}"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Q</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass     sex  sibsp  parch embarked  survived\n",
       "0       3  female      1      1        Q         0\n",
       "1       2  female      1      2        S         1\n",
       "2       3    male      1      0        S         1\n",
       "3       2    male      1      1        C         0\n",
       "4       1  female      1      0        C         1"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'survived'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.5882352941176471, 0.4117647058823529]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.31195000000000001"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob[1][\"embarked\"][\"C\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "T =  predict(prob, test3, col,L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>sex</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>embarked</th>\n",
       "      <th>survived</th>\n",
       "      <th>predict</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Q</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pclass     sex  sibsp  parch embarked  survived  predict\n",
       "0       3  female      1      1        Q         0        1\n",
       "1       2  female      1      2        S         1        1\n",
       "2       3    male      1      0        S         1        0\n",
       "3       2    male      1      1        C         0        1\n",
       "4       1  female      1      0        C         1        1"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "165\n",
      "209\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "78.94736842105263"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "naiveaccuracy(T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'pclass': []}\n",
      "{'pclass': [], 'name': [], 'sex': [], 'sibsp': [], 'parch': [], 'ticket': [], 'embarked': [], 'survived': []}\n"
     ]
    }
   ],
   "source": [
    "d = {}\n",
    "f = {}\n",
    "# ls = [2,2,5]\n",
    "# ls2 = [5,7,8]\n",
    "features = train.columns.values\n",
    "features = features.tolist()\n",
    "for feature in train:\n",
    "    d[i] = []\n",
    "    f[feature] = []\n",
    "    \n",
    "print(d)\n",
    "print(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: {'pclass': [{1: 'a'}, {2: 'b'}], 'sex': [{1: 'a'}, {2: 'b'}], 'sibsp': [{1: 'a'}, {2: 'b'}], 'parch': [{1: 'a'}, {2: 'b'}], 'embarked': [{1: 'a'}, {2: 'b'}], 'survived': [{1: 'a'}, {2: 'b'}]}, 1: {'pclass': [{1: 'a'}, {2: 'b'}], 'sex': [{1: 'a'}, {2: 'b'}], 'sibsp': [{1: 'a'}, {2: 'b'}], 'parch': [{1: 'a'}, {2: 'b'}], 'embarked': [{1: 'a'}, {2: 'b'}], 'survived': [{1: 'a'}, {2: 'b'}]}}\n"
     ]
    }
   ],
   "source": [
    "ls = list(set(train3[\"survived\"]))\n",
    "d = {}\n",
    "f = {}\n",
    "\n",
    "features = train3.columns.values\n",
    "features = features.tolist()\n",
    "for i in ls:\n",
    "    d[i] = []\n",
    "    for feature in features:\n",
    "        f.update({feature:[{1:\"a\"},{2:\"b\"}]})\n",
    "    d[i] = f\n",
    "    \n",
    "    \n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def binning(df):\n",
    "    col = input(\"Pick a column to bin: \")\n",
    "    colmin = min(df[col])\n",
    "    print(colmin)\n",
    "    colmax = max(df[col])\n",
    "    print(colmax)\n",
    "    diff = colmax - colmin \n",
    "    print(diff)\n",
    "    numb = int(input(\"number of bins: \"))\n",
    "    bins = diff / numb\n",
    "    ls = []\n",
    "    \n",
    "    for n in range(1,numb+1):\n",
    "        bb = str(\"b\") + str(n)\n",
    "#         ls.append(bb)\n",
    "        bb = colmin + (bins*n) \n",
    "        ls.append(int(bb))\n",
    "#         print(bb)\n",
    "    print(ls)\n",
    "    t = str(\"0-\")\n",
    "    for row in df[col]:\n",
    "        \n",
    "        for i in range(0,len(ls)):\n",
    "            \n",
    "            if row < ls[i]:\n",
    "                s = t + str(ls[i])\n",
    "                df[col] = df[col].replace(row, s)\n",
    "                t = str(ls[i]) + \"-\"\n",
    "                \n",
    "        \n",
    "        \n",
    "    print(df.head())  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pick a column to bin: sibsp\n",
      "0\n",
      "8\n",
      "8\n",
      "number of bins: 5\n",
      "[1, 3, 4, 6, 8]\n",
      "   pclass                                             name     sex   age  \\\n",
      "0       1                    Allen, Miss. Elisabeth Walton  female  29.0   \n",
      "2       1                     Allison, Miss. Helen Loraine  female   2.0   \n",
      "3       1             Allison, Mr. Hudson Joshua Creighton    male  30.0   \n",
      "4       1  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)  female  25.0   \n",
      "5       1                              Anderson, Mr. Harry    male  48.0   \n",
      "\n",
      "  sibsp  parch  ticket      fare embarked  survived  \n",
      "0   0-1      0   24160  211.3375        S         1  \n",
      "2   8-3      2  113781  151.5500        S         0  \n",
      "3   8-3      2  113781  151.5500        S         0  \n",
      "4   8-3      2  113781  151.5500        S         0  \n",
      "5   0-1      0   19952   26.5500        S         1  \n"
     ]
    }
   ],
   "source": [
    "binning(clean)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}