Building a Spam filter (Naive Bayes).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b9bc044d-7881-4516-91cf-e68e6332a5b5",
   "metadata": {},
   "source": [
    "# Building a Spam filter with Naive Bayes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d782e91-a1a7-4afa-bb8d-44973fc79b75",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "- In this project, we're going to build a Spam filter for SMS messages. \n",
    "- Our main public is telephone provider companies.\n",
    "- Our main goal is achieve the best accuracy possible using the multinomial Naive Bayes algorithm. This is an supervised machine learning model. Naive Bayes algorithm is indicated when working with small datasets, it generalizes well with them, unlike more complex models like neural networks.\n",
    "- To classify messages as spam or non-spam, the computer:\n",
    "    1. Learns how humans classify messages;\n",
    "    2. Uses that knowledge to estimate probabilities for new messages being spam or non-spam;\n",
    "    3. Classifies a new message based on these values:\n",
    "        - the probability for spam is greater — spam,\n",
    "        - the probability for non-spam is greater — non-spam,\n",
    "        - the two probability values are equal — we may need a human to classify the message.\n",
    "- We're going to build the classificator, preparing the constants, the parameters and calculating the probabilites. In addition, we're going to use the Naive Bayes model of Scikit-learn library. In the end, we'll compare the accuracies to know which way performs better."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea18deb2-2b28-4a6f-b054-9d48375c25d7",
   "metadata": {},
   "source": [
    "## Introducing the Data\n",
    "- We'll use a dataset of 5,572 SMS that were already classified by humans. The dataset was put together by Tiago A. Almeida and José María Gómez Hidalgo, and it can be downloaded from the [UCI Machine Learning Repository.](https://archive.ics.uci.edu/ml/datasets/sms+spam+collection)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "946b0a07-bdb7-49d8-957f-4e1e31a9395b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing the libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import matplotlib.style as style\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df3d9c5-cfe8-4431-b734-cb66e4efd630",
   "metadata": {},
   "source": [
    "- The dataset doesn't have a header row, which means we need to use the header=None parameter, otherwise the first row will be wrongly used as the header row. We used the names=['Label', 'SMS'] parameter to name the columns as Label and SMS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f10e2164-5fdb-48e8-a8e8-a8ca42302c79",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing the dataset\n",
    "sms_set = pd.read_csv('SMSSpamCollection', sep='\\t', header=None, names=['Label', 'SMS'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bfaf5e3c-6cd4-499e-aea6-ab6dec895c8c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham  Go until jurong point, crazy.. Available only ...\n",
       "1   ham                      Ok lar... Joking wif u oni...\n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3   ham  U dun say so early hor... U c already then say...\n",
       "4   ham  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# knowing the dataset\n",
    "sms_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c727063e-056a-416a-b668-5b3abd39b9e0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 5572 entries, 0 to 5571\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   Label   5572 non-null   object\n",
      " 1   SMS     5572 non-null   object\n",
      "dtypes: object(2)\n",
      "memory usage: 87.2+ KB\n"
     ]
    }
   ],
   "source": [
    "# checking null values\n",
    "sms_set.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0310acbd-d0e0-4571-aedf-4eb1b7d83a9c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     4825\n",
       "spam     747\n",
       "Name: Label, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# counting the labels\n",
    "sms_set['Label'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "212d7283-fb59-4791-bdb3-d16ee84c72f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     86.593683\n",
       "spam    13.406317\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# counting the percentage of the labels\n",
    "sms_set['Label'].value_counts(normalize=True) * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cf29864-8445-4887-ba7a-6345da7f2a66",
   "metadata": {},
   "source": [
    "- In this first look at the dataset, we notice the absence of null values, and two types of label: \"ham\"(which means non-spam) with 86,6% of the rows and spam, with 13,4%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c4e941e-297e-49ec-a346-735baa824592",
   "metadata": {},
   "source": [
    "### Training and Test set\n",
    "- Next, before we start working on our filter, we need to design how we will test it. To do that, first we'll split our dataset into two categories:\n",
    "    - A training set, that will be used to \"train\" the computer how to classify the messages;\n",
    "    - A test set, that will be used to test the performance of our spam filter.\n",
    "- Our training set will have 80% of the dataset (4,458 messages) and our test set will have 20% of the dataset (1,114 messages)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "edaa05fc-fd8d-45a4-a365-0180ba4dca40",
   "metadata": {},
   "outputs": [],
   "source": [
    "# randomize the dataset\n",
    "data_randomized = sms_set.sample(frac=1, random_state=1)\n",
    "\n",
    "# calculate index for split\n",
    "training_test_index = round(len(data_randomized) * 0.8)\n",
    "\n",
    "# training/Test split\n",
    "training_set = data_randomized[:training_test_index].reset_index(drop=True) \n",
    "test_set = data_randomized[training_test_index:].reset_index(drop=True)\n",
    "# drop parameter to avoid the old index being added as a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e5f1a295-29ee-4b98-abee-878f0c43f2af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     86.54105\n",
       "spam    13.45895\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the training set\n",
    "training_set['Label'].value_counts(normalize=True) * 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3fb45d3e-9c0c-42d7-a1fc-b9823290c02d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     86.804309\n",
       "spam    13.195691\n",
       "Name: Label, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the test set\n",
    "test_set['Label'].value_counts(normalize=True) * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "861a1dae-065c-45a9-96bd-d3524b5518f7",
   "metadata": {},
   "source": [
    "- As we want both splits of the dataset (the training and the test set) to be good representations of the whole dataset, we have to check if the percentages of the two types of label (ham and spam) are similar to the ones we found earlier. And we got arround 86% and 13% for both. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9751c71a-2044-4481-8f50-9fb285f7d3c6",
   "metadata": {},
   "source": [
    "## Data Cleaning\n",
    "- On this section, we'll clean the dataset to prepare to the use of the Naive Bayes algorithm. First, we'll remove punctuation and turn lower case the wors in the messages. Then, transform each message in a list, create a vocabulary list with the unique words and create a dictionary with the word counts per sms. The final step is concatenate the DataFrame with the count of words and the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "28dbb2f6-5f86-464b-9823-5d8f0baf29e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Suporte\\AppData\\Local\\Temp/ipykernel_1748/2879400168.py:2: FutureWarning: The default value of regex will change from True to False in a future version.\n",
      "  training_set['SMS'] = training_set['SMS'].str.replace('\\W', ' ').str.lower()\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>yep  by the pretty sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>yes  princess  are you going to make me moan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>welp apparently he retired</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>havent</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>i forgot 2 ask ü all smth   there s a card on ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham                       yep  by the pretty sculpture\n",
       "1   ham      yes  princess  are you going to make me moan \n",
       "2   ham                         welp apparently he retired\n",
       "3   ham                                            havent \n",
       "4   ham  i forgot 2 ask ü all smth   there s a card on ..."
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# removing punctuation and turning lower case the words\n",
    "training_set['SMS'] = training_set['SMS'].str.replace('\\W', ' ').str.lower()\n",
    "training_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4bc0e73f-ac6f-419e-9952-29c95902a7c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yep, by, the, pretty, sculpture]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yes, princess, are, you, going, to, make, me,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>[welp, apparently, he, retired]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>[havent]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>[i, forgot, 2, ask, ü, all, smth, there, s, a,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS\n",
       "0   ham                  [yep, by, the, pretty, sculpture]\n",
       "1   ham  [yes, princess, are, you, going, to, make, me,...\n",
       "2   ham                    [welp, apparently, he, retired]\n",
       "3   ham                                           [havent]\n",
       "4   ham  [i, forgot, 2, ask, ü, all, smth, there, s, a,..."
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transforming each message from the SMS column into a list\n",
    "training_set['SMS'] = training_set['SMS'].str.split()\n",
    "training_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0567deb3-b11a-46be-89c8-b77f740ee9d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7783"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# crating a vocabulary list with the unique words of the column\n",
    "vocabulary = []\n",
    "for m in training_set['SMS']:\n",
    "    for w in m:\n",
    "        vocabulary.append(w)\n",
    "vocabulary = list(set(vocabulary))\n",
    "\n",
    "# number of unique words\n",
    "len(vocabulary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e46a38ed-821b-4382-b8e6-228a789b029f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trouser</th>\n",
       "      <th>education</th>\n",
       "      <th>usmle</th>\n",
       "      <th>violated</th>\n",
       "      <th>miss</th>\n",
       "      <th>169</th>\n",
       "      <th>tag</th>\n",
       "      <th>tcr</th>\n",
       "      <th>realy</th>\n",
       "      <th>01223585334</th>\n",
       "      <th>...</th>\n",
       "      <th>worlds</th>\n",
       "      <th>diapers</th>\n",
       "      <th>hmph</th>\n",
       "      <th>edison</th>\n",
       "      <th>adventure</th>\n",
       "      <th>southern</th>\n",
       "      <th>2</th>\n",
       "      <th>name1</th>\n",
       "      <th>kickoff</th>\n",
       "      <th>worry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 7783 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   trouser  education  usmle  violated  miss  169  tag  tcr  realy  \\\n",
       "0        0          0      0         0     0    0    0    0      0   \n",
       "1        0          0      0         0     0    0    0    0      0   \n",
       "2        0          0      0         0     0    0    0    0      0   \n",
       "3        0          0      0         0     0    0    0    0      0   \n",
       "4        0          0      0         0     0    0    0    0      0   \n",
       "\n",
       "   01223585334  ...  worlds  diapers  hmph  edison  adventure  southern  2  \\\n",
       "0            0  ...       0        0     0       0          0         0  0   \n",
       "1            0  ...       0        0     0       0          0         0  0   \n",
       "2            0  ...       0        0     0       0          0         0  0   \n",
       "3            0  ...       0        0     0       0          0         0  0   \n",
       "4            0  ...       0        0     0       0          0         0  2   \n",
       "\n",
       "   name1  kickoff  worry  \n",
       "0      0        0      0  \n",
       "1      0        0      0  \n",
       "2      0        0      0  \n",
       "3      0        0      0  \n",
       "4      0        0      0  \n",
       "\n",
       "[5 rows x 7783 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# creating a dictionary with the word counts per sms\n",
    "word_counts_per_sms = {unique_word: [0] * len(training_set['SMS']) for unique_word in vocabulary}\n",
    "for index, sms in enumerate(training_set['SMS']):\n",
    "    for word in sms:\n",
    "        word_counts_per_sms[word][index]+=1\n",
    "        \n",
    "word_counts = pd.DataFrame(word_counts_per_sms)\n",
    "word_counts.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "de1b1061-2925-43a0-a457-6280b463be67",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "      <th>trouser</th>\n",
       "      <th>education</th>\n",
       "      <th>usmle</th>\n",
       "      <th>violated</th>\n",
       "      <th>miss</th>\n",
       "      <th>169</th>\n",
       "      <th>tag</th>\n",
       "      <th>tcr</th>\n",
       "      <th>...</th>\n",
       "      <th>worlds</th>\n",
       "      <th>diapers</th>\n",
       "      <th>hmph</th>\n",
       "      <th>edison</th>\n",
       "      <th>adventure</th>\n",
       "      <th>southern</th>\n",
       "      <th>2</th>\n",
       "      <th>name1</th>\n",
       "      <th>kickoff</th>\n",
       "      <th>worry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yep, by, the, pretty, sculpture]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>[yes, princess, are, you, going, to, make, me,...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ham</td>\n",
       "      <td>[welp, apparently, he, retired]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>[havent]</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>[i, forgot, 2, ask, ü, all, smth, there, s, a,...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 7785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS  trouser  \\\n",
       "0   ham                  [yep, by, the, pretty, sculpture]        0   \n",
       "1   ham  [yes, princess, are, you, going, to, make, me,...        0   \n",
       "2   ham                    [welp, apparently, he, retired]        0   \n",
       "3   ham                                           [havent]        0   \n",
       "4   ham  [i, forgot, 2, ask, ü, all, smth, there, s, a,...        0   \n",
       "\n",
       "   education  usmle  violated  miss  169  tag  tcr  ...  worlds  diapers  \\\n",
       "0          0      0         0     0    0    0    0  ...       0        0   \n",
       "1          0      0         0     0    0    0    0  ...       0        0   \n",
       "2          0      0         0     0    0    0    0  ...       0        0   \n",
       "3          0      0         0     0    0    0    0  ...       0        0   \n",
       "4          0      0         0     0    0    0    0  ...       0        0   \n",
       "\n",
       "   hmph  edison  adventure  southern  2  name1  kickoff  worry  \n",
       "0     0       0          0         0  0      0        0      0  \n",
       "1     0       0          0         0  0      0        0      0  \n",
       "2     0       0          0         0  0      0        0      0  \n",
       "3     0       0          0         0  0      0        0      0  \n",
       "4     0       0          0         0  2      0        0      0  \n",
       "\n",
       "[5 rows x 7785 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# concatenating the word_counts with the training_set\n",
    "training_set_clean = pd.concat([training_set, word_counts], axis=1)\n",
    "training_set_clean.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ed18a64-b771-4851-b3d8-aeb566f788fd",
   "metadata": {},
   "source": [
    "## Implementation\n",
    "- Now, we're ready for the implementation of the Naive Bayes algorithm to classify the messages. It works upon the probability it gets from these two equations: ![](nbeq1.png)\n",
    "- To calculate P(wi|Spam) and P(wi|Ham) inside the formulas we need other two equations: ![](nbeq2.png)\n",
    "- where we need the constants:\n",
    "    - Nwi|Spam</sub> — the number of times the word wi occurs in spam messages;\n",
    "    - Nwi|Ham</sub> — the number of times the word wi occurs in ham messages;\n",
    "    - NSpam — total number of words in spam messages;\n",
    "    - NHam — total number of words in ham messages;\n",
    "    - NVocabulary — total number of unique words in the vocabulary;\n",
    "    - α — a smoothing parameter (Laplace smoother)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f015893-b16c-4aa7-9caa-c903299d677d",
   "metadata": {},
   "source": [
    "### 1. Calculating the constants\n",
    "- We will start calculating constants for the Naive Bayes algorithm: probability of spam, probability of non-spam, number of words in spam messages, number of words in non-spam messages and total number of vocabulary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "404a3a13-7f11-4838-a5d9-32ea4ab76456",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Isolating spam and ham messages first\n",
    "spam_messages = training_set_clean[training_set_clean['Label'] == 'spam']\n",
    "ham_messages = training_set_clean[training_set_clean['Label'] == 'ham']\n",
    "\n",
    "# P(Spam) and P(Ham)\n",
    "p_spam = len(spam_messages) / len(training_set_clean)\n",
    "p_ham = len(ham_messages) / len(training_set_clean)\n",
    "\n",
    "# N_Spam\n",
    "n_words_per_spam_message = spam_messages['SMS'].apply(len)\n",
    "n_spam = n_words_per_spam_message.sum()\n",
    "\n",
    "# N_Ham\n",
    "n_words_per_ham_message = ham_messages['SMS'].apply(len)\n",
    "n_ham = n_words_per_ham_message.sum()\n",
    "\n",
    "# N_Vocabulary\n",
    "n_vocabulary = len(vocabulary)\n",
    "\n",
    "# Laplace smoothing\n",
    "alpha = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eff2f02d-de03-4308-908d-606d47ba74b7",
   "metadata": {},
   "source": [
    "### 2. Calculating the parameters\n",
    "- Now that we have the constant terms calculated above, we can move on with calculating the parameters P(wi|Spam) and P(wi|Ham). Each parameter will thus be a conditional probability value associated with each word in the vocabulary.\n",
    "- The Naive Bayes algorithm is very fast compared to other algorithms because we beforehand calculate the both probabilities for each individual word, which remains constant for every new message. When a new message comes in, most of the needed computations are already done, which enables the algorithm to almost instantly classify the new message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f601ff16-01da-426e-9ded-99c44d873a6b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initiate parameters\n",
    "parameters_spam = {unique_word:0 for unique_word in vocabulary}\n",
    "parameters_ham = {unique_word:0 for unique_word in vocabulary}\n",
    "\n",
    "# calculate parameters\n",
    "for word in vocabulary:\n",
    "    n_word_given_spam = spam_messages[word].sum()   # spam_messages already defined above\n",
    "    p_word_given_spam = (n_word_given_spam + alpha) / (n_spam + alpha*n_vocabulary)\n",
    "    parameters_spam[word] = p_word_given_spam\n",
    "    \n",
    "    n_word_given_ham = ham_messages[word].sum()   # ham_messages already defined above\n",
    "    p_word_given_ham = (n_word_given_ham + alpha) / (n_ham + alpha*n_vocabulary)\n",
    "    parameters_ham[word] = p_word_given_ham"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32003062-19e8-4fb7-923b-5f1f001e66f8",
   "metadata": {},
   "source": [
    "### 3. Preparing the classificator\n",
    "- Now, we're ready to create a function that classifies the new messages. After calculating the equations above, the classificator will compare the values of P(Spam|w1, w2, ..., wn) and P(Ham|w1, w2, ..., wn), and return:\n",
    "    - 'ham': if P(Ham|w1, w2, ..., wn) > P(Spam|w1, w2, ..., wn);\n",
    "    - 'spam': if P(Ham|w1, w2, ..., wn) < P(Spam|w1, w2, ..., wn);\n",
    "    - 'needs human classification': if P(Ham|w1, w2, ..., wn) = P(Spam|w1, w2, ..., wn)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "10a6d058-c1f4-43ae-adbb-cb7ada26ceda",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing regular expressions\n",
    "import re\n",
    "\n",
    "# function of the classificator\n",
    "def classify(message):\n",
    "    \"\"\"\n",
    "    Classifies the given message based on the parameters previouly calculated and compares\n",
    "    the two values and classifies the message as spam or ham, or requires \n",
    "    human classification. \n",
    "    \n",
    "    Arg: message to be classified (str).\n",
    "    \n",
    "    Returns: classification of the message (str).\n",
    "    \"\"\"\n",
    "    message = re.sub('\\W', ' ', message)\n",
    "    message = message.lower()\n",
    "    message = message.split()\n",
    "\n",
    "    p_spam_given_message = p_spam\n",
    "    p_ham_given_message = p_ham\n",
    "\n",
    "    for word in message:\n",
    "        if word in parameters_spam:\n",
    "            p_spam_given_message *= parameters_spam[word]\n",
    "\n",
    "        if word in parameters_ham:\n",
    "            p_ham_given_message *= parameters_ham[word]\n",
    "\n",
    "    if p_ham_given_message > p_spam_given_message:\n",
    "        return 'ham'\n",
    "    elif p_spam_given_message > p_ham_given_message:\n",
    "        return 'spam'\n",
    "    else:\n",
    "        return 'needs human classification'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a124689f-be33-4d61-a476-781e321e2cda",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Later i guess. I needa do mcat study too.\n",
      "ham\n",
      "Get your garden ready for summer with a FREE selection of summer bulbs and seeds worth £33:50 only with The Scotsman this Saturday. To stop go2 notxt.co.uk\n",
      "spam\n"
     ]
    }
   ],
   "source": [
    "# testing the classification\n",
    "print(test_set['SMS'][0])\n",
    "print(classify(test_set['SMS'][0]))\n",
    "\n",
    "print(test_set['SMS'][30])\n",
    "print(classify(test_set['SMS'][30]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e46cdc3-b5fe-4c2a-9c8f-96ab60b535a5",
   "metadata": {},
   "source": [
    "## Measuring the accuracy\n",
    "- In this section, we'll use the classificator in the test set, create a new column called 'Predicted' and calculate the accuracy of our spam filter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7a6adc59-3f2a-4577-911a-9044d973306f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "      <th>Predicted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Later i guess. I needa do mcat study too.</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>But i haf enuff space got like 4 mb...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Had your mobile 10 mths? Update to latest Oran...</td>\n",
       "      <td>spam</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>All sounds good. Fingers . Makes it difficult ...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>All done, all handed in. Don't know if mega sh...</td>\n",
       "      <td>ham</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Label                                                SMS Predicted\n",
       "0   ham          Later i guess. I needa do mcat study too.       ham\n",
       "1   ham             But i haf enuff space got like 4 mb...       ham\n",
       "2  spam  Had your mobile 10 mths? Update to latest Oran...      spam\n",
       "3   ham  All sounds good. Fingers . Makes it difficult ...       ham\n",
       "4   ham  All done, all handed in. Don't know if mega sh...       ham"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# creating the new column 'Predicted' with the results of the classification\n",
    "test_set['Predicted'] = test_set['SMS'].apply(classify)\n",
    "test_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2b8ed7d7-fa3c-4798-8ac6-cc2e0863bdf8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1100\n",
      "1114\n",
      "98.74\n"
     ]
    }
   ],
   "source": [
    "# Calculating the accuracy of the spam filter\n",
    "correct = 0\n",
    "total = len(test_set)        # number of sms in the test set\n",
    "for row in test_set.iterrows():\n",
    "    if row[1]['Predicted']==row[1]['Label']:\n",
    "        correct+=1\n",
    "accuracy = correct/total*100\n",
    "\n",
    "print(correct)\n",
    "print(total)\n",
    "print(round(accuracy, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c8aeb94-f1e0-4d0f-b1f9-84e9e60a2aad",
   "metadata": {},
   "source": [
    "- In 1114 classified messages, 1100 were correct. It is a very good number, 98.74%. In the next section, we'll look at the messages that were incorrectly classified (14) to understand the errors."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d97effb2-11e4-435f-a43b-98500e7dff7d",
   "metadata": {},
   "source": [
    "## Analysing the errors\n",
    "- Let's read and analyse the 14 messages that got a wrong classification. We'll divide them into three categories: \n",
    "    - False spam: the ones classificated as 'spam', but were 'ham';\n",
    "    - False ham: the ones classificated as 'ham', but were 'spam';\n",
    "    - Needs human classification: the ones that the classificator couldn't identify."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3be77e08-f3f1-430a-8ba9-a1a7d401b9ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "# classificating the wrong results\n",
    "false_spam = test_set[(test_set['Predicted']=='spam')&(test_set['Label']=='ham')]\n",
    "false_ham = test_set[(test_set['Predicted']=='ham')&(test_set['Label']=='spam')]\n",
    "needs_human_classification = test_set[test_set['Predicted']=='needs human classification']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "7a7aaf1e-fa98-489e-a9ff-fb44e4f1f323",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Label                                                SMS Predicted\n",
      "152   ham                  Unlimited texts. Limited minutes.      spam\n",
      "159   ham                                       26th OF JULY      spam\n",
      "284   ham                             Nokia phone is lovly..      spam\n",
      "302   ham                   No calls..messages..missed calls      spam\n",
      "319   ham  We have sent JD for Customer Service cum Accou...      spam\n"
     ]
    }
   ],
   "source": [
    "# checking 'false spam'\n",
    "print(false_spam)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "203f567d-d3e6-4e20-a757-8ea21d5140c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Label                                                SMS Predicted\n",
      "114  spam  Not heard from U4 a while. Call me now am here...       ham\n",
      "135  spam  More people are dogging in your area now. Call...       ham\n",
      "504  spam  Oh my god! I've found your number again! I'm s...       ham\n",
      "546  spam  Hi babe its Chloe, how r u? I was smashed on s...       ham\n",
      "741  spam  0A$NETWORKS allow companies to bill for SMS, s...       ham\n",
      "876  spam           RCT' THNQ Adrian for U text. Rgds Vatian       ham\n",
      "885  spam                                      2/2 146tf150p       ham\n",
      "953  spam  Hello. We need some posh birds and chaps to us...       ham\n"
     ]
    }
   ],
   "source": [
    "# checking 'false ham'\n",
    "print(false_ham)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "50d77633-8dd6-4186-bc8e-112ef5bee66e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Label                                                SMS  \\\n",
      "293   ham  A Boy loved a gal. He propsd bt she didnt mind...   \n",
      "\n",
      "                      Predicted  \n",
      "293  needs human classification  \n"
     ]
    }
   ],
   "source": [
    "# checking false 'needs human classification'\n",
    "print(needs_human_classification)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fd95f95-8794-4b8e-9631-20bedeb1c241",
   "metadata": {},
   "source": [
    "- Analysing the wrong cases, we got five false spam (when the classification is spam, but actually is ham), eight false ham (the opposite case, when the predicted label gets ham, but it is spam) and one case of unclear classification (needs human classification). \n",
    "- The lenght of the messages can affect the classification. Ususally short messages are labeled as spam. And the presence of suspicious ad-style words, like \"call\", \"contact\", \"free\" or \"phone\", can affect the precision of the classificator. A third aspect that can affect the accuracy is the absence of the words in the new message in the vocabulary of the classificator. We think that was the reason of the case of unclear classification. This message is long and full of slangs and abbreviations that probably aren't in the vocabulary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3677439-54e2-493e-89f1-f2fb43357aad",
   "metadata": {},
   "source": [
    "## Using Scikit-learn library\n",
    "- In this section, we'll keep the same model, but using Scikit-learn library. This is one of the most popular machine learning libraries. Scikit-learn contains five types of the Naive Bayes model: the Gaussian, the Bernoulli, the Multinomial, the Complement and the Categorical.\n",
    "- We'll use the Multinomial, the most indicated one for document or text classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "82ffd802-a2c8-4660-98fe-b12cab7d07d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# make a copy of the set\n",
    "skl_set = sms_set.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e2630aa-58d6-4068-aa5f-e9a561708d84",
   "metadata": {},
   "source": [
    "- To run the model in the library, we need to convert the label column into numeric:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "9d24a53b-e0e2-4867-9e15-614b6cde73e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>SMS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                                SMS\n",
       "0      0  Go until jurong point, crazy.. Available only ...\n",
       "1      0                      Ok lar... Joking wif u oni...\n",
       "2      1  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3      0  U dun say so early hor... U c already then say...\n",
       "4      0  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# converting the labels to numeric\n",
    "skl_set['Label'] = np.where(skl_set['Label']=='spam',1, 0)\n",
    "skl_set.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "4af5b9b5-cd0d-46ac-bffd-d529abc45adc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing the methods, functions and classes\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.naive_bayes import MultinomialNB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0b0c252-81d8-4dcb-944b-da9a5d2e41b4",
   "metadata": {},
   "source": [
    "- Now, we're going to split the data into training and test set. By default, sklearn splits in the ratio of 70:30, but we're changing into 80:20 to make it more similar to the classificator we build above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "51808d5a-0c08-424b-a4f6-244ee94049d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# splitting the data\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(skl_set['SMS'], \n",
    "                                                    skl_set['Label'], \n",
    "                                                    random_state=0,\n",
    "                                                    test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13e41297-178c-45f7-a79c-2804620b595f",
   "metadata": {},
   "source": [
    "- Our next step is transform the messages to the vectorized form, using the CountVectorizer class. We need to do this beacuse the machine learning algorithms works doing computation, which is possible with numerical values.\n",
    "- We also will change the regular expression that filters the message, again to look similar to our previous classificator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "6a0b4ebd-fbb6-4b3a-a984-d24ac646376e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4457, 45)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# extracting features\n",
    "vectorizer = CountVectorizer(token_pattern='\\W').fit(X_train)\n",
    "X_train_vectorized = vectorizer.transform(X_train)\n",
    "X_train_vectorized.toarray().shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "817d1985-8df0-4554-99a4-1217423c1196",
   "metadata": {},
   "source": [
    "- Now, we train the instantiate the model (MultinomialNB) and fit the model to the training data. In addition, we'll set the alpha to 1, like the previous classificator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "120e0e54-f58c-4859-bb0d-e752e73e7c28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# instantiate the model and fit the trainng data\n",
    "model = MultinomialNB(alpha=1)\n",
    "model.fit(X_train_vectorized, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e45b9e6-4119-49e8-a431-9e3a6ba5ba6e",
   "metadata": {},
   "source": [
    "- The next step is to use the model to make predictions and evaluate the accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "65617694-ceb4-4ddd-b7bd-9ba9390f0a51",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 94.17040358744394 %\n"
     ]
    }
   ],
   "source": [
    "# predict and measure the accuracy\n",
    "predictions = model.predict(vectorizer.transform(X_test))\n",
    "print(\"Accuracy:\", 100 * sum(predictions == Y_test) / len(predictions), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f4e3492-2dd5-4959-8065-7ce20224cbdf",
   "metadata": {},
   "source": [
    "- Doing this way, we got 94,17% of accuracy, less than our previous implementation with 98,74%. Let's change the parameters we choose to see if we can improve our accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db822895-e32b-4bc0-b5e2-e6145c32b442",
   "metadata": {},
   "source": [
    "### Changing parameters\n",
    "- We'll use the same model before, but now with some changes:\n",
    "    - The default train/test ratio: 70:30;\n",
    "    - The default token_pattern of the CountVectorized class: r\"(?u)/b/W/W+b\";\n",
    "    - Unigram and bigram in the CountVectorized class: ngram_range=(1, 2);\n",
    "    - Alpha = 0.1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "48e83047-a90c-4b4f-a904-e0f4b23733d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 98.77961234745155 %\n"
     ]
    }
   ],
   "source": [
    "# splitting the data\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(skl_set['SMS'], \n",
    "                                                    skl_set['Label'], \n",
    "                                                    random_state=0)\n",
    "\n",
    "# extracting features\n",
    "vectorizer = CountVectorizer(ngram_range=(1, 2)).fit(X_train)\n",
    "X_train_vectorized = vectorizer.transform(X_train)\n",
    "X_train_vectorized.toarray().shape\n",
    "\n",
    "# instantiate the model and fit the trainng data\n",
    "model = MultinomialNB(alpha=0.1)\n",
    "model.fit(X_train_vectorized, Y_train)\n",
    "\n",
    "# predict and measure the accuracy\n",
    "predictions = model.predict(vectorizer.transform(X_test))\n",
    "print(\"Accuracy:\", 100 * sum(predictions == Y_test) / len(predictions), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a07c0e80-12c3-4674-aca7-c0770a6c2308",
   "metadata": {},
   "source": [
    "- We got an accuracy of 98.77%, the best among the three classifications. The first one with 98,74% and the second with 94.17%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6096ae29-6203-4d8f-8e41-14c75595b951",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd1d85fa-21ed-4138-8734-eb795251520d",
   "metadata": {},
   "source": [
    "- In this project, we created a spam filter based on the Naive Bayes algorithm (multinomial type). After test three ways of classification, we got one using the scikit-learn library that reaches 98.77% of accuracy, a high number.\n",
    "- The filter takes a message and classifies as spam or ham. \n",
    "- Some of the advantages of this filter are: it can be trained on a per-user basis and it can perform particularly well in avoiding false positives.\n",
    "- The disavantages are: some filters could be susceptible to Bayesian poisoning (when a spammer send out emails with large amounts of legitimate text), spammers can transform the words used normally in large quantities and another technique used is to replace text with pictures, either directly included or linked.\n",
    "- More information about Naive Bayes spam filter can be found [here](https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering#Discussion)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}