Generate_Bag_of_Words.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generating Bag Of Words(Step by Step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know **Neural Network** or **Machine Learning Algorithm** need features to train. \n",
    "**Bag of Words(BOW)** is a method to extract features from text. \n",
    "\n",
    "**STEPS:**\n",
    "\n",
    "**Document**>>**Clean Text**>>**Tokenize**>>**Build Vocabulary**>>**Generate Vectors**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Joe waited for the train, The train was late, Mary and Samantha took the bus, I looked for Mary and Samantha at the bus station, Mary and Samantha arrived at the bus station early but waited until noon for the bus,\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "document = \"Joe waited for the train. The train was late. Mary and Samantha took the bus. I looked for Mary and Samantha at the bus station. Mary and Samantha arrived at the bus station early but waited until noon for the bus.\"\n",
    "\n",
    "# removing punctuation and separating sentence with comma(,)\n",
    "remove_punc_text = re.sub(r'[^\\w\\s]',',',document)\n",
    "print(remove_punc_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Joe waited for the train', 'The train was late', 'Mary and Samantha took the bus', 'I looked for Mary and Samantha at the bus station', 'Mary and Samantha arrived at the bus station early but waited until noon for the bus,']\n"
     ]
    }
   ],
   "source": [
    "clean_text = remove_punc_text.split(\", \")\n",
    "print(clean_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tokenize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "word_extraction function will split sentence into words and ignore stopwords.\n",
    "Here we give a small stopwords set('a', 'the', 'is')\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "def word_extraction(sentence):\n",
    "    ignore = [\"a\", \"the\", \"is\"]\n",
    "    words = re.sub(\"[^\\w]\", \" \",  sentence).split()\n",
    "    cleaned_text = [w.lower() for w in words if w not in ignore]\n",
    "    return cleaned_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['and', 'arrived', 'at', 'bus', 'but', 'early', 'for', 'i', 'joe', 'late', 'looked', 'mary', 'noon', 'samantha', 'station', 'the', 'took', 'train', 'until', 'waited', 'was']\n"
     ]
    }
   ],
   "source": [
    "# applying tokenization to our all sentences\n",
    "def tokenize(sentences):\n",
    "    words = []\n",
    "    for sentence in sentences:\n",
    "        w = word_extraction(sentence)\n",
    "        words.extend(w)\n",
    "        \n",
    "    words = sorted(list(set(words)))\n",
    "    return words\n",
    "\n",
    "tokens = tokenize(clean_text)\n",
    "print(tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build Vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word List for Document \n",
      "['and', 'arrived', 'at', 'bus', 'but', 'early', 'for', 'i', 'joe', 'late', 'looked', 'mary', 'noon', 'samantha', 'station', 'the', 'took', 'train', 'until', 'waited', 'was'] \n",
      "\n",
      "Length of our vocabulary is: 21\n"
     ]
    }
   ],
   "source": [
    "vocab = tokens\n",
    "print(\"Word List for Document \\n{0} \\n\".format(vocab));\n",
    "print(\"Length of our vocabulary is:\", len(vocab))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating Vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Joe waited for the train\n",
      "[0. 0. 0. 0. 0. 0. 1. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 1. 0.]\n",
      "\n",
      "The train was late\n",
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 1. 0. 1. 0. 0. 1.]\n",
      "\n",
      "Mary and Samantha took the bus\n",
      "[1. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 1. 0. 1. 0. 0. 1. 0. 0. 0. 0.]\n",
      "\n",
      "I looked for Mary and Samantha at the bus station\n",
      "[1. 0. 1. 1. 0. 0. 1. 1. 0. 0. 1. 1. 0. 1. 1. 0. 0. 0. 0. 0. 0.]\n",
      "\n",
      "Mary and Samantha arrived at the bus station early but waited until noon for the bus,\n",
      "[1. 1. 1. 2. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 1. 1. 0.]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "import numpy as np\n",
    "\n",
    "for sentence in clean_text:\n",
    "        words = word_extraction(sentence)\n",
    "        bag_vector = np.zeros(len(vocab))\n",
    "        for w in words:\n",
    "            for i,word in enumerate(vocab):\n",
    "                if word == w: \n",
    "                    bag_vector[i] += 1\n",
    "                    \n",
    "        print(\"{0}\\n{1}\\n\".format(sentence,np.array(bag_vector)))\n",
    "        \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explanation\n",
    "\n",
    "How it works:\n",
    "\n",
    "*step 1:* Initiaze a zero vector array with len(vocab)\n",
    "\n",
    "our here: `[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]`\n",
    "\n",
    "*step 2:* enumerate the vocabulary\n",
    "\n",
    "Our here: \n",
    "\n",
    "`0 : and, 1 : arrived, 2 : at, 3 : bus, 4 : but, 5 : early, 6 : for,\n",
    "7 : i, 8 : joe, 9 : late, 10 : looked, 11 : mary, 12 : noon, 13 : samantha, \n",
    "14 : station, 15 : the, 16 : took, 17 : train, 18 : until, 19 : waited, 20 : was`\n",
    "\n",
    "*step 3:* Compare each word with this enumeration list and increament vector element value\n",
    "\n",
    "such as: \n",
    "\n",
    "for sentence:\n",
    "\n",
    "`Joe waited for the train`\n",
    "\n",
    "`[0. 0. 0. 0. 0. 0. 1. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 1. 0.]`\n",
    "\n",
    "\n",
    "word `joe` position is 8. so we increament vector value to 1 at position 8.\n",
    "same for other words.\n",
    "\n",
    "**Now use these feature vectors for you Machine Learning Model.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}