(200361138) Rohitkumar Keswani Q123.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv                               # csv reader\n",
    "from sklearn.svm import LinearSVC\n",
    "from nltk.classify import SklearnClassifier\n",
    "from random import shuffle\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load data from a file and append it to the rawData\n",
    "def loadData(path, Text=None):\n",
    "    with open(path, encoding='utf8') as f:\n",
    "        reader = csv.reader(f, delimiter='\\t')\n",
    "        for line in reader:\n",
    "            if line[0] == \"DOC_ID\":  # skip the header\n",
    "                continue\n",
    "            (Id, Text, Label) = parseReview(line)\n",
    "            rawData.append((Id, Text, Label))\n",
    "\n",
    "\n",
    "def splitData(percentage):\n",
    "    # A method to split the data between trainData and testData \n",
    "    dataSamples = len(rawData)\n",
    "    halfOfData = int(len(rawData)/2)\n",
    "    trainingSamples = int((percentage*dataSamples)/2)\n",
    "    for (_, Text, Label) in rawData[:trainingSamples] + rawData[halfOfData:halfOfData+trainingSamples]:\n",
    "        trainData.append((toFeatureVector(preProcess(Text)),Label))\n",
    "    for (_, Text, Label) in rawData[trainingSamples:halfOfData] + rawData[halfOfData+trainingSamples:]:\n",
    "        testData.append((toFeatureVector(preProcess(Text)),Label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert line from input file into an id/text/label tuple\n",
    "def parseReview(reviewLine):\n",
    "    # Should return a triple of an integer, a string containing the review, and a string indicating the label\n",
    "    # DESCRIBE YOUR METHOD IN WORDS\n",
    "    return (reviewLine[0], reviewLine[8], reviewLine[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the function parseReview with revieLine as an argument it returns a triple consisting of an integer, a string which is the review and a string which is a label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TEXT PREPROCESSING AND FEATURE VECTORIZATION\n",
    "import re, nltk\n",
    "# Input: a string of one review\n",
    "def preProcess(text):\n",
    "    # Should return a list of tokens\n",
    "    # DESCRIBE YOUR METHOD IN WORDS\n",
    "    tokens=text.split(' ')\n",
    "    return tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preprocessing of text is done in the above preProcess function with text as an argument and splitting the data over a white space with the help of split method and returning it as tokens."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "featureDictglobal = {} # A global dictionary of features\n",
    "\n",
    "def toFeatureVector(tokens):\n",
    "    # Should return a dictionary containing features as keys, and weights as values\n",
    "    # DESCRIBE YOUR METHOD IN WORDS\n",
    "    localfeatureDict={}\n",
    "    for t in tokens:\n",
    "        try:\n",
    "            featureDictglobal[t] = featureDictglobal[t]+1\n",
    "            localfeatureDict[t] = localfeatureDict[t]+1\n",
    "        except KeyError:\n",
    "            featureDictglobal[t] = 1\n",
    "            localfeatureDict[t] = 1\n",
    "    return localfeatureDict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A global dictionary of features is created as featureDict and featureDictLocal is created for local dictionary.\n",
    "toFeatureVector function with tokens as argument is used to create a dictionary with features as keys and weights as value in it. Where weight will be the occurence of the word in the text. if its the first occurence it will assigned as one and will increase one as the frequency increases in the text in both the local and global dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TRAINING AND VALIDATING OUR CLASSIFIER\n",
    "def trainClassifier(trainData):\n",
    "    print(\"Training Classifier...\")\n",
    "    pipeline =  Pipeline([('svc', LinearSVC())])\n",
    "    return SklearnClassifier(pipeline).train(trainData)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above function training and validating of our classifier is done by making a linearSVC pipeline and the function returning SklearnClassifier on the training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "from sklearn.metrics import precision_recall_fscore_support\n",
    "from sklearn.metrics import accuracy_score\n",
    "crossValidateActual=[]\n",
    "def crossValidate(dataset, folds):\n",
    "    shuffle(dataset)\n",
    "    cv_results = []\n",
    "    foldSize = int(len(dataset)/folds)\n",
    "    # DESCRIBE YOUR METHOD IN WORDS\n",
    "    for i in range(0,len(dataset),foldSize):\n",
    "        TestData = dataset[i:i+foldSize]\n",
    "        TrainData = dataset[:i] + dataset[i+foldSize:]\n",
    "        classifier = trainClassifier(TrainData)\n",
    "        Actual = [x[1] for x in TestData]\n",
    "        PredictedLabels = predictLabels(TestData, classifier)\n",
    "        cv_results = (precision_recall_fscore_support(Actual, PredictedLabels, average='weighted'))\n",
    "    print(\"Accuracy: %f\" % accuracy_score(Actual,PredictedLabels))\n",
    "    return cv_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above function crossValidate there are two arguments one is the input dataset and second is the fold. In this function we have taken 10 folds. Then the shuffle function is used to reorganize the data and then the foldsize is calculated by dividing the length of the dataset by the number of folds. Then a for loop is used from 0 to the length of the dataset and step argument as a third argument which is the foldsize. This loop is used for TestData and TrainData and trainClassifier function is used on the training data. The Actual are the actual labels  while the PredictLabels predicts the labels from the data and finally the crossvalidation results are returned by the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# PREDICTING LABELS GIVEN A CLASSIFIER\n",
    "\n",
    "def predictLabels(reviewSamples, classifier):\n",
    "    return classifier.classify_many(map(lambda t: t[0], reviewSamples))\n",
    "\n",
    "def predictLabel(reviewSample, classifier):\n",
    "    return classifier.classify(toFeatureVector(preProcess(reviewSample)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above funtions does the prediction of the labels given a classifier where in the first function it takes the review sample whereas in the second fucntion it takes the preprocessed review sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Now 0 rawData, 0 trainData, 0 testData\n",
      "Preparing the dataset...\n",
      "Now 21000 rawData, 0 trainData, 0 testData\n",
      "Preparing training and test data...\n",
      "After split, 21000 rawData, 16800 trainData, 4200 testData\n",
      "Training Samples: \n",
      "16800\n",
      "Features: \n",
      "89043\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\rohit\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\sklearn\\svm\\_base.py:977: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Training Classifier...\n",
      "Accuracy: 0.648214\n",
      "Precision: 0.650270\n",
      "Recall: 0.648214\n",
      "F Score:0.648442\n"
     ]
    }
   ],
   "source": [
    "# MAIN\n",
    "\n",
    "# loading reviews\n",
    "# initialize global lists that will be appended to by the methods below\n",
    "rawData = []          # the filtered data from the dataset file (should be 21000 samples)\n",
    "trainData = []        # the pre-processed training data as a percentage of the total dataset (currently 80%, or 16800 samples)\n",
    "testData = []         # the pre-processed test data as a percentage of the total dataset (currently 20%, or 4200 samples)\n",
    "\n",
    "# the output classes\n",
    "fakeLabel = 'fake'\n",
    "realLabel = 'real'\n",
    "\n",
    "# references to the data files\n",
    "reviewPath = 'amazon_reviews.txt'\n",
    "\n",
    "# Do the actual stuff (i.e. call the functions we've made)\n",
    "# We parse the dataset and put it in a raw data list\n",
    "print(\"Now %d rawData, %d trainData, %d testData\" % (len(rawData), len(trainData), len(testData)),\n",
    "      \"Preparing the dataset...\",sep='\\n')\n",
    "loadData(reviewPath) \n",
    "\n",
    "# We split the raw dataset into a set of training data and a set of test data (80/20)\n",
    "# You do the cross validation on the 80% (training data)\n",
    "# We print the number of training samples and the number of features before the split\n",
    "print(\"Now %d rawData, %d trainData, %d testData\" % (len(rawData), len(trainData), len(testData)),\n",
    "      \"Preparing training and test data...\",sep='\\n')\n",
    "splitData(0.8)\n",
    "# We print the number of training samples and the number of features after the split\n",
    "print(\"After split, %d rawData, %d trainData, %d testData\" % (len(rawData), len(trainData), len(testData)),\n",
    "      \"Training Samples: \", len(trainData), \"Features: \", len(featureDictglobal), sep='\\n')\n",
    "\n",
    "# QUESTION 3 - Make sure there is a function call here to the\n",
    "# crossValidate function on the training set to get your results\n",
    "validationResults=crossValidate(trainData,10)\n",
    "print(\"Precision: %f\\nRecall: %f\\nF Score:%f\" % validationResults[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In tha above Main Function the ouput classes are created fakelabel and reallabel. Global lists are created which will be appended by the method which are rawdata trandata and testdata. Now the dataset is parsed into the raw data list. Then the raw Dataset is split into training data and testdata into 80 and 20 percentage respectively. And the data is printed as the number of data in raw data in training data and testing data before and after spliting. And the crossValidate function is called on the training data and the results are printed which are Precision, recall and F Score."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluate on test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "({'This': 1, 'assortment': 1, 'is': 1, 'really': 1, \"Hershey's\": 1, 'at': 1, 'their': 1, 'best.': 1, 'The': 1, 'little': 1, 'ones': 1, 'are': 1, 'always': 1, 'excited': 1, 'whenever': 1, 'the': 1, 'holidays': 1, 'come': 1, 'because': 1, 'of': 1, 'this.': 1}, '__label1__')\n",
      "Training Classifier...\n",
      "Done training!\n",
      "Precision: 0.615498\n",
      "Recall: 0.615476\n",
      "F Score:0.615458\n",
      "Accuracy: 0.615476\n"
     ]
    }
   ],
   "source": [
    "# Finally, check the accuracy of your classifier by training on all the tranin data\n",
    "# and testing on the test set\n",
    "# Will only work once all functions are complete\n",
    "functions_complete = True  # set to True once you're happy with your methods for cross val\n",
    "if functions_complete:\n",
    "    print(testData[0])   # have a look at the first test data instance\n",
    "    classifier = trainClassifier(trainData)  # train the classifier\n",
    "    TTrue = [t[1] for t in testData]   # get the ground-truth labels from the data\n",
    "    Pred = predictLabels(testData, classifier)  # classify the test data to get predicted labels\n",
    "    finalScores = precision_recall_fscore_support(TTrue, Pred, average='weighted') # evaluate\n",
    "    print(\"Done training!\")\n",
    "    print(\"Precision: %f\\nRecall: %f\\nF Score:%f\" % finalScores[:3])\n",
    "    print(\"Accuracy: %f\" % accuracy_score(TTrue, Pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the accuracy of the classifier is checked by training on all the training data and then testing on the test set.\n",
    "This will work when all the functions are complete and if the functions are complete the first instance of the data is printed and the classifier is trained on the training data, And next is the getting of the ground-truth labels form the data and lastly labels are predicted on the test data and results are printed which are Precision, Recall and F Score."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Questions 4 and 5\n",
    "Once you're happy with your functions for Questions 1 to 3, it's advisable you make a copy of this notebook to make a new notebook, and then within it adapt and improve all three functions in the ways asked for in questions 4 and 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}