07_Multi-Class_Text_Classification_with_PySpark_and_Doc2Vec.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multi-Class Text Classification with PySpark and Doc2Vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we utilize Apache Spark's machine learning library (MLlib) with PySpark to tackle NLP problem and how to simulate Doc2Vec inside Spark envioronment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apache Spark is a famous distributed competiting system to to scale up any data processing solutions. Spark also provides a Machine-learning powered library called 'MLlib'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before going ahead, we need to know what is ‘Doc2Vec’. It is an NLP model to describe a text or document. It converts a text into a vector of numerical features to be used in any ML algorithm. Basically, it is a feature engineering technique. It tries to understand the context of documents by random sampling of words and trains a neural network with those. Hidden layer vectors of the neural network become document vectors a.k.a ‘Doc2Vec’. There is another technique called ‘Word2Vec’ which also works on similar principals. But instead of documents/texts, it works on word corpus and provides vectors for words. \\\n",
    "\n",
    "Reference: https://medium.com/wisio/a-gentle-introduction-to-doc2vec-db3e8c0cce5e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Set-up PySpark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#we use the findspark library to locate spark on our local machine\n",
    "import findspark\n",
    "findspark.init('C:/Users/bokhy/spark/spark-2.4.6-bin-hadoop2.7')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# PySpark local setup \n",
    "from pyspark.sql import SparkSession, SQLContext\n",
    "from pyspark import SparkContext\n",
    "MAX_MEMORY = \"5g\"\n",
    "spark = SparkSession.builder \\\n",
    "                    .appName('multi_class_text_classifiter')\\\n",
    "                    .config(\"spark.executor.memory\", MAX_MEMORY) \\\n",
    "                    .config(\"spark.driver.memory\", MAX_MEMORY) \\\n",
    "                    .getOrCreate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "            <div>\n",
       "                <p><b>SparkSession - in-memory</b></p>\n",
       "                \n",
       "        <div>\n",
       "            <p><b>SparkContext</b></p>\n",
       "\n",
       "            <p><a href=\"http://DESKTOP-L9PS4IG:4040\">Spark UI</a></p>\n",
       "\n",
       "            <dl>\n",
       "              <dt>Version</dt>\n",
       "                <dd><code>v2.4.6</code></dd>\n",
       "              <dt>Master</dt>\n",
       "                <dd><code>local[*]</code></dd>\n",
       "              <dt>AppName</dt>\n",
       "                <dd><code>multi_class_text_classifiter</code></dd>\n",
       "            </dl>\n",
       "        </div>\n",
       "        \n",
       "            </div>\n",
       "        "
      ],
      "text/plain": [
       "<pyspark.sql.session.SparkSession at 0x2539731f908>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spark"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the ‘Sentence Classification Set’ from the UCI Machine Learning Repository. This one contains a total of 3297 labeled sentences spread across different files\n",
    "Data can be downloaded from this [Link](https://archive.ics.uci.edu/ml/datasets/Sentence+Classification)\n",
    "When downloded, the zip file give 3 different folders: labeled articles, unlabeled articles, and word lists.\n",
    "We are going to only use 'labeled articles' folder in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.sql.types import StructType, StructField, StringType\n",
    "\n",
    "schema = StructType([\n",
    "    StructField(\"value\", StringType(), True)\n",
    "])\n",
    "total_df = spark.createDataFrame([], schema)\n",
    "\n",
    "for file_name in os.listdir(\"./data/labeled_articles\"):\n",
    "    df = spark.read.option(\"header\", \"true\").text('./data/labeled_articles/' + file_name)\n",
    "    total_df = total_df.union(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3297"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Total number of records\n",
    "total_df.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------------------+\n",
      "|               value|\n",
      "+--------------------+\n",
      "|    ### abstract ###|\n",
      "|MISC\tThe Minimum ...|\n",
      "|MISC\tIf the under...|\n",
      "|MISC\tFor MDL, in ...|\n",
      "|AIMX\tWe show that...|\n",
      "|OWNX\tWe derive a ...|\n",
      "|OWNX\tThis implies...|\n",
      "|OWNX\tWe discuss t...|\n",
      "|### introduction ###|\n",
      "|MISC\t``Bayes mixt...|\n",
      "|CONT\tIn many case...|\n",
      "|MISC\tThe MDL or M...|\n",
      "|MISC\tIn practice,...|\n",
      "|MISC\tHow good are...|\n",
      "|MISC\tThis questio...|\n",
      "|MISC\tIn many case...|\n",
      "|MISC\tIn particula...|\n",
      "|MISC\tAssume that ...|\n",
      "|MISC\tThen for Bay...|\n",
      "|MISC\tThis corresp...|\n",
      "+--------------------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "total_df.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset contains unuseful texts or characters like ‘### abstract ###’ & ‘### introduction ###’, or ' '' ' . \n",
    "This dataset is not yet divided into separate ‘label’ & ‘content’ column which is very common for classification problems. So, this has to be cleaned & divided into ‘label’ & ‘content’ columns for us to use for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "def process_line(x):\n",
    "    line = x['value']\n",
    "    parts = re.split(\"\\s+\",line,1)\n",
    "    sub_parts = re.split('--', parts[0])\n",
    "    parts_1 = ''\n",
    "    if len(sub_parts) > 1:\n",
    "        parts_1 = sub_parts[1] + ' ' + parts[1]\n",
    "    else:\n",
    "        parts_1 = parts[1]\n",
    "    return ([sub_parts[0],parts_1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Spark, they use speical dataframe object called 'RDD'.\n",
    "RDD was the primary user-facing API in Spark since its inception. At the core, an RDD is an immutable distributed collection of elements of your data, partitioned across nodes in your cluster that can be operated in parallel with a low-level API that offers transformations and actions. \n",
    "So we turn the data into RDD format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_rdd = total_df.rdd.filter(lambda x : x['value'] not in ['### introduction ###','### abstract ###']).map(lambda x : process_line(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We precessed the data where now '1' is the label and '2' is the actual text for our problem. Now we can use this dataset for actual problem-solving."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----+--------------------+\n",
      "|  _1|                  _2|\n",
      "+----+--------------------+\n",
      "|MISC|The Minimum Descr...|\n",
      "|MISC|If the underlying...|\n",
      "|MISC|For MDL, in gener...|\n",
      "|AIMX|We show that this...|\n",
      "|OWNX|We derive a new u...|\n",
      "|OWNX|This implies a sm...|\n",
      "|OWNX|We discuss the ap...|\n",
      "|MISC|``Bayes mixture\",...|\n",
      "|CONT|In many cases how...|\n",
      "|MISC|The MDL or MAP (m...|\n",
      "|MISC|In practice, the ...|\n",
      "|MISC|How good are the ...|\n",
      "|MISC|This question has...|\n",
      "|MISC|In many cases, an...|\n",
      "|MISC|In particular the...|\n",
      "|MISC|Assume that the o...|\n",
      "|MISC|Then for Bayes mi...|\n",
      "|MISC|This corresponds ...|\n",
      "|MISC|For the MDL predi...|\n",
      "|MISC|Note that in orde...|\n",
      "+----+--------------------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "input_df = input_rdd.toDF()\n",
    "input_df.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----+-----+\n",
      "|  _1|count|\n",
      "+----+-----+\n",
      "|BASE|   61|\n",
      "|OWNX|  867|\n",
      "|CONT|  170|\n",
      "|MISC| 1825|\n",
      "|AIMX|  194|\n",
      "+----+-----+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Check the number of each categories in our data\n",
    "input_df.groupBy('_1').count().show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Basic Text Cleaning\n",
    "Before jumping into ‘Doc2Vec’ processing, basic text cleaning is necessary. A typical text cleaning involves the following steps\n",
    "1. Conversion to lowercase \\\n",
    "2. Removal of punctuations \\\n",
    "3. Removal of integers, numbers \\\n",
    "4. Removal of extra spaces \\\n",
    "5. Removal of tags (like html, p>, etc) \\\n",
    "6. Removal of stop words (like ‘and’, ‘to’, ‘the’ etc) \\\n",
    "7. Stemming (Conversion of words to root form)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Genism is a famous library for text cleaning\n",
    "import gensim.parsing.preprocessing as gsp\n",
    "from gensim import utils\n",
    "\n",
    "from pyspark.sql.functions import udf\n",
    "from pyspark.sql.types import StringType\n",
    "\n",
    "filters = [\n",
    "           gsp.strip_tags, \n",
    "           gsp.strip_punctuation,\n",
    "           gsp.strip_multiple_whitespaces,\n",
    "           gsp.strip_numeric,\n",
    "           gsp.remove_stopwords, \n",
    "           gsp.strip_short, \n",
    "           gsp.stem_text\n",
    "          ]\n",
    "\n",
    "def clean_text(x):\n",
    "    s = x[1]\n",
    "    s = s.lower()\n",
    "    s = utils.to_unicode(s)\n",
    "    for f in filters:\n",
    "        s = f(s)\n",
    "    return (x[0],s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Let's compare the text Before/After the text-cleaning we perform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The Minimum Description Length principle for online sequence estimation/prediction in a proper learning setup is studied'"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# BEFORE\n",
    "input_rdd.take(1)[0][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'minimum descript length principl onlin sequenc estim predict proper learn setup studi'"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# AFTER\n",
    "clean_text(input_rdd.take(1)[0])[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Though the ‘cleaned’ sentence is not grammatically correct anymore, still it holds the context which is very essential for ‘Doc2Vec’ processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Clean all texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----+--------------------+\n",
      "|  _1|                  _2|\n",
      "+----+--------------------+\n",
      "|MISC|minimum descript ...|\n",
      "|MISC|underli model cla...|\n",
      "|MISC|mdl gener loss bo...|\n",
      "|AIMX|case model class ...|\n",
      "|OWNX|deriv new upper b...|\n",
      "|OWNX|impli small bound...|\n",
      "|OWNX|discuss applic ma...|\n",
      "|MISC|bay mixtur solomo...|\n",
      "|CONT|case bay mixtur c...|\n",
      "|MISC|mdl map maximum p...|\n",
      "|MISC|practic mdl estim...|\n",
      "|MISC|good predict bay ...|\n",
      "|MISC|question attract ...|\n",
      "|MISC|case import quali...|\n",
      "|MISC|particular squar ...|\n",
      "|MISC|assum outcom spac...|\n",
      "|MISC|bay mixtur predic...|\n",
      "|MISC|correspond instan...|\n",
      "|MISC|mdl predictor los...|\n",
      "|MISC|note order mdl co...|\n",
      "+----+--------------------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cleaned_rdd = input_rdd.map(lambda x : clean_text(x))\n",
    "cleaned_df = cleaned_rdd.toDF()\n",
    "cleaned_df.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Crete a Model (ML Pipeline)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As of now, Apache Spark does not provide any API for ‘Doc2Vec’. But it provides a ‘Word2Vec’ transformer. It is based on the ‘Skip-Gram’ approach.\n",
    "\n",
    "Let’s say, for our use case, one sentence has 5 words. Then, for example, a typical ‘Word2Vec’ will convert each word into a feature vector of size 100. In this case, a ‘Doc2Vec’ representation will be average of all these 100 length vectors and its length will also be 100. This is a simplified ‘average-out’ scheme of the ‘Doc2Vec’ model. We will use this average schemed ‘Word2Vec’ of Apache Spark as our ‘Doc2Vec’ model.\n",
    "\n",
    "Our Machine Learning pipeline will consist of two stages\n",
    "\n",
    "- A Tokenizer\n",
    "- A ‘Word2Vec’ model\n",
    "\n",
    "We will use Apache Spark Pipeline API for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.feature import Word2Vec\n",
    "from pyspark.ml import Pipeline\n",
    "from pyspark.ml.feature import Tokenizer\n",
    "\n",
    "tokenizer = Tokenizer(inputCol=\"_2\", outputCol=\"tokens\")\n",
    "w2v = Word2Vec(vectorSize=300,\n",
    "               minCount=0, \n",
    "               inputCol=\"tokens\",\n",
    "               outputCol=\"features\")\n",
    "\n",
    "doc2vec_pipeline = Pipeline(stages=[tokenizer,w2v])\n",
    "doc2vec_model = doc2vec_pipeline.fit(cleaned_df)\n",
    "doc2vecs_df = doc2vec_model.transform(cleaned_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----+--------------------+--------------------+--------------------+\n",
      "|  _1|                  _2|              tokens|            features|\n",
      "+----+--------------------+--------------------+--------------------+\n",
      "|MISC|minimum descript ...|[minimum, descrip...|[-0.0113720915590...|\n",
      "|MISC|underli model cla...|[underli, model, ...|[-0.0156475625963...|\n",
      "|MISC|mdl gener loss bo...|[mdl, gener, loss...|[-0.0095312943061...|\n",
      "|AIMX|case model class ...|[case, model, cla...|[-0.0196333309092...|\n",
      "|OWNX|deriv new upper b...|[deriv, new, uppe...|[-0.0164283528251...|\n",
      "|OWNX|impli small bound...|[impli, small, bo...|[-0.0218414339236...|\n",
      "|OWNX|discuss applic ma...|[discuss, applic,...|[-0.0189581960350...|\n",
      "|MISC|bay mixtur solomo...|[bay, mixtur, sol...|[-0.0167323263778...|\n",
      "|CONT|case bay mixtur c...|[case, bay, mixtu...|[-0.0094780717699...|\n",
      "|MISC|mdl map maximum p...|[mdl, map, maximu...|[-0.0191538601990...|\n",
      "|MISC|practic mdl estim...|[practic, mdl, es...|[-0.0215153907483...|\n",
      "|MISC|good predict bay ...|[good, predict, b...|[-0.0180142346769...|\n",
      "|MISC|question attract ...|[question, attrac...|[-0.0177000655482...|\n",
      "|MISC|case import quali...|[case, import, qu...|[-0.0189265827163...|\n",
      "|MISC|particular squar ...|[particular, squa...|[-0.0144605820532...|\n",
      "|MISC|assum outcom spac...|[assum, outcom, s...|[-0.0378455093596...|\n",
      "|MISC|bay mixtur predic...|[bay, mixtur, pre...|[-0.0166334967980...|\n",
      "|MISC|correspond instan...|[correspond, inst...|[2.55577196367084...|\n",
      "|MISC|mdl predictor los...|[mdl, predictor, ...|[-0.0105661017402...|\n",
      "|MISC|note order mdl co...|[note, order, mdl...|[-0.0191857390878...|\n",
      "+----+--------------------+--------------------+--------------------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Doc2Vec contents\n",
    "doc2vecs_df.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 'features' column is the actual ‘Doc2Vec’ dense vectors. We have used ‘Doc2Vec’ of size 300. Generally, the preferred size is kept between 100 and 300."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Train a Model and Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Dataset Count: 2332\n",
      "Test Dataset Count: 785\n"
     ]
    }
   ],
   "source": [
    "# Train/Test Split\n",
    "w2v_train_df, w2v_test_df = doc2vecs_df.randomSplit([0.75, 0.25], seed = 623)\n",
    "\n",
    "print(\"Training Dataset Count: \" + str(w2v_train_df.count()))\n",
    "print(\"Test Dataset Count: \" + str(w2v_test_df.count()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Our model will make predictions and score on the test set, and then we then look at the top accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1. Random-Forest Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Spark MLlib does not understand typical categorical variables. For that our class labels (column '1') have to be converted into numeric values. 'StringIndexer' function from PyPspark does that for us.\n",
    "\n",
    "Here also, we have to build a pipeline with the following stages:\n",
    "- StringIndexer (input = '1', output = 'label')\n",
    "- RandomForest Classifier (label column = 'label', features column = 'features'. This 'features' is coming from 'Doc2Vec' transformation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.feature import StringIndexer\n",
    "from pyspark.ml.classification import RandomForestClassifier\n",
    "from pyspark.ml.evaluation import MulticlassClassificationEvaluator\n",
    "\n",
    "si = StringIndexer(inputCol=\"_1\", outputCol=\"label\")\n",
    "rf_classifier = RandomForestClassifier(labelCol=\"label\", \n",
    "                                       featuresCol=\"features\")\n",
    "\n",
    "# Build Pipeline\n",
    "rf_classifier_pipeline = Pipeline(stages=[si,rf_classifier])\n",
    "\n",
    "# Start Training\n",
    "rfModel = rf_classifier_pipeline.fit(w2v_train_df)\n",
    "\n",
    "# Prediction on Test-set\n",
    "rf_predictions = rfModel.transform(w2v_test_df)\n",
    "\n",
    "# Evalutation the model\n",
    "rf_model_evaluator = MulticlassClassificationEvaluator(\n",
    "    labelCol=\"label\", predictionCol=\"prediction\", metricName=\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 0.626752\n"
     ]
    }
   ],
   "source": [
    "accuracy = rf_model_evaluator.evaluate(rf_predictions)\n",
    "print(\"Accuracy = %g\" % (accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2. Logistic-Regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark.ml.classification import LogisticRegression\n",
    "\n",
    "lr_classifier = LogisticRegression(family=\"multinomial\", \n",
    "                                   maxIter=20, \n",
    "                                   regParam=0.3, \n",
    "                                   elasticNetParam=0)\n",
    "\n",
    "# Build Pipeline\n",
    "lr_classifier_pipeline = Pipeline(stages=[si,lr_classifier])\n",
    "\n",
    "# Start Training\n",
    "lrModel = lr_classifier_pipeline.fit(w2v_train_df)\n",
    "\n",
    "# Prediction on Test-set\n",
    "lr_predictions = lrModel.transform(w2v_test_df)\n",
    "\n",
    "# Evalutation the model\n",
    "lr_model_evaluator = MulticlassClassificationEvaluator(\n",
    "    labelCol=\"label\", predictionCol=\"prediction\", metricName=\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test-set Accuracy = 0.568153\n"
     ]
    }
   ],
   "source": [
    "accuracy = lr_model_evaluator.evaluate(lr_predictions)\n",
    "print(\"Test-set Accuracy = %g\" % (accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We can fine-tune by changing parameters in LogisticRegression() model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}