07. Advanced SVMs - R.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 7 - Advanced Support Vector Machines\n===\n\nSupport vector machines (SVMs) let us predict categories. In this exercise, we will be using SVM, paying attention to the key steps as we go:  formatting data correctly, splitting the data into training and test sets, training an SVM model using the training set, and then evaluating and visualising the SVM model using the test set.\n\nWe will be looking at __prions__: misfolded proteins that are associated with several fatal neurodegenerative diseases (kind of like Daleks, if you have seen Doctor Who). Looking at examples of protein mass and weight, we will build a predictive model to detect prions in blood samples.\n\n#### Run the code below to load the required libraries for this exercise."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Load `tidyverse` package\nsuppressMessages(install.packages(\"tidyverse\"))\nsuppressMessages(library(\"tidyverse\"))\nsuppressMessages(install.packages(\"e1071\"))\nsuppressMessages(library(\"e1071\"))\nsuppressMessages(install.packages(\"magrittr\"))\nsuppressMessages(library(\"magrittr\"))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n---\n\nLet's load the required R packages and the prion data for this exercise.\n\n**In the code below, complete the data loading step, by replacing `<prionDataset>` with `prion_data`, and running the code.**"
    },
    {
      "metadata": {
        "scrolled": true,
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <prionDataset> WITH prion_data\n###\n<prionDataset> <- read.csv(\"Data/PrionData.csv\")\n###\n\n# Check the structure of `prion_data`\nstr(prion_data)\nhead(prion_data)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "It appears that we have an extra column `X` in `prion_data` that contains the row number. By default, R has labelled the column `X` because the input didn't have a column name (it was blank). This behaviour happens regularly when exporting data sets from a program like Microsoft Excel and then importing them into R.\n\nLet's get rid of the first column from `prion_data`, and then check that it has been successfully removed. We will use the `select` function from the `dplyr` package together with the `-` symbol to \"minus\" the `X` column from our dataset.\n\n> **N.B. We have used a different assignment symbol `%<>%` from the `magrittr` package in the code below. The `magrittr` assignment symbol `%<>%` is a combination of the `magrittr` pipe symbol `%>%` and the base R assignment symbol `<-`. It takes the variable on the left hand side of the `%<>%` symbol, and updates the value of the variable with the result of the right hand side. So the object on the left hand side acts as both the initial value and the resulting value.**\n\n#### Replace `<removeColumn>` with `-X` to remove the excess column X, then run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <removeColumn> WITH -X\n###\nprion_data %<>% select(<removeColumn>)\n###\nstr(prion_data)\nhead(prion_data)\n\n# Check frequency of `prion_status` in `prion_data`\nprion_data %>%\ngroup_by(prion_status) %>%\nsummarise(n = n()) %>% \nmutate(freq = n/sum(n))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Excellent, we have successfully removed column `X` from `prion_data`!\n\nNow, looking at the output of `str` and `head`, we can observe that `prion_data` is a `data.frame` that contains 485 observations and 3 variables stored in the following columns:\n\n* `mass` is the first *feature*;\n* `weight` is the second *feature*;\n* `prion_status` is the *label* (or category).\n\nOf the 485 observations, 375 (77.31%) are non-prions, and 110 (22.68%) are prions.\n\nStep 2\n---\n\nLet's graph `prion_data` to better understand the features and labels.\n\n**In the cell below replace:**\n\n**1. `<xData>` with `mass`**\n\n**2. `<yData>` with `weight`**\n\n**3. `<colorData>` with `prion_status`**\n\n** then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "prion_data  %>% \n###\n# REPLACE <xData> WITH mass AND <yData> WITH weight AND <colorData> WITH prion_status\n###\nggplot(aes(x = <xData> , y = <yData> , colour = <colorData> )) +\n###\ngeom_point() +\nggtitle(\"Classification plot for prion data\") +\n# Create labels for x-axis, y-axis, and legend\nlabs(x = \"Mass\", y = \"Weight\", colour = \"Prion status\") +\n# Align title to centre\ntheme(plot.title = element_text(hjust = 0.5))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 3\n---\n\nTo create a SVM model, let's split our data into training and test sets. We'll start by checking the total number of instances in our data set. If we go back to the output from `str(prion_data)` in Step 2, we have 485 observations and 3 variables.\n\nSo, let's use 400 examples for our `training` set, and the remainder for our `test` set.\n\nWe will use the `slice` function to select the first 400 rows from `prion_data`\n\n#### Replace `<selectData>` with `1:400`, and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <selectData> WITH 1:400\n###\ntrain_prion <- slice(prion_data, <selectData>)\n###\nstr(train_prion)\n\n# Check percentage of samples that are prions\ntrain_prion %>%\ngroup_by(prion_status) %>%\nsummarise(n = n()) %>% \nmutate(freq = n/sum(n))\n\n# Create test set using the remaining examples\ntest_prion <- slice(prion_data, 401:n())\nstr(test_prion)\n\n# Check percentage of samples that are prions\ntest_prion %>%\ngroup_by(prion_status) %>%\nsummarise(n = n()) %>% \nmutate(freq = n/sum(n))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Well done! Let's look at a summary of our training data to get a better idea of what we're dealing with.\n\n#### Replace `<trainDataset>` with `train_prion` and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <trainDataset> WITH train_prion\n###\nsummary(<trainDataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Using the `summary` function, we observe our training data contains 314 non-prions and 86 prions out of a total of 400 observations. This looks right, because the scatter plot we created in Step 2 showed us the majority of observations have 'non-prion' status.\n\nLet's take a look at `test_prion` too, using the `summary` function again.\n\n#### Replace `<testDataset>` with `test_prion` and run the code."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <testDataset> WITH test_prion\n###\nsummary(<testDataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Looking good! Alright, now to make a support vector machine.\n\nStep 4\n---\n\nBelow we will make an SVM similar to the previous exercise. Remember the syntax for SVMs using the `e1071::svm` function:\n\n**`svm_model <- svm(x = x, y = y, data = dataset)`**\n\nwhere `x` represents the features (a matrix), and `y` represents the labels (factors).\n\nAlternatively, we can use the following syntax for the `svm` function:\n\n**`model <- svm(formula = y ~ x, data = dataset)`**\n\nwhere `y` represents the labels/categories, and `x` represents the features. Note if you have multiple `x` features in the dataset, you can simply type `.` in the `formula` argument to refer to everything in the data set except the y argument. Let's try out this syntax using the training data as our input.\n\n**In the code below, create an SVM model using the `train_prion` data using the `svm` function with the `formula` argument.**\n\n#### Replace `<dataSelection>` with `prion_status ~ .`, then run the code.\n\nNote: the `prion_status` on the left hand side of the formula selects our labels, and the `.` on the right hand side of the formula selects our features. In this case, the `.` selects all the features in our dataset `train_prion`."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <dataSelection> WITH prion_status ~ .\n###\nSVM_Model <- svm(formula = <dataSelection> , data = train_prion)\n###\n\nprint(\"Model ready!\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Well done! We've made a SVM model using our training set `train_prion`.\n\nStep 5\n---\n\nLet's create some custom functions to graph and evaluate SVM models. We will use these functions throughout the remainder of this exercise. You do not need to edit the code block below.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to prepare functions for later use\n\n# Create a custom function named `Graph_SVM` to plot an SVM model\n\nGraph_SVM <- function(model, data_set){\n    grid <- expand.grid(\"mass\" = seq(min(data_set$mass), max(data_set$mass), length.out = 100),\n                        \"weight\" = seq(min(data_set$weight), max(data_set$weight), length.out = 100))\n    preds <- predict(model, grid)\n    df <- data.frame(grid, preds)\n    ggplot() +\n    geom_tile(data = df, aes(x = mass, y = weight, fill = preds)) +\n    geom_point(data = data_set, aes(x = mass, y = weight, shape = prion_status, \n                                    colour = prion_status), \n               alpha = 0.75) +\n    scale_colour_manual(values = c(\"grey10\", \"grey50\")) +\n    labs(title = paste(\"SVM model prediction\"), x = \"Mass\", y = \"Weight\",\n         fill = \"Prediction\", shape = \"Prion status\", colour = \"Prion status\") +\n    theme(plot.title = element_text(hjust = 0.5))\n    }\n\n# Create another custom function named `Evaluate_SVM` to evaluate the SVM model, print results to screen,\n# and run the `Graph_SVM` custom function\nEvaluate_SVM <- function(model, data_set){\n    predictions <- predict(model, data_set)\n    total <- 0\n    for(i in 1:nrow(data_set)){\n    if(toString(predictions[i]) == data_set[i, \"prion_status\"]){\n        total = total + 1\n        }\n        }\n    # Print results to screen\n    print(\"SVM Model Evaluation\")\n    print(paste0(\"Model name: \", deparse(substitute(model))))\n    print(paste0(\"Dataset: \", deparse(substitute(data_set))))\n    print(paste0(\"Accuracy: \", total/nrow(data_set)*100, \"%\"))\n    print(paste0(\"Number of samples: \", nrow(data_set)))\n    \n    # Call our custom function for graphing SVM model\n    Graph_SVM(model, data_set)\n}\n\nprint(\"Custom function ready!\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Excellent! Now that we have created the custom function `Evaluate_SVM` (which incorporates the `Graph_SVM` function) let's evaluate our SVM model on the training data. \n\nIn the code below, we will change the inputs to the `Evaluate_SVM` function, where the first argument is the SVM model we will evaluate, and the second argument is the dataset we will evaluate the SVM model with.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model` **\n\n** 2. `<dataset>` with `train_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <svmModel> WITH SVM_Model AND <dataset> WITH train_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 6\n---\n\nThe SVM has performed reasonably well separating our training data set into two. Now let's take a look at our test set.\n\nIn the code below, we will use our custom function `Evaluate_SVM` to evaluate `SVM_Model` on the test set.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model` **\n\n** 2. `<dataset>` with `test_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <svmModel> WITH SVM_Model AND <dataset> WITH test_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "That's a good result. \n\nConclusion\n---\n\nWell done! We've taken a data set, tidied it, prepared it into training and test sets, created an SVM based on the training set, and evaluated the SVM model using the test set.\n\nYou can go back to the course now, or you can try using different kernels with your SVM below.\n\nOPTIONAL: Step 8\n---\n\nWant to have a play around with different kernels for your SVM models? It's really easy!\n\nThe standard kernel is a radial basis kernel. But there's a few more you can choose from: `linear`, `polynomial`, and `sigmoid`. Let's try them out.\n\nIf you want to use a linear kernel, all you need to do is add `kernel = \"linear\"` to your model. Like this:\n\n`SVM_Model <- svm(formula = y ~ x, data = dataset, kernel = \"linear\")`\n\nGive it a go with all the different kernels below. The first kernel, `linear`, has been done for you.\n\n**Run the code below**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to make a linear SVM\n\n# Make a linear SVM model\nSVM_Model_Linear <- svm(prion_status ~ . , data = train_prion, kernel = \"linear\")\nprint(\"Model ready\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now we have created the linear SVM model, let's evaluate it on our training and test sets using our custom function we created earlier, `Evaluate_SVM`. Remember the inputs to `Evaluate_SVM` are the SVM model followed by the data you wish to evaluate the model on.\n\nIn the code blocks below, we will change the inputs to our `Evaluate_SVM` function to the appropriate variable names to evaluate the linear SVM model on the training and test sets.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model_Linear` **\n\n** 2. `<dataset>` with `train_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Evaluate linear SVM model on training set\n\n###\n# REPLACE <svmModel> WITH SVM_Model_Linear AND <dataset> WITH train_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "And now for the test set.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model_Linear` **\n\n** 2. `<dataset>` with `test_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Evaluate linear SVM model on test set\n\n###\n# REPLACE <svmModel> WITH SVM_Model_Linear AND <dataset> WITH test_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "You can see the hyperplane is a linear line! Compare the linear SVM model results to the radial SVM model results to see the difference for yourself!\n\n## Now let's try a sigmoid kernel.\n\n** In the cell below replace: **\n\n** 1. `<kernelSelection>` with `\"sigmoid\"` **\n\n** 2. `<svmModel>` with `SVM_Model_Sigmoid` **\n\n** 3. `<dataset>` with `train_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <kernelSelection> WITH \"sigmoid\" (INCLUDING THE QUOTATION MARKS)\n###\nSVM_Model_Sigmoid <- svm(prion_status ~ . , data = train_prion, kernel = <kernelSelection>)\n###\n\n# Evaluate sigmoid SVM model on training set\n###\n# REPLACE <svmModel> WITH SVM_Model_Sigmoid AND <dataset> WITH train_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "And now for the test set.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model_Sigmoid` **\n\n** 2. `<dataset>` with `test_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Evaluate sigmoid SVM model on test set\n###\n# REPLACE <svmModel> WITH SVM_Model_Sigmoid AND <dataset> WITH test_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Perhaps a sigmoid kernel isn't a good idea for this data set....\n\n## Let's try a sigmoid kernel instead.\n\n** In the cell below replace: **\n\n** 1. `<kernelSelection>` with `\"polynomial\"` **\n\n** 2. `<svmModel>` with `SVM_Model_Sigmoid` **\n\n** 3. `<dataset>` with `train_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <kernelSelection> WITH \"polynomial\" (INCLUDING THE QUOTATION MARKS)\n###\nSVM_Model_Poly <- svm(prion_status ~ . , data = train_prion, kernel = <kernelSelection>)\n\n# Evaluate polynomial SVM model on training set\n###\n# REPLACE <svmModel> WITH SVM_Model_Poly AND <dataset> WITH train_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "And now for the test set.\n\n** In the cell below replace: **\n\n** 1. `<svmModel>` with `SVM_Model_Poly` **\n\n** 2. `<dataset>` with `test_prion` **\n\n** Then __run the code__. **"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Evaluate polynomial SVM model on test set\n###\n# REPLACE <svmModel> WITH SVM_Model_Poly AND <dataset> WITH test_prion\n###\nEvaluate_SVM(<svmModel>, <dataset>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "If we were to carry on analysing prions like this, a polynomial SVM looks like a good choice (based on the performance of the different models on `test_prion`). If the data set was more complicated, we could try different degrees for the polynomial to see which one was the most accurate. This is part of __`tuning`__ a model.\n\nWell done!"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "r",
      "display_name": "R",
      "language": "R"
    },
    "language_info": {
      "mimetype": "text/x-r-source",
      "name": "R",
      "pygments_lexer": "r",
      "version": "3.4.1",
      "file_extension": ".r",
      "codemirror_mode": "r"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}