GeneralExemplars/StatsWithR/Data_Cleaning__Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## This Notebook - Goals - FOR EDINA\n",
    "\n",
    "**What?:**\n",
    "- R notebook demonstrating a typical data cleaning process\n",
    "- Use of health data\n",
    "\n",
    "**Who?:**\n",
    "- Data Science and statistics students and academics \n",
    "- Anyone dealing with data\n",
    "\n",
    "**Why?:**\n",
    "- Demonstrate a notebook in a non-python language\n",
    "\n",
    "**Noteable features to exploit:**\n",
    "- R notebook\n",
    "- Visualisation in R\n",
    "- Sequential steps\n",
    "\n",
    "**How?:**\n",
    "- Use of R notebook in wide ranging application\n",
    "\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data cleaning using R\n",
    "\n",
    "This notebook provides a brief overview of the initial steps of dealing with data in an R notebook.\n",
    "\n",
    "This tutorial is aimed at people who have some background in data science but are not necessarily familiar with R or R notebooks.\n",
    "\n",
    "Broadly speaking, \"data cleaning\" refers to the processing necessary to get from the downloaded data to data fit for analysis. This can be broken down to the following stages:\n",
    "- Raw data to technically correct data.\n",
    "- Technically correct data to consistent data.\n",
    "\n",
    "<em><strong>Note:</strong> Exploratory data analysis (EDA) and data cleaning are different things, but they should be done together, iteratively. This notebook addresses some of the ways to start exploring your data, but the EDA that follows will be different for every data set. The better the data is understood, the better the cleaning will be!</em>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "# Acquiring data\n",
    "\n",
    "R comes with some [toy datasets](https://www.rdocumentation.org/packages/datasets/versions/3.6.2) built in, these are fairly small datasets suitable for toy examples such as this.\n",
    "\n",
    "Of course you can also import your own data, just store your file in the same folder as your notebook and load (as below).\n",
    "\n",
    "This tutorial will use a csv file from the [UCI machine learning repository](https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29) (open license) which contains data about breast cancer. \n",
    "\n",
    "First, load the data into a data frame. The variable name used is <code>raw_cervical</code>. This variable will hold the raw data, and will not be altered so that each stage of data cleaning can be reused if desired, like a checkpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load readr library to notebook (pre-installed on Noteable)\n",
    "library(readr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning message:\n",
      "“Missing column names filled in: 'X33' [33]”\n",
      "Parsed with column specification:\n",
      "cols(\n",
      "  .default = col_double(),\n",
      "  diagnosis = \u001b[31mcol_character()\u001b[39m,\n",
      "  X33 = \u001b[31mcol_character()\u001b[39m\n",
      ")\n",
      "\n",
      "See spec(...) for full column specifications.\n",
      "\n",
      "Warning message:\n",
      "“569 parsing failures.\n",
      "row col   expected     actual                     file\n",
      "  1  -- 33 columns 32 columns 'data/cervical_data.csv'\n",
      "  2  -- 33 columns 32 columns 'data/cervical_data.csv'\n",
      "  3  -- 33 columns 32 columns 'data/cervical_data.csv'\n",
      "  4  -- 33 columns 32 columns 'data/cervical_data.csv'\n",
      "  5  -- 33 columns 32 columns 'data/cervical_data.csv'\n",
      "... ... .......... .......... ........................\n",
      "See problems(...) for more details.\n",
      "”\n"
     ]
    }
   ],
   "source": [
    "# load the data to the notebook\n",
    "raw_cervical = readr::read_csv(\"data/cervical_data.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# wholly unnecessary but I keep forgetting the syntax\n",
    "#write.csv(raw_cervical, \"data/raw_data.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The warning box that appears after running the above cell provides useful information. R tries to inform the reader that there's something strange about the way the data frame is organised:<br>\n",
    "<code>“Missing column names filled in: 'X33' [33]”</code><br>\n",
    "This provides a prompt for further investigation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking at the data\n",
    "\n",
    "Below are some useful functions to get the general picture of the data, allowing an insight into potential issues that need to be identified and solved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "569"
      ],
      "text/latex": [
       "569"
      ],
      "text/markdown": [
       "569"
      ],
      "text/plain": [
       "[1] 569"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "33"
      ],
      "text/latex": [
       "33"
      ],
      "text/markdown": [
       "33"
      ],
      "text/plain": [
       "[1] 33"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# output number of rows in data, then columns\n",
    "nrow(raw_cervical)\n",
    "ncol(raw_cervical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A tibble: 6 × 5</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>row</th><th scope=col>col</th><th scope=col>expected</th><th scope=col>actual</th><th scope=col>file</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>1</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "\t<tr><td>2</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "\t<tr><td>3</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "\t<tr><td>4</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "\t<tr><td>5</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "\t<tr><td>6</td><td>NA</td><td>33 columns</td><td>32 columns</td><td>'data/cervical_data.csv'</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A tibble: 6 × 5\n",
       "\\begin{tabular}{lllll}\n",
       " row & col & expected & actual & file\\\\\n",
       " <int> & <chr> & <chr> & <chr> & <chr>\\\\\n",
       "\\hline\n",
       "\t 1 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\t 2 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\t 3 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\t 4 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\t 5 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\t 6 & NA & 33 columns & 32 columns & 'data/cervical\\_data.csv'\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A tibble: 6 × 5\n",
       "\n",
       "| row &lt;int&gt; | col &lt;chr&gt; | expected &lt;chr&gt; | actual &lt;chr&gt; | file &lt;chr&gt; |\n",
       "|---|---|---|---|---|\n",
       "| 1 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "| 2 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "| 3 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "| 4 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "| 5 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "| 6 | NA | 33 columns | 32 columns | 'data/cervical_data.csv' |\n",
       "\n"
      ],
      "text/plain": [
       "  row col expected   actual     file                    \n",
       "1 1   NA  33 columns 32 columns 'data/cervical_data.csv'\n",
       "2 2   NA  33 columns 32 columns 'data/cervical_data.csv'\n",
       "3 3   NA  33 columns 32 columns 'data/cervical_data.csv'\n",
       "4 4   NA  33 columns 32 columns 'data/cervical_data.csv'\n",
       "5 5   NA  33 columns 32 columns 'data/cervical_data.csv'\n",
       "6 6   NA  33 columns 32 columns 'data/cervical_data.csv'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# print a few lines that show what the warning describes above\n",
    "# notice expected vs actual differ in number of columns in data\n",
    "head(problems(raw_cervical))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The table above shows that there is one more column than expected present in the data.\n",
    "\n",
    "Time to do a little digging into the issue..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at the first couple of rows of data using the <code>head()</code> function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A tibble: 6 × 33</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>id</th><th scope=col>diagnosis</th><th scope=col>radius_mean</th><th scope=col>texture_mean</th><th scope=col>perimeter_mean</th><th scope=col>area_mean</th><th scope=col>smoothness_mean</th><th scope=col>compactness_mean</th><th scope=col>concavity_mean</th><th scope=col>concave points_mean</th><th scope=col>⋯</th><th scope=col>texture_worst</th><th scope=col>perimeter_worst</th><th scope=col>area_worst</th><th scope=col>smoothness_worst</th><th scope=col>compactness_worst</th><th scope=col>concavity_worst</th><th scope=col>concave points_worst</th><th scope=col>symmetry_worst</th><th scope=col>fractal_dimension_worst</th><th scope=col>X33</th></tr>\n",
       "\t<tr><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>⋯</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>  842302</td><td>M</td><td>17.99</td><td>10.38</td><td>122.80</td><td>1001.0</td><td>0.11840</td><td>0.27760</td><td>0.3001</td><td>0.14710</td><td>⋯</td><td>17.33</td><td>184.60</td><td>2019.0</td><td>0.1622</td><td>0.6656</td><td>0.7119</td><td>0.2654</td><td>0.4601</td><td>0.11890</td><td>NA</td></tr>\n",
       "\t<tr><td>  842517</td><td>M</td><td>20.57</td><td>17.77</td><td>132.90</td><td>1326.0</td><td>0.08474</td><td>0.07864</td><td>0.0869</td><td>0.07017</td><td>⋯</td><td>23.41</td><td>158.80</td><td>1956.0</td><td>0.1238</td><td>0.1866</td><td>0.2416</td><td>0.1860</td><td>0.2750</td><td>0.08902</td><td>NA</td></tr>\n",
       "\t<tr><td>84300903</td><td>M</td><td>19.69</td><td>21.25</td><td>130.00</td><td>1203.0</td><td>0.10960</td><td>0.15990</td><td>0.1974</td><td>0.12790</td><td>⋯</td><td>25.53</td><td>152.50</td><td>1709.0</td><td>0.1444</td><td>0.4245</td><td>0.4504</td><td>0.2430</td><td>0.3613</td><td>0.08758</td><td>NA</td></tr>\n",
       "\t<tr><td>84348301</td><td>M</td><td>11.42</td><td>20.38</td><td> 77.58</td><td> 386.1</td><td>0.14250</td><td>0.28390</td><td>0.2414</td><td>0.10520</td><td>⋯</td><td>26.50</td><td> 98.87</td><td> 567.7</td><td>0.2098</td><td>0.8663</td><td>0.6869</td><td>0.2575</td><td>0.6638</td><td>0.17300</td><td>NA</td></tr>\n",
       "\t<tr><td>84358402</td><td>M</td><td>20.29</td><td>14.34</td><td>135.10</td><td>1297.0</td><td>0.10030</td><td>0.13280</td><td>0.1980</td><td>0.10430</td><td>⋯</td><td>16.67</td><td>152.20</td><td>1575.0</td><td>0.1374</td><td>0.2050</td><td>0.4000</td><td>0.1625</td><td>0.2364</td><td>0.07678</td><td>NA</td></tr>\n",
       "\t<tr><td>  843786</td><td>M</td><td>12.45</td><td>15.70</td><td> 82.57</td><td> 477.1</td><td>0.12780</td><td>0.17000</td><td>0.1578</td><td>0.08089</td><td>⋯</td><td>23.75</td><td>103.40</td><td> 741.6</td><td>0.1791</td><td>0.5249</td><td>0.5355</td><td>0.1741</td><td>0.3985</td><td>0.12440</td><td>NA</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A tibble: 6 × 33\n",
       "\\begin{tabular}{lllllllllllllllllllll}\n",
       " id & diagnosis & radius\\_mean & texture\\_mean & perimeter\\_mean & area\\_mean & smoothness\\_mean & compactness\\_mean & concavity\\_mean & concave points\\_mean & ⋯ & texture\\_worst & perimeter\\_worst & area\\_worst & smoothness\\_worst & compactness\\_worst & concavity\\_worst & concave points\\_worst & symmetry\\_worst & fractal\\_dimension\\_worst & X33\\\\\n",
       " <dbl> & <chr> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & ⋯ & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <chr>\\\\\n",
       "\\hline\n",
       "\t   842302 & M & 17.99 & 10.38 & 122.80 & 1001.0 & 0.11840 & 0.27760 & 0.3001 & 0.14710 & ⋯ & 17.33 & 184.60 & 2019.0 & 0.1622 & 0.6656 & 0.7119 & 0.2654 & 0.4601 & 0.11890 & NA\\\\\n",
       "\t   842517 & M & 20.57 & 17.77 & 132.90 & 1326.0 & 0.08474 & 0.07864 & 0.0869 & 0.07017 & ⋯ & 23.41 & 158.80 & 1956.0 & 0.1238 & 0.1866 & 0.2416 & 0.1860 & 0.2750 & 0.08902 & NA\\\\\n",
       "\t 84300903 & M & 19.69 & 21.25 & 130.00 & 1203.0 & 0.10960 & 0.15990 & 0.1974 & 0.12790 & ⋯ & 25.53 & 152.50 & 1709.0 & 0.1444 & 0.4245 & 0.4504 & 0.2430 & 0.3613 & 0.08758 & NA\\\\\n",
       "\t 84348301 & M & 11.42 & 20.38 &  77.58 &  386.1 & 0.14250 & 0.28390 & 0.2414 & 0.10520 & ⋯ & 26.50 &  98.87 &  567.7 & 0.2098 & 0.8663 & 0.6869 & 0.2575 & 0.6638 & 0.17300 & NA\\\\\n",
       "\t 84358402 & M & 20.29 & 14.34 & 135.10 & 1297.0 & 0.10030 & 0.13280 & 0.1980 & 0.10430 & ⋯ & 16.67 & 152.20 & 1575.0 & 0.1374 & 0.2050 & 0.4000 & 0.1625 & 0.2364 & 0.07678 & NA\\\\\n",
       "\t   843786 & M & 12.45 & 15.70 &  82.57 &  477.1 & 0.12780 & 0.17000 & 0.1578 & 0.08089 & ⋯ & 23.75 & 103.40 &  741.6 & 0.1791 & 0.5249 & 0.5355 & 0.1741 & 0.3985 & 0.12440 & NA\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A tibble: 6 × 33\n",
       "\n",
       "| id &lt;dbl&gt; | diagnosis &lt;chr&gt; | radius_mean &lt;dbl&gt; | texture_mean &lt;dbl&gt; | perimeter_mean &lt;dbl&gt; | area_mean &lt;dbl&gt; | smoothness_mean &lt;dbl&gt; | compactness_mean &lt;dbl&gt; | concavity_mean &lt;dbl&gt; | concave points_mean &lt;dbl&gt; | ⋯ ⋯ | texture_worst &lt;dbl&gt; | perimeter_worst &lt;dbl&gt; | area_worst &lt;dbl&gt; | smoothness_worst &lt;dbl&gt; | compactness_worst &lt;dbl&gt; | concavity_worst &lt;dbl&gt; | concave points_worst &lt;dbl&gt; | symmetry_worst &lt;dbl&gt; | fractal_dimension_worst &lt;dbl&gt; | X33 &lt;chr&gt; |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "|   842302 | M | 17.99 | 10.38 | 122.80 | 1001.0 | 0.11840 | 0.27760 | 0.3001 | 0.14710 | ⋯ | 17.33 | 184.60 | 2019.0 | 0.1622 | 0.6656 | 0.7119 | 0.2654 | 0.4601 | 0.11890 | NA |\n",
       "|   842517 | M | 20.57 | 17.77 | 132.90 | 1326.0 | 0.08474 | 0.07864 | 0.0869 | 0.07017 | ⋯ | 23.41 | 158.80 | 1956.0 | 0.1238 | 0.1866 | 0.2416 | 0.1860 | 0.2750 | 0.08902 | NA |\n",
       "| 84300903 | M | 19.69 | 21.25 | 130.00 | 1203.0 | 0.10960 | 0.15990 | 0.1974 | 0.12790 | ⋯ | 25.53 | 152.50 | 1709.0 | 0.1444 | 0.4245 | 0.4504 | 0.2430 | 0.3613 | 0.08758 | NA |\n",
       "| 84348301 | M | 11.42 | 20.38 |  77.58 |  386.1 | 0.14250 | 0.28390 | 0.2414 | 0.10520 | ⋯ | 26.50 |  98.87 |  567.7 | 0.2098 | 0.8663 | 0.6869 | 0.2575 | 0.6638 | 0.17300 | NA |\n",
       "| 84358402 | M | 20.29 | 14.34 | 135.10 | 1297.0 | 0.10030 | 0.13280 | 0.1980 | 0.10430 | ⋯ | 16.67 | 152.20 | 1575.0 | 0.1374 | 0.2050 | 0.4000 | 0.1625 | 0.2364 | 0.07678 | NA |\n",
       "|   843786 | M | 12.45 | 15.70 |  82.57 |  477.1 | 0.12780 | 0.17000 | 0.1578 | 0.08089 | ⋯ | 23.75 | 103.40 |  741.6 | 0.1791 | 0.5249 | 0.5355 | 0.1741 | 0.3985 | 0.12440 | NA |\n",
       "\n"
      ],
      "text/plain": [
       "  id       diagnosis radius_mean texture_mean perimeter_mean area_mean\n",
       "1   842302 M         17.99       10.38        122.80         1001.0   \n",
       "2   842517 M         20.57       17.77        132.90         1326.0   \n",
       "3 84300903 M         19.69       21.25        130.00         1203.0   \n",
       "4 84348301 M         11.42       20.38         77.58          386.1   \n",
       "5 84358402 M         20.29       14.34        135.10         1297.0   \n",
       "6   843786 M         12.45       15.70         82.57          477.1   \n",
       "  smoothness_mean compactness_mean concavity_mean concave points_mean ⋯\n",
       "1 0.11840         0.27760          0.3001         0.14710             ⋯\n",
       "2 0.08474         0.07864          0.0869         0.07017             ⋯\n",
       "3 0.10960         0.15990          0.1974         0.12790             ⋯\n",
       "4 0.14250         0.28390          0.2414         0.10520             ⋯\n",
       "5 0.10030         0.13280          0.1980         0.10430             ⋯\n",
       "6 0.12780         0.17000          0.1578         0.08089             ⋯\n",
       "  texture_worst perimeter_worst area_worst smoothness_worst compactness_worst\n",
       "1 17.33         184.60          2019.0     0.1622           0.6656           \n",
       "2 23.41         158.80          1956.0     0.1238           0.1866           \n",
       "3 25.53         152.50          1709.0     0.1444           0.4245           \n",
       "4 26.50          98.87           567.7     0.2098           0.8663           \n",
       "5 16.67         152.20          1575.0     0.1374           0.2050           \n",
       "6 23.75         103.40           741.6     0.1791           0.5249           \n",
       "  concavity_worst concave points_worst symmetry_worst fractal_dimension_worst\n",
       "1 0.7119          0.2654               0.4601         0.11890                \n",
       "2 0.2416          0.1860               0.2750         0.08902                \n",
       "3 0.4504          0.2430               0.3613         0.08758                \n",
       "4 0.6869          0.2575               0.6638         0.17300                \n",
       "5 0.4000          0.1625               0.2364         0.07678                \n",
       "6 0.5355          0.1741               0.3985         0.12440                \n",
       "  X33\n",
       "1 NA \n",
       "2 NA \n",
       "3 NA \n",
       "4 NA \n",
       "5 NA \n",
       "6 NA "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# output the first few rows of data with headers\n",
    "head(raw_cervical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The column names can also be accessed directly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'id'</li><li>'diagnosis'</li><li>'radius_mean'</li><li>'texture_mean'</li><li>'perimeter_mean'</li><li>'area_mean'</li><li>'smoothness_mean'</li><li>'compactness_mean'</li><li>'concavity_mean'</li><li>'concave points_mean'</li><li>'symmetry_mean'</li><li>'fractal_dimension_mean'</li><li>'radius_se'</li><li>'texture_se'</li><li>'perimeter_se'</li><li>'area_se'</li><li>'smoothness_se'</li><li>'compactness_se'</li><li>'concavity_se'</li><li>'concave points_se'</li><li>'symmetry_se'</li><li>'fractal_dimension_se'</li><li>'radius_worst'</li><li>'texture_worst'</li><li>'perimeter_worst'</li><li>'area_worst'</li><li>'smoothness_worst'</li><li>'compactness_worst'</li><li>'concavity_worst'</li><li>'concave points_worst'</li><li>'symmetry_worst'</li><li>'fractal_dimension_worst'</li><li>'X33'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'id'\n",
       "\\item 'diagnosis'\n",
       "\\item 'radius\\_mean'\n",
       "\\item 'texture\\_mean'\n",
       "\\item 'perimeter\\_mean'\n",
       "\\item 'area\\_mean'\n",
       "\\item 'smoothness\\_mean'\n",
       "\\item 'compactness\\_mean'\n",
       "\\item 'concavity\\_mean'\n",
       "\\item 'concave points\\_mean'\n",
       "\\item 'symmetry\\_mean'\n",
       "\\item 'fractal\\_dimension\\_mean'\n",
       "\\item 'radius\\_se'\n",
       "\\item 'texture\\_se'\n",
       "\\item 'perimeter\\_se'\n",
       "\\item 'area\\_se'\n",
       "\\item 'smoothness\\_se'\n",
       "\\item 'compactness\\_se'\n",
       "\\item 'concavity\\_se'\n",
       "\\item 'concave points\\_se'\n",
       "\\item 'symmetry\\_se'\n",
       "\\item 'fractal\\_dimension\\_se'\n",
       "\\item 'radius\\_worst'\n",
       "\\item 'texture\\_worst'\n",
       "\\item 'perimeter\\_worst'\n",
       "\\item 'area\\_worst'\n",
       "\\item 'smoothness\\_worst'\n",
       "\\item 'compactness\\_worst'\n",
       "\\item 'concavity\\_worst'\n",
       "\\item 'concave points\\_worst'\n",
       "\\item 'symmetry\\_worst'\n",
       "\\item 'fractal\\_dimension\\_worst'\n",
       "\\item 'X33'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'id'\n",
       "2. 'diagnosis'\n",
       "3. 'radius_mean'\n",
       "4. 'texture_mean'\n",
       "5. 'perimeter_mean'\n",
       "6. 'area_mean'\n",
       "7. 'smoothness_mean'\n",
       "8. 'compactness_mean'\n",
       "9. 'concavity_mean'\n",
       "10. 'concave points_mean'\n",
       "11. 'symmetry_mean'\n",
       "12. 'fractal_dimension_mean'\n",
       "13. 'radius_se'\n",
       "14. 'texture_se'\n",
       "15. 'perimeter_se'\n",
       "16. 'area_se'\n",
       "17. 'smoothness_se'\n",
       "18. 'compactness_se'\n",
       "19. 'concavity_se'\n",
       "20. 'concave points_se'\n",
       "21. 'symmetry_se'\n",
       "22. 'fractal_dimension_se'\n",
       "23. 'radius_worst'\n",
       "24. 'texture_worst'\n",
       "25. 'perimeter_worst'\n",
       "26. 'area_worst'\n",
       "27. 'smoothness_worst'\n",
       "28. 'compactness_worst'\n",
       "29. 'concavity_worst'\n",
       "30. 'concave points_worst'\n",
       "31. 'symmetry_worst'\n",
       "32. 'fractal_dimension_worst'\n",
       "33. 'X33'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       " [1] \"id\"                      \"diagnosis\"              \n",
       " [3] \"radius_mean\"             \"texture_mean\"           \n",
       " [5] \"perimeter_mean\"          \"area_mean\"              \n",
       " [7] \"smoothness_mean\"         \"compactness_mean\"       \n",
       " [9] \"concavity_mean\"          \"concave points_mean\"    \n",
       "[11] \"symmetry_mean\"           \"fractal_dimension_mean\" \n",
       "[13] \"radius_se\"               \"texture_se\"             \n",
       "[15] \"perimeter_se\"            \"area_se\"                \n",
       "[17] \"smoothness_se\"           \"compactness_se\"         \n",
       "[19] \"concavity_se\"            \"concave points_se\"      \n",
       "[21] \"symmetry_se\"             \"fractal_dimension_se\"   \n",
       "[23] \"radius_worst\"            \"texture_worst\"          \n",
       "[25] \"perimeter_worst\"         \"area_worst\"             \n",
       "[27] \"smoothness_worst\"        \"compactness_worst\"      \n",
       "[29] \"concavity_worst\"         \"concave points_worst\"   \n",
       "[31] \"symmetry_worst\"          \"fractal_dimension_worst\"\n",
       "[33] \"X33\"                    "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# output variable names\n",
    "colnames(raw_cervical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you glance over the column names, the last one looks out of place. Looking again at the table produced by <code>head()</code>, the first few rows of this column seem to be NA values. \n",
    "\n",
    "You can attain summary information for individual variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. \n",
       "  6.981  11.700  13.370  14.127  15.780  28.110 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# output summary statistics for radius_mean variable\n",
    "summary(raw_cervical$radius_mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or indeed the whole data set, although this can become quite unwieldy. This has been commented out, so just remove the <code>#</code> at the beginning of the line to run the function again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# output summary statistics for whole data set\n",
    "#summary(raw_cervical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even from these few cells, we have a lot more information than we did to begin with!\n",
    "\n",
    "There are 569 samples, and 33 columns. We have the names for each column, as well as the first few samples for each. We have identified that the last column is not expected, and only seems to contain NA values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contextualising data - using what's given\n",
    "\n",
    "Above, there are a range of tools used to examine what is present in the data. Additional context can (and should) be added by reading the author's description (available [here](https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29)) of the data. \n",
    "\n",
    "Key information given about the data:\n",
    "\n",
    "Context:\n",
    "- Features computed from image of a [fine needle aspirate](https://en.wikipedia.org/wiki/Fine-needle_aspiration) of a breast mass.\n",
    "- Features describe characteristics of cell nuceli in image.\n",
    "- 569 instances\n",
    "- 32 attributes\n",
    "- Class distribution: 357 benign, 212 malignant\n",
    "- No missing values\n",
    "\n",
    "Attributes:\n",
    "- ID in field 1\n",
    "- Predicting field 2, diagnosis - B=benign, M=malignant\n",
    "- Columns 3-32 contain the mean, standard error and \"worst\"/largest values of 10 different features. Where \"worst\" is mean of three largest values. \n",
    "- The 10 features are:\n",
    "    - radius (mean distance from center to perimeter)\n",
    "    - texture (standard deviation of grey-scale image values)\n",
    "    - perimeter\n",
    "    - area\n",
    "    - smoothness (local variation in radius lengths)\n",
    "    - compactness $\\frac{{perimeter}^2}{area - 1.0}$\n",
    "    - concavity (severity of concave portions of contour)\n",
    "    - number of concave portions of contour\n",
    "    - symmetry\n",
    "    - fractal dimension $({coastline approximation}) - 1$\n",
    "- Columns 3-32 are organised as follows:\n",
    "    - 3-12: mean for each feature\n",
    "    - 13-22: standard error for each feature\n",
    "    - 23-32: \"worst\" value for each feature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "# Cleaning - from raw to technically correct data\n",
    "\n",
    "In essence, the getting the data to be \"technically correct\" involves identifying errors and inconsistencies in your data. The names, types and labels in the data need to make sense. Luckily, this dataset is not too messy, but if it were, this step would ensure that R could access the data so that we can do more with it!\n",
    "\n",
    "For example, if a height variable is stored as a string \"1\", this step would involve ensuring that R reads the value as a number, not a string of characters.\n",
    "\n",
    "The goal of this step:\n",
    "- The data is stored in a data.frame.\n",
    "- The columns of the data.frame are suitable.\n",
    "- Each column uses an appropriate type to represent the values in it.\n",
    "    - Numeric data stored as <code>numeric/integer</code>.\n",
    "    - Text data stored as <code>character</code> (for single characters and strings).\n",
    "    - Categorical data stored as <code>factor/ordered</code>.\n",
    "    - True/false data stored as <code>logical</code>.\n",
    "    \n",
    "Steps to reach the goal:\n",
    "- Removing unused/irrelevant columns/rows / cols with 0 variance.\n",
    "- Renaming variables for consistency and meaning.\n",
    "- Encode missing values appropriately (if required).\n",
    "- Remove duplicates (if this doesnt cause loss of information).\n",
    "- Fix dates (common bugbear).\n",
    "- Fix strings (another common bugbear).\n",
    "- Recoding vectors (eg: use 0 and 1 instead of yes/no)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Irrelevant columns/rows and columns with 0 variance\n",
    "\n",
    "Time to revisit the weird 33rd column from earlier.\n",
    "\n",
    "By checking the description of the data, as well as other people's comments online on [kaggle](https://www.kaggle.com/uciml/breast-cancer-wisconsin-data/data) it seems like column 33 shouldn't be there, and got read in by mistake. \n",
    "\n",
    "<b>Why did this happen?</b><br> By opening the csv file (just click on the file name in Noteable) I can see that the first line (header) ends with a comma, meaning that its still trying to read another heading and thinks there are 32 columns.\n",
    "\n",
    "Seems safe to get rid of it, since the mistake is understood and confirmed by others. \n",
    "\n",
    "Removing the **irrelevant column**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the last column of the data (will print many NA's) - uncomment to run\n",
    "#raw_cervical$X33"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove col 33, X33 (meaningless)\n",
    "nas_cervical <- cbind(raw_cervical[-33])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! The unnecessary column is gone. Bear in mind that the data frame without NA's is stored in the <code>nas_cervical</code> variable, the <code>raw_cervical</code> is left as it was. From now on, use the <code>nas_cervical</code> data frame!\n",
    "\n",
    "The original data description claims that there is no missing data at all in the dataset. Hopefully, that's the case now. To double check:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/latex": [],
      "text/markdown": [],
      "text/plain": [
       "integer(0)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# prints nothing because none found!\n",
    "# apply function used to check every column (dimension 2)\n",
    "apply(is.na(nas_cervical),2,which)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can also sometimes be useful to check for infinity and NaN (not a number) values in the data. This is just an extension of what has already been done here, and follows the same idea.\n",
    "\n",
    "If NA values are found in your data, libraries [naniar](https://cran.r-project.org/web/packages/naniar/vignettes/getting-started-w-naniar.html) and [knitr](https://yihui.org/knitr/) provide useful functions to visualise missing data.\n",
    "\n",
    "Sometimes, the purpose of data cleaning is to get it ready to answer a specific question about the variables. In these cases, it is sometimes appropriate to remove columns or samples that are unrelated to the question. \n",
    "\n",
    "Since the purpose here is to understand the data, no rows or columns need to be deleted on this basis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Empty rows and columns can be removed by using the <code>remove_empty()</code> function in the [janitor](https://cran.r-project.org/web/packages/janitor/vignettes/janitor.html) package."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Columns with 0 variance are not very informative. If the variable changes nothing, its completely constant, why should it bulk up the data?\n",
    "\n",
    "The following cell checks for 0 variance columns in the numerical data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/latex": [],
      "text/markdown": [],
      "text/plain": [
       "named integer(0)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# extract only numerical columns\n",
    "num_cervical <- cbind(nas_cervical[-2])\n",
    "\n",
    "# check for 0 variance columns\n",
    "which(apply(num_cervical,2, var)==0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "see also <code>remove_constant()</code> from the [janitor](https://cran.r-project.org/web/packages/janitor/vignettes/janitor.html) package to remove 0 variance columns from data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable names\n",
    "\n",
    "This is also a good time to check for unhelpful or confusing variable names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'id'</li><li>'diagnosis'</li><li>'radius_mean'</li><li>'texture_mean'</li><li>'perimeter_mean'</li><li>'area_mean'</li><li>'smoothness_mean'</li><li>'compactness_mean'</li><li>'concavity_mean'</li><li>'concave points_mean'</li><li>'symmetry_mean'</li><li>'fractal_dimension_mean'</li><li>'radius_se'</li><li>'texture_se'</li><li>'perimeter_se'</li><li>'area_se'</li><li>'smoothness_se'</li><li>'compactness_se'</li><li>'concavity_se'</li><li>'concave points_se'</li><li>'symmetry_se'</li><li>'fractal_dimension_se'</li><li>'radius_worst'</li><li>'texture_worst'</li><li>'perimeter_worst'</li><li>'area_worst'</li><li>'smoothness_worst'</li><li>'compactness_worst'</li><li>'concavity_worst'</li><li>'concave points_worst'</li><li>'symmetry_worst'</li><li>'fractal_dimension_worst'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'id'\n",
       "\\item 'diagnosis'\n",
       "\\item 'radius\\_mean'\n",
       "\\item 'texture\\_mean'\n",
       "\\item 'perimeter\\_mean'\n",
       "\\item 'area\\_mean'\n",
       "\\item 'smoothness\\_mean'\n",
       "\\item 'compactness\\_mean'\n",
       "\\item 'concavity\\_mean'\n",
       "\\item 'concave points\\_mean'\n",
       "\\item 'symmetry\\_mean'\n",
       "\\item 'fractal\\_dimension\\_mean'\n",
       "\\item 'radius\\_se'\n",
       "\\item 'texture\\_se'\n",
       "\\item 'perimeter\\_se'\n",
       "\\item 'area\\_se'\n",
       "\\item 'smoothness\\_se'\n",
       "\\item 'compactness\\_se'\n",
       "\\item 'concavity\\_se'\n",
       "\\item 'concave points\\_se'\n",
       "\\item 'symmetry\\_se'\n",
       "\\item 'fractal\\_dimension\\_se'\n",
       "\\item 'radius\\_worst'\n",
       "\\item 'texture\\_worst'\n",
       "\\item 'perimeter\\_worst'\n",
       "\\item 'area\\_worst'\n",
       "\\item 'smoothness\\_worst'\n",
       "\\item 'compactness\\_worst'\n",
       "\\item 'concavity\\_worst'\n",
       "\\item 'concave points\\_worst'\n",
       "\\item 'symmetry\\_worst'\n",
       "\\item 'fractal\\_dimension\\_worst'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'id'\n",
       "2. 'diagnosis'\n",
       "3. 'radius_mean'\n",
       "4. 'texture_mean'\n",
       "5. 'perimeter_mean'\n",
       "6. 'area_mean'\n",
       "7. 'smoothness_mean'\n",
       "8. 'compactness_mean'\n",
       "9. 'concavity_mean'\n",
       "10. 'concave points_mean'\n",
       "11. 'symmetry_mean'\n",
       "12. 'fractal_dimension_mean'\n",
       "13. 'radius_se'\n",
       "14. 'texture_se'\n",
       "15. 'perimeter_se'\n",
       "16. 'area_se'\n",
       "17. 'smoothness_se'\n",
       "18. 'compactness_se'\n",
       "19. 'concavity_se'\n",
       "20. 'concave points_se'\n",
       "21. 'symmetry_se'\n",
       "22. 'fractal_dimension_se'\n",
       "23. 'radius_worst'\n",
       "24. 'texture_worst'\n",
       "25. 'perimeter_worst'\n",
       "26. 'area_worst'\n",
       "27. 'smoothness_worst'\n",
       "28. 'compactness_worst'\n",
       "29. 'concavity_worst'\n",
       "30. 'concave points_worst'\n",
       "31. 'symmetry_worst'\n",
       "32. 'fractal_dimension_worst'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       " [1] \"id\"                      \"diagnosis\"              \n",
       " [3] \"radius_mean\"             \"texture_mean\"           \n",
       " [5] \"perimeter_mean\"          \"area_mean\"              \n",
       " [7] \"smoothness_mean\"         \"compactness_mean\"       \n",
       " [9] \"concavity_mean\"          \"concave points_mean\"    \n",
       "[11] \"symmetry_mean\"           \"fractal_dimension_mean\" \n",
       "[13] \"radius_se\"               \"texture_se\"             \n",
       "[15] \"perimeter_se\"            \"area_se\"                \n",
       "[17] \"smoothness_se\"           \"compactness_se\"         \n",
       "[19] \"concavity_se\"            \"concave points_se\"      \n",
       "[21] \"symmetry_se\"             \"fractal_dimension_se\"   \n",
       "[23] \"radius_worst\"            \"texture_worst\"          \n",
       "[25] \"perimeter_worst\"         \"area_worst\"             \n",
       "[27] \"smoothness_worst\"        \"compactness_worst\"      \n",
       "[29] \"concavity_worst\"         \"concave points_worst\"   \n",
       "[31] \"symmetry_worst\"          \"fractal_dimension_worst\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# read column names\n",
    "colnames(nas_cervical) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the variables are named helpfully. If the variable names need to be changed, you can use the <code>clean_names()</code> function from the [janitor](https://cran.r-project.org/web/packages/janitor/vignettes/janitor.html) package."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Duplicates\n",
    "\n",
    "R has some useful functions for dealing with duplicates in data. Packages are available, but base R packages work fine here.\n",
    "\n",
    "Here <code>anyDuplicated()</code> is used, but it may be worth looking at <code>duplicated()</code> and <code>unique()</code>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0"
      ],
      "text/latex": [
       "0"
      ],
      "text/markdown": [
       "0"
      ],
      "text/plain": [
       "[1] 0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check for duplicates\n",
    "anyDuplicated(nas_cervical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[anyDuplicated()](https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/duplicated) returns 0, so no duplicate rows in data. easy. If there were, [unique()](https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/unique) could be used to reduce the data frame."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fix classes of each column - recode factors\n",
    "\n",
    "common activities at this stage include:\n",
    "- Checking if there are dates that are inevitably wrongly input can be fixed somehow.\n",
    "- Fix strings to make sense because I guarantee they do not right now.\n",
    "- Recode categorical data to usable factors maybe numbers maybe booleans not sure will get back to you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In R, there are 2 main ways to retrieve class information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get class for each column - uncomment to run\n",
    "#lapply(nas_cervical,class)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# display structure of object - uncomment to run\n",
    "#str(nas_cervical)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'M'</li><li>'B'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'M'\n",
       "\\item 'B'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'M'\n",
       "2. 'M'\n",
       "3. 'M'\n",
       "4. 'M'\n",
       "5. 'M'\n",
       "6. 'M'\n",
       "7. 'M'\n",
       "8. 'M'\n",
       "9. 'M'\n",
       "10. 'M'\n",
       "11. 'M'\n",
       "12. 'M'\n",
       "13. 'M'\n",
       "14. 'M'\n",
       "15. 'M'\n",
       "16. 'M'\n",
       "17. 'M'\n",
       "18. 'M'\n",
       "19. 'M'\n",
       "20. 'B'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       " [1] \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\" \"M\"\n",
       "[20] \"B\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# look at first 20 values in column\n",
    "head(nas_cervical$diagnosis, 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>1</li><li>0</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 1\n",
       "\\item 0\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 1\n",
       "2. 1\n",
       "3. 1\n",
       "4. 1\n",
       "5. 1\n",
       "6. 1\n",
       "7. 1\n",
       "8. 1\n",
       "9. 1\n",
       "10. 1\n",
       "11. 1\n",
       "12. 1\n",
       "13. 1\n",
       "14. 1\n",
       "15. 1\n",
       "16. 1\n",
       "17. 1\n",
       "18. 1\n",
       "19. 1\n",
       "20. 0\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       " [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# forces numeric 0 or 1 for values\n",
    "nas_cervical$diagnosis <- gsub('M', \"1\", nas_cervical$diagnosis)\n",
    "nas_cervical$diagnosis <- gsub('B', \"0\", nas_cervical$diagnosis)\n",
    "nas_cervical$diagnosis <- as.numeric(nas_cervical$diagnosis)\n",
    "\n",
    "# print first 20 values of column\n",
    "head(nas_cervical$diagnosis, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<em><strong>Note:</strong> when you do something like this, make sure its clear to the reader of your work what has been done. 1 and 0 mean nothing out of context, but as the person writing the code, you understand that 1 represents M in the original data, and 0 represents B.</em>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dates and strings\n",
    "\n",
    "It is fairly common to have problems with dates and strings in data. This is not a problem in this dataset, but this should be checked. the [lubridate](https://lubridate.tidyverse.org) package provides good support for this"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "# Cleaning - from technically correct to consistent data\n",
    "\n",
    "Just because the data is technically correct doesn't mean it makes sense. This is another reason why context is important.\n",
    "\n",
    "For data to be consistent, it should be ready for inference.\n",
    "\n",
    "Goal for this step:\n",
    "- Data is stored in a suitable data frame, ready for inference!\n",
    "\n",
    "Steps to reach the goal:\n",
    "- Variable constraints addressed (eg: can age=-35?).\n",
    "- Missing values - addressing strategies to get the most from data with missing values.\n",
    "- Dealing with outliers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable Constraints\n",
    "\n",
    "Can someone's age be -35? Can someone be married, divorced, single and pregnant all at the same time?\n",
    "\n",
    "These are the sorts of questions to ask at this stage.\n",
    "\n",
    "This step is very subject specific. This absolutely requires subject knowledge. As the data analyist, it can be helpful to ask people in that discipline if the values make sense. In this case, the data is well documented and reviewed, there are no variable constraints of note.\n",
    "\n",
    "This step isnt included in this guide because it can be a long step, and this data is quite well documented. Generally speaking though, this is a useful step in ensuring that variables are correctly labelled and contain useful information that actually makes sense. Correcting variable constraint violations is an art - not to be taken lightly.\n",
    "\n",
    "Checking out the [editrules](https://cran.r-project.org/web/packages/editrules/index.html) package is a good place to start. In particular, look at the <code>editset()</code> and <code>violatedEdits()</code> functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outliers\n",
    "\n",
    "There are many different ways to deal with outliers (again, context dependant). You may wish to remove all rows with missing values, remove irrlevant variables then remove rows with missing values, or employ some other method.\n",
    "\n",
    "Outliers:<br>\n",
    "This step has two stages: identify and then treat. It is not as simple as removal, because this will affect the data in (possibly) unintended ways. Previous steps involved changing the way the data was represented. This step involves meaningfully changing what is happening in the data, this is a delicate operation. \n",
    "\n",
    "<b>What is an outlier?</b><br>\n",
    "An unusual bit of data. \"Unusual\" is (obviously) quite a context sensitive term. The idea of an outlier *at all* depends on the data that you have.\n",
    "\n",
    "One common way to look for outliers is to use boxplots:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>25.22</li><li>22.27</li><li>24.25</li><li>23.27</li><li>27.22</li><li>23.29</li><li>28.11</li><li>23.21</li><li>23.51</li><li>25.73</li><li>22.01</li><li>27.42</li><li>23.09</li><li>24.63</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 25.22\n",
       "\\item 22.27\n",
       "\\item 24.25\n",
       "\\item 23.27\n",
       "\\item 27.22\n",
       "\\item 23.29\n",
       "\\item 28.11\n",
       "\\item 23.21\n",
       "\\item 23.51\n",
       "\\item 25.73\n",
       "\\item 22.01\n",
       "\\item 27.42\n",
       "\\item 23.09\n",
       "\\item 24.63\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 25.22\n",
       "2. 22.27\n",
       "3. 24.25\n",
       "4. 23.27\n",
       "5. 27.22\n",
       "6. 23.29\n",
       "7. 28.11\n",
       "8. 23.21\n",
       "9. 23.51\n",
       "10. 25.73\n",
       "11. 22.01\n",
       "12. 27.42\n",
       "13. 23.09\n",
       "14. 24.63\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       " [1] 25.22 22.27 24.25 23.27 27.22 23.29 28.11 23.21 23.51 25.73 22.01 27.42\n",
       "[13] 23.09 24.63"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAC9FBMVEUAAAABAQECAgIDAwME\nBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUW\nFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJyco\nKCgpKSkqKiorKyssLCwtLS0vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7\nOzs8PDw9PT0+Pj4/Pz9AQEBCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1O\nTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19g\nYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFy\ncnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OE\nhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWW\nlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eo\nqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm7\nu7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fJycnKysrLy8vMzMzNzc3O\nzs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g\n4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy\n8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7///8JRHuGAAAACXBIWXMA\nABJ0AAASdAHeZh94AAAgAElEQVR4nO3de2BU1Z3A8ZOQEEkIEAo+ePhW1EqpwFarUBRBoaJ9\naKu1FbGorSiytUuprrhSZEvXd2lrt1qrQi0qttatFUHLVndFBbSiFtSVFRDkWZVHSHL/2Xtn\n8qQwe3V+51zz+30/fzA3k5uZc/B8neTmJLgIQNFc1gMANCAkQAAhAQIICRBASIAAQgIEEBIg\ngJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIg\ngJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIg\ngJBCedi5DvHNA85VBHzWj/50gQfa3hFS8f7dNao87Ov/udezCEk1Qipec0iJq/Z2VmNIK26+\n+faAYxMP6Z0O7pWiRqQUIRUvCWnk6NGjRx6dlHTfXs5qDCkw8ZBuc4S0J4RUvCSkzbmjpzo5\nd8JezlIS0omEtEeEVLyWkKJLnOsU3zT8ekTPsup/uK0ud+cvB1bWjFr82zZfI13t3GeS9zXl\nteuOU3qW9Rx0w7stD3unc0OiBwZ0Oujq2mj5Gd2qTv1L7u5XLzm8onrQrbui3Z4nd/qi4d2q\nTprfamwPJAOaN6RL9bAn83dsnf6ZmvJ9R9yVfMj8EucWxreb9nNuYvQL546JfnNil+rhT0dR\nq5Baf8To/OevE6X/Cts/Qipeq5Cuda4svvla4xdMn2+I3/hO7rDimkIh1Q5r/IhDVzY/7P3O\nHTu3JLnzsjd6JDc9t8T3PrRP/sSTt+/2PMnpj3dM3urwRMvY4qfr/NPcOaWPJm8v7d34ISds\niN+a5Nzh8eNcFH/o9ug+53r9KPe+ssdbBtr2IwhpbwipeK1Cins4Oop+Hy/bn7x0Z5lzc6Po\nv5NVP++R08oKhRSv9aN+/cwfv+Tc55ofNj7zgD5jJnSNP+D0XhM/Ez/MzVH0RvzJ43dfW/w5\n56bs9jzJ6QcPmDIiPu8fotYPUlH9zV/PjB+l7874tSeu4pCf/nZy/CGj4/fu6J88zpMlrmJZ\nFP0mPrXigl/f0Nm5g2qbB9r2I17+Xfzws//zjVB/te0HIRWvKaS6FZfFR7dF0Y9Hj07+nz3G\nuW9E0fj4peT9+HO3owqFNNa5G+Ob2nMn/Gt908PGZ7qz45eg+Gaf16Od/Zw7I4omODcsft+7\nnV319rbPk5w+ZHvuVaq0Nmr9IJfGtw/Gt3+Ion9xrsvq+M174zefi29frHBl/32kczc1njou\nyr20ucebB7rbR6x1fI20R4RUvDaXv4fubL7/cudGRNHR+eUZTSsU0hXOHXjPO20fNlnYz8Zx\nVTh3bvzmVc4NiKLDnLt6e2yoc0+0fZ7k9IXxW0/Et//T5kFeim/rujh3bRR9yrmxyd11Nc5d\nnxzc5Fylc8MbGk9dGt/uqnbuuuaB7vYRhLQXhFS8ViF1/6fkS5do/pmHVuTePiWK4s/FZiT3\nzSsU0pLK5OzDLnqoruVhk4W9Lb7tk3+AG+P3Rw2lLc91a9vnSU7/W3zeisZ0mh+kPPcad1wS\nRENZ/pUvik7I1xk1DE9GvbrNqZ927sKmge7+EYS0F4RUvCSkUWeeeWb8/+yzcnf8JL6j6uhP\n9UgWeEN+yUfRHwtetXvyk/k6Dn6m+WGbvtqPX4R+Gt/cnoT0fqvXvqltnqf59P/dPaRuuYOT\nnPtK7sPvyL0Z9/P53MHP47tO28OpjY+3+0cQ0l4QUvGavka6M759ML59L355OS9+Lfl2boHv\n49wNyVlzdw9pQHLvHU3fXWp45vrTurr811N5ewqpoYNztzSd0PZ59hpSRfJ5W/IyMy73+vKj\n3P2fce5rye2a7kmT97R+vvyprV6RWn8EIe0FIRWvKaSGzzp3wJYoWhS/vSR++5TcAj8ydykg\niqa0CWmGc/smC/yS1t+mrftdTe6SQN6eQkoerfnac9vn2WtIuYWffOETf4kzoLGf2vjNHyYH\npzk3psR1XdV46ork1M65Uxsfb7ePIKS9IKTiNV/+Xhq/XoyPv3LJXyV4Of56ZmgUXRB/xrQp\nfvno0yak5CLY76PolcrcvdtuGDsm9+XJCOcebnrYPYZ0uXO9PoiTO+/C773d9nn2HtKkKHdt\nO7kUcX38ueD/RrlP6Epei3KfhB6545vOndyQP/V7Uf4y4cLmx9vtI96J37n3jbmGEVLxWr6P\nNDFebU9Fq+OVfcZLj/Tu51yX/1r3VPzegff/anD8//nSqHl9vl4SL9BLrur+2Xxe8f/3v/SH\n5xb9S7mrWNf0sHsM6fVOzp34H49/0blP1rV9nj2HFAfUoeOk393Yxbl+dVG0Oc758FkPfbfC\nuW/G7/1rlXNP5jY23Jx7vrLyqU/+5BP5Uxsfb7ePqCt3bsjcxwP9zbYjhFS8lpC2HhD/H357\n8rIR6/Vmr9wlgQtzb1X9OP6jviWPS3P3HvHnOL341eClPo1XEErvbH7YPYYUPZC/TOd6vxK1\nfZ49h3Rf/Cnk9Nxp++Q2/jTvU/ji9riK43PX56I58XtfTp6v5nstp/7dzoYv5i5Inp4cjvb9\nV9r+EFLxWu1smO2SrQK1PzymU+9vro7m9yvrc39Uf2O/jvt++S8v5y9PN63PuhlHdOx98bq3\n43vjT9Wid6YN2q+88qiLl7U87J5DipaPO6Si8tir488W2z7PnkP6hXNHRb8csE/NmKX5O7b+\nYHDX8gO+8EhyPM25nhuTg5HOHVcbP0BVdEdy6rLWT9/mI6Lo7bO67XPIdA9/i+0cIaERP8lX\nDEJCI0IqBiGhESEVg5DQiJCKQUhoREjFICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRA\nACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRA\nACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAgKEtPQ5oF1Z+uFXuf+QFjugnVn8oZe5\n/5Cedju9PwcgaKd7+kN/DCEBuyEkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQII\nCRBASIAAQgIEEBIggJAAAYRkybLJo0ZNXpb1KFQiJENmdBj63e8O7TAj63FoREh2PNBxXnIz\nr+ODWY9EIUKy49NX5W+vOi7bcahESGa85/4rf/BMyfvZjkQjQjJjjXstf/CqW5PtSDQiJDNq\n9/mP/MGj+9RmOxKNCMmOL41oSG7qT/1S1iNRiJDseLXreWujaO15XV/LeiQKEZIhzx/rDj7Y\nHft81uPQiJAsqX/+7rufq896FCoREiCAkAABhAQIICRAACEBAggJEEBIgABCsmXXrqxHoBQh\nGbLrxgEVFQNupCUPCMmOHcN7Tp8/f3qP4TuyHolChGTH9Qe8ldy8tf/1WY9EIUIyo6H37fmD\n2/s0ZDsSjQjJjA3uxfzBMrcx25FoREhmrHcvb7jzH//xzo1/ceuzHos+hGRGfc9vd+k1Zkyv\nLt/qyad24gjJjq+WXFsXRXX/XPLVrEeiECHZcVL3w3/16qt3H979pKxHohAhmbGtwx8m9XCu\nx6Q/dNie9Vj0ISQzVie/127t2ih6za3Oeiz6EJIZ2zsszB8sKGNrgzhCsuOU8/O355+S7ThU\nIiQ7nim/Lv5r3Xld+TNZj0QhQjLkt917jji1Z/ffZT0OjQjJkq2zp0yZvTXrUahESIAAQgIE\nEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIE\nEBIggJAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkUxreeKMh6zHoREiGbPlW\nZ+eqv7Ul63FoREh2bP7k0fe/+eb9Rx+7OeuRKERIdkzsl3st2nLklVmPRCFCMqP+E7/KH/zq\nE/XZjkQjQjJjnXs5f/CyW5ftSDQiJDM2umX5g6VuY7Yj0YiQzGg48Jb8wS0HZjsQlQjJjhv2\nXZncrNz3hqxHohAh2bFzVM0/P/roNTWja7MeiUKEZEjdrOOrq4+fVZf1ODQiJFsa2CHkByEB\nAggJEEBIpqx67LFVWY9BpyxC2vnswjcKn0FIXqwY5qqq3LAVWY9Do6AhTVuY/PmzGufcwCWF\nTiQkH97a77Rl9fXLRu7Pi5K8oCG5yfEfj7qKL1xyouu6ssCJhOTD+Z/NfQOp9rNfz3ggGoUP\n6Yiuy+M/Hyq5sMCJhOTBrqp5+YN5VbuyHYlGwUNa776fOz6rd4ETCcmDNe7V/MErbm22I9Eo\neEir3L2542vKC5xISB5sdYvzB8+6v2U7Eo2Ch1TXdUbueFz3AicSkg/HTM3fXntMtuNQKWxI\n5y5e8e6Uwz+ID1+pOqPAiYTkw12Vf0pu/lR5V9YjUShsSHkPRtHsqtJnC5xISF5MKvvyzJlf\nLpuU9Tg0ChrSL2+eOvGCs4YtiKJZvX9f6ERC8mPhhYMGjV2Y9ShUymiL0Ht//+s3/qdXTbPO\nbnvxzwGEk9leu01v7nbHrofnNhvv3pN4DiCUsCEtG3XQSY0/Vza50KP8jJDQvgQN6c8VrrLc\nfW5TckxI0CRoSKPLH27YcVP54PcjQoIuQUPqe37y54KOo+oIKRtvP/7421mPQaegIZVfm7u5\nx11BSFlYebLr1MmdXGjfPT6ioCH1GZO/neJmElJ4q/YfsaS+fsmIA/h5JHlBQ7qi5PbcT8Q0\nXOCuvJyQQvv6CfmfRzrh6xkPRKOgIW040A3PHTRc4RwhBdb880gPdeY324kL+32kd7/d9E/z\nPHQYIQW2lp9H8ujj+VuECMmDVj+PtDXbkWhESHYcc13+dio/jySPkOz4RdWi5GZR1Z0ZD0Qj\nQrKjYWLZOf/2b+eUTcx6IBoRkiVPfGPgwG88kfUoVCIkQ+puG1xZOfg2Ln57QEh27Dyt+9TH\nHpva/TR+/FgeIdkxfb/cr1x/Y7/pWY9EIUIyo6HvrfmDW/vyr42JIyQzNrpl+YOlbmO2I9GI\nkMxY517OH/zFrc92JBoRkhn1Pe7OH9zd4+9/hxOKREh2TDpic3Kz+Qh+Q6Q8QrJjS/9+c1au\nnN2v/5asR6IQIRmydUJX57pOYO+3B4Rky1tvZT0CpQgJEEBIgABCAgQQEiCAkAABhAQIICRA\nACEBAggJEEBIhjTMOb1v39Pn8GN9HhCSHXXnVk64997LKs/jt5/IIyQ7bq55Mbl5seaWrEei\nECHZcegP87c/PCzbcahESGZsdi/kD15wm7MdiUaEZMY7bnn+YLl7J9uRaERIZtR1m3PX4Orq\nwXfN6cbvbBBHSHaM71zqevZ0pZ0vznokChGSHRNc55FDhozs7C7PeiQKEZIdlTUlJc6VlNRU\nZT0ShQjJjHWu/LHtL7yw/bFyty7rsehDSGYsc+d8MG/atHkfnO1ezHos+hCSGavd6ft1GTKk\ny36n8a+ayyMkM9Y4N2lbFG270rk1WY9FH0Iyo7bUHfLI+vWPHOJKd2U9Fn0IyYwdJfuVlDpX\nGt/syHos+hCSGatdt1PGDx06/uRubnXWY9GHkMzYVnr3INejhxt8d+m2rMeiDyHZcdLF0fK5\nc5dH40/KeiQKEZIdC8purY+i+lvKFmQ9EoUIyZB7Kg8955xDK+/JehwaEZIla2ddeumP+W6s\nD4QECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAkQAAhAQIIyZSNixZtzHoMOhGSIas+\nX1JWVvL5VVmPQyNCsmNNn6FP79z59NC+/BIheYRkx7hBuV96smPQuKxHohAhmVFX/UD+YG4X\n/hFZcYRkxlr3Sv7gFX7TqjxCMmOLez5/8Jzbku1INCIkO478QbRjyZId0Q/6ZT0ShQjJjp90\nHlnmXNnIzj/NeiQKEZId67qUnHrjjaeWdFmf9UgUIiQ7Lu3/wNlHHXX23P7fynokChGSGXXd\n5uQP+FfNPSAkM95xy/MHy9072Y5EI0IyY5Nbkj94wW3OdiQaEZIdh87M3848LNtxqERIdtxU\n81Jy81LNzVmPRCFCsqPuK1VXzJ59edVX2Wonj5AMabhvZO/eI+9ryHocGhESIICQ2pc5w4tz\n7LFFPsCcrP8GPqYIqX1ZNLk4/fsX+QCLsv4b+JgiJFvGjs16BEoRki2E5Akh2UJInhCSLYTk\nCSHZsoiLBX4QEiCAkAABhAQIICRb5rAzwQ9CsoWrdp4Qki2E5Akh2UJInhCSLYTkCSHZQkie\nEJIt7GzwhJAAAYQECCAkQAAh2cLOBk8IyRau2nlCSLYQkieEZAsheUJIthCSJ4RkCyF5Qki2\nsLPBE0ICBBASIICQAAGEZAs7GzwhJFu4aucJIdlCSJ4Qki2E5Akh2UJInhCSLYTkCSHZws4G\nT0KH1PD6/HnzFqz6f84iJLQzYUPa9J19Xc6B128rdB4hoZ0JGtKaQ9wRY6fOnHnNub3cpzYV\nOJGQ0M4EDemi8rmNR3WzSiYWOJGQfGFngydBQ9p/XMvxV/oWOJGQfOGqnSdBQyqf3nJ8XccC\nJxKSL4TkSdCQDjqn5fjMgwucSEi+EJInQUOaWPKjHfmj9691kwucSEi+EJInQUPafJyrPmXs\nhMsuGFbphhRKhZB8ISRPwn4faedNAzok30YqP/7ndYXOIyRf2NngSfAtQtv/+vzzK3bu6T1L\nn2v2fUJC+5LZXrtNb+52x8oy18rfJJ4DCCVsSMtGHXTSrPwndZMLPQqf2qGdCRrSnytcZbn7\nXG5zECFlgp0NngQNaXT5ww07biof/H5ESBnhqp0nQUPqe37y54KOo+oIKSOE5EnYLULX5m7u\ncVcQUkYIyZOgIfUZk7+d4mYSUjYIyZOgIV1RcnttcttwgbvyckLKAiF5EjSkDQe64bmDhiuc\nI6QssLPBk7DfR3r321c2Hj10GCFBEX6LECCAkAABhGQLOxs8ISRbuGrnCSHZQkieEJIthOQJ\nIdlCSJ4Qki2E5Akh2cLOBk8ICRBASIAAQgIEEJIt7GzwhJBs4aqdJ4RkCyF5Qki2EJInhGQL\nIXlCSLYQkieEZAs7GzwhJEAAIQECCAkQQEi2sLPBE0Kyhat2nhCSLYTkCSHZQkieEJIthOQJ\nIdlCSJ4Qki3sbPCEkAABhAQIICRAACHZws4GTwjJFq7aeUJIthCSJ4RkCyF5Qki2EJInhGQL\nIXlCSLaws8ETQgIEEBIggJAAAYRkCzsbPCEkW7hq5wkh2UJInhCSLYTkCSHZQkieEJIthOQJ\nIdnCzgZPCAkQQEiAAEICBBCSLexs8ISQbOGqnSeEZAsheUJIthCSJ4RkCyF5Qki2EJInhGQL\nOxs8ISRAACEBAggJEEBItrCzwRNCsoWrdp4Qki2E5Akh2UJInhCSLYTkCSHZQkieEJIt7Gzw\nhJAAAYQECCAkQAAh2cLOBk8IyRau2nlCSLYQkieEZAsheUJIthCSJ4RkCyF5Qki2sLPBE0IC\nBBASIICQAAGEZAs7GzwhJFu4aucJIdlCSJ4Qki2E5Akh2UJInhCSLYTkCSHZws4GTwgJEEBI\ngABCAgQQki3sbPCEkGzhqp0nhGQLIXlCSLYQkieEZAsheUJIthCSJ4RkCzsbPCEkQAAhAQII\nCRBASLaws8ETQrKFq3aeEFJYs4dnq1evjAcwO+v/Ap4QUlhj+0/O1Ne+lu3z99f6ikhIYVn/\n1Ert/AkpLLULKSW18yeksNQupJTUzp+QwlK7kFJSO39CCkvtQkpJ7fwJKSy1CykltfMnpLDU\nLqSU1M6fkMJSu5BSUjt/QgpL7UJKSe38swhp57ML3yh8BiFppXb+QUOatjD582c1zrmBSwqd\nSEhaqZ1/0JDc5PiPR13FFy450XVdWeBEQtJK7fzDh3RE1+Xxnw+VXFjgRELSSu38g4e03n0/\nd3xW7wInEpJWaucfPKRV7t7c8TXlBU4kJK3Uzj94SHVdZ+SOx3UvcCIhaaV2/mFDOnfxinen\nHP5BfPhK1RkFTiQkrdTOP2xIeQ9G0eyq0mcLnEhIWqmdf9CQfnnz1IkXnDVsQRTN6v37QicS\nklZq55/RFqH36v/urq1Xt/xk/0hCUkrt/DPba7dhxW53rP/q2c0Gur9JPMfHkNqFlJLa+WcW\n0uRCj8KndlqpnT8hhaV2IaWkdv6EFJbahZSS2vkHDWlgK/sTkkVq5x80pNLSimYdCMkitfMP\nGtLk6pZLdXxqZ5La+QcNqfbTg2qbjgnJJLXzD3uxYXmnq5oOCckktfMPfNVu68amo6dmFDiN\nkLRSO39+i1BYahdSSmrnT0hhqV1IKamdPyGFpXYhpaR2/oQUltqFlJLa+RNSWGoXUkpq509I\nYaldSCmpnT8hhaV2IaWkdv6EFJbahZSS2vkTUlhqF1JKaudPSGGpXUgpqZ0/IYWldiGlpHb+\nhBSW2oWUktr5E1JYahdSSmrnT0hhqV1IKamdPyGFpXYhpaR2/oQUltqFlJLa+RNSWGoXUkpq\n509IYaldSCmpnT8hhaV2IaWkdv6EFJbahZSS2vkTUlhqF1JKaudPSGGpXUgpqZ0/IYWldiGl\npHb+hBSW2oWUktr5E1JYahdSSmrnT0hhqV1IKamdPyGFpXYhpaR2/oQUltqFlJLa+RNSWGoX\nUkpq509IYaldSCmpnT8hhaV2IaWkdv6EFJbahZSS2vkTUlhjj7zYtCMJqQUhfXRjnXGE1IKQ\nPjpCyvq/gCeEFBYhZf1fwBNCCmvsGc+ZdgYhtSCkj07tVauU1M6fkMJSu5BSUjt/QgpL7UJK\nSe38CSkstQspJbXzJ6Sw1C6klNTOn5DCUruQUlI7f0IKS+1CSknt/AkpLLULKSW18yeksNQu\npJTUzp+QwlK7kFJSO39CCkvtQkpJ7fwJKSy1CykltfMnpLDULqSU1M6fkMJSu5BSUjt/QgpL\n7UJKSe38CSkstQspJbXzJ6Sw1C6klNTOn5DCUruQUlI7f0IKS+1CSknt/AkpLLULKSW18yek\nsNQupJTUzp+QwlK7kFJSO39CCkvtQkpJ7fwJKSy1CykltfMnpLDULqSU1M6fkMJSu5BSUjt/\nQgpL7UJKSe38CSkstQspJbXzJ6Sw1C6klNTOn5DCUruQUlI7f0IKS+1CSknt/AkpLLULKSW1\n8yeksNQupJTUzp+QwlK7kFJSO39CCkvtQkpJ7fwJKSy1CykltfMnpLDULqSU1M6fkMJSu5BS\nUjt/QgpL7UJKSe38CSkstQspJbXzJ6Sw1C6klNTOn5DCUruQUlI7f0IKa7wzbnzW/wU8IaSw\nNsw3bkPW/wU8ISRAACHZMnVq1iNQipBsUfvFftYIyRZC8oSQbCEkTwjJFkLyhJBsueiirEeg\nFCHZsmZN1iNQipAAAYQECCAkQAAh2cLOBk8IyRYuf3tCSLYQkieEZAsheUJIthCSJ4RkCzsb\nPCEkW9jZ4AkhAQIICRBASIAAQrKFnQ2eEJItXP72hJBsISRPCMkWQvKEkGwhJE8IyRZ2NnhC\nSLaws8ETQgIEEBIggJAAAYRkCzsbPCEkW7j87Qkh2UJInoQOqeH1+fPmLVj1/5xFSL4Qkidh\nQ9r0nX3z/yTvgddvK3QeIflCSJ4EDWnNIe6IsVNnzrzm3F7uU5sKnEhIvrCzwZOgIV1UPrfx\nqG5WycQCJxKSL+xs8CRoSPuPazn+St8CJxIS2pmgIZVPbzm+rmOBEwkJ7UzQkA46p+X4zIML\nnEhIaGeChjSx5Ec78kfvX+smFziRkHxhZ4MnQUPafJyrPmXshMsuGFbphhRKhZB84fK3J2G/\nj7TzpgEdkm8jlR//87pC5xGSL4TkSfAtQtv/+vzzK3bu4R1v9Tu0WU9C8oSQPMlsr92GFbvd\nUXvPHc2ucntKDcUjJE8yC2lyoUd5mpA8YWeDJ4RkCzsbPCEkQEDQkAa2sj8hQZGgIZWWVjTr\nQEhQJGhIk6tbLtXxqV0m2NngSdCQaj89qLbpmJAyweVvT8JebFje6aqmQ0LKBCF5Eviq3daN\nTUdPzShwGiH5QkiefDx/ixAh+UJInhCSLexs8ISQbGFngyeEBAggJEAAIQECCMkWdjZ4Qki2\ncPnbE0KyhZA8ISRbCMkTQrKFkDwhJFvY2eAJIdnCzgZPCAkQQEiAAEICBBCSLexs8ISQbOHy\ntyeEZAsheUJIthCSJ4RkCyF5Qki2sLPBE0KyhZ0NnhASIICQAAGEBAggJFvY2eAJIdnC5W9P\nCMkWQvKEkGwhJE8IyRZC8oSQbGFngyeEZAs7GzwhJEAAIQECCAkQQEi2sLPBE0KyhcvfnhCS\nLYTkCSHZQkieEJIthOQJIdnCzgZPCMkWdjZ4QkiAAEICBBASIICQbGFngyeEZAuXvz0hJFsI\nyRNCsoWQPCEkWwjJE0KyhZ0NnhCSLexs8ISQAAGEBAggJEAAIdnCzgZPCMkWLn97Qki2EJIn\nhGQLIXlCSLYQkieEZAs7GzwhJFvY2eAJIQECCAkQQEiAAEKyhZ0NnhCSLVz+9oSQbCEkTwjJ\nFkLyhJBsISRPCKl9mVZTnIqKIh9gWtZ/Ax9ThNS+rJlfnN/8psgHYGfEnhESIICQAAGEBAgg\nJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAgg\nJEAAIQECPp4hLXZAO7P4Qy9z/yFFS58D2pWlH36VBwgJ0I+QAAGEBAggJEAAIQECCAkQQEiA\nAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiA\nAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiA\nAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiA\nAEICBHWvY7MAAADiSURBVBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAgg\nJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAgg\nJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAgg\nJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAgg\nJEAAIQECCAkQQEiAAEICBBASIOD/AFa+YICxMX8uAAAAAElFTkSuQmCC",
      "text/plain": [
       "Plot with title “Radius mean boxplot”"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# find outlier points for the radius_mean variable in the data.\n",
    "boxplot.stats(nas_cervical$radius_mean)$out\n",
    "\n",
    "boxplot(nas_cervical$radius_mean, main=\"Radius mean boxplot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also plot data that meets a certain condition. The next plot only uses data for which the diagnosis variable is 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Attaching package: ‘dplyr’\n",
      "\n",
      "\n",
      "The following objects are masked from ‘package:stats’:\n",
      "\n",
      "    filter, lag\n",
      "\n",
      "\n",
      "The following objects are masked from ‘package:base’:\n",
      "\n",
      "    intersect, setdiff, setequal, union\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# import for filtering functions\n",
    "library(dplyr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# select data with diagnoses of 1\n",
    "melignant <- dplyr::filter(nas_cervical, diagnosis==1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>27.22</li><li>28.11</li><li>27.42</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 27.22\n",
       "\\item 28.11\n",
       "\\item 27.42\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 27.22\n",
       "2. 28.11\n",
       "3. 27.42\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 27.22 28.11 27.42"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAC6FBMVEUAAAABAQECAgIDAwME\nBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUW\nFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJyco\nKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6\nOjo7Ozs8PDw9PT0+Pj4/Pz9AQEBCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExN\nTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5f\nX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhqampra2tsbGxtbW1ubm5vb29wcHBxcXFy\ncnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx+fn5/f3+AgICBgYGCgoKDg4OEhISF\nhYWGhoaHh4eIiIiJiYmKioqLi4uNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eY\nmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamq\nqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm7u7u8vLy9\nvb2+vr6/v7/AwMDCwsLDw8PExMTFxcXGxsbHx8fJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR\n0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODi4uLj4+Pk\n5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX2\n9vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7///8OoQXVAAAACXBIWXMAABJ0AAASdAHeZh94\nAAAdRklEQVR4nO3dfYBVZZ3A8WeAmUEQEVYtBURTUEuLxJfK1zQzCNTctrR2wZekEtANF0ml\nlZQydX2rbHVN0zVfwFJqlURLLdEV1tQM1KUkTcVMFJmGOf/uOffOO8PdR+/v9xzn9/t+/uDe\nuXPuvefB35eZOTxRyADULZR9AoAFhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRA\nACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRA\nACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQUip3hDAwv7kthOaE\n7/rO3y7xifZ3hFS/fw/thuz2j7/a4lGEZBoh1a8zpMKcLR3VHtKqSy+9IuG5iYf0wsDwVF1n\nZBQh1a8I6ZOTJ0/+5F5FSTdu4aj2kBITD+nyQEh9IaT6FSG9Wrl3/1YhfHQLRxkJ6SBC6hMh\n1a8rpGxGCFvlN203H7X9oGEHXN5aefC6iUNGTFr+kx4/I50TwoHF5zry+tvVR2w/aPv9Lnyp\n62WvDeGQ7LYJW409pyV7csq2Qz/x28rDv5uxe/Ow/S77W9brfSqH//LIbYcevLTbud1WnNDi\nQ7YZdvh91Qdeu+DAEY07HPUfxVOWNoSwLL9d954QZmfXhPD+7JaDthl25INZ1i2k7s+YXP3+\ndbb0b2H/R0j16xbSeSEMym++0P4D06fb8g++VrnbfG6tkFoOb3/G+1Z3vuyPQ9j71obiwa8+\nu11xs/1f8kcXDa4e+PENvd6nOPyepuKjgb/oOrf87bb+XuWYAUuKj1eMan/KR1/OPzozhN3z\n1zklf+qG7MYQdvpO5XOD7uk60Z7PIKQtIaT6dQsp72GvLLsrH9vvPn7toBBuzbLfFFO/+M6j\nB9UKKZ/1PW9+6O7jQzis82XzI3ccPfX04fkTPrXT7APzl7k0y57Nv3k86+nlh4Uwr9f7FIfv\nMmHeUflxB2TdX6R52Kk3X5S/ypiN+deevIpdv/eTuflTJueffWuf4nXuawjNK7PslvzQ5mk3\nX7h1CGNbOk+05zOe+Gn+8jf96tkkv6/9CiHVryOk1lVfze9dnmVXTp5c/Jk9NYR/yrIv5V9K\n1uffu+1ZK6TpIVyc37SccPq3NnW8bH5k+Gz+JSi/GfxMtnGPEKZk2ekhHJ5/7qWtw7ANPd+n\nOPyQDZWvUgNasu4v8uX89vb89udZ9q8hbPN8/uGP8g8fyW//pzkM+s34EC5pP/TkrPKlLdzT\neaK9nvGnwM9IfSKk+vW4/H3oxs7HZ4ZwVJbtVR3PbEGtkGaFsPMNL/R82WKwH87jag7hhPzD\nOSFMyLLdQjhnQ+7QEH7R832Kw4sfeH6R3/5vjxd5PL9t3SaE87LsQyFMLx5uHRHC+cWdS0IY\nEsKRbe2Hrshv/zYshG90nmivZxDSFhBS/bqFNPJfih9dsqXHvK+58vERWZZ/L7aweGxxrZAe\nG1Icvdspi1q7XrYY7Dfz29HVF7g4/3zWNqDrvS7r+T7F4X/Nj1vVnk7nizRWvsbtWwTRNqj6\nlS/LPlqtM2s7sjjr53sc+uEQTuo40d7PIKQtIKT6FSFNOuaYY/I/s4+tPPDd/IGhe31ou2LA\n26ojn2V317xqd98HqnXs8lDny3b8tJ9/EfpefnNFEdL6bl/75vd4n87D/9A7pG0rdw4O4XOV\np19d+TDv59OVOz/IHzq6j0PbX6/3MwhpCwipfh0/I12b396e376ef3k5Mf9a8pXKgA8O4cLi\nqFt7hzShePTqjr9danvo/KOHh+rPU1V9hdQ2MIR/6zig5/tsMaTm4vu24svMyZWvL9+pPH5g\nCF8obteOLJq8ofv7VQ/t9hWp+zMIaQsIqX4dIbV9LIQd/5Jlv8w/fiz/+IjKgI+vXArIsnk9\nQloYwg7FgM/o/te0rT8dUbkkUNVXSMWrdV577vk+WwypMvjFDz75jzgT2vtpyT/8dnHn6BCm\nNoTha9oPXVUcunXl0PbX6/UMQtoCQqpf5+XvFfnXiy/lP7lUrxI8kf88c2iWTcu/Y1qXf/kY\n3SOk4iLYXVn21JDKo29eOH1q5ceTo0K4o+Nl+wxpZgg7vZEnd+JJZ/+x5/tsOaQzs8q17eJS\nxPn594J/yCrf0DU8nVW+CR3/1qkhfLyteujZWfUy4bLO1+v1jBfyT255Y65jhFS/rr9Hmp1P\n2/3Z8/lkT3n8zlF7hLDNr1+8P//sxB9fv3/+5/yArHM+n2nIB3TGnJEfq+aV/7l//M8f+eW/\nNobmFztets+QntkqhIN+ds9nQvhAa8/36TukPKCBTWf+9OJtQtijNctezXPe/apFZzWHcGr+\n2d8PDeG+ysaGSyvvN6hx/n3f/bvqoe2v1+sZrY0hHHLrPYl+Z/sRQqpfV0iv7Zj/Cb+h+LKR\n2+m5nSqXBE6qfDT0yvyXTV15fLny6LgH8vTyrwaPj26/gjDg2s6X7TOk7LbqZbow6qms5/v0\nHdKN+beQF1QOG1zZ+NO5T+EzG/IqPlK5Ppf9Z/7ZJ4r3G3F216Gb7Wz4TOWC5KeKu5PVf0/7\nHUKqX7edDTeFYqtAy7ffv9WoU5/Plu4xaPSPs00X79G0w9//9onq5emO+WxdOK5p1Gkv/jF/\nNP9WLXthwX7vaRyy52kru16275CyJ0/etXnI3ufk3y32fJ++Q7omhD2z6yYMHjF1RfWB1765\n//DGHY+7s7i/IITtXynufDKEfVvyFxiaXV0curL72/d4Rpb98dhtB+96gcLvYj9HSGjH/5Kv\nHoSEdoRUD0JCO0KqByGhHSHVg5DQjpDqQUiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiA\nAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiA\nAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBCQIacUjQL+y4u1PuX5IywPQzyx/22Ou\nH9KDYaP6ewCCNoYH3/ZzCAnohZAAAYQECCAkQAAhAQIICRBASIAAQgIEEBIggJAAAYQECCAk\nQAAhAQIICRBASIAAQgIEEJInK+dOmjR3ZdlnYRIhObJw4KFnnXXowIVln4dFhOTHbU2Li5vF\nTbeXfSYGEZIfH55TvZ2zb7nnYRIhufF6+HX1zkMN68s9E4sIyY214enqnd+FteWeiUWE5EbL\n4J9V7ywZ3FLumVhESH4cf1RbcbPpE8eXfSYGEZIfvxt+4p+y7E8nDn+67DMxiJAceXTvsMsu\nYe9Hyz4PiwjJk02P/vCHj2wq+yxMIiRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEB\nAggJEEBIgABC8mT1BZ///AWryz4LkwjJkSubJsyYMaHpyrLPwyJC8mPJoOuLm+sHLSn7TAwi\nJD8OmFW9nXlAuedhEiG5sb7hgeqdBxreKPdMLCIkN57v+Oe4ng7Pl3smFhGSGxub7s6yTZuy\n7L+a+N0VR0h+TJly2X5Dhux32aenlH0mBhGSH48OHHz23Xef3Tzwv8s+E4MIyY9vjhzTtM8+\nTWNGfrPsMzGIkNxoG3N5y7LLL1/WctmYtrLPxR5CcuOV0P7/1bcyvFLumVhESG68GJ6o3vlt\n+HO5Z2IRIbmxabsfVu9ctx3/2Ko4QvLjzHGvFjevjvvnss/EIELy4y8f3OOm1atvHP/B18o+\nE4MIyZHXZm4bwrYz6UgBIfmyZk3ZZ2AUIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQEC\nCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQEC\nCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQECCAkQQEiAAEICBBASIICQAAGEBAggJEAAIQEC\nCAkQQEiAAELqX66dWJ+xY+t8gWvL/h14lyKk/mX5t+ozcWKdL7C87N+BdylC8mX69LLPwChC\n8oWQlBCSL4SkhJB8ISQlhOTLci4W6CAkQAAhAQIICRBASL5cc03ZZ2AUIfnCVTslhOQLISkh\nJF8ISUkZIW18eNmztY8gJC2EpCRpSAuWFb9+f0QIYeJjtQ4kJC2EpCRpSGFu/suS0HzcjIPC\n8NU1DiQkLexsUJI+pHHDn8x/XdRwUo0DCQn9TPKQ/hy+Xrl/7KgaBxIS+pnkIa0JP6rcP7ex\nxoGEhH4meUitwxdW7p88ssaBhKSFnQ1K0oZ0wvJVL83b/Y387lNDp9Q4kJC0cNVOSdqQqm7P\nspuGDni4xoGEpIWQlCQN6bpL58+eduzh92bZVaPuqnUgIWkhJCUlbRF6fdNmD7Vcf3WnOYSk\nhJCUlLbXbt1zvR5Ys+f7Om0fXpd4D2yGkJSkDWnlpLEHX9VauTu31qt8n5CUsLNBSdKQHmgO\nQxrDYeuK+4QES5KGNLnxjra3Lmncf31GSLAlaUhjvlj8em/TpFZCgi1JQ2o8r3JzQ5hFSCVh\nZ4OSpCGNnlq9nRcuIqRycNVOSdKQZjVc0VLctk0LZ8wkpDIQkpKkIb28cziycqdtVgiEVAZC\nUpL275Fe+soZ7fcW7UZIZSAkJe/Of0WIkLQQkhJC8oWdDUoICRBASIAAQgIEEJIv7GxQQki+\ncNVOCSH5QkhKCMkXQlJCSL4QkhJC8oWQlBCSL+xsUEJIgABCAgQQEiCAkHxhZ4MSQvKFq3ZK\nCMkXQlJCSL4QkhJC8oWQlBCSL4SkhJB8YWeDEkICBBASIICQAAGE5As7G5QQki9ctVNCSL4Q\nkhJC8oWQlBCSL4SkhJB8ISQlhOQLOxuUEBIggJAAAYQECCAkX9jZoISQfOGqnRJC8oWQlBCS\nL4SkhJB8ISQlhOQLISkhJF/Y2aCEkAABhAQIICRAACH5ws4GJYTkC1ftlBCSL4SkhJB8ISQl\nhOQLISkhJF8ISQkh+cLOBiWEBAggJEAAIQECCMkXdjYoISRfuGqnhJB8ISQlhOQLISkhJF8I\nSQkh+UJISgjJF3Y2KCEkQAAhAQIICRBASL6ws0EJIfnCVTslhOQLISkhJF8ISQkh+UJISgjJ\nF0JSQki+sLNBCSEBAggJEEBIgABC8oWdDUoIyReu2ikhJF8ISQkh+UJISgjJF0JSQki+EJIS\nQvKFnQ1KCAkQQEiAAEICBBCSL+xsUEJIvnDVTgkh+UJISgjJF0JSQki+EJISQvKFkJQQki/s\nbFBCSIAAQgIEEBIggJB8YWeDEkLyhat2SgjJF0JSQki+EJISQvKFkJQQki+EpISQfGFngxJC\nAgQQEiCAkAABhOQLOxuUEJIvXLVTQki+EJISQvKFkJQQki+EpISQfCEkJYTkCzsblBASIICQ\nAAGEBAggJF/Y2aCEkHzhqp0SQvKFkJQQki+EpISQfCEkJYTkCyEpISRf2NmghJAAAYQECCAk\nQAAh+cLOBiWE5AtX7ZQQki+EpISQfCEkJYTkCyEpISRfCEkJIfnCzgYlhAQIICRAACEBAgjJ\nF3Y2KCEkX7hqp4SQfCEkJYTkCyEpISRfCEkJIflCSEoIyRd2NighJEAAIQECCAkQQEi+sLNB\nSeqQ2p5ZunjxvWv+n6MISQtX7ZSkDWnd13YIFTuf/2at4whJCyEpSRrS2l3DuOnzL7ro3BN2\nCh9aV+NAQtJCSEqShnRK463t91qvaphd40BC0kJISpKG9N6Tu+5/bkyNAwlJCyEpSRpS4wVd\n97/RVONAQtLCzgYlSUMa+w9d94/ZpcaBhIR+JmlIsxu+81b13vrzwtwaBxIS+pmkIb26bxh2\nxPTTvzrt8CHhkFqpEBL6mbR/j7TxkgkDi79GavzID1prHUdIWtjZoCT5FqENv3/00VUb+/hE\n609u7fQlQlLCVTslpe21W/dcrweee++ITkPCXyXeA5shJCVpQ1o5aezBV1W/qZtb61X41k4L\nISlJGtIDzWFIYzissjmIkEpBSEqShjS58Y62ty5p3H99RkglISQlSUMa88Xi13ubJrUSUknY\n2aAk7Rah8yo3N4RZhARbkoY0emr1dl64iJBgStKQZjVc0VLctk0LZ8wkJBiSNKSXdw5HVu60\nzQqBkMrAzgYlaf8e6aWvnNF+b9FuhFQGrtop4V8R8oWQlBCSL4SkhJB8ISQlhJTWktPKNX58\nySewpOz/AkoIKa3pJQ/ycceV+/7jrX5FJKS0vH9rZXb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJSW\n2UGKZHb9hJSW2UGKZHb9hJSW2UGKZHb9hJTW9CmPuDaFkLoQ0js3PThHSF0I6Z0jpLL/Cygh\npLQIqez/AkoIKa3p+8x1bR9C6kJI75zZq1aRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iRzK6fkNIyO0iR\nzK6/jJA2Przs2dpHEJJVZtefNKQFy4pfvz8ihDDxsVoHEpJVZtefNKQwN/9lSWg+bsZBYfjq\nGgcSklVm158+pHHDn8x/XdRwUo0DCckqs+tPHtKfw9cr948dVeNAQrLK7PqTh7Qm/Khy/9zG\nGgcSklVm1588pNbhCyv3Tx5Z40BCssrs+tOGdMLyVS/N2/2N/O5TQ6fUOJCQrDK7/rQhVd2e\nZTcNHfBwjQMNhzT+NNfGE1KXdxzSdZfOnz3t2MPvzbKrRt1V60C7Id35WefuLPu/gJKStgi9\nvmmzh9ruX9ppttmQYFRpe+1eXtXrgWcGh27elHgPbGb+/LLPwKjSQppb61UeDBsl3gObMfvD\nftkIyRdCUkJIvhCSkqQhTezmvYRUBkJSkjSkAQOaOw0kpDKcckrZZ2BU0pDmDuu6VMe3dqVY\nu7bsMzAqaUgtH96vpeM+IcGStBcbntxqTsddQoIlia/avfZKx737F9Y4jJDQz7w7/xUhQtLC\nzgYlhOQLl7+VEJIvhKSEkHwhJCWE5AshKSEkX9jZoISQfGFngxJCAgQQEiCAkAABhOQLOxuU\nEJIvXP5WQki+EJISQvKFkJQQki+EpISQfGFngxJC8oWdDUoICRBASIAAQgIEEJIv7GxQQki+\ncPlbCSH5QkhKCMkXQlJCSL4QkhJC8oWdDUoIyRd2NighJEAAIQECCAkQQEi+sLNBCSH5wuVv\nJYTkCyEpISRfCEkJIflCSEoIyRd2NighJF/Y2aCEkAABhAQIICRAACH5ws4GJYTkC5e/lRCS\nL4SkhJB8ISQlhOQLISkhJF/Y2aCEkHxhZ4MSQgIEEBIggJAAAYTkCzsblBCSL1z+VkJIvhCS\nEkLyhZCUEJIvhKSEkHxhZ4MSQvKFnQ1KCAkQQEiAAEICBBCSL+xsUEJIvnD5Wwkh+UJISgjJ\nF0JSQki+EJISQvKFnQ1KCMkXdjYoISRAACEBAggJEEBIvrCzQQkh+cLlbyWE5AshKSEkXwhJ\nCSH5QkhKCMkXdjYoISRf2NmghJAAAYQECCAkQAAh+cLOBiWE5AuXv5UQki+EpISQfCEkJYTk\nCyEpISRf2NmghJB8YWeDEkICBBASIICQAAGE5As7G5QQki9c/lZCSL4QkhJC8oWQlBCSL4Sk\nhJB8YWeDEkLyhZ0NSggJEEBIgABCAgQQki/sbFBCSL5w+VsJIflCSEoIyRdCUkJIvhCSEkLy\nhZ0NSgjJF3Y2KCEkQAAhAQIICRBASL6ws0EJIfnC5W8lhOQLISkhJF8ISQkh+UJISgjJF3Y2\nKCEkX9jZoISQAAGEBAggJEAAIfnCzgYlhOQLl7+VEJIvhKSEkHwhJCWE5AshKSGk/mXBiPo0\nN9f5AgvK/h14lyKk/mXt0vrcckudL8DOiL4REiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQ\nEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAt6dIS0PQD+z/G2P\nuX5I2YpHgH5lxduf8gQhAfYREiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQI\nICRAACEBAggJEEBIgABCAgQQEiCAkAABhAQIICRAACEBAggJEEBIgABCAgT8H1FSPZlgbtjm\nAAAAAElFTkSuQmCC",
      "text/plain": [
       "Plot with title “Radius mean boxplot”"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 420,
       "width": 420
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# boxplot result\n",
    "boxplot.stats(melignant$radius_mean)$out\n",
    "boxplot(melignant$radius_mean, main=\"Radius mean boxplot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "\n",
    "## Summary\n",
    "\n",
    "This notebook has covered the basics of looking at and cleaning data. Additionally, some links have been provided to useful libraries!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R with Stan",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}