Car Prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Predicting Car Prices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What the project is about:** In this project, I practice using a machine learning workflow to predict a car's market price using its attributes. The data set I will be working with contains information on various cars. For each car, I have information about the technical aspects of the vehicle such as the motor's displacement, the weight of the car, the miles per gallon, how fast the car accelerates, and more. \n",
    "\n",
    "If you are interested in learning more, you can read more about the data set [here](https://archive.ics.uci.edu/ml/datasets/automobile) and can download it directly from [here](https://archive.ics.uci.edu/ml/machine-learning-databases/autos/imports-85.data). \n",
    "\n",
    "**Goal of the project:** My primary objective here is to explore the fundamentals of machine learning using the k-nearest neighbors algorithm, while applying this algorithm to a real-world context."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to the Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'll start by reading in the dataset and examining the first few rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "pd.options.display.max_columns = 99"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>symboling</th>\n",
       "      <th>normalized-losses</th>\n",
       "      <th>make</th>\n",
       "      <th>fuel-type</th>\n",
       "      <th>aspiration</th>\n",
       "      <th>num-of-doors</th>\n",
       "      <th>body-style</th>\n",
       "      <th>drive-wheels</th>\n",
       "      <th>engine-location</th>\n",
       "      <th>wheel-base</th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "      <th>curb-weight</th>\n",
       "      <th>engine-type</th>\n",
       "      <th>num-of-cylinders</th>\n",
       "      <th>engine-size</th>\n",
       "      <th>fuel-system</th>\n",
       "      <th>bore</th>\n",
       "      <th>stroke</th>\n",
       "      <th>compression-rate</th>\n",
       "      <th>horsepower</th>\n",
       "      <th>peak-rpm</th>\n",
       "      <th>city-mpg</th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>?</td>\n",
       "      <td>alfa-romero</td>\n",
       "      <td>gas</td>\n",
       "      <td>std</td>\n",
       "      <td>two</td>\n",
       "      <td>convertible</td>\n",
       "      <td>rwd</td>\n",
       "      <td>front</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>dohc</td>\n",
       "      <td>four</td>\n",
       "      <td>130</td>\n",
       "      <td>mpfi</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>13495</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>?</td>\n",
       "      <td>alfa-romero</td>\n",
       "      <td>gas</td>\n",
       "      <td>std</td>\n",
       "      <td>two</td>\n",
       "      <td>convertible</td>\n",
       "      <td>rwd</td>\n",
       "      <td>front</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>dohc</td>\n",
       "      <td>four</td>\n",
       "      <td>130</td>\n",
       "      <td>mpfi</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>?</td>\n",
       "      <td>alfa-romero</td>\n",
       "      <td>gas</td>\n",
       "      <td>std</td>\n",
       "      <td>two</td>\n",
       "      <td>hatchback</td>\n",
       "      <td>rwd</td>\n",
       "      <td>front</td>\n",
       "      <td>94.5</td>\n",
       "      <td>171.2</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "      <td>2823</td>\n",
       "      <td>ohcv</td>\n",
       "      <td>six</td>\n",
       "      <td>152</td>\n",
       "      <td>mpfi</td>\n",
       "      <td>2.68</td>\n",
       "      <td>3.47</td>\n",
       "      <td>9.0</td>\n",
       "      <td>154</td>\n",
       "      <td>5000</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>164</td>\n",
       "      <td>audi</td>\n",
       "      <td>gas</td>\n",
       "      <td>std</td>\n",
       "      <td>four</td>\n",
       "      <td>sedan</td>\n",
       "      <td>fwd</td>\n",
       "      <td>front</td>\n",
       "      <td>99.8</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2337</td>\n",
       "      <td>ohc</td>\n",
       "      <td>four</td>\n",
       "      <td>109</td>\n",
       "      <td>mpfi</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>10.0</td>\n",
       "      <td>102</td>\n",
       "      <td>5500</td>\n",
       "      <td>24</td>\n",
       "      <td>30</td>\n",
       "      <td>13950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>164</td>\n",
       "      <td>audi</td>\n",
       "      <td>gas</td>\n",
       "      <td>std</td>\n",
       "      <td>four</td>\n",
       "      <td>sedan</td>\n",
       "      <td>4wd</td>\n",
       "      <td>front</td>\n",
       "      <td>99.4</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2824</td>\n",
       "      <td>ohc</td>\n",
       "      <td>five</td>\n",
       "      <td>136</td>\n",
       "      <td>mpfi</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>8.0</td>\n",
       "      <td>115</td>\n",
       "      <td>5500</td>\n",
       "      <td>18</td>\n",
       "      <td>22</td>\n",
       "      <td>17450</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   symboling normalized-losses         make fuel-type aspiration num-of-doors  \\\n",
       "0          3                 ?  alfa-romero       gas        std          two   \n",
       "1          3                 ?  alfa-romero       gas        std          two   \n",
       "2          1                 ?  alfa-romero       gas        std          two   \n",
       "3          2               164         audi       gas        std         four   \n",
       "4          2               164         audi       gas        std         four   \n",
       "\n",
       "    body-style drive-wheels engine-location  wheel-base  length  width  \\\n",
       "0  convertible          rwd           front        88.6   168.8   64.1   \n",
       "1  convertible          rwd           front        88.6   168.8   64.1   \n",
       "2    hatchback          rwd           front        94.5   171.2   65.5   \n",
       "3        sedan          fwd           front        99.8   176.6   66.2   \n",
       "4        sedan          4wd           front        99.4   176.6   66.4   \n",
       "\n",
       "   height  curb-weight engine-type num-of-cylinders  engine-size fuel-system  \\\n",
       "0    48.8         2548        dohc             four          130        mpfi   \n",
       "1    48.8         2548        dohc             four          130        mpfi   \n",
       "2    52.4         2823        ohcv              six          152        mpfi   \n",
       "3    54.3         2337         ohc             four          109        mpfi   \n",
       "4    54.3         2824         ohc             five          136        mpfi   \n",
       "\n",
       "   bore stroke  compression-rate horsepower peak-rpm  city-mpg  highway-mpg  \\\n",
       "0  3.47   2.68               9.0        111     5000        21           27   \n",
       "1  3.47   2.68               9.0        111     5000        21           27   \n",
       "2  2.68   3.47               9.0        154     5000        19           26   \n",
       "3  3.19   3.40              10.0        102     5500        24           30   \n",
       "4  3.19   3.40               8.0        115     5500        18           22   \n",
       "\n",
       "   price  \n",
       "0  13495  \n",
       "1  16500  \n",
       "2  16500  \n",
       "3  13950  \n",
       "4  17450  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = ['symboling', 'normalized-losses', 'make', 'fuel-type', 'aspiration', 'num-of-doors', 'body-style', \n",
    "        'drive-wheels', 'engine-location', 'wheel-base', 'length', 'width', 'height', 'curb-weight', 'engine-type', \n",
    "        'num-of-cylinders', 'engine-size', 'fuel-system', 'bore', 'stroke', 'compression-rate', 'horsepower', 'peak-rpm', 'city-mpg', 'highway-mpg', 'price']\n",
    "cars = pd.read_csv('imports-85.data', names=cols)\n",
    "\n",
    "cars.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Select only the columns with continuous values from - https://archive.ics.uci.edu/ml/machine-learning-databases/autos/imports-85.names\n",
    "continuous_values_cols = ['normalized-losses', 'wheel-base', 'length', 'width', 'height', 'curb-weight', 'engine-size', 'bore', 'stroke', 'compression-rate', 'horsepower', 'peak-rpm', 'city-mpg', 'highway-mpg', 'price']\n",
    "numeric_cars = cars[continuous_values_cols]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>normalized-losses</th>\n",
       "      <th>wheel-base</th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "      <th>curb-weight</th>\n",
       "      <th>engine-size</th>\n",
       "      <th>bore</th>\n",
       "      <th>stroke</th>\n",
       "      <th>compression-rate</th>\n",
       "      <th>horsepower</th>\n",
       "      <th>peak-rpm</th>\n",
       "      <th>city-mpg</th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>?</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>130</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>13495</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>?</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>130</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>?</td>\n",
       "      <td>94.5</td>\n",
       "      <td>171.2</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "      <td>2823</td>\n",
       "      <td>152</td>\n",
       "      <td>2.68</td>\n",
       "      <td>3.47</td>\n",
       "      <td>9.0</td>\n",
       "      <td>154</td>\n",
       "      <td>5000</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>164</td>\n",
       "      <td>99.8</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2337</td>\n",
       "      <td>109</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>10.0</td>\n",
       "      <td>102</td>\n",
       "      <td>5500</td>\n",
       "      <td>24</td>\n",
       "      <td>30</td>\n",
       "      <td>13950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>164</td>\n",
       "      <td>99.4</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2824</td>\n",
       "      <td>136</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>8.0</td>\n",
       "      <td>115</td>\n",
       "      <td>5500</td>\n",
       "      <td>18</td>\n",
       "      <td>22</td>\n",
       "      <td>17450</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  normalized-losses  wheel-base  length  width  height  curb-weight  \\\n",
       "0                 ?        88.6   168.8   64.1    48.8         2548   \n",
       "1                 ?        88.6   168.8   64.1    48.8         2548   \n",
       "2                 ?        94.5   171.2   65.5    52.4         2823   \n",
       "3               164        99.8   176.6   66.2    54.3         2337   \n",
       "4               164        99.4   176.6   66.4    54.3         2824   \n",
       "\n",
       "   engine-size  bore stroke  compression-rate horsepower peak-rpm  city-mpg  \\\n",
       "0          130  3.47   2.68               9.0        111     5000        21   \n",
       "1          130  3.47   2.68               9.0        111     5000        21   \n",
       "2          152  2.68   3.47               9.0        154     5000        19   \n",
       "3          109  3.19   3.40              10.0        102     5500        24   \n",
       "4          136  3.19   3.40               8.0        115     5500        18   \n",
       "\n",
       "   highway-mpg  price  \n",
       "0           27  13495  \n",
       "1           27  16500  \n",
       "2           26  16500  \n",
       "3           30  13950  \n",
       "4           22  17450  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_cars.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I can't have any missing values if I want to use them for predictive modeling. Based on the data set preview above, I can see that the `normalized-losses` column contains missing values represented using `\"?\"`. I'll go ahead and replace these values and look for the presence of missing values in other numeric columns. I will also [rescale](https://en.wikipedia.org/wiki/Feature_scaling#Rescaling) the values in the numeric columns so they all range from 0 to 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>normalized-losses</th>\n",
       "      <th>wheel-base</th>\n",
       "      <th>length</th>\n",
       "      <th>width</th>\n",
       "      <th>height</th>\n",
       "      <th>curb-weight</th>\n",
       "      <th>engine-size</th>\n",
       "      <th>bore</th>\n",
       "      <th>stroke</th>\n",
       "      <th>compression-rate</th>\n",
       "      <th>horsepower</th>\n",
       "      <th>peak-rpm</th>\n",
       "      <th>city-mpg</th>\n",
       "      <th>highway-mpg</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>130</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>13495</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>88.6</td>\n",
       "      <td>168.8</td>\n",
       "      <td>64.1</td>\n",
       "      <td>48.8</td>\n",
       "      <td>2548</td>\n",
       "      <td>130</td>\n",
       "      <td>3.47</td>\n",
       "      <td>2.68</td>\n",
       "      <td>9.0</td>\n",
       "      <td>111</td>\n",
       "      <td>5000</td>\n",
       "      <td>21</td>\n",
       "      <td>27</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>94.5</td>\n",
       "      <td>171.2</td>\n",
       "      <td>65.5</td>\n",
       "      <td>52.4</td>\n",
       "      <td>2823</td>\n",
       "      <td>152</td>\n",
       "      <td>2.68</td>\n",
       "      <td>3.47</td>\n",
       "      <td>9.0</td>\n",
       "      <td>154</td>\n",
       "      <td>5000</td>\n",
       "      <td>19</td>\n",
       "      <td>26</td>\n",
       "      <td>16500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>164</td>\n",
       "      <td>99.8</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.2</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2337</td>\n",
       "      <td>109</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>10.0</td>\n",
       "      <td>102</td>\n",
       "      <td>5500</td>\n",
       "      <td>24</td>\n",
       "      <td>30</td>\n",
       "      <td>13950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>164</td>\n",
       "      <td>99.4</td>\n",
       "      <td>176.6</td>\n",
       "      <td>66.4</td>\n",
       "      <td>54.3</td>\n",
       "      <td>2824</td>\n",
       "      <td>136</td>\n",
       "      <td>3.19</td>\n",
       "      <td>3.40</td>\n",
       "      <td>8.0</td>\n",
       "      <td>115</td>\n",
       "      <td>5500</td>\n",
       "      <td>18</td>\n",
       "      <td>22</td>\n",
       "      <td>17450</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  normalized-losses  wheel-base  length  width  height  curb-weight  \\\n",
       "0               NaN        88.6   168.8   64.1    48.8         2548   \n",
       "1               NaN        88.6   168.8   64.1    48.8         2548   \n",
       "2               NaN        94.5   171.2   65.5    52.4         2823   \n",
       "3               164        99.8   176.6   66.2    54.3         2337   \n",
       "4               164        99.4   176.6   66.4    54.3         2824   \n",
       "\n",
       "   engine-size  bore stroke  compression-rate horsepower peak-rpm  city-mpg  \\\n",
       "0          130  3.47   2.68               9.0        111     5000        21   \n",
       "1          130  3.47   2.68               9.0        111     5000        21   \n",
       "2          152  2.68   3.47               9.0        154     5000        19   \n",
       "3          109  3.19   3.40              10.0        102     5500        24   \n",
       "4          136  3.19   3.40               8.0        115     5500        18   \n",
       "\n",
       "   highway-mpg  price  \n",
       "0           27  13495  \n",
       "1           27  16500  \n",
       "2           26  16500  \n",
       "3           30  13950  \n",
       "4           22  17450  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_cars = numeric_cars.replace('?', np.nan)\n",
    "numeric_cars.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "normalized-losses    41\n",
       "wheel-base            0\n",
       "length                0\n",
       "width                 0\n",
       "height                0\n",
       "curb-weight           0\n",
       "engine-size           0\n",
       "bore                  4\n",
       "stroke                4\n",
       "compression-rate      0\n",
       "horsepower            2\n",
       "peak-rpm              2\n",
       "city-mpg              0\n",
       "highway-mpg           0\n",
       "price                 4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric_cars = numeric_cars.astype('float')\n",
    "numeric_cars.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "normalized-losses    37\n",
       "wheel-base            0\n",
       "length                0\n",
       "width                 0\n",
       "height                0\n",
       "curb-weight           0\n",
       "engine-size           0\n",
       "bore                  4\n",
       "stroke                4\n",
       "compression-rate      0\n",
       "horsepower            2\n",
       "peak-rpm              2\n",
       "city-mpg              0\n",
       "highway-mpg           0\n",
       "price                 0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Because `price` is the column I want to predict, I'll remove any rows with missing `price` values.\n",
    "numeric_cars = numeric_cars.dropna(subset=['price'])\n",
    "numeric_cars.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Replace missing values in other columns using column means.\n",
    "numeric_cars = numeric_cars.fillna(numeric_cars.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "normalized-losses    0\n",
       "wheel-base           0\n",
       "length               0\n",
       "width                0\n",
       "height               0\n",
       "curb-weight          0\n",
       "engine-size          0\n",
       "bore                 0\n",
       "stroke               0\n",
       "compression-rate     0\n",
       "horsepower           0\n",
       "peak-rpm             0\n",
       "city-mpg             0\n",
       "highway-mpg          0\n",
       "price                0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Confirm that there are no more missing values!\n",
    "numeric_cars.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Normalize all columnns to range from 0 to 1 except the target column.\n",
    "price_col = numeric_cars['price']\n",
    "numeric_cars = (numeric_cars - numeric_cars.min())/(numeric_cars.max() - numeric_cars.min())\n",
    "numeric_cars['price'] = price_col"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Univariate Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the dataset has been cleaned, I'll start with some univariate k-nearest neighbors models. Starting with simple models before moving to more complex models helps me to structure my code workflow and understand the features better.\n",
    "\n",
    "First, I'll create a function that encapsulates the training and simple validation process. From there, I will use this function to train and test univariate models using the different numeric columns in the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "engine-size          3238.462830\n",
       "horsepower           4037.037713\n",
       "curb-weight          4401.118255\n",
       "highway-mpg          4630.026799\n",
       "width                4704.482590\n",
       "city-mpg             4766.422505\n",
       "length               5427.200961\n",
       "wheel-base           5461.553998\n",
       "compression-rate     6610.812153\n",
       "bore                 6780.627785\n",
       "normalized-losses    7330.197653\n",
       "peak-rpm             7697.459696\n",
       "stroke               8006.529545\n",
       "height               8144.441043\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsRegressor\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "def knn_train_test(train_col, target_col, df):\n",
    "    knn = KNeighborsRegressor()\n",
    "    np.random.seed(1)\n",
    "        \n",
    "    # Randomize order of rows in data frame.\n",
    "    shuffled_index = np.random.permutation(df.index)\n",
    "    rand_df = df.reindex(shuffled_index)\n",
    "\n",
    "    # Divide number of rows in half and round.\n",
    "    last_train_row = int(len(rand_df) / 2)\n",
    "    \n",
    "    # Select the first half and set as training set.\n",
    "    # Select the second half and set as test set.\n",
    "    train_df = rand_df.iloc[0:last_train_row]\n",
    "    test_df = rand_df.iloc[last_train_row:]\n",
    "    \n",
    "    # Fit a KNN model using default k value.\n",
    "    knn.fit(train_df[[train_col]], train_df[target_col])\n",
    "    \n",
    "    # Make predictions using model.\n",
    "    predicted_labels = knn.predict(test_df[[train_col]])\n",
    "\n",
    "    # Calculate and return RMSE.\n",
    "    mse = mean_squared_error(test_df[target_col], predicted_labels)\n",
    "    rmse = np.sqrt(mse)\n",
    "    return rmse\n",
    "\n",
    "rmse_results = {}\n",
    "train_cols = numeric_cars.columns.drop('price')\n",
    "\n",
    "# For each column (minus `price`), train a model, return RMSE value\n",
    "# and add to the dictionary `rmse_results`.\n",
    "for col in train_cols:\n",
    "    rmse_val = knn_train_test(col, 'price', numeric_cars)\n",
    "    rmse_results[col] = rmse_val\n",
    "\n",
    "# Create a Series object from the dictionary so \n",
    "# I can easily view the results, sort, etc\n",
    "rmse_results_series = pd.Series(rmse_results)\n",
    "rmse_results_series.sort_values()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like I'll need to modify the `knn_train_test()` function I wrote to accept a parameter for the `k` value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'bore': {1: 8602.58848450066,\n",
       "  3: 6984.239489480916,\n",
       "  5: 6780.627784685976,\n",
       "  7: 6878.097965921532,\n",
       "  9: 6866.808502038413},\n",
       " 'city-mpg': {1: 5901.143574354764,\n",
       "  3: 4646.746408727155,\n",
       "  5: 4766.422505090134,\n",
       "  7: 5232.523034167316,\n",
       "  9: 5465.209492527533},\n",
       " 'compression-rate': {1: 8087.205346523092,\n",
       "  3: 7375.063685578359,\n",
       "  5: 6610.812153159129,\n",
       "  7: 6732.801282941515,\n",
       "  9: 7024.485525463435},\n",
       " 'curb-weight': {1: 5264.290230758878,\n",
       "  3: 5022.318011757233,\n",
       "  5: 4401.118254793124,\n",
       "  7: 4330.608104418053,\n",
       "  9: 4632.044474454401},\n",
       " 'engine-size': {1: 3258.4861059962027,\n",
       "  3: 2840.562805643501,\n",
       "  5: 3238.4628296477176,\n",
       "  7: 3563.086774256415,\n",
       "  9: 3831.8244149840766},\n",
       " 'height': {1: 8904.04645636071,\n",
       "  3: 8277.609643045525,\n",
       "  5: 8144.441042663747,\n",
       "  7: 7679.598124393773,\n",
       "  9: 7811.03606291223},\n",
       " 'highway-mpg': {1: 6025.594966720739,\n",
       "  3: 4617.305019788554,\n",
       "  5: 4630.026798588056,\n",
       "  7: 4796.061440186946,\n",
       "  9: 5278.358056953987},\n",
       " 'horsepower': {1: 4170.054848037801,\n",
       "  3: 4020.8492630885394,\n",
       "  5: 4037.0377131537603,\n",
       "  7: 4353.811860277134,\n",
       "  9: 4515.135617419103},\n",
       " 'length': {1: 4628.45550121557,\n",
       "  3: 5129.8358210721635,\n",
       "  5: 5427.2009608367125,\n",
       "  7: 5313.427720847974,\n",
       "  9: 5383.054514833446},\n",
       " 'normalized-losses': {1: 7846.750605148984,\n",
       "  3: 7500.5698123109905,\n",
       "  5: 7330.197653434445,\n",
       "  7: 7756.421586234123,\n",
       "  9: 7688.096096891432},\n",
       " 'peak-rpm': {1: 9511.480067750124,\n",
       "  3: 8537.550899973421,\n",
       "  5: 7697.4596964334805,\n",
       "  7: 7510.294160083481,\n",
       "  9: 7340.041341263401},\n",
       " 'stroke': {1: 9116.495955406906,\n",
       "  3: 7338.68466990294,\n",
       "  5: 8006.529544647101,\n",
       "  7: 7803.937796804327,\n",
       "  9: 7735.554366079291},\n",
       " 'wheel-base': {1: 4493.734068810494,\n",
       "  3: 5120.161506064513,\n",
       "  5: 5461.553997873057,\n",
       "  7: 5448.1070513823315,\n",
       "  9: 5738.405685192312},\n",
       " 'width': {1: 4559.257297950061,\n",
       "  3: 4606.413692169901,\n",
       "  5: 4704.482589704386,\n",
       "  7: 4571.485046194653,\n",
       "  9: 4652.914172067787}}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def knn_train_test(train_col, target_col, df):\n",
    "    np.random.seed(1)\n",
    "        \n",
    "    # Randomize order of rows in data frame.\n",
    "    shuffled_index = np.random.permutation(df.index)\n",
    "    rand_df = df.reindex(shuffled_index)\n",
    "\n",
    "    # Divide number of rows in half and round.\n",
    "    last_train_row = int(len(rand_df) / 2)\n",
    "    \n",
    "    # Select the first half and set as training set.\n",
    "    # Select the second half and set as test set.\n",
    "    train_df = rand_df.iloc[0:last_train_row]\n",
    "    test_df = rand_df.iloc[last_train_row:]\n",
    "    \n",
    "    k_values = [1,3,5,7,9]\n",
    "    k_rmses = {}\n",
    "    \n",
    "    for k in k_values:\n",
    "        # Fit model using k nearest neighbors.\n",
    "        knn = KNeighborsRegressor(n_neighbors=k)\n",
    "        knn.fit(train_df[[train_col]], train_df[target_col])\n",
    "\n",
    "        # Make predictions using model.\n",
    "        predicted_labels = knn.predict(test_df[[train_col]])\n",
    "\n",
    "        # Calculate and return RMSE.\n",
    "        mse = mean_squared_error(test_df[target_col], predicted_labels)\n",
    "        rmse = np.sqrt(mse)\n",
    "        \n",
    "        k_rmses[k] = rmse\n",
    "    return k_rmses\n",
    "\n",
    "k_rmse_results = {}\n",
    "\n",
    "# For each column (minus `price`), train a model, return RMSE value\n",
    "# and add to the dictionary `rmse_results`.\n",
    "train_cols = numeric_cars.columns.drop('price')\n",
    "for col in train_cols:\n",
    "    rmse_val = knn_train_test(col, 'price', numeric_cars)\n",
    "    k_rmse_results[col] = rmse_val\n",
    "\n",
    "k_rmse_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's visualize the results using a line plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVNX/x/EXq+CGOyigmIKKiqAiauaugHtmaJrLN7XU\nyq2yvbRvP7Fvau5pLrlkLqW54x5pbqjgvuACCSOgIrIzLHN/f1wdd0OcYQb9PB+P+2C4zNz5DOV9\nc+459xwLRVEUhBBCiKdkaeoChBBCFE0SIEIIIQpEAkQIIUSBSIAIIYQoEAkQIYQQBSIBIoQQokCM\nHiBubm40aNAAHx8fmjRpAkBSUhIdO3akVq1a+Pv7k5ycrH9+cHAw7u7u1KlTh+3bt+v3h4eH4+Xl\nhYeHB6NHjzZ22UIIIf6F0QPE0tKS0NBQIiIiCAsLA2DSpEm0b9+e8+fP07ZtW4KDgwE4c+YMq1ev\n5uzZs4SEhDBixAju3KYyfPhwFi5cSGRkJJGRkWzbts3YpQshhHgCoweIoijodLr79q1fv56BAwcC\nMHDgQNatWwfAhg0b6NOnD9bW1ri5ueHu7k5YWBjx8fGkpqbi6+sLwIABA/SvEUIIYRpGDxALCws6\ndOiAr68vCxYsACAhIQFHR0cAnJycuHbtGgAajQZXV1f9a52dndFoNGg0GlxcXPT7XVxc0Gg0xi5d\nCCHEE1gb+w327dtH5cqVuX79ur7fw8LC4r7nPPi9EEII82f0AKlcuTIAFStWpEePHoSFheHo6Khv\nhcTHx1OpUiVAbXHExMToXxsbG4uzs/Nj9z+KhJEQQhTM006NaNRLWBkZGaSlpQGQnp7O9u3bqV+/\nPt26dWPx4sUALFmyhO7duwPQrVs3Vq5cSXZ2NlFRUVy8eJEmTZrg5OSEg4MDYWFhKIrC0qVL9a95\nFEVRZFMUvv76a5PXYC6b/C7kdyG/iydvBWHUFkhCQgKvvvoqFhYW5Obm0q9fPzp27Ejjxo0JCgpi\n0aJFVKtWjdWrVwPg6elJUFAQnp6e2NjYMGfOHH2LYvbs2QwaNIisrCw6depEQECAMUsXQgjxL4wa\nINWrV+fYsWMP7S9Xrhw7d+585Gs+/fRTPv3004f2N2rUiJMnTxq8RiGEEAUjd6I/x1q3bm3qEsyG\n/C7ukt/FXfK7eDYWSkEvfpkpCwuLAl/PE0KIF1VBzp3SAhFCCFEgEiBCCPGciL4Vzalrpwrt/Yx+\nH4gQQgjjSM9OJzQ6lG2XtrHt0jaSMpP4pMUn1KtUr1DeX/pAhBCiiFAUhZPXTrL14la2XdpGmCaM\nRpUb4V/DH/+a/ng7eWNpUbALSwU5d0qACCGEGbuRcYMdl3aw7dI2tl/ajr2NPQE1AvCv6U8btzaU\nKlbKIO8jAYIEiBCi6FIUhdiUWI5cPUKYJoydUTuJTIykVbVW+lZGzXI1jfLeEiBIgAghio641DiO\nXD2ibnHqV0VR8HX2pXHlxrSp3obmrs2xtbI1ei0SIEiACCHul5GTgSZFQ2xK7H1bfHo8Flhga2WL\nrZUtNpY2+sf3bjZW9++3s7ajhE0JitsUp4RtCUrYlKCE7e3vbz8uZlXsoYldr6df52jc0buBcfUI\nmbmZNK7SmMaVG6tfqzTGpbSLSSaFlQBBAkSIF018Wjynrp16KCDubGnZaTiXdsaltIu6lXLB1cEV\np5JOAGTnZeu3nLyc+77X79ep+7W5WrLyskjPTicjJ4P0nPSHHqfnpJOry70vULLzsknVptKoSqP7\nwsKtjJvZzCAuAYIEiBDPs1xdLqeunWJ/zH79lpSVRAPHBlR1qKoPCdfSrvrHFYpXKPSTdK4uVw2V\n2+FiYWGBWxm3Ao+QKgwSIEiACPE8ScpM4mDsQTUsYvdzWHMY59LONHdpTnNXdatVoZZZn5gLQ1pu\nLnHZ2VzNzsbBygrvUk8/MksCBAkQIYqyqKQo/oz+U9+6iE2JxdfZl2YuzWju2pymLk0pZ1/O1GUW\nmtQ7waDV6gMiTqtVv96zP1dRqGJrS5VixXi9YkXev2cJ8PySAEECRIiiJicvh/Xn1zP3yFxOJJyg\nY42O+sCo71gfa8sXa8KMxJwc5l29yhyNhqTcXKoUK0ZlW1uq2NpSuVgx9evtsLizr7SV1TNfpivI\nufPF+i8jhDAb/9z6h/nh81kYsRCP8h4MazSMnnV6Usy6mKlLM4nzGRlMi41l5bVrvFqhAlu8vKhf\nooTZdLI/igSIEKLQ5OnyCLkYwtwjczkQe4A367/JrgG78KzoaerSTEJRFEJv3WJqbCyHUlIYVqIE\nZxMScFq/Ho4eBSsrqFgRKlRQt0c9Ll8ebGwAyMmBmzfB0bFw6pdLWEIIo4tLjWNhxELmh8/HqaQT\nwxoNo3e93hS3KW7q0kwiW6dj1T//MDU6mqzMTMbu38+bS5dir9WCn5+6NW4MFhZw4wZcv37/13sf\n37yJrkRJbllXICq1AnEtguiya8xT1ySXsIQQZkOn6NgdtZu5R+ayK2oXQZ5B/NH7DxpWbmjq0gpf\nbi6cPEni4cPMS0tj1ksvUTcqioknT+JftiyWfn6wdy9UraqGRj6dPAkzp+vY8dst+rS8zn+63qCR\nr4MRP8j9pAUihDCom5k3WXxsMXOPzMXO2o7hjYfTz6sfpYuVNnVphevSJdiyBUJCiLx0iWn9+rHS\nz48emZmMdnHBq0ED/aWnp5GXB5s2wfTpcP48DB8Ob78NlSo9W7nSAhFCmMzRq0eZc3gOa8+tpbN7\nZxb3WEwzl2Zm3Qn8tGJiICsLatZ8REMhKwv++gtCQtTgSE1F07Mnn48cyZaSJRnm7MyZKlVwKlaw\nQQK3bsGiRTBrlhoWI0dCr15ga/xpsh5LAkQIUWBZuVmsPr2a2Ydnk5CWwLDGwzj/3nkqlXjGP4fN\nTEICfPst/PorFC+uXpFq2RK61I2iXXYIlY+FYLHnL/DygsBA0leu5PsyZZh59SrDqlThUtWqlLIu\n2On23DmYORNWrIDAQPWrn5+BP2ABSYAIIZ7a5aTLzD0yl5+P/UzjKo354pUv6OTeCStLK1OXZlAp\nKTB5MsyeDf37w7kT2VQ4s4eUVSEoW7ZguSGRndaBbOFNtC2X4N2+LGnNEpifdZmWWVmEN25MNTu7\nfL9fdjbEx0NcHERFweLFEBGhXqI6dQqqVDHeZy0ICRAhRL7k6fLYenErc47M4VDsIQZ5D+LA4ANG\nW5/ClLRa+PFHCA4Gf3+I2HWTqlvmQuOZULUqDp07w4Zl0LAhPS0t8dPA3INJTLI4SvoZSyzn1iW5\nogMrW0KrVlCvnjpgKi4Orl5Vv977+M7XW7fUIbiVK6th0acPrFsHT5FBhUo60YUQT5SUmcT88PnM\nPTKXcvbleNf3XfrU64O9jb2pSzO4vDxYvhy++ko96X8/Ioo6W3+AX36B7t3hgw/UH9zjQkYG4y5f\nJiI1le9q1CCoYkWuX7dg717Ys0ftFjl3Tu23uBMMlSs/+nGFCuqtH6YgU5kgASKEIZ25foYuv3bh\n5aovM7LJSHydfU1dklEoCmzeDJ9+CqVKwexBh/HZNRl27YIhQ9Qe6weuHyXl5PDff/5haXw8H1Wt\nyihnZ+xMdfY3AAkQJECEMJQdl3bQb20/JneczIAGA0xdjtHs3w8ffwxJiToW9NyC397vsYiOhjFj\nYPBgNVHukaPT8ePVq3z7zz/0rFCBCdWr42jKoVAGIsN4hRAG8dPRn/jqz6/4Peh3WlZraepyDE6n\nU2/C++orOBOexc/tf+HlG1OwCCkOH34Ir78OD4yaupGdzeabN5n4zz+42dmxu0ED6pUsaaJPYB4k\nQIQQenm6PD7e+TEbIzey9z97cS/vXqjvr9Go/RAODuof/pbPsMxHSoo6kunyZXW793F0NLiXS+TH\nBj/ycs4sLBIawpw50Lq1/gYPRVE4kZ7O5sRENiUmcjo9nXZlyzKtZk0Cy5c3yOct6uQSlhACgPTs\ndPqt7UeyNpk1QWuMvu5GXp46NPXvv+9uWq064ig5GdLT1RApU0YNlDJl7m73fu/goB7rwbDIyICX\nXoLq1dWv+q1yJjU3/oDtzClqx/jYsfqO8Yy8PHYnJbEpMZHNN29ia2FBl/Ll6VK+PC3LlKHYsySa\nmZM+ECRAhCgITYqGbiu74eXoxbwu87C1Mvw1/YwMCAu7GxYHD6qjj15+GVq0ULcaNe7e4Z2Xp7Yi\nbt1St+Tku48f/N7C4m5A3AmMSpUeuFtcUWD1arXDo3Fj+N//4KWX+Ccri82JiWxOTGRvcjKNSpWi\nS/nydC5XjlrFiz9Xd9I/iQQIEiBCPK1j8cfouqIrIxqP4JMWnxjshHntGuzbdzcwTp1Sb9S+ExbN\nm6szkheKw4dh9GjIzIQffuBM48YsT0hgY2IicdnZBJYrR5fy5elYtixlCjA/1fNAAgQJECGexsbz\nG3lrw1vM6TSH1+u+/szHi42F335T/9A/e1YNiTuB4esL9oV964hGo47N3bmT68HBrGzThiXXrhGX\nnU0/R0derVCBJqVLY/WCtDKeRAIECRAh8kNRFKYfms7/9v2PP3r/gZ9LwSdXiouD33+HVavgzBno\n0QOCgqBduwJNNmsYGRkweTLaOXPY/OmnLG3ZktC0NLqUL88AJyfalS0rofEACRAkQIT4N7m6XEaF\njOKvf/5iU99NuJVxe+pjXLsGa9aooXH8OHTtCr17Q4cOpp0dFp0OZcUKDs+fz5JevVjl5UW9UqUY\n6OTEaxUrUrqAExq+CCRAkAAR4klStCkE/RYEwKpeq3Cwy//iQzduwNq16uWpI0egc2e1peHvbx5z\nNcUeOMCy9etZ6uNDrqMjA9zc6O/oiFuhXzcrmsw2QHQ6HY0aNcLV1ZUNGzYwYcIE5s+fT6XbK6BM\nnDiRgIAAAIKDg1m0aBHW1tZMnz6djh07AhAeHs6gQYPIysqiU6dOTJs27dEfSAJEiEeKTIyk56qe\ntKzWkhmBM7C2fPiv8dxcdaXU+Pi7W1ycOp/TwYMQEKC2NAIDTdCfcYdWC4mJ+m1nSgrfpaQQ7uBA\nL2BgixY0K1PmhRk9ZShmeyf69OnTqVu3LikpKfp9Y8eOZezYsfc97+zZs6xevZqzZ88SGxtL+/bt\nuXDhAhYWFgwfPpyFCxfi6+tLp06d2LZtG/7+/oVRvhBF3u9nfmf45uEMqvpfvG+9w4xpFsTHq+tc\n3BsWN29C+fLg5HT/Nniw2vooUcKARel06ljcmzfVLTHx7tc7240b93+fmKgGSLlyXHF3Z2yfPoS7\nuPBtejobu3fH7oFpR4RxGT1AYmNj2bJlC59//jlTp07V739U0q1fv54+ffpgbW2Nm5sb7u7uhIWF\nUa1aNVJTU/H1VSdyGzBgAOvWrZMAEeJf5OTlMPS3j1l37g+KrQ9hu64xsZ53g6Fu3buPHR3V2WAN\n0k0QF6d2kly/fjcgHtySk9U7BcuVu7uVLasWUb48uLtD06bq4zv7ypdHW6IEk2NjmRoTw0gXF5a5\numJfhCcxLMqMHiBjxozh+++/Jzk5+b79s2bNYtmyZTRu3JgpU6bg4OCARqOhWbNm+uc4Ozuj0Wiw\ntrbGxcVFv9/FxQWNRmPs0oUosm7dgrm/apgYGYQ2uSxDyh9l2NJy1K9v5DeOilJv0Fu1Sr3L280N\nPDzuD4k7W5kyTz13eUhiIiOPHMGzeHGONGpEdenfMCmjBsjmzZtxdHTE29ub0NBQ/f4RI0bw1Vdf\nYWFhwRdffMEHH3zAggULDPa+48eP1z9u3bo1rVu3NtixhTBXeXmwYwcsWQIbT+8kr1t/guqNZP7A\nj7G1MfIUHGfOwKRJ6lrg77wD588b9C7BqMxMxly8yOmMDKbXrEknmYvqmYWGht53Xi4IowbIvn37\n2LBhA1u2bCEzM5PU1FQGDBjA0qVL9c8ZOnQoXbt2BdQWR0xMjP5nsbGxODs7P3b/49wbIEI8786e\nVUNj2TKo4qzD6fWJlGw0h19fW07b6m2N++aHD6vL9u3fD6NGqYt3O+R/ZNe/yczL438xMcyIjWWs\nqysrPT2L9Job5uTBP64nTJjw9AdRCkloaKjStWtXRVEUJS4uTr9/6tSpyhtvvKEoiqKcPn1a8fb2\nVrRarXL58mWlRo0aik6nUxRFUfz8/JRDhw4pOp1OCQwMVEJCQh75PoX4kYQwmdRURZkzR1GaNFGU\nypUVZdw4RdkXfkMJ/CVQabGohRKbHGu8N9fpFOXPPxWlQwdFcXVVlBkzFCU93cBvoVPWX7+uVD9w\nQHnt5EklOjPToMcXDyvIudMkd9WMGzeOY8eOYWlpiZubG/PmzQPA09OToKAgPD09sbGxYc6cOfqh\neLNnz75vGO+dYb9CvEh0OrWl8dlnav/y+PHqzXsRCYd5/bfX6eXZi+B2wdhYGeEW8DvL9k2cqI6O\n+uQTePNNg985eDEjg1EXL3IpM5O5Hh50LGfcWYFFwcmNhEIUEX//rc4HaGMD06aBn586mvHHIz8y\nPnQ8c7vMpWednoZ/45wcdQzvxInq9LaffQavvWbQxbtzdDr2p6Twx/Xr/JKQwLiqVRnt4oLtczx9\nurkx2/tAhBAFFx2tzkB+4AB89x306aOex9Oy03hn0zucunaKfW/te/ziT9nZ6qSCV65ATIw6fDY1\nFdLS1K93tnu/v/dxdjY0a6b2dQQGPjBHesFptFpCEhMJuXmTXUlJ1LC3J7BcOY41boyLOdzaLv6V\ntECEMFOpqeo5+6ef1P7pDz6A4sVBm6tl+6XtfLLrE/yqNGF28//DPu46/POPGhL3bv/8o96LUbky\nVKsGLi7q8NlSpdStZMm7jx/8/s5je3uDhEaOTse+5GRCbt4k5OZNNFotHcqWJbB8efzLlsWpWDED\n/NZEQZntVCaFSQJEFHV5eeqoqi++UPs3Jk6Eik7Z7Li0gzXHV5C8dT0DL5WkVawVDgnJ6sm9WjV1\nq1r17nbn+8qVn+nuwFs5OaTl5VHM0hJbS0uKWVhQzNIyX1OFxGZl6QNjd1ISNe3tCSxfnsBy5WhS\nqhTWconKbEiAIAEiira//oIxY9Q/+r+fmk1yuV38EfEr6Zv+oP/F4rQ+nY5lrdrYvt5HTZfq1Z95\n2KxOUYjLzuZSZubdLSuLS5mZXMzMJEdRcLCyQqsoaHU6tDod2YqCze0guRMo+u3296l5eSRkZ9Ox\nXDkCy5XDv1w5HE06Va94EgkQJEBE0XT5MowbB2FHcnjzy93cKv4L2Rv/4I0LdrQ4l47i0xC7oDfg\n1VfhCfdAPUmuTsfe5GTOZGTcFxaXs7IoZWVFDXt7atrbU8Penhp2dupXe3sq2tg81NpQFIXsewJF\nq9PdFzBanQ5bS0salCwp624UERIgSICIoiU1Ff77f7nM3fonvgFL8bz0B70jbfC9mImuxcvYB/WF\nbt2e6a7u0+npLI6PZ3lCAs7FitGwZMn7wuIlOztKyToZLzwJECRARNGg06n9HJ/M2s6rVd5k0IUs\nfGJzyG3XmhJ9BkCnTs90aSoxJ4cVCQksSUjgqlbLACcnBjo6Utug0+mK54kM4xWiCNi3D94fewvf\n4r3Yf+UvHGw9qRA8Hvz9KVa8eIGPm6PTsfXmTRbHx7MrKYlO5cvzf9Wry/KtwmgkQIQoJDEx8PE4\nBSImsDR5IsVLl6XS0jWUDOz2TMc9kZamv0RV096eQU5OLKpdGwe5LCWMTP4PE8LIMjLg++/h8IwN\nfFt8CHa5t8ia8AUvDf3yvvsrFEUh757Oaf3Xex/fs+9UejpLEhJIzMlhgKMje3188HiGFowQT0v6\nQIQwEkVRl8VYPvowX1oNpVLWKfYMbkfPCaspae9Ajk7H8oQEvo+J4XJWFlqdDktQ77WwtMTWwkJ/\n38Wj9rna2fGmoyNtypTBUi5RiWcknehIgAjzcPQoTBt6moHR42ig/MkM/1L4f7uSFjXboNXp+Dku\nju9iYqhhZ8fn1arRtHRpbC0tpa9CmIwECBIgwrTi4+GH9y/TaNN4AqzXM7W1jux3hvCl/0QUS1vm\nx8Xx/ZUrNChZki+qVaOZAdfOEOJZyCgsIUxoxx9pxPT9mC9ZwfqASnRoUYWZfZZQx7EhM65e5YeY\nGF52cGBD/fo0LFXK1OUK8cwkQIQwgN+mxlDn466Uf7kkjVtbEtQqiPXNPuan+Ot0PnSIjmXLsrNB\nA+qVLGnqUoUwGAkQIZ6BosCc9zbRY1E/FvhbsK1HHeYEbGdndhnqHomge4UK7JPRUeI5JQEiRAHk\n6fLYFBnCts/GM2FrOL++H0DH0f9HSkYpekXH06dSSY42aoSbvb2pSxXCaKQTXYinEJMcw8KIhSw4\nuoB3Nlgw+EgG1hs3YtmiMV1PnaK6nR2Ta9SgiqxtIYoYGYWFBIgwvFxdLiEXQph3dB77Y/bTu3pv\nOo+NpU5GHC5HNxDnVBb/Eyd4rWJF/q969XytkyGEuZEAQQJEGM6V5CssDF/IwoiFVHWoytuN3uYV\nXRtuvdIf68oVqR+xlOPo6HLyJJ9Wrcp7Li6mLlmIAivIuVOWAxPiAXm6PEaGjMRnng9JWUmE9Ath\n/+D9vKJpipVfO3KaNMfr/G/sztHif+IEM9zdJTzEC0k60YW4R1ZuFn3X9CVZm8zlkZdxsFNv9Ds1\nbSdOY/sSOfg7ms//D8sTEhh78SJr6tbllTJlTFy1EKYhASLEbUmZSXRf2R3n0s5seW0LxazVjvBj\nI36iytyvuPzdbzT7sCWTr1xhpkbDbm9v6sr6GuIFJgEiBBCbEkvALwF0eKkDU/ynYGlhCXl5nPD/\niFKhm0n4fS+NX63J2EuX2JmUxD4fH1zs7ExdthAmJQEiXnhnrp8hcHkg7/m+x4fNP1Q7E1PTiGz8\nBulX0il58ADuPg68ceYM8dnZ7PX2poyNjanLFsLkJEDEC23flX30XN2TyR0m079BfwCUjEwu1+7E\neW1Nml5ci20lCDhxgkq2tmzz8sLOysrEVQthHmQYr3hhrT+3niEbh7Ds1WUE1AxQd+bmct7rdS7F\n2dP80i+kl8gh8MQJ2pQpww81a8q6G+K5JbPxCpFP84/O56vQr9jSdwu+zr7qTkXhXNsR3IhKw+fc\nKq4WyyQw/ATvOTvzoaur3CAoxAMkQMQLRVEU/rvnvyw5voQ9g/bgXt5d/7Nzvb9GezAcp0N/EumQ\nQdCx00ypUYM3nZxMWLEQ5ksCRLww8nR5vLvlXcI0Yex7ax9OJe8Gw4Uxs7Fdu4KskL9ZX+EW35++\nwvI6dWhfrpwJKxbCvEmAiBdCZk4mfdf2JVWbSuigUEoXK63/2T+Tf6PEjGBOLvmL+ZWv8881LYca\nNaKaDNMV4olkKhPx3LuZeZMOyzpgb23Pln5b7guPuOW7KfHxu2yctoFR7jeoYGPD3z4+Eh5C5IOM\nwhLPNZ2iw3e+L62qtWJyx8nqDYK3Je0KR+cfwLfBq/mluQ2Ta9RgoPR3iBeUjMIS4gGbIjdhaWHJ\nlI5T7htFlXb8EhlduzN46lIuNrFjZ716NJDlZoV4KhIg4rk2ef9kPmz24X3hkR2TwNkufeg7dx51\nfJw5Uru23FkuRAEUSh+ITqejYcOGdOvWDYCkpCQ6duxIrVq18Pf3Jzk5Wf/c4OBg3N3dqVOnDtu3\nb9fvDw8Px8vLCw8PD0aPHl0YZYsiLkwTxpXkK7zm+Zp+n+5WCr/1e5+Os75l0Ct1WVevnoSHEAVU\nKAEyffp0PD099d9PmjSJ9u3bc/78edq2bUtwcDAAZ86cYfXq1Zw9e5aQkBBGjBihvyY3fPhwFi5c\nSGRkJJGRkWzbtq0wShdF2JQDUxjddDTWlmpDOy8zi48+/Z73x/yHX/18+bx6NbmzXIhnYPQAiY2N\nZcuWLQwZMkS/b/369QwcOBCAgQMHsm7dOgA2bNhAnz59sLa2xs3NDXd3d8LCwoiPjyc1NRVfX/WO\n4QEDBuhfI8SjRN+KZtflXQz2GQzALa2WDvNXENLAk73NWhHoJPd3CPGsjB4gY8aM4fvvv7/vGnRC\nQgKOjo4AODk5ce3aNQA0Gg2urq765zk7O6PRaNBoNLjcs+Kbi4sLGo3G2KWLImzawWkM9hlMSduS\nHE1JocGm7ZSPvk5Ix+7UdSpu6vKEeC4YtRN98+bNODo64u3tTWho6GOfJ3MMiYJSFIVrOTlEZ2Xp\nt/NpySzLqoqbox8z9+6leFoWH/60jteDp1LtJQkPIQzFqAGyb98+NmzYwJYtW8jMzCQ1NZX+/fvj\n5OSkb4XEx8dTqVIlQG1xxMTE6F8fGxuLs7PzY/c/zvjx4/WPW7duTevWrQ3+2UThu5WTw/y4OKJu\nB8XlzCz+ycqimGJFuWw7SqTaYXXdjvjUo9il3iBpvSeTL35Gh9y9ZKzfQY2GDqb+CEKYjdDQ0Cf+\nYZ8fhXYj4V9//cWUKVPYsGED48aNo3z58nz88cd89913JCUlMWnSJM6cOUO/fv04dOgQGo2GDh06\ncOHCBSwsLGjatCkzZszA19eXzp07M3LkSAICAh7+QHIj4XOr896zhF/KwTaiPCmRdqRdtMPJohjV\nKlnj6gqurlDZJZtvU6uzoPZSOn/3FTYujlgsWQKlSpm6fCHMWpG5kfCTTz4hKCiIRYsWUa1aNVav\nXg2Ap6cnQUFBeHp6YmNjw5w5c/SXt2bPns2gQYPIysqiU6dOjwwP8fz6en0SW7NvMUXrS9M31MBw\ncoIH13ZacmwFfda78OqY/8DgwfDll2ApM/YIYQwylYkwa4oCk6bm8aXzESZXr8FovwpPeK7CZ/+p\nyoQNqdgu/BlefbUQKxWiaCvIuVMCRJit3Fx47z1YWyaKxq+ls8W33uOfnJdH9LA+WK1bj8uuI1h4\neRVeoUI8Bwpy7pS2vTBLKSnQpQucTktH1+UqC7zcH//kpCTo3Jmb+3ez77cfJDyEKCQSIMLsxMRA\nixbgVl1BGRvJhOpuVClW7NFPPnsW/Py4Xq0iPQYVo+crQwuzVCFeaBIgwqyEh0OzZjBwIDT8Mo5c\nRcewKlUe/eSNG6FVK/jsMz4MtGREs5HYWtkWbsFCvMCkD0SYjY0b4a23YO5caNE1m/qHD7OzQQO8\nHpxmXVFovWZXAAAgAElEQVQgOBjmzIHff0fj6Ur9H+tzaeQlytqXNU3xQhRxRWYYrxAPmjlTzYRN\nm8DPD/qeuch/nJweDo/0dPjPf+DKFQgLgypVmLnzE/p79ZfwEKKQSYAIk8rLg7FjYccO2L8f3Nxg\n282bHExJYcHtyTP1Dh2Cd94Bb28IDQU7O1K1qSwIX8DhoYdNUb4QL7Qn9oHs3r1b/zgqKuq+n61d\nu9Y4FYkXRlqaeqvGqVN3wyMjL48RkZHMcXenuJUV5OTAypXQtCm88YY6rvfnn+H2muWLIhbRtnpb\nqpetbtoPI8QL6Il9IA0bNiQ8PPyhx4/63lxIH0jRcPUqdO0KDRqofR62t/u+P718meisLFY4OsJP\nP8Hs2eDuDqNHq+N677n1PFeXS80ZNVnVaxV+Ln4m+iRCPB8M3gdy78EePLCcpEVBZWdDYCD07Alf\nfQV3JmM+mZbGwpgYTqxZA8uWqc2TzZvVlHmENWfWUNWhqoSHECbyxAC5d5r1B6dclynYRUFNnAhV\nq94THjodui1bePvWLb7duhWnWrXg/Hm4PUvzoyiKwpQDU/j8lc8Lr3AhxH2eGCCXL1+mW7duKIqi\nfwzqP94H+0SEyI9jx9TRt8eOgUVaKixeDDNnMs/fH6suXRiycCE87qbBe+y9spdbWbfoWqur8YsW\nQjzSE/tA/vrrrye+uFWrVgYv6FlJH4j5yskBX18YMwYG3poO33wDbdtyddQoGlhYEOrtTd0SJfJ1\nrG4rutHZvTPvNH7HyFUL8WIw+mSKOTk5nDp1CmdnZ/0iUOZGAsR8ffONOhJ308jtWAx+C/bsgZde\nIuj0aTzs7fn2pZfydZzzN87TcnFLokdFY29jb+SqhXgxGHwyxWHDhnH69GkAkpOTadCgAQMGDMDH\nx4cVK1YUvFLxwjl+HGbNggXfxmPxn0FqJ/lLL7E5MZGItDQ+r1Yt38eaemAqwxsPl/AQwsSeGCB7\n9+6lbt26APz88894eHhw8uRJjh49yv/+979CKVAUfTk56s3jkybqqDyuPwwdCm3akJ6Xx7uRkcz1\n8MD+wZWhHuNa+jV+O/MbI3xHGLlqIcS/eWKA2NrenZhux44d9OjRAwAnJyfjViWeK999pw6o+k/C\nJNBq1VUCga+jomhZpgztyuZ/CpI5h+fwuufrVCphnpdQhXiRPHEUVpkyZdi0aRPOzs7s27ePhQsX\nApCbm0tmZmahFCiKtpMnYfp0ODVvHxYjZsCRI2BtTURqKssSEjj14HQlT5CZk8mPR35kz6A9RqxY\nCJFfTwyQefPmMXLkSOLj45k2bZq+5bFr1y46d+5cKAWKois3V710NfWLmziO6Qvz54OLC3tu3aL/\n2bNMrlGDirb5n3596fGl+Dn7UatCLSNWLYTIL5nOXej9qNFwNiODCW5ulLWxeebjBQfDn7sVtpXs\niUW1auRMncqE6GgWxsezsFYtOpUvn+9j6RQddWbX4acuP9HKzfyGjwtR1Bl8KpORI0c+8cUzZsx4\nqjcT5mtWbCxTYmNpX7YsdQ8fZnKNGrxRqVKBZxw4fRqmToXz78/GYv0VLi1eTL+ICMra2BDRqBFO\n+bhZ8F5bLmyhlG0pWlZrWaB6hBCG98QWiK2tLfXq1SMoKIgqVao8lE4DBw40eoFPS1ogT+9HjYbv\nrlwh1NsbN3t7DiYn805kJJVsbfnR3Z2axYs/1fFyc6F5c/jY/xg953bgl507GZuWxhfVqvG+szOW\nBQilNkvaMLThUPrW7/vUrxVC/DuDt0Di4uL47bffWLVqFdbW1vTu3ZtevXpRpkyZZypUmI+frl4l\n+J7wAGjq4MDRRo2YrtHQNDycUS4ujKtalWKW+VsBecoUcCyRRrsNA+m3ZAnHtVp2NmhAgwcXh8qn\n8LhwLt68yOuerxfo9UII48h3H0hsbCwrV65k6tSpfPfdd/Tv39/YtRWItEDyb1FcHF9HR/NngwaP\nbWVcycri/QsXOJ+RwY8eHrT5lyG3Z86oy5T/2vsj3vZvSWCtWkyuUUNd26OA3lz7Jg0cG/DRyx8V\n+BhCiCcz2lQm4eHhrFixgh07dtCoUSM++OADPD09C1yoMUmA5M+S+Hg+v3yZ3d7eeOTjEtX6Gzd4\n/8IF2pQp89jRU7m50PwVHXV6b2RbdfipUSO6ubg8U52xKbF4/ejF5VGXKWMnLV8hjMXgAfLVV1+x\nefNm6tSpQ58+fQgICMDa2rxXwZUA+Xe/xMfz8eXL7GrQgNr5nLwQIC03l6+jo1mWkMDE6tV5q3Ll\n+/ozPp2eyUKHcBqkHGVJy5ZU8fZ+5lo/3vEx2jwt0wKmPfOxhBCPZ/AAsbS0pHr16hS//RfqnRE5\niqJgYWHBiRMnnqFc45AAebIVCQl8cOkSOxs0wPMpwuNex1JTGRYZibWFBXM9PKhXsiRTIhIYd+UC\nE3f+zkd1a2M5bNgz15qWnYbbNDcODz0sS9YKYWQG70SXNT+eL6uvXWPspUvs8PJ6dHhkZEBCgro4\n+RNGSnmXKsW+hg356epV2hw/jmfx4oRdzObXtdvonRMLM34wSL2LIhbRpnobCQ8hzFSBbyRctWoV\nvXv3NnQ9z0xaII+25vp13o2MZNvjRkPdvAnt20NsLOTlQePG6ubrq25VqjwyVOK1Wt5bdYMa08KY\nlDQWi4gIMMAovTxdHu4z3VnecznNXJs98/GEEE9m8BZIeno68+bN49KlS9StW5dhw4axfv16Pv/8\nc9zd3c0yQMTD1l2/zojISLZ6eT0xPOLrtWfiy99R7FYCla8eoer6w7gtnE/NpLfJw4qzJXw5bd+Y\nYza+hFs2Jj6vIlptMSpl5rLK+h0sNm8wSHgArDu3DqeSThIeQpixJ7ZAXnvtNUqVKkWzZs3YsWMH\nMTEx2NnZMX36dLwN0EFqDNICud/GGzcYcv48W7y8aFSq1MNPSEyE9u3JeKUjHmsmMWSoBZUrq6vK\n6jdbBYfkK5S7fISyFw9T8txhip89ilK6DDk+vtheuYhVvzdg3DiD1f3yopcZ03QMvTx7GeyYQojH\nM3gnupeXl76jPC8vj8qVK3PlyhXs7OyerVIjkgC5a0tiIoPOnWNz/fr4li798BNuh4fSoSOvnp9E\nTXcLJk/O58F1Orh4UZ1d9+ZNGDEC8nmj4b85GHuQvmv6cuH9C1hZFvz+ESFE/hn8Eta9Q3atrKxw\ncXEx6/AQd227eZNB586xoV69J4YH/v7MrRpMzC4LVv/2FG9gaQkeHupmYFMPTGWU3ygJDyHM3BNb\nIFZWVpS4PVpHURQyMzMpXry4fhhvSkpKoRWaX9ICgejMTHzDw1lXrx4vOzg8/ITERGjXDgICOP1m\nMK3bWPD331DLDGZJj0qKovH8xkSPiqZUsUdcchNCGIXBWyB5eXnPVJAofIqiMOT8eT5wcXl0eNy4\nobY8AgPJ+moib/hZMGmSeYQHwIxDMxjsM1jCQ4giwLxvKxdPbd7Vq6Tk5fGhq+vDP7wnPJg4kXGj\nLKhdG956q/DrfJTkrGSWHF/C8WHHTV2KECIfDNPr+RharRY/Pz98fHyoW7cun332GQATJkzAxcWF\nhg0b0rBhQ7Zu3ap/TXBwMO7u7tSpU4ft27fr94eHh+Pl5YWHhwejR482ZtlFVnRmJl9ERbG4dm2s\nH+zQvnFDvWzVqRNMnMjmLRZs2ADz5j3xnsFCNT98PoHugbg6PCL8hBDmRzGy9PR0RVEUJTc3V/Hz\n81P+/vtvZfz48cqUKVMeeu6ZM2cUb29vJScnR4mKilJq1Kih6HQ6RVEUpUmTJkpYWJiiKIoSGBio\nbN269ZHvVwgfySzpdDqlXUSEEhwd/fAPr19XFC8vRfn0U0XR6ZSrVxXFyUlR9uwp/DofJzs3W3Gd\n6qoc0RwxdSlCvJAKcu40agsE0M+jpdVq0el0lL09HbjyiM6a9evX06dPH6ytrXFzc8Pd3Z2wsDDi\n4+NJTU3F19cXgAEDBrBu3Tpjl16kzLt6ldRHXbq60/Lo0gX+7//QKRYMHAhvvw2vvGKaWh/l9zO/\n81LZl2hUpZGpSxFC5JPRA0Sn0+Hj44OTkxOtW7fWTwM/a9YsvL29GTJkCMnJyQBoNBpc7zkBOjs7\no9Fo0Gg0uNwzLbiLiwsajcbYpRcZ0ZmZfBkdzc8PXrq6Nzy+/RYsLJg6FdLT4csvTVfvgxRFYerB\nqYxtNtbUpQghnoLRA8TS0pKIiAhiY2PZs2cPf/31FyNGjODy5cscO3YMJycnPvjgA2OXYVBHUlIY\ndPYsN7KzTV0KiqIw+Px5PnR1vX+CxBs3oG1b6NpVHx5Hj8J338Hy5WBOs/LvvbKX5Kxkunh0MXUp\nQoinUGinkdKlS9O5c2eOHDlCq1at9PuHDh1K165dAbXFERMTo/9ZbGwszs7Oj93/OOPHj9c/bt26\nNa1btzbIZ8jIy+Pr6GiWxsfTskwZOp88ya4GDShpwrPxvKtXScvL44N7F27KyYGAADU8/vtfsLAg\nLQ3eeANmzlQn2zUnUw9MZUzTMVhaGP3vGSHEbaGhoYSGhj7bQQzdEXOv69evK7du3VIURVEyMjKU\nV155Rdm5c6cSFxenf87UqVOVN954Q1EURTl9+rTi7e2taLVa5fLly/d1ovv5+SmHDh1SdDqdEhgY\nqISEhDzyPY31kf68eVOpefCg0uf0aSVBq1V0Op0y+OxZpeOxY4o2L88o7/lvojIylAp//62cTku7\n/wdTpihKx46Kcvt3pyiK8tZbijJwYOHWlx+RNyKViv+rqKRnp5u6FCFeaAU5dxr1T+e4uDgGDhyI\noijodDr69+9Pu3btGDBgAMeOHcPS0hI3NzfmzZsHgKenJ0FBQXh6emJjY8OcOXP0i1jNnj2bQYMG\nkZWVRadOnQgICDBm6XrJubl8fOkSmxITmePhQbcKFfQ/m+vhwetnzjDg7FmWe3piVYjjYXWPu3QV\nGwsTJ8KBA/rxuatXw549EB5eaOXl27SD03i70dsUt/n3ZXWFEOalwOuBmCtDTmWy6cYNhl+4QGC5\ncvzvpZcoY2Pz0HOy8vIIPHkSz+LFmeXurg88Y5ur0fBzfDz7fHzu7zgPCoI6dWDCBAD++UddzmPz\nZvWrOUnMSKTmzJqcffcsTiWdTF2OEC80g8/GWxQZIkCuZ2cz6uJFDqWkML9WLdreHnr8OCm5ubQ+\ndoxu5cszvrrxV8+7M9fVHm9v6tzb+ti2TZ0V99QpsLcnNxdat4Zu3Qw607rBTNw7kQs3L/Bz959N\nXYoQL7yCnDul1/IeiqKwIiGB+ocPU8XWlpO+vv8aHgClra0J8fJi+bVrzIqNNWqN9166ui88srLg\nvfdg1iywtwfUK1nFisGHHxq1pALR5mqZFTaLMU3HmLoUIUQBmdFgTtOKzcpi+IULRGdlsaF+fZo8\nagr0J3C0tWW7lxevRERQ3saGNxwdjVLnI0ddgTo+18tLnecK2LcP5sxR+z0MtEyHQa08tZJ6lerh\n5ehl6lKEEAX0wgeITlGYHxfHF1FRvOfszJq6dbEt4Bm3ur09IV5etD9+nHI2NviXK2fQWqMzM/kq\nOpo93t7393tcvKiOz42IAODWLejXD376SV3K3Nxk5mTyzZ5v+KnLT6YuRQjxDF74PpDFcXH8ePUq\nC2vVot6j1gsvgH3JyfQ4dYqN9erR9FFTqheATlHocPw4/uXKMa5q1bs/UBS11dGuHXz0EYoCffpA\nxYrq1SxzNCF0AieunWBN0BpTlyKEuE060Xn6X0KOToelhYXBh+BuTkxk8Llz7Pb2vn+YbQH9qNGw\nJD6efQ0b3l/rmjXw9ddq68PGhkWL4IcfICxM3xViVi4nXabJ/CaEvxNOVYeq//4CIUShkADBvFYk\n/CU+ns+iovjbx4eqz7AU8GNHXaWmgqcn/PorvPIK589DixYQGgp16z57/YamKApdV3SlRdUWfNLi\nE1OXI4S4h8FXJBTP5k0nJ27k5NDx+HH2+vhQ0db2qY+RrdMx+Px5Pnpw1BWo93q0awevvIJWq05V\n8s035hkeABsjN3Lx5kXW9l5r6lKEEAYgLZBC8Pnly2xPSmJ3gwaUesS8WTpFIVarJTIjg8jMzPu+\nxmq1tC1blo31699/6erkSTU8Tp2CSpUYOxaiomDtWvNZIOpeGTkZ1J1TlwVdF9DupXamLkcI8QC5\nhIV5BoiiKLwTGcnlzEzGu7lxITPzvqC4lJlJWWtrPIoXx8Pe/r6v1e3sHh4VptNBy5bw5pswbBhb\nt8LQoXDsGJQvb5rP+G++3P0lF25eYGWvlaYuRQjxCBIgmGeAAOQpCm+fP8/p9PSHgqKmvf3Tzei7\neLF6k8eBAyTcsMLHR+0GMdCkwwZ3IfECzRY24/iw4ziXfvwsykII05EAwXwDxGBu3lQ7zjdvRufT\niE6doHFjdckPc6QoCgHLA+j4Ukc+aF601n0R4kUiU5m8CD79FHr1gkaNmDYNkpPVUbzmau3ZtWhS\nNIz0G2nqUoQQBiajsIqSgwdh40Y4c4ajRyE4WL3f4xGTBJuF9Ox0xmwbw7JXl2FjZaZFCiEKTFog\nRUVurjrT7vffk2ZdhjfegBkzoBAm/y2w/+75Ly2rtaSVW6t/f7IQosiRFkhR8eOP4OAAffsycjC8\n/LJ634e5Onv9LAsjFnJy+ElTlyKEMBIJkKIgLk69Q3DPHlausuDvv81zdcE7FEXhvZD3+OKVL2Sh\nKCGeYxIgRcEHH8CQIUTZ1eH992HrVjDQvI9Gsfr0am5k3ODdJu+auhQhhBFJgJgznQ6mTIH9+8n9\ncT79AuHjj6FRI1MX9nip2lQ+2P4Bq3qtwtpS/vcS4nkm/8LNVWIiDBwIN25AaCgTJpegVCkYO9bU\nhT3ZhL8m0KFGB16u+rKpSxFCGJmMwjJHf/8NPj5Quzbs2cNf/7ixYAEsWWKeqwveceraKZYeX8p3\n7b8zdSlCiEIgLRBzotOpS9NOnw4LFkCXLiQmQv/+sGgROJlxf7SiKLy75V3Gtx5PpRKVTF2OEKIQ\nSICYi2vX1KRIT4fDh8HVlbw86NsXgoL0S52brV9P/kpadhrvNHrH1KUIIQqJGV8QeYGEhkLDhmrv\neGgouLoC6hQl2dkwaZJJq/tXyVnJfLTjI+Z0moOVpZWpyxFCFBJpgZhSXh783/+pNwkuXgz+/vof\nrV8PS5fCkSPwNBP1msLXoV/TxaMLfi5+pi5FCFGIzPzU9ByLj4d+/dR+j6NHoUoV/Y/On1fX99i4\nESqZcXdCmCaMyfsnsy9mH8eHHTd1OUKIQiaXsExh5071klWLFurje8IjNRV69lQbJn5m+Ae9TtGx\n8fxGWi1uxeu/vU5z1+ace/ccFYpXMHVpQohCJuuBFNStW2oHhZWVullb3//1UeNtc3PVdcwXLYJl\ny6Bt2/t+rChqh7mDgzoIy5xk5Wbxy4lfmHJgCvbW9nzU/CNer/u63CwoxHOiIOdO+ddfEDt2qM0E\ne3s1FPLyHv4KD4eKoqjNivBwcHR86LBTpkB0NOzdW7gf50luZt7kx8M/MuvwLHycfJjdaTZt3Npg\nYY4LrwshCpUEyNM6d07tu9i8WV2X/HF0uodDJS8PypWDR5x8d++GyZPV9T3s7IxYfz5FJUXxw8Ef\n+OXEL3Sv3Z0d/XdQr1I9U5clhDAjEiBPIzERunZVb/Z7UniAegnL1jZfh42JUTNp+XKoWtUAdT6D\nw5rDTD4wmZ2XdzK04VBODj8p65gLIR5J+kDyKzsbOnaEJk3gf/8z2GGzstQsev11+Ogjgx32qekU\nHSM2j2DLhS2MbjqaIQ2HULpYadMVJIQoVAU5d0qA5IeiwNtvq3eLr12r9mcYyNChan/86tWPvLJV\nKPJ0eQzeMJioW1Fs7ruZkrZmPFe8EMIopBPdWH74Qe2c2LfPoOExf756yEOHTBceubpcBq4bSEJa\nAlv6bqGEbQnTFCKEKHKkBfJvNm1SWx8HDxq0gyIsDLp0UUdc1aplsMM+lZy8HPqu7UuqNpU/ev+B\nvY29aQoRQpictEAM7eRJeOst2LDBoOFx7Zra5/HTT6YLD22ult6/9yZPyWNdn3XYWZvB0C8hRJFi\n1DvRtVotfn5++Pj4ULduXT777DMAkpKS6NixI7Vq1cLf35/k5GT9a4KDg3F3d6dOnTps375dvz88\nPBwvLy88PDwYPXq0MctWXbsG3brBtGnQtKnBDpubC336wJtvQo8eBjvsU8nKzeK11a9haWHJmqA1\nEh5CiAIxaoAUK1aMP//8k4iICE6cOMHu3bvZt28fkyZNon379pw/f562bdsSHBwMwJkzZ1i9ejVn\nz54lJCSEESNG6JtUw4cPZ+HChURGRhIZGcm2bduMV3hWFrz6qnqW79vXoIf+9FOwsYFvvjHoYfMt\nMyeT7iu7U8K2BKt6rcLWKn9DjYUQ4kFGnwurePHigNoa0el0lC1blvXr1zNw4EAABg4cyLp16wDY\nsGEDffr0wdraGjc3N9zd3QkLCyM+Pp7U1FR8fX0BGDBggP41Bqco6tCoKlXUaUcMaMkS+P13+PVX\ng/bF51t6djpdVnShYvGKLO+5HBsrm8IvQgjx3DB6gOh0Onx8fHBycqJ169Z4enqSkJCA4+2pPJyc\nnLh27RoAGo0G19trYQA4Ozuj0WjQaDS4uLjo97u4uKDRaIxT8KRJcPasQdePTUiA3r3h22/VadrL\nlzfIYZ9KqjaVwOWBVHWoypIeS2QOKyHEMzP6WcTS0pKIiAhSUlLw9/cnNDT0oXmUDD2v0vjx4/WP\nW7duTevWrfP3wrVrYc4cdcTV7ZbTs1AUdU2PceNg0CB1yQ97Ewx0Ss5KJnB5IPUq1WNul7lYWsgk\nzEK86EJDQwkNDX2mYxTan6GlS5emU6dOHDlyBEdHR30rJD4+nkq3F71wdnYmJiZG/5rY2FicnZ0f\nu/9x7g2QfAsPh3fega1b4QnHzq/oaPVw165BSIg6e7spJGUmEbA8AN8qvswInCHhIYQAHv7jekIB\nLtkb9Wxy48YN/QirzMxMduzYgY+PD926dWPx4sUALFmyhO7duwPQrVs3Vq5cSXZ2NlFRUVy8eJEm\nTZrg5OSEg4MDYWFhKIrC0qVL9a8xiKtXoXt3dWXARo2e6VB5eerArcaNoU0b9X4PU4VHYkYi7Ze1\np7lLc2YGzpTwEEIYlFFbIHFxcQwcOBBFUdDpdPTv35927drh4+NDUFAQixYtolq1aqxevRoAT09P\ngoKC8PT0xMbGhjlz5ugvb82ePZtBgwaRlZVFp06dCAgIMEyRGRlqeAwbBr16PdOhTp+GwYOhWDHY\nvx88PAxTYkFcT79O+2XtCagRwKT2k2T6dSGEwcmd6L/+Clu2qAs8FfAkq9VCcDDMnq12lA8darD+\n96d2Lf0aS48vZfbh2bxZ/02+afONhIcQ4l/JZIoUcCoTna7AZ/yDB9VWR82aav+7AbpPnlqeLo/t\nl7azIGIBu6N206N2D4b4DOHlqi8XfjFCiCJJAoTCW9I2LQ0+/1ydRXf6dHVqksL+Qz8qKYqfj/3M\nz8d+pnLJygxpOIQ+9frINOxCiKcmc2EVkg0bYORIaNUKTp0q3Ps6tLla1p1bx4KIBRyLP0bfen3Z\n3HczXo5ehVeEEEIgAfJUoqLU4IiMhAULoH37wnvvkwknWRixkOUnl+Pt5M1gn8H0qN1D5rESQpiM\nBEg+aLXw/ffq8NyxY9XpSIoVK5z33h21m093fYomRcNbPm8RNiSM6mWrF86bCyHEE0iA/Ivt2+G9\n98DTE44cATe3wnnfFG0K43aMY/OFzUzzn0aP2j2wsjTBBFpCCPEYEiCPERurtjaOHIGZM6Fz58J7\n720Xt/H2prfp8FIHTg0/hYOdQ+G9uRBC5JPcmvyAnByYPBm8vaF2bfXmwMIKj1tZtxi8fjDvbHqH\n+V3ns6DbAgkPIYTZkhbIPfbsgREjwMUFDhwAd/fCe+9NkZsYtmkY3Wp14+Twk5QqVqrw3lwIIQpA\nAgR1uvWPPoLQUPjhB+jZs/Du6biZeZNRW0exP2Y/y15dRpvqbQrnjYUQ4hm98JewQkKgXj1wcoIz\nZ+C11wovPP44+wf15tSjvH15Tgw7IeEhhChSXvg70WNiICUF6tY1YlEPuJ5+nfdD3ic8LpxF3RfR\nomqLwntzIYR4hILcif7Ct0BcXQsvPBRFYfXp1XjN9cK1tCvHhx2X8BBCFFkvfB9Iri6X7Lxsits8\n+wqET6LN1TJk4xCOXj3Kut7r8HPxM+r7CSGEsb3wLZBfT/5Ky59bokkx0hrrqDcFdvq1E+nZ6Rx9\n+6iEhxDiufDCB0h/r/70rNOTpgubcvTqUYMfPy41jlaLW1GrfC1+e/037G1MsCi6EEIYwQsfIBYW\nFnz2ymdMD5hOwPIA1pxZY7BjRyZG8vKil+lVpxezO82WqUiEEM+VF74P5I6edXriVsaN7iu7c+7G\nOT575bNnWsnvUOwheqzqwbdtvmVww8EGrFQIIczDCz+M90FXU6/SfWV3aleozfyu8ws0XfqWC1sY\nuG4gP3f/mS4eXQpcixBCFBYZxmsAVUpV4a9Bf6HN1dJuaTuupV97qtcvPraYt9a/xcY3Nkp4CCGe\naxIgj1Dcpjgre62kffX2+C3w42TCyX99jaIoBO8NZsJfEwgdFEpTl6aFUKkQQpiOXML6FytOrmDU\n1lEs6r7osS2KPF0eo7eOZs+VPYT0C6FKqSoGe38hhCgMBTl3SoDkw8HYg/Rc1ZMPm3/ImKZj7utc\nz8rNYsAfA7iecZ11vdfJ9OtCiCJJAgTjBAjAleQrdF3RlSZVmjC782xsrWxJzkqmx6oeVCxekWWv\nLqOYdSGtcyuEEAYmnehGVNWhKvve2se1jGv4/+LPyYSTtFzckvqV6rOy10oJDyHEC0cC5CmUtC3J\n2qC1+FbxpcHcBvSt15fpAdOxtJBfoxDixSOXsAooNiUWl9IuRn8fIYQoDNIHQuEFiBBCPE+kD0QI\nIYKjYYMAAAhySURBVEShkQARQghRIBIgQgghCkQCRAghRIFIgAghhCgQCRAhhBAFIgEihBCiQIwa\nILGxsbRt25a6detSv359Zs6cCcCECRNwcXGhYcOGNGzYkK1bt+pfExwcjLu7O3Xq1GH79u36/eHh\n4Xh5eeHh4cHo0aONWbYQQoj8UIwoLi5OiYiIUBRFUVJTUxUPDw/l7Nmzyvjx45UpU6Y89PwzZ84o\n3t7eSk5OjhIVFaXUqFFD0el0iqIoSpMmTZSwsDBFURQlMDBQ2bp16yPf08gfqUj5888/TV2C2ZDf\nxV3yu7hLfhd3FeTcadQWiJOTE97e3gCULFmSOnXqoNFo7gTXQ89fv349ffr0wdraGjc3N9zd3QkL\nCyM+Pp7U1FR8fX0BGDBgAOvWrTNm6c+F0NBQU5dgNuR3cZf8Lu6S38WzKbQ+kOjoaI4dO4afnx8A\ns2bNwtvbmyFDhpCcnAyARqPB1dVV/xpnZ2c0Gg0ajQYXl7vzTrm4uOiDSAghhGkUSoCkpaXRq1cv\npk+fTsmSJRkxYgSXL1/m2LFjODk58cEHHxRGGUIIIQzJ8FfS7peTk6P4+/sr06ZNe+TPo6Ojlfr1\n6yuKoijBwcHKpEmT9D/z9/dXDh48qMTFxSm1a9fW71+xYoUybNiwRx4PkE022WSTrQDb07LGyN56\n6y08PT0ZNWqUfl98fDxOTk4ArF27lnr16gHQrVs3+vXrx5gxY9BoNFy8eJEmTZpgYWGBg4MDYWFh\n+P5/e/cX0tQbxgH8q2lDKiLKlkk1opZYa65tYRoSxJYVaYFouzCL6CYqpIu6DqLyIorAIIKcA2u7\nMQbSpLIL20BMY3gRZCWsvJgDG4WJtPT9XYiHtFXzlHv77Xw/V/tz3J7z8hwf3rPzvMduh8fjwblz\n55J+n+BKvEREabGgBSQUCqGtrQ0mkwkWiwVZWVm4cuUK7t+/j3A4jOzsbBgMBty5cwcAUFxcjNra\nWhQXFyM3Nxe3b99W7j/e3NyM48ePY2JiAgcOHEBlZeVChk5ERL+RcfcDISKi9MiYTvTOzk4UFRXB\naDSiqalJdjhSGQwGmM1mWCwW7Ny5U3Y4aXXy5Eno9Xps375deS0ej8PpdGLLli3Yt2+fctVfpks2\nFr9q4s1kc5uab926BUCbuaGmwftnMmIGMjU1BaPRiK6uLqxduxZ2ux1erxdFRUWyQ5Ni48aN6O/v\nx4oVK2SHknbBYBBLly7FsWPHMDAwAAC4ePEiVq5ciQsXLqCpqQnxeBzXrl2THOnCSzYWly5dwrJl\ny3D+/HnJ0aVXNBpFNBpFSUkJxsbGYLVa4ff70dLSornc+NlY+Hy+eedGRsxAent7sXnzZmzYsAG5\nubk4evQo/H6/7LCkEUJgampKdhhS7N69+4fC6ff70dDQAABoaGjQTBNqsrEAtHmhSbKm5uHhYU3m\nxnwbvH8lIwrI3AZErTcaZmVlweFwwG634+7du7LDkS4Wi0Gv1wOYPnhisZjkiORK1sSrJTNNzaWl\npRgZGdF0bqTS4P0rGVFAaLZQKISXL1/i0aNHaG5uRjAYlB3SP2Xmyj4tmtvEq7VTWXObmufmgpZy\n43cN3qnkRkYUkMLCQrx//155Pjw8jMLCQokRyVVQUAAAyM/Px5EjR9Db2ys5Irn0ej1GRkYATJ//\nXb16teSI5MnPz1f+SZ46dQovXryQHFH6fPv2DTU1Naivr0d1dTUA7eZGsrFQkxsZUUDsdjvevn2L\nSCSCr1+/wuv1oqqqSnZYUoyPj2NsbAwA8OXLFzx+/Fhp1NQKIcSsc7lVVVVwu90AgNbWVuWA0YK5\nYxGNRpXH3zfxakGypmat5sbPGrxnpJwb8+5d/0cFAgFhNBrFpk2bxNWrV2WHI83Q0JAwm82ipKRE\nbNu2TXNj4XK5REFBgVi8eLFYt26duHfvnvj48aPYu3evMBqNwuFwiHg8LjvMtEg2FvX19cJkMgmz\n2Syqq6tFNBqVHWZaBINBkZ2drRwbFotFBAIBMTo6qrnc+NlYqMmNjLiMl4iI0i8jTmEREVH6sYAQ\nEZEqLCBERKQKCwgREanCAkJERKqwgBARkSosIES/EYlEYDKZ/vnPJEo3FhCiFCzEGklaWneJMhML\nCNE8DA0NYceOHejv75/1usvlQiAQUJ6fOHEC7e3tiEQiqKiogM1mg81mQ09Pzw+f2drairNnzyrP\nDx06hO7ubgDAkydPUFZWBpvNhrq6OoyPjy/QnhHNHwsIUYoGBwdRU1MDj8cDq9U66726ujr4fD4A\nQCKRwLNnz3Dw4EHo9Xo8ffoUfX198Hq9swrF95LNRkZHR3H58mV0dXWhr68PVqsV169f//s7RqRS\njuwAiP4PYrEYDh8+jPb29qR3uty/fz8aGxuRSCQQCARQUVEBnU6Hz58/48yZMwiHw1i0aBHevHmT\n8nf29PTg1atXKC8vhxACiUQCu3bt+pu7RfRHWECIUrB8+XKsX78ez58/T1pAdDod9uzZg87OTvh8\nPrhcLgDAjRs3sGbNGgwMDGBychJ5eXk//G1OTs6sO0hOTEwAmF5J1+l0oq2tbYH2iujP8BQWUQp0\nOh0ePnwIj8eDBw8eJN2mtrYWLS0tCAaDqKysBAB8+vRJuT+Lx+PB5OSksv3MOqYGgwHhcBhCCHz4\n8EG5f0tpaSlCoRDevXsHYHqp/vnMYIgWGgsIUYry8vLQ0dGBmzdvoqOj44f3nU4nuru74XA4kJMz\nPbk/ffo03G43LBYLBgcHsWTJEmX7md89ysvLYTAYsHXrVjQ2Niq/r6xatQputxsulwtmsxllZWV4\n/fp1GvaUKDVczp2IiFThDISIiFRhASEiIlVYQIiISBUWECIiUoUFhIiIVGEBISIiVVhAiIhIFRYQ\nIiJS5T9ZB5mSRhR+4AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe5a7a35080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "for k,v in k_rmse_results.items():\n",
    "    x = list(v.keys())\n",
    "    y = list(v.values())\n",
    "    \n",
    "    plt.plot(x,y)\n",
    "    plt.xlabel('k value')\n",
    "    plt.ylabel('RMSE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `engine-size` column has the lowest RMSE for all k values and the minimum RMSE for this column is at `k = 3`. The k value with the minimum RMSE differs for each column. It would appear that the univariate k-nearest neighbors algorithm predicts better for lower prices. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multivariate Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, I'll modify the `knn_train_test()` function I wrote above to work with multiple columns in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "engine-size          3346.484586\n",
      "horsepower           4219.377860\n",
      "width                4618.910560\n",
      "curb-weight          4730.075815\n",
      "highway-mpg          5069.469256\n",
      "length               5176.394904\n",
      "city-mpg             5202.409003\n",
      "wheel-base           5252.392462\n",
      "compression-rate     7166.073599\n",
      "bore                 7222.472445\n",
      "normalized-losses    7624.407151\n",
      "stroke               8000.240467\n",
      "peak-rpm             8119.365233\n",
      "height               8163.346266\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# Compute average RMSE across different `k` values for each feature.\n",
    "feature_avg_rmse = {}\n",
    "for k,v in k_rmse_results.items():\n",
    "    avg_rmse = np.mean(list(v.values()))\n",
    "    feature_avg_rmse[k] = avg_rmse\n",
    "series_avg_rmse = pd.Series(feature_avg_rmse)\n",
    "sorted_series_avg_rmse = series_avg_rmse.sort_values()\n",
    "print(sorted_series_avg_rmse)\n",
    "\n",
    "sorted_features = sorted_series_avg_rmse.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'2 best features': {5: 2949.8817277180374},\n",
       " '3 best features': {5: 3580.7376651928435},\n",
       " '4 best features': {5: 3487.340917327035},\n",
       " '5 best features': {5: 3410.2170133901805},\n",
       " '6 best features': {5: 3478.510890118539}}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def knn_train_test(train_cols, target_col, df):\n",
    "    np.random.seed(1)\n",
    "    \n",
    "    # Randomize order of rows in data frame.\n",
    "    shuffled_index = np.random.permutation(df.index)\n",
    "    rand_df = df.reindex(shuffled_index)\n",
    "\n",
    "    # Divide number of rows in half and round.\n",
    "    last_train_row = int(len(rand_df) / 2)\n",
    "    \n",
    "    # Select the first half and set as training set.\n",
    "    # Select the second half and set as test set.\n",
    "    train_df = rand_df.iloc[0:last_train_row]\n",
    "    test_df = rand_df.iloc[last_train_row:]\n",
    "    \n",
    "    k_values = [5]\n",
    "    k_rmses = {}\n",
    "    \n",
    "    for k in k_values:\n",
    "        # Fit model using k nearest neighbors.\n",
    "        knn = KNeighborsRegressor(n_neighbors=k)\n",
    "        knn.fit(train_df[train_cols], train_df[target_col])\n",
    "\n",
    "        # Make predictions using model.\n",
    "        predicted_labels = knn.predict(test_df[train_cols])\n",
    "\n",
    "        # Calculate and return RMSE.\n",
    "        mse = mean_squared_error(test_df[target_col], predicted_labels)\n",
    "        rmse = np.sqrt(mse)\n",
    "        \n",
    "        k_rmses[k] = rmse\n",
    "    return k_rmses\n",
    "\n",
    "k_rmse_results = {}\n",
    "\n",
    "for nr_best_feats in range(2,7):\n",
    "    k_rmse_results['{} best features'.format(nr_best_feats)] = knn_train_test(\n",
    "        sorted_features[:nr_best_feats],\n",
    "        'price',\n",
    "        numeric_cars\n",
    "    )\n",
    "\n",
    "k_rmse_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best result is with two best features: `engine-size` and `horsepower`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameter Tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the above results, I can now optimize the model that performed the best."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'2 best features': {1: 2783.6204237227344,\n",
       "  2: 2657.7963807419765,\n",
       "  3: 2792.586573031673,\n",
       "  4: 2891.5329686923255,\n",
       "  5: 2949.8817277180374,\n",
       "  6: 3096.402601694776,\n",
       "  7: 3164.681969020496,\n",
       "  8: 3413.228359192009,\n",
       "  9: 3748.6716603306486,\n",
       "  10: 4080.7125057341937,\n",
       "  11: 4215.6372280600335,\n",
       "  12: 4275.421524277872,\n",
       "  13: 4373.901683035496,\n",
       "  14: 4424.285137239815,\n",
       "  15: 4539.505493095937,\n",
       "  16: 4667.307671446768,\n",
       "  17: 4729.605305844226,\n",
       "  18: 4790.556632159094,\n",
       "  19: 4824.3866193292615,\n",
       "  20: 4840.850914693829,\n",
       "  21: 4837.429062000271,\n",
       "  22: 4831.16988267597,\n",
       "  23: 4861.679492959275,\n",
       "  24: 4903.346008862579},\n",
       " '3 best features': {1: 3399.8148100410203,\n",
       "  2: 3497.191103423058,\n",
       "  3: 3333.6966577570593,\n",
       "  4: 3355.8842294742026,\n",
       "  5: 3580.7376651928435,\n",
       "  6: 3732.943016673517,\n",
       "  7: 3639.9439408462786,\n",
       "  8: 3747.4209132113137,\n",
       "  9: 3986.593913133887,\n",
       "  10: 4005.354888715163,\n",
       "  11: 4121.687230061635,\n",
       "  12: 4255.700651624227,\n",
       "  13: 4328.476829895253,\n",
       "  14: 4332.216494947217,\n",
       "  15: 4388.225713011904,\n",
       "  16: 4408.838883583756,\n",
       "  17: 4404.781029718083,\n",
       "  18: 4447.577705091259,\n",
       "  19: 4537.049753345422,\n",
       "  20: 4592.444230865941,\n",
       "  21: 4636.731219491763,\n",
       "  22: 4721.248544133379,\n",
       "  23: 4787.943506313775,\n",
       "  24: 4802.894378990491},\n",
       " '4 best features': {1: 2952.725686581471,\n",
       "  2: 3131.704952720018,\n",
       "  3: 3129.692821910155,\n",
       "  4: 3241.4320776448717,\n",
       "  5: 3487.340917327035,\n",
       "  6: 3637.0381471429987,\n",
       "  7: 3606.195077860286,\n",
       "  8: 3809.9307026308247,\n",
       "  9: 3875.274902378068,\n",
       "  10: 3997.1583055842293,\n",
       "  11: 4162.564050411074,\n",
       "  12: 4289.486490995821,\n",
       "  13: 4368.061602779942,\n",
       "  14: 4416.304772968801,\n",
       "  15: 4434.013914355171,\n",
       "  16: 4441.4634909198785,\n",
       "  17: 4512.996303789127,\n",
       "  18: 4523.575629742228,\n",
       "  19: 4534.834065236792,\n",
       "  20: 4620.211598150367,\n",
       "  21: 4688.356509517293,\n",
       "  22: 4731.46717779913,\n",
       "  23: 4763.535312989311,\n",
       "  24: 4751.601375872476},\n",
       " '5 best features': {1: 2824.7061233282866,\n",
       "  2: 2915.6731645496975,\n",
       "  3: 3012.4204546509704,\n",
       "  4: 3202.8876051367483,\n",
       "  5: 3410.2170133901805,\n",
       "  6: 3618.4509432660384,\n",
       "  7: 3622.6290209234803,\n",
       "  8: 3848.635835654326,\n",
       "  9: 3977.8149139381726,\n",
       "  10: 3994.8132211260104,\n",
       "  11: 4159.843526607947,\n",
       "  12: 4294.3389473154875,\n",
       "  13: 4380.848359486949,\n",
       "  14: 4466.368754416089,\n",
       "  15: 4522.420711094978,\n",
       "  16: 4536.427578452413,\n",
       "  17: 4587.098443664006,\n",
       "  18: 4622.107837952761,\n",
       "  19: 4612.890107622797,\n",
       "  20: 4632.693976139521,\n",
       "  21: 4712.917548435062,\n",
       "  22: 4676.301064518744,\n",
       "  23: 4691.189310956096,\n",
       "  24: 4755.990767231825}}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def knn_train_test(train_cols, target_col, df):\n",
    "    np.random.seed(1)\n",
    "    \n",
    "    # Randomize order of rows in data frame.\n",
    "    shuffled_index = np.random.permutation(df.index)\n",
    "    rand_df = df.reindex(shuffled_index)\n",
    "\n",
    "    # Divide number of rows in half and round.\n",
    "    last_train_row = int(len(rand_df) / 2)\n",
    "    \n",
    "    # Select the first half and set as training set.\n",
    "    # Select the second half and set as test set.\n",
    "    train_df = rand_df.iloc[0:last_train_row]\n",
    "    test_df = rand_df.iloc[last_train_row:]\n",
    "    \n",
    "    k_values = [i for i in range(1, 25)]\n",
    "    k_rmses = {}\n",
    "    \n",
    "    for k in k_values:\n",
    "        # Fit model using k nearest neighbors.\n",
    "        knn = KNeighborsRegressor(n_neighbors=k)\n",
    "        knn.fit(train_df[train_cols], train_df[target_col])\n",
    "\n",
    "        # Make predictions using model.\n",
    "        predicted_labels = knn.predict(test_df[train_cols])\n",
    "\n",
    "        # Calculate and return RMSE.\n",
    "        mse = mean_squared_error(test_df[target_col], predicted_labels)\n",
    "        rmse = np.sqrt(mse)\n",
    "        \n",
    "        k_rmses[k] = rmse\n",
    "    return k_rmses\n",
    "\n",
    "k_rmse_results = {}\n",
    "\n",
    "for nr_best_feats in range(2,6):\n",
    "    k_rmse_results['{} best features'.format(nr_best_feats)] = knn_train_test(\n",
    "        sorted_features[:nr_best_feats],\n",
    "        'price',\n",
    "        numeric_cars\n",
    "    )\n",
    "\n",
    "k_rmse_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's view these results as a visual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fe5b530f198>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYVdX6wPEvCIimIqiAAooDqKgIKqCVQ86zVkrOWmqZ\nt0xtupWa9uuKlpqzOQ9lTul1BKe6qFGGipg5oQIJR3AWUJnP+v1x8iQKiHgm9f08z37CffbwnhNn\nv6y91l6vlVJKIYQQQjwia3MHIIQQ4skkCUQIIUSxSAIRQghRLJJAhBBCFIskECGEEMUiCUQIIUSx\nGD2BeHp60rBhQ/z9/QkMDATgxo0btG/fntq1a9OhQwdSUlL024eEhODl5UXdunXZvXu3fn1UVBS+\nvr54e3szevRoY4cthBDiIYyeQKytrQkPD+fo0aNERkYCMGXKFNq2bcuZM2do3bo1ISEhAJw8eZL1\n69dz6tQpwsLCGDlyJHcfU3n77bdZunQpMTExxMTEsGvXLmOHLoQQohBGTyBKKbRabZ51W7ZsYfDg\nwQAMHjyYzZs3A7B161b69OmDjY0Nnp6eeHl5ERkZSXJyMmlpaQQEBAAwaNAg/T5CCCHMw+gJxMrK\ninbt2hEQEMCSJUsAuHTpEi4uLgC4urpy+fJlADQaDR4eHvp93dzc0Gg0aDQa3N3d9evd3d3RaDTG\nDl0IIUQhbIx9goiICCpXrsyVK1f0/R5WVlZ5trn/30IIISyf0RNI5cqVAahUqRI9e/YkMjISFxcX\nfSskOTkZZ2dnQNfiSEhI0O+bmJiIm5tbgevzI8lICCGK51GnRjTqLaw7d+5w69YtAG7fvs3u3btp\n0KAB3bt3Z8WKFQCsXLmSHj16ANC9e3fWrl1LVlYWcXFxnDt3jsDAQFxdXXFwcCAyMhKlFKtWrdLv\nkx+llCxK8fnnn5s9BktZ5LOQz0I+i8KX4jBqC+TSpUu8/PLLWFlZkZOTQ//+/Wnfvj1NmjQhODiY\nZcuWUa1aNdavXw+Aj48PwcHB+Pj4YGtry/z58/Utinnz5jFkyBAyMjLo3LkzHTt2NGboQgghHsKo\nCaR69epER0c/sN7JyYm9e/fmu88nn3zCJ5988sD6xo0bc/z4cYPHKIQQonjkSfSnWKtWrcwdgsWQ\nz+If8ln8Qz6Lx2Olinvzy0JZWVkV+36eEEI8q4pz7TT6KCwhhOl5enry119/mTsMYYGqVatGfHy8\nQY4lLRAhnkLyPRAFKeh3ozi/M9IHIoQQolgkgQghhCgWSSBCCCGKRRKIEMJirVy5kubNm5vl3OPG\njaNSpUpUqVLFLOd/EkgCEUKYVKtWrShVqhTlypWjbNmy1K1bt9DtjTG/nbW1NbGxsQW+npCQwIwZ\nMzh9+jQXL158rHPt27cvzyzjTxNJIEIIk7KysmL+/PmkpqaSlpbGqVOnzBJDYf766y8qVqxIhQoV\nHvtcSqnHSoK5ubmPHYOxSAIRQpjcowwX1Wq1vPvuu5QvXx4fHx9+/vln/WupqakMGzaMKlWq4OHh\nwfjx4/XHPn/+PK1ataJ8+fI4OzvTt29fAFq2bIlSCl9fX8qVK8eGDRvynO+nn36iffv2XLx4kXLl\nyvHGG28AcPDgQV544QUcHR3x9/dn3759+n1WrFiBj48P5cqVo1atWixatAjQTSjbuXNnLl68SNmy\nZSlXrhzJycm8/vrrTJgwQb///a2U6tWr89VXX9GwYUPKlCmDVqslKSmJXr164ezsTM2aNZkzZ45+\n+0OHDhEQEICDgwOVK1fmgw8+KPLn+1jUU+YpfEtCPDJL/h60atVKOTs7q0qVKqkXX3xRhYeHF7jt\nihUrlI2NjZo1a5bKyclR69atUw4ODurGjRtKKaV69uyp3n77bZWenq6uXLmigoKC1KJFi5RSSvXt\n21dNnjxZKaVUZmamioiI0B/XyspKxcbGFnje8PBw5eHhof+3RqNRFSpUUDt37lRKKbV3715VoUIF\ndfXqVaWUUqGhoSouLk4ppdT+/ftV6dKl1dGjR/M9llJKDRkyRI0fP77A83l6eip/f3+l0WhURkaG\n0mq1qnHjxurLL79UOTk5Ki4uTtWsWVPt3r1bKaVUs2bN1Pfff6+UUur27dvq999/L/C9FfS7UZzf\nGWmBCPEMsrIyzFIcX331FbGxsWg0GoYPH063bt2Ii4srcHsXFxdGjRpFiRIlCA4Opnbt2uzYsYPL\nly8TFhbGN998g729PRUrVmT06NGsXbsWAFtbW/766y80Gg12dnY8//zzeY6rHqEV9P3339OlSxc6\ndOgAQJs2bWjSpAmhoaEAdOrUCU9PTwCaN29O+/btOXDgwKN8LA947733qFKlCiVLluTQoUNcvXqV\nzz77jBIlSuDp6cmwYcPyvNdz585x7do1SpcuTWBg4GOdu6gkgQjxDFLKMEtxBAQE8Nxzz2Fra8ug\nQYN44YUX9Bfi/NxfPK5atWpcvHiRv/76i+zsbCpXroyTkxOOjo6MGDGCK1euAPD111+j1WoJDAyk\nQYMGLF++vHgBo+sTWb9+PU5OTvpzRUREkJSUBEBYWBjNmjWjQoUKODo6EhYWxtWrV4t9PiBPGe+7\nifDe84eEhOjLgS9btowzZ85Qp04dgoKC2LFjx2Odu6hkLiwhhFk9bAoNjUaT598XLlygR48eeHh4\nYG9vz7Vr1/LtpHZ2dtb3RURERNC2bVtatmxJjRo1HjlGDw8PBg0axMKFCx94LSsri169evH999/T\no0cPrK2tefnll/XvKb/YnnvuOe7cuaP/991EdK979/Pw8KBGjRqcOXMm3/hq1qzJDz/8AMDGjRvp\n1asX169fp1SpUo/2Rh+RtECEECaTkpLC7t27yczMJDc3l9WrV3PgwIFCC8RdunSJOXPmkJOTw4YN\nGzh9+jSdO3fG1dWV9u3bM2bMGNLS0lBKERsby/79+wH48ccf9cmnfPnyWFtbY22tu+S5uroWOoz3\nfgMGDGDbtm3s3r0brVZLRkYG+/bt4+LFi2RlZZGVlUXFihWxtrYmLCyM3bt36/d1cXHh2rVrpKam\n6tf5+fkRGhrKjRs3SE5OZtasWYWePzAwkLJly/LVV1+RkZFBbm4uJ06c4PDhwwCsXr1a3+JxcHDA\nyspK/16NSRKIEMJksrOzGTduHM7OzlSqVIl58+axZcsWatWqVeA+TZs25ezZs1SsWJHx48ezceNG\nHB0dAVi1ahVZWVn4+Pjg5ORE7969SU5OBnQjk4KCgihXrhw9e/Zk9uzZ+n6KiRMnMmjQIJycnPjx\nxx8fGre7uztbtmxh8uTJVKpUiWrVqjFt2jS0Wi1lypRh9uzZ9O7dGycnJ9auXZun5Hbt2rXp27cv\nNWrUwMnJieTkZAYOHIivry+enp507NiRPn365Dnf/a0Wa2trtm/fTnR0NNWrV8fZ2Znhw4frk9LO\nnTupV68e5cqVY8yYMaxbt46SJUs+/H/IY5LZeIV4Csn3QBREZuMVQghhdpJAhBBCFIskECGEEMUi\nCUQIIUSxSAIRQghRLJJAhBBCFIskECGEEMUiCUQIIUSxSAIRQlgsKWlr2SSBCCFMJisri2HDhuHp\n6YmDgwONGjVi586dhe4jJW0tlyQQIYTJ5OTkULVqVQ4cOEBKSgr/93//R3BwMBcuXDBpHFLS1jAk\ngQghTKZ06dJMmDBB/xd5ly5dqF69OkeOHClwHylpKyVtTeYpfEtCPLIn5XuQnJysSpUqpc6cOZPv\n61LS1rJL2kpBKSGeQVaTDNOvoD4v/oy/OTk5DBgwgCFDhuDt7V3gdndL2gIEBwczffp0duzYQbt2\n7QgLCyMlJYWSJUtib2/P6NGjWbx4McOHD89T0tbNzc1oJW0HDhxIp06d9NveW9LWz8/vUT6SPO6W\ntAWIjIzUl7QF8pS0bdeuXZ6SthUqVDBZSVtJIEI8gx7nwm+Q8yvFgAEDKFmyZJ5bMfkpSknbu8dU\nSlG1alVAV9J23LhxBAYG4uTkxNixY3n99deLFe/dkrbbtm3TnysnJ4fWrVsDupK2X3zxBTExMWi1\nWtLT0/H19S3Wue4qqKTt3fNrtVpatGgB6Erajh8/njp16lCjRg0mTJhAly5dHuv8RSEJRAhhckOH\nDuXq1auEhoZSokSJQreVkrZS0lYIIQAYMWIEp0+fZuvWrdjZ2T10eylpKyVthRCCCxcusGjRIqKj\no3FxcdGPTFqzZk2B+0hJ22e8pK1Wq6Vx48Z4eHiwdetWJk2axOLFi3F2dgZg8uTJdOzYEYCQkBCW\nLVuGjY0Ns2bNon379gBERUUxZMgQMjIy6Ny5MzNnzsz/DUkpTyHkeyAK9MSVtJ01axb16tXLs27s\n2LFERUURFRWlTx6nTp1i/fr1nDp1irCwMEaOHKl/Q2+//TZLly4lJiaGmJgYdu3aZYrQhRBCFMDo\nCSQxMZHQ0FCGDRuWZ31+mW7Lli306dMHGxsbPD098fLyIjIykuTkZNLS0ggICABg0KBBbN682dih\nCyGEKITRE8iYMWP4+uuvH7inN3fuXPz8/Bg2bBgpKSmAbrTFvU9jurm5odFo0Gg0eYa0ubu7PzAy\nQwghhGkZdRjvjh07cHFxwc/Pj/DwcP36kSNHMmHCBKysrBg3bhzvv/8+S5YsMdh5J06cqP+5VatW\ntGrVymDHFsJS5ebCnj2wcqW5IxFPgvDw8DzX5eIwagKJiIhg69athIaGkp6eTlpaGoMGDWLVqlX6\nbYYPH063bt0AXYsjISFB/1piYiJubm4Fri/IvQlEiKfdqVO6pPHdd1DFTYtr78mw1txRCUt3/x/X\nkyZNeuRjGPUW1uTJk7lw4QKxsbGsXbuW1q1bs2rVKv0wO4BNmzZRv359ALp3787atWvJysoiLi6O\nc+fOERgYiKurKw4ODkRGRqKUYtWqVXmGyQnxrLl1CxYsgKAgaNMGlIIN269R6b2u3KwoA0yEaZjl\nSfSPPvqI6OhorK2t8fT01D/d6ePjQ3BwMD4+Ptja2jJ//nx938m8efPyDOO9O3JLiGeJVqtraXz6\nKTRtChMnQrt2cPTSIXpv6E0vn16EtAnB7o2HP6AnxOMyyXMgpiTj38XT6pdfYPRosLWFmTN1rQ+l\nFAsOL2Bi+ES+7fotr9R9BZDvgSjYE/cciBCi+OLj4bXXoF8/eP99+PVXXfK4lXWLAf8dwMIjC4l4\nI0KfPJ4mUtLWskkCEcJCpaXpblU1aQL168Pp09C3L2TlZrLtzDaClgRR0tqOg6+E4ZV4B7Zuhblz\n4aOPzB16kZw9e5ZSpUoxaNCgQreTkraWS2bjFcLC5ObqRlWNG6fr3zh2DCq5ZrHn/B42HltDys4t\nDD5fhl8TS+Bw6Uew2gjVqumWqlV1yxPgnXfeMVndivs9aSVtHzZjsblIC0QIC7JvHwQEwNKl8ON/\ns+gzLoxJEQN5/XUnSgx5nfnD/sua49507zIWhx+3Q2IipKbC8eOwfTvMnw///re538ZDrV27FkdH\nR9q0afPQbaWkrZS0NZmn8C2JZ8D580q9+qpSHtWy1CdLdqq3fxighvZ9Tu1tUkFllLFX6c2fV2rO\nHKUSE4t0PEv+HqSkpChvb2+l0WjUxIkT1cCBAwvcVkraWnZJW2mBCGFGaWnw0b9z8HtlDzdqDaRH\ngCOdZ/fhm2EbmHMtiDYjplIy9gL2+yPgnXegkAdoH4mVlWGWYpgwYQLDhw8vcuf03ZK2JUqUIDg4\nmNq1a7Njxw4uX75MWFgY33zzDfb29lSsWJHRo0ezdq3uKcp7S9ra2dkZraQtQKdOnfRTxd9b0vZx\n3C1pW7JkSQ4dOqQvaVuiRIk8JW3vvte7JW1Lly5tsluDkkCEMAOtFpYvh1qtdpN6ogq7M14ldO6P\nTMlqzov/nk/Ji5cotesnGDoUKlUyfABKGWZ5RNHR0ezdu5fRo0cXeZ+ilLR1cnLC0dGRESNGcOXK\nFUBX0lar1RIYGEiDBg1Yvnz5I8d7192Stk5OTvpzRURE6CsJhoWF0axZMypUqICjoyNhYWH6Ak/F\nVVBJ27vnDwkJ4fLly4CupO2ZM2eoU6cOQUFB7Nix47HOXVTSiS6EiUVEwLtjbxJQuhe/XtiHg50P\nFUMmQocOlCxdutjHzdZq2Xn9OivumenB0uzbt4+//vqLqlWropTi1q1b5ObmcvLkSX11vftJSVsp\naSvEMy8hAfr1VcwbOpFVF5z5+OJxXFZtpOJvx+Dll6GYyeOPW7cYe+4c7r/9xpQLF+jo5GTgyA3n\nrbfe4vz580RHR3Ps2DFGjBhB165d85SAvZ+UtLXckrbSAhHCyO7cga+/hkOzt/Jl6WHY59wkY9I4\nagwfn6cfQSlFrlJkKUWmVvvPf+/9+Z51f96+zcpLl7iWnc0gFxcO+Pvj/XcSetNcb/Yh7O3tsbe3\n1/+7TJky2Nvb41RI0ru3pK2rq+sDJW0//vhjfHx8uHXrFjVq1ODjjz8GdCOTRo8eTWpqKi4uLvmW\ntM3IyGDRokX06tWr0LjvlrT98MMP6du3LzY2NgQGBrJgwYI8JW2zsrLo1q1bgSVttVotJ0+eZODA\ngezduxdPT0+qV6/O66+/zvTp0/X7FFTSduzYsVSvXp2srCxq167Nl19+CehK2o4dO5b09HSqVav2\ndJW0NSWZwkFYCqVg3TpYPfoQ40sMxznjT/YPbcMrk9ZTppQD2Votqy9d4uuEBGIzMsjUarEG7Kyt\nKWltjZ2Vle7nu/+9b52HvT0DXFx4qXx5rO+74Mj3QBTEkFOZSAIRwgiOHIGZw08wOP4jGqr/MbtD\nWTp8uZYXa71EplbL8qQkpiYkUNPens+qVaNpuXLYWVtTwkBPXcv3QBREEkgh5IsjzCk5Gb55N5bG\n2yfS0WYLM1ppyXprGOM7TEZZ27E4KYmvL1ygYZkyjKtWjWYODkaJQ74HoiCGTCDSByKEgez57y0S\n+n3MeNawpaMz7V6swpw+K6nr0ojZFy/yTUICLzg4sLVBAxqVLWvucIV4bJJAhDCADTMSqPtxNyq8\nUIYmrawJbhnMlmYfsyj5Cl1+/532jo7sbdiQ+mXKmDtUIQxGEogQj0EpmP/Odnou68+SDlbs6lmX\n+R13szerPPUOH6VHxYpE3DM6SoiniSQQIYohV5vL9pgwdn06kUk7o/jh3Y60H/0fUu+UpVd8Mn2c\ny3CkcWM8jfwglxDmJJ3oQjyChJQElh5dypIjS3hrqxVDD9/BZts2rF9sQrc//6S6vT3TatakignG\n4BdGvgeiIDIKqxDyxRGGlqPNIexsGAuPLOTXhF95rfprdBmbSN07Sbgf2UqSqyMd/viDVytV4j/V\nqxulANKjku+BKIiUtBXCBC6kXODz/32O50xPQn4JoZdPLw61O8qw10/gYW9H9b/2cbJiWV44epR3\n3dyYXKOGRSSPp4mUtLVskkCEuE+uNpdRYaPwX+jPjYwbhPUP49ehv9Jc05QSQW3IDnwe3zMb+Dk7\nkw5//MFsLy/euWfmVFG4gQMHUrlyZRwcHKhZsyb/+c9/Ct1eStpaLulEF+IeGTkZ9NvYj5TMFGJH\nxeJgr3vQ78+Ze3Ed24+YoVN5fvHrrL50ibHnzrGxXj2aly9v5qifLJ988gmLFy/G3t6emJgYWrRo\nQZMmTfS1NkxBStoahrRAhPjbjfQbtP+uPSVtShLaL1SfPKJHLsJ57ABip26g2aIhTLtwgU9jY/nZ\nz0+SRzH4+PjoJ1RUSmFra0ulQmqeSElbKWlrMk/hWxImkJCSoOrNq6dGh41Wudpc3cqcHHWszRh1\nroS3+mNjjMrVatXos2dV/chIlZCebt6AH8LSvwcjR45UpUuXVjY2NmrBggUFbiclbS27pK3cwhLP\nvJNXTtJpdSfeCXiHD57/QDcaJe0WMU36cvvCbcoc/A0vfwf6njxJclYWB/z8KG9ra+6wH4tVeLhB\njqNatSrWfvPmzWPu3Lns37+fV199lcaNGxMQEJDvtndL2gIEBwczffp0duzYQbt27QgLCyMlJYWS\nJUtib2/P6NGjWbx4McOHD89T0tbNzc1oJW0HDhxIp06d9NveW9LWz8/vUT8avbslbQEiIyP1JW2B\nPCVt27Vrl6ekbYUKFUxW0lYSiHimRVyI4JX1rzCt3TQGNhwIgLqTTmydzpzJrEXTc5uwc4aOf/yB\ns50du3x9sbfQ+9GPorgXfkOysrKiZcuW9O7dmzVr1hSYQIpS0hZ0CUEpRdWqVQFdSdtx48YRGBiI\nk5MTY8eO5fXXXy9WrHdL2m7btk1/rpycHFq3bg3oStp+8cUXxMTEoNVqSU9Px9fXt1jnuqugkrZ3\nz6/VamnRogWgK2k7fvx46tSpQ40aNZgwYQJdunR5rPMXhSQQ8czacnoLw7YN47uXv6NjrY66lTk5\nxDTpx/k77rQ4v4Tbz2XTNvoPXipfnm9q1Xqg7oZ4fDk5OZQuZKoXKWkrJW2FsCiLjyxmxI4RhPYL\n/Sd5KMXp1iO5EncL/+gVXCyZzvNRUQx0cWGmJA+DuHLlCuvWreP27dtotVp27drFhg0b8lTwu5+U\ntLXckraW3dNWDE/hWxIGpNVq1aTwSarGrBoq5mpMntdO9R6vom0bq7NRqSr8xg3l/Msv6rukJDNF\n+ngs9Xtw5coV1bJlS+Xo6KjKly+vAgIC1NatWwvcfsWKFerFF19U7777rnJwcFC1a9dWe/fu1b+e\nmpqq3n77beXu7q7Kly+vGjVqpNatW6eUUuqjjz5Sbm5uqmzZsqpWrVpqyZIl+v0WLlyoKleurBwd\nHdWGDRseOG9+Hd+RkZGqZcuWysnJSTk7O6uuXbuqhIQEpZRS8+fPVy4uLsrR0VENGjRI9e3bN08n\n+dChQ1WFChWUo6OjSkpKUhkZGeq1115T5cqVUw0bNlQzZ87Mc77q1aurn376Kc/5k5KSVN++fZWr\nq6tycnJSzZo1028zYMAA5ezsrMqWLavq169f6Gda0O9GcX5nZCoT8czI1ebyr9B/EamJJLR/KK5l\nXPWvnR0zjxJzZpIa9gs/1cni6wsX+L5uXdoWUqvbksn3QBRECkoJ8YjSs9Ppt6kfaZlphA8Jp1zJ\ncvrX/pq2gedmh3B85T4WV77CX5cz+b1xY6r9/ayCECJ/0gcinnrX06/T7rt2lLIpRWj/0DzJI2n1\nzzz38b/YNnMr73ldpaKtLb/4+0vyEKII5BaWeKpplZaAxQG0rNaSae2nYW31z99MN36KQtuhI1+G\nrOf7522ZVrMmg11dCznak0O+B6IgcgtLiCLaHrMdaytrprefnmdY5K1j57nTrQdDZ6ziXKA9e+vX\np6GUmxXikUgCEU+1ab9O44NmH+RJHlkJlzjVtQ/9vl1IXX83Dtep88Q/WS6EOZikD0Sr1dKoUSO6\nd+8OwI0bN2jfvj21a9emQ4cOpKSk6LcNCQnBy8uLunXr5hlLHRUVha+vL97e3owePdoUYYsnXKQm\nkgspF3jV51X9Ou3NVDb0f5f2c79kSPN6bK5fX5KHEMX1yAN/i2HGjBmqf//+qlu3bkop3fjsqVOn\nKqWUmjJlivr444+VUkqdOHFC+fn5qezsbP1kYVqtVimlVGBgoIqMjFRKKdWpUyf9pGb3M9FbEk+A\n4A3B6pvfvtH/O+dOuho7Ypxy3BSqQpOumTEy46tWrZoCZJHlgaVatWr5/s7Ao187jd4CSUxMJDQ0\nlGHDhunXbdmyhcGDBwMwePBgNm/eDMDWrVvp06cPNjY2eHp64uXlRWRkJMnJyaSlpennyhk0aJB+\nHyHyE38znp9if2Ko/1AAbmZm0m7xGsIa+nCgWUs6uT6Zz3cUVXx8vH5uKFlkuXeJj4832O+Z0RPI\nmDFj+Prrr/Pcg7506RIuLi6AbkqBy5cvA7o5b+6dE9/NzQ2NRoNGo8kzsZi7u/sD8+MIca+ZB2cy\n1H8oZezKcCQ1lYbbd1Mh/gph7XtQz7XgeZeEEEVn1E70HTt24OLigp+fH+GFTB8tdaRFcSmluJyd\nTXxGhn45cyuF7zKq4ukSxJwDByh9K4MPFm2md8gMqtWQ5CGEoRg1gURERLB161ZCQ0NJT08nLS2N\ngQMH4urqqm+FJCcn4+zsDOhaHAkJCfr9ExMTcXNzK3B9QSZOnKj/uVWrVrSygKmrxeO7mZ3N4qQk\n4v5OFLHpGfyVkUFJVQKnLHueS7OnxBV7ktOOYJ92lRtbfJh27lPa5RzgzpY91GzkYO63IITFCA8P\nL/QP+6Iw2YOE+/btY/r06WzdupWPPvqIChUq8PHHHzN16lRu3LjBlClTOHnyJP379+f3339Ho9HQ\nrl07zp49i5WVFU2bNmX27NkEBATQpUsXRo0aRceOHR98Q/IA1VOry4FTRJ3Pxu5oBVJj7Ll1zh5X\nq5JUc7bBwwM8PKCyexZfplVnSZ1VdJk6AVt3F6xWroSyZc0dvhAW7Yl5kPDf//43wcHBLFu2jGrV\nqrF+/XpAVys5ODgYHx8fbG1tmT9/vv721rx58xgyZAgZGRl07tw53+Qhnl6fb7nBzqybTM8MoGlf\nXcJwdYX7azutjF5Dny3uvDzmdRg6FMaPB1NMay3EM0imMhEWTSmYMiOX8W6HmVa9JqODKhayreLT\n16syaWsadkuXw8svmzBSIZ5sxbl2SgIRFisnB955BzaVj6PJq7cJDahf8Ma5ucSP6EOJzVtw/+kw\nVo9ZTlSIZ01xrp3SthcWKTUVunaFE7duo+16kSW+XgVvfOMGdOnC9V9/JmLDN5I8hDARSSDC4iQk\nwIsvgmd1hRobw6TqnlQpWTL/jU+dgqAgrlSrRM8hJXml+XBThirEM00SiLAoUVHQrBkMHgyNxieR\no7SMqFIl/423bYOWLeHTT/mgkzUjm43CroSdaQMW4hkmfSDCYmzbBm+8Ad9+Cy92y6LBoUPsbdgQ\n3/unWVcKQkJg/nz48Uc0Ph40WNCA86PO41jK0TzBC/GEe2KG8QpxvzlzdDlh+3YICoJ+J8/xuqvr\ng8nj9m14/XW4cAEiI6FKFebs/TcDfQdK8hDCxCSBCLPKzYWxY2HPHvj1V/D0hF3Xr3MwNZUlf0+e\nqff77/A9MJJ8AAAgAElEQVTWW+DnB+HhYG9PWmYaS6KWcGj4IXOEL8QzrdA+kJ9//ln/c1xcXJ7X\nNm3aZJyIxDPj1i3doxp//vlP8riTm8vImBjme3lRukQJyM6GtWuhaVPo21c3rnf5cvi7Zvmyo8to\nXb011R2rm/fNCPEMKrQPpFGjRkRFRT3wc37/thTSB/JkuHgRunWDhg11fR52f/d9fxIbS3xGBmtc\nXGDRIpg3D7y8YPRo3bjeex49z9HmUGt2Ldb1WkeQe5CZ3okQTweD94Hce7D7DywXaVFcWVnQqRO8\n8gpMmAB3J2M+fusWSxMS+GPjRvjuO13zZMcOXZbJx8aTG6nqUFWShxBmUmgCuXea9funXJcp2EVx\nTZ4MVavekzy0WrShobx58yZf7tyJa+3acOYM/D1Lc36UUkz/bTqfNf/MdIELIfIoNIHExsbSvXt3\nlFL6n0H35b2/T0SIooiO1o2+jY4Gq1tpsGIFzJnDwg4dKNG1K8OWLoWCHhq8x4ELB7iZcZNutbsZ\nP2ghRL4K7QPZt29foTu3bNnS4AE9LukDsVzZ2RAQAGPGwOCbs+CLL6B1ay6+9x4NrawI9/Oj3nPP\nFelY3dd0p4tXF95q8paRoxbi2WD0yRSzs7P5888/cXNz0xeBsjSSQCzXF1/oRuJuH7Ubq6FvwP79\nUKMGwSdO4F2qFF/WqFGk45y5eoYWK1oQ/148pWxLGTlqIZ4NBp9MccSIEZw4cQKAlJQUGjZsyKBB\ng/D392fNmjXFj1Q8c44dg7lzYcmXyVi9PkTXSV6jBjuuXePorVt8Vq1akY8147cZvN3kbUkeQphZ\noQnkwIED1KtXD4Dly5fj7e3N8ePHOXLkCF999ZVJAhRPvuxs3cPjUyZrqfzRQBg+HF56idu5ufwr\nJoZvvb0pdX9lqAJcvn2ZDSc3MDJgpJGjFkI8TKEJxM7un4np9uzZQ8+ePQFwdXU1blTiqTJ1qm5A\n1euXpkBmpq5KIPB5XBwtypenjWPRpyCZf2g+vX164/ycZd5CFeJZUugorPLly7N9+3bc3NyIiIhg\n6dKlAOTk5JCenm6SAMWT7fhxmDUL/lwYgdXI2XD4MNjYcDQtje8uXeLP+6crKUR6djoLDi9g/5D9\nRoxYCFFUhSaQhQsXMmrUKJKTk5k5c6a+5fHTTz/RpUsXkwQonlw5ObpbVzPGXcdlTD9YvBjc3dl/\n8yYDT51iWs2aVLIr+vTrq46tIsgtiNoVaxsxaiFEUcl07kJvgUbDqTt3mOTpiaOt7WMfLyQE/vez\nYleZV7CqVo3sGTOYFB/P0uRkltauTecKFYp8LK3SUndeXRZ1XURLT8sbPi7Ek87gU5mMGjWq0J1n\nz579SCcTlmtuYiLTExNp6+hIvUOHmFazJn2dnYs948CJEzBjBpx5dx5WWy5wfsUK+h89iqOtLUcb\nN8a1CA8L3iv0bChl7crSolqLYsUjhDC8QlsgdnZ21K9fn+DgYKpUqfJAdho8eLDRA3xU0gJ5dAs0\nGqZeuEC4nx+epUpxMCWFt2JicLazY4GXF7VKl36k4+XkwPPPw8cdonnl23Z8v3cvY2/dYly1arzr\n5oZ1MZLSSytfYnij4fRr0O+R9xVCPJzBWyBJSUls2LCBdevWYWNjw2uvvUavXr0oX778YwUqLMei\nixcJuSd5ADR1cOBI48bM0mhoGhXFe+7ufFS1KiWti1YBefp0cHnuFm22Dqb/ypUcy8xkb8OGNLy/\nOFQRRSVFce76OXr79C7W/kII4yhyH0hiYiJr165lxowZTJ06lYEDBxo7tmKRFkjRLUtK4vP4eP7X\nsGGBrYwLGRm8e/YsZ+7cYYG3Ny89ZMjtyZO6MuU/vPYhb3ZoQafatZlWs6autkcxDdg0gIYuDfnw\nhQ+LfQwhROGMNpVJVFQUa9asYc+ePTRu3Jj3338fHx+fYgdqTJJAimZlcjKfxcbys58f3kW4RbXl\n6lXePXuWl8qXL3D0VE4OPN9cS93XtrGrOixq3Jju7u6PFWdiaiK+C3yJfS+W8vbS8hXCWAyeQCZM\nmMCOHTuoW7cuffr0oWPHjtjYWHYVXEkgD/d9cjIfx8byU8OG1Cni5IUAt3Jy+Dw+nu8uXWJy9eq8\nUblynv6MT2als9QhioapR1jZogVV/PweO9aP93xMZm4mMzvOfOxjCSEKZvAEYm1tTfXq1Sn991+o\nd0fkKKWwsrLijz/+eIxwjUMSSOHWXLrE++fPs7dhQ3weIXncKzotjRExMdhYWfGttzf1y5Rh+tFL\nfHThLJP3/siH9epgPWLEY8d6K+sWnjM9OTT8kJSsFcLIDN6JLjU/ni7rL19m7Pnz7PH1zT953LkD\nly7pipMXMlLKr2xZIho1YtHFi7x07Bg+pUsTeS6LHzbt4rXsRJj9jUHiXXZ0GS9Vf0mShxAWqtgP\nEq5bt47XXnvN0PE8NmmB5G/jlSv8KyaGXQWNhrp+Hdq2hcREyM2FJk10S0CAbqlSJd+kkpyZyTvr\nrlJzZiRTbozF6uhRMMAovVxtLl5zvFj9ymqaeTR77OMJIQpn8BbI7du3WbhwIefPn6devXqMGDGC\nLVu28Nlnn+Hl5WWRCUQ8aPOVK4yMiWGnr2+hySO5flsmvzCVkjcvUfniYapuOYTn0sXUuvEmuZTg\n1HMBnCjVhGjbAKKsm5CcW4nMzJI4p+ewzuYtrHZsNUjyANh8ejOuZVwleQhhwQptgbz66quULVuW\nZs2asWfPHhISErC3t2fWrFn4GaCD1BikBZLXtqtXGXbmDKG+vjQuW/bBDa5dg7ZtudO8Pd4bpzBs\nuBWVK+uqyuoXO4VDygWcYg/jeO4QZU4fovSpI6hy5cn2D8DuwjlK9O8LH31ksLhfWPYCY5qOoZdP\nL4MdUwhRMIN3ovv6+uo7ynNzc6lcuTIXLlzA3t7+8SI1Ikkg/wi9do0hp0+zo0EDAsqVe3CDv5OH\nateel89MoZaXFdOmFfHgWi2cO6ebXff6dRg5Eor4oOHDHEw8SL+N/Tj77llKWBf/+REhRNEZ/BbW\nvUN2S5Qogbu7u0UnD/GPXdevM+T0abbWr19o8qBDB76tGkLCT1as3/AIJ7C2Bm9v3WJgM36bwXtB\n70nyEMLCFdoCKVGiBM/9PVpHKUV6ejqlS5fWD+NNTU01WaBFJS0QiE9PJyAqis316/OCg8ODG1y7\nBm3aQMeOnBgQQquXrPjlF6htAbOkx92Io8niJsS/F0/ZkvncchNCGIXBWyC5ubmPFZAwPaUUw86c\n4X139/yTx9WrupZHp05kTJhM3yArpkyxjOQBMPv32Qz1HyrJQ4gngGU/Vi4e2cKLF0nNzeUDD48H\nX7wneTB5Mh+9Z0WdOvDGG6aPMz8pGSmsPLaSYyOOmTsUIUQRGKbXswCZmZkEBQXh7+9PvXr1+PTT\nTwGYNGkS7u7uNGrUiEaNGrFz5079PiEhIXh5eVG3bl12796tXx8VFYWvry/e3t6MHj3amGE/seLT\n0xkXF8eKOnWwub9D++pV3W2rzp1h8mR2hFqxdSssXFjoM4MmtThqMZ28OuHhkE/yE0JYHmVkt2/f\nVkoplZOTo4KCgtQvv/yiJk6cqKZPn/7AtidPnlR+fn4qOztbxcXFqZo1ayqtVquUUiowMFBFRkYq\npZTq1KmT2rlzZ77nM8FbskharVa1OXpUhcTHP/jilStK+foq9cknSmm16uJFpVxdldq/3/RxFiQr\nJ0t5zPBQhzWHzR2KEM+k4lw7jdoCAfTzaGVmZqLVanH8ezpwlU9nzZYtW+jTpw82NjZ4enri5eVF\nZGQkycnJpKWlERAQAMCgQYPYvHmzsUN/oiy8eJG0/G5d3W15dO0K//kPWmXF4MHw5pvQvLl5Ys3P\njyd/pIZjDRpXaWzuUIQQRWT0BKLVavH398fV1ZVWrVrpp4GfO3cufn5+DBs2jJSUFAA0Gg0e91wA\n3dzc0Gg0aDQa3O+ZFtzd3R2NRmPs0J8Y8enpjI+PZ/n9t67uTR5ffglWVsyYAbdvw/jx5ov3fkop\nZhycwdhmY80dihDiERg9gVhbW3P06FESExPZv38/+/btY+TIkcTGxhIdHY2rqyvvv/++scMwqMOp\nqQw5dYqrWVnmDgWlFEPPnOEDD4+8EyRevQqtW0O3bvrkceQITJ0Kq1eDJc3Kf+DCAVIyUujq3dXc\noQghHoHJLiPlypWjS5cuHD58mJYtW+rXDx8+nG7dugG6FkdCQoL+tcTERNzc3ApcX5CJEyfqf27V\nqhWtWrUyyHu4k5vL5/HxrEpOpkX58nQ5fpyfGjakjBmvxgsvXuRWbi7v31u4KTsbOnbUJY//+z+w\nsuLWLejbF+bM0U22a0lm/DaDMU3HYG1l9L9nhBB/Cw8PJzw8/PEOYuiOmHtduXJF3bx5Uyml1J07\nd1Tz5s3V3r17VVJSkn6bGTNmqL59+yqllDpx4oTy8/NTmZmZKjY2Nk8nelBQkPr999+VVqtVnTp1\nUmFhYfme01hv6X/Xr6taBw+qPidOqEuZmUqr1aqhp06p9tHRKjM31yjnfJi4O3dUxV9+USdu3cr7\nwvTpSrVvr9Tfn51SSr3xhlKDB5s2vqKIuRqjKn1VSd3Oum3uUIR4phXn2mnUP52TkpIYPHgwSim0\nWi0DBw6kTZs2DBo0iOjoaKytrfH09GThwoUA+Pj4EBwcjI+PD7a2tsyfP19fxGrevHkMGTKEjIwM\nOnfuTMeOHY0Zul5KTg4fnz/P9mvXmO/tTfeKFfWvfevtTe+TJxl06hSrfXwoYcLxsNqCbl0lJsLk\nyfDbb/rxuevXw/79EBVlsvCKbObBmbzZ+E1K2z68rK4QwrIUux6IpTLkVCbbr17l7bNn6eTkxFc1\nalDe1vaBbTJyc+l0/Dg+pUsz18tLn/CM7VuNhuXJyUT4++ftOA8Ohrp1YdIkAP76S1fOY8cO3X8t\nybU716g1pxan/nUK1zKu5g5HiGeawWfjfRIZIoFcycrivXPn+D01lcW1a9P676HHBUnNyaFVdDTd\nK1RgYnXjV8+7O9fVfj8/6t7b+ti1Szcr7p9/QqlS5ORAq1bQvbtBZ1o3mMkHJnP2+lmW91hu7lCE\neOYV59opvZb3UEqx5tIlGhw6RBU7O44HBDw0eQCUs7EhzNeX1ZcvMzcx0agx3nvrKk/yyMiAd96B\nuXOhVClAdyerZEn44AOjhlQsmTmZzI2cy5imY8wdihCimCxoMKd5JWZk8PbZs8RnZLC1QQMC85sC\nvRAudnbs9vWl+dGjVLC1pa+Li1HizHfUFejG5/r66ua5AiIiYP58Xb+Hgcp0GNTaP9dS37k+vi6+\n5g5FCFFMz3wC0SrF4qQkxsXF8Y6bGxvr1cOumFfc6qVKEebrS9tjx3CytaWDk5NBY41PT2dCfDz7\n/fzy9nucO6cbn3v0KAA3b0L//rBoka6UuaVJz07ni/1fsKjrInOHIoR4DM98H8iKpCQWXLzI0tq1\nqZ9fvfBiiEhJoeeff7Ktfn2a5jelejFolaLdsWN0cHLio6pV/3lBKV2ro00b+PBDlII+faBSJd3d\nLEs0KXwSf1z+g43BG80dihDib9KJzqN/CNlaLdZWVgYfgrvj2jWGnj7Nz35+eYfZFtMCjYaVyclE\nNGqUN9aNG+Hzz3WtD1tbli2Db76ByEh9V4hFib0RS+DiQKLeiqKqQ9WH7yCEMAlJIFhWRcLvk5P5\nNC6OX/z9qfoYpYALHHWVlgY+PvDDD9C8OWfOwIsvQng41Kv3+PEbmlKKbmu68WLVF/n3i/82dzhC\niHsYvCKheDwDXF25mp1N+2PHOODvTyU7u0c+RpZWy9AzZ/jw/lFXoHvWo00baN6czEzdVCVffGGZ\nyQNgW8w2zl0/x6bXNpk7FCGEAUgLxAQ+i41l940b/NywIWXzmTdLqxSJmZnE3LlDTHp6nv8mZmbS\n2tGRbQ0a5L11dfy4Lnn8+Sc4OzN2LMTFwaZNllMg6l53su9Qb349lnRbQpsabcwdjhDiPnILC8tM\nIEop3oqJITY9nYmenpxNT8+TKM6np+NoY4N36dJ4lyqV57/V7e0fHBWm1UKLFjBgAIwYwc6dMHw4\nREdDhQrmeY8PM/7n8Zy9fpa1vdaaOxQhRD4kgWCZCQQgVynePHOGE7dvP5AoapUq9Wgz+q5YoXvI\n47ffuHS1BP7+um4QA006bHBnr52l2dJmHBtxDLdyBc+iLIQwH0kgWG4CMZjr13Ud5zt2oPVvTOfO\n0KSJruSHJVJK0XF1R9rXaM/7zz9ZdV+EeJbIVCbPgk8+gV69oHFjZs6ElBTdKF5LtenUJjSpGkYF\njTJ3KEIIA5NRWE+Sgwdh2zY4eZIjRyAkRPe8Rz6TBFuE21m3GbNrDN+9/B22JSw0SCFEsUkL5EmR\nk6Obaffrr7llU56+fWH2bDDB5L/F9n/7/48W1VrQ0rPlwzcWQjxxpAXypFiwABwcoF8/Rg2FF17Q\nPfdhqU5dOcXSo0s5/vZxc4cihDASSSBPgqQk3ROC+/ezdp0Vv/ximdUF71JK8U7YO4xrPk4KRQnx\nFJME8iR4/30YNow4+7q8+y7s3AkGmvfRKNafWM/VO1f5V+C/zB2KEMKIJIFYMq0Wpk+HX38lZ8Fi\n+neCjz+Gxo3NHVjB0jLTeH/3+6zrtQ4ba/n1EuJpJt9wS3XtGgweDFevQng4k6Y9R9myMHasuQMr\n3KR9k2hXsx0vVH3B3KEIIYxMRmFZol9+AX9/qFMH9u9n31+eLFkCK1daZnXBu/68/Cerjq1iatup\n5g5FCGEC0gKxJFqtrjTtrFmwZAl07cq1azBwICxbBq4W3B+tlOJfof9iYquJOD/nbO5whBAmIAnE\nUly+rMsUt2/DoUPg4UFuLvTrB8HB+lLnFuuH4z9wK+sWbzV+y9yhCCFMxIJviDxDwsOhUSNd73h4\nOHh4ALopSrKyYMoUs0b3UCkZKXy450Pmd55PCesS5g5HCGEi0gIxp9xc+M9/dA8JrlgBHTroX9qy\nBVatgsOH4VEm6jWHz8M/p6t3V4Lcg8wdihDChCz80vQUS06G/v11/R5HjkCVKvqXzpzR1ffYtg2c\nLbg7IVITybRfpxGREMGxEcfMHY4QwsTkFpY57N2ru2X14ou6n+9JHmlp8MoruoZJkAX+Qa9VWrad\n2UbLFS3pvaE3z3s8z+l/naZi6YrmDk0IYWJSD6S4bt7UdVCUKKFbbGzy/je/8bY5Obo65suWwXff\nQevWeV5WStdh7uCgG4RlSTJyMvj+j++Z/tt0StmU4sPnP6R3vd7ysKAQT4niXDvl218ce/bomgml\nSumSQm7ug/+FB5OKUrpmRVQUuLg8cNjp0yE+Hg4cMO3bKcz19OssOLSAuYfm4u/qz7zO83jJ8yWs\nLLHwuhDCpCSBPKrTp3V9Fzt26OqSF0SrfTCp5OaCkxPkc/H9+WeYNk1X38Pe3ojxF1HcjTi+OfgN\n3//xPT3q9GDPwD3Ud65v7rCEEBZEEsijuHYNunXTPexXWPIA3S0sO7siHTYhQZeTVq+GqlUNEOdj\nOKQ5xLTfprE3di/DGw3n+NvHpY65ECJf0gdSVFlZ0L49BAbCV18Z7LAZGbpc1Ls3fPihwQ77yLRK\ny8gdIwk9G8ropqMZ1mgY5UqWM19AQgiTKs61UxJIUSgFb76pe1p80yZdf4aBDB+u649fvz7fO1sm\nkavNZejWocTdjGNHvx2UsbPgueKFEEYhnejG8s03us6JiAiDJo/Fi3WH/P138yWPHG0OgzcP5tKt\nS4T2C+U5u+fME4gQ4okjLZCH2b5d1/o4eNCgHRSRkdC1q27EVe3aBjvsI8nOzabfpn6kZabx39f+\nSynbUuYJRAhhdtICMbTjx+GNN2DrVoMmj8uXdX0eixaZL3lk5mTy2o+vkaty2dxnM/Y2FjD0Swjx\nRDHqk+iZmZkEBQXh7+9PvXr1+PTTTwG4ceMG7du3p3bt2nTo0IGUlBT9PiEhIXh5eVG3bl12796t\nXx8VFYWvry/e3t6MHj3amGHrXL4M3bvDzJnQtKnBDpuTA336wIAB0LOnwQ77SDJyMnh1/atYW1mz\nMXijJA8hRLEYNYGULFmS//3vfxw9epQ//viDn3/+mYiICKZMmULbtm05c+YMrVu3JiQkBICTJ0+y\nfv16Tp06RVhYGCNHjtQ3qd5++22WLl1KTEwMMTEx7Nq1y3iBZ2TAyy/rrvL9+hn00J98Ara28MUX\nBj1skaVnp9NjbQ+es3uOdb3WYVeiaEONhRDifkafC6t06dKArjWi1WpxdHRky5YtDB48GIDBgwez\nefNmALZu3UqfPn2wsbHB09MTLy8vIiMjSU5OJi0tjYCAAAAGDRqk38fglNINjapSRTftiAGtXAk/\n/gg//GDQvvgiu511m65rulKpdCVWv7Ia2xK2pg9CCPHUMHoC0Wq1+Pv74+rqSqtWrfDx8eHSpUu4\n/D2Vh6urK5cvXwZAo9Hg8XctDAA3Nzc0Gg0ajQZ3d3f9end3dzQajXECnjIFTp0yaP3YS5fgtdfg\nyy9107RXqGCQwz6StMw0Oq3uRFWHqqzsuVLmsBJCPDajX0Wsra05evQoqampdOjQgfDw8AfmUTL0\nvEoTJ07U/9yqVStatWpVtB03bYL583Ujrv5uOT0OpXQ1PT76CIYM0ZX8KGWGgU4pGSl0Wt2J+s71\n+bbrt1hbySTMQjzrwsPDCQ8Pf6xjmOzP0HLlytG5c2cOHz6Mi4uLvhWSnJyM899FL9zc3EhISNDv\nk5iYiJubW4HrC3JvAimyqCh46y3YuRMKOXZRxcfrDnf5MoSF6WZvN4cb6TfouLojAVUCmN1ptiQP\nIQTw4B/Xk4pxy96oV5OrV6/qR1ilp6ezZ88e/P396d69OytWrABg5cqV9OjRA4Du3buzdu1asrKy\niIuL49y5cwQGBuLq6oqDgwORkZEopVi1apV+H4O4eBF69NBVBmzc+LEOlZurG7jVpAm89JLueQ9z\nJY9rd67R9ru2PO/+PHM6zZHkIYQwKKO2QJKSkhg8eDBKKbRaLQMHDqRNmzb4+/sTHBzMsmXLqFat\nGuvXrwfAx8eH4OBgfHx8sLW1Zf78+frbW/PmzWPIkCFkZGTQuXNnOnbsaJgg79zRJY8RI6BXr8c6\n1IkTMHQolCwJv/4K3t6GCbE4rty+Qtvv2tKxZkemtJ0i068LIQxOnkT/4QcIDdUVeCrmRTYzE0JC\nYN48XUf58OEG639/ZJdvX2bVsVXMOzSPAQ0G8MVLX0jyEEI8lEymSDGnMtFqi33FP3hQ1+qoVUvX\n/26A7pNHlqvNZff53Sw5uoSf436mZ52eDPMfxgtVXzB9MEKIJ5IkEExX0vbWLfjsM90surNm6aYm\nMfUf+nE34lgevZzl0cupXKYywxoNo0/9PjINuxDikclcWCaydSuMGgUtW8Kff5r2uY7MnEw2n97M\nkqNLiE6Opl/9fuzotwNfF1/TBSGEEEgCeSRxcbrEERMDS5ZA27amO/fxS8dZenQpq4+vxs/Vj6H+\nQ+lZp6fMYyWEMBtJIEWQmQlff60bnjt2rG46kpIlTXPun+N+5pOfPkGTquEN/zeIHBZJdcfqpjm5\nEEIUQhLIQ+zeDe+8Az4+cPgweHqa5rypmal8tOcjdpzdwcwOM+lZpyclrM0wgZYQQhRAEkgBEhN1\nrY3Dh2HOHOjSxXTn3nVuF29uf5N2Ndrx59t/4mDvYLqTCyFEEcmjyffJzoZp08DPD+rU0T0caKrk\ncTPjJkO3DOWt7W+xuNtilnRfIslDCGGxpAVyj/37YeRIcHeH334DLy/TnXt7zHZGbB9B99rdOf72\nccqWLGu6kwshRDFIAkE33fqHH0J4OHzzDbzyiume6biefp33dr7Hrwm/8t3L3/FS9ZdMc2IhhHhM\nz/wtrLAwqF8fXF3h5El49VXTJY//nvov9efXp0KpCvwx4g9JHkKIJ8oz/yR6QgKkpkK9ekYM6j5X\nbl/h3bB3iUqKYlmPZbxY9UXTnVwIIfJRnCfRn/kWiIeH6ZKHUor1J9bj+60vHuU8ODbimCQPIcQT\n65nvA8nR5pCVm0Vp28evQFiYzJxMhm0bxpGLR9j82maC3IOMej4hhDC2Z74F8sPxH2ixvAWaVCPV\nWEf3UGDnHzpzO+s2R948IslDCPFUeOYTyEDfgbxS9xWaLm3KkYtHDH78pLQkWq5oSe0KtdnQewOl\nbM1QFF0IIYzgmU8gVlZWfNr8U2Z1nEXH1R3ZeHKjwY4dcy2GF5a9QK+6vZjXeZ5MRSKEeKo8830g\nd71S9xU8y3vSY20PTl89zafNP32sSn6/J/5Oz3U9+fKlLxnaaKgBIxVCCMvwzA/jvd/FtIv0WNuD\nOhXrsLjb4mJNlx56NpTBmwezvMdyunp3LXYsQghhKjKM1wCqlK3CviH7yMzJpM2qNly+ffmR9l8R\nvYI3trzBtr7bJHkIIZ5qkkDyUdq2NGt7raVt9bYELQni+KXjD91HKUXIgRAm7ZtE+JBwmro3NUGk\nQghhPnIL6yHWHF/DezvfY1mPZQW2KHK1uYzeOZr9F/YT1j+MKmWrGOz8QghhCsW5dkoCKYKDiQd5\nZd0rfPD8B4xpOiZP53pGTgaD/juIK3eusPm1zTL9uhDiiSQJBOMkEIALKRfotqYbgVUCmddlHnYl\n7EjJSKHnup5UKl2J717+jpI2JqpzK4QQBiad6EZU1aEqEW9EcPnOZTp834Hjl47TYkULGjg3YG2v\ntZI8hBDPHEkgj6CMXRk2BW8ioEoADb9tSL/6/ZjVcRbWVvIxCiGePXILq5gSUxNxL+du9PMIIYQp\nSB8IpksgQgjxNJE+ECGEECYjCUQIIUSxSAIRQghRLJJAhBBCFIskECGEEMUiCUQIIUSxSAIRQghR\nLEZNIImJibRu3Zp69erRoEED5syZA8CkSZNwd3enUaNGNGrUiJ07d+r3CQkJwcvLi7p167J79279\n+vJjDsUAAAgMSURBVKioKHx9ffH29mb06NHGDFsIIURRKCNKSkpSR48eVUoplZaWpry9vdWpU6fU\nxIkT1fTp0x/Y/uTJk8rPz09lZ2eruLg4VbNmTaXVapVSSgUGBqrIyEillFKdOnVSO3fuzPecRn5L\nT5T//e9/5g7BYshn8Q/5LP4hn8U/inPtNGoLxNXVFT8/PwDKlClD3bp10Wg0dxPXA9tv2bKFPn36\nYGNjg6enJ15eXkRGRpKcnExaWhoBAQEADBo0iM2bNxsz9KdCeHi4uUOwGPJZ/EM+i3/IZ/F4TNYH\nEh8fT3R0NEFBQQDMnTsXPz8/hg0bRkpKCgAajQYPDw/9Pm5ubmg0GjQaDe7u/8w75e7urk9EQggh\nzMMkCeTWrVv06tWLWbNmUaZMGUaOHElsbCzR0dG4urry/vvvmyIMIYQQhmT4O2l5ZWdnqw4dOqiZ\nM2fm+3p8fLxq0KCBUkqpkJAQNWXKFP1rHTp0UAcPHlRJSUmqTp06+vVr1qxRI0aMyPd4gCyyyCKL\nLMVYHpUNRvbGG2/g4+PDe++9p1+XnJyMq6srAJs2baJ+/foAdO/enf79+zNmzBg0Gg3nzp0jMDAQ\nKysrHBwciIyMJCAggFWrVjFq1Kh8z6dkJl4hhDAJoyaQiIgIVq9eTYMGDfD398fKyorJkyfzww8/\nEB0djbW1NZ6enixcuBAAHx8fgoOD8fHxwdbWlvnz5+vrj8+bN48hQ4aQkZFB586d6dixozFDF0II\n8RBPXT0QIYQQpvHUPIm+c+dO6tSpg7e3N1OnTjV3OGbl6elJw4YN8ff3JzAw0NzhmNTQoUNxcXHB\n19dXv+7GjRu0b9+e2rVr06FDB/2ov6ddfp9FYQ/xPs3uf6h59uzZwLP5u1GcB7wL8lS0QLRaLd7e\n3vz0009UqVKFgIAA1q5dS506dcwdmlnUqFGDI0eO4OjoaO5QTO6XX36hTJky/H979xPS9B/HcfxZ\nWUMiOpQtkcpDeSmZtgmmIUFMikgLJNvBxEOXKJAOdQ6C6hB16RTkHETuYgykRf8ONkHMQjwEWQkr\nD3NgUphEyz6/gzhSV81vuW+/fV+Pk3Nze38/vOeL73d7f78nTpxgeHgYgPPnz7NhwwbOnTvHlStX\nmJyc5PLlyzZXuvwyrcWFCxdYt24dZ8+etbm63EokEiQSCSoqKpiamsLr9RKJROjo6HBcb/xsLcLh\n8JJ7Iy/2QAYGBtixYwfbtm1j9erVHD9+nEgkYndZtjHG8P37d7vLsMXevXsXBWckEqG1tRWA1tZW\nxwyhZloLcOYXTTINNY+NjTmyN5Y64P0reREgCwcQnT5ouGLFCvx+P1VVVdy8edPucmyXTCZxu93A\n7JsnmUzaXJG9Mg3xOsncUHN1dTXj4+OO7o1sBrx/JS8CRObr6+vjxYsX3Lt3jxs3bhCLxewu6Z8y\n980+J1o4xOu0Q1kLh5oX9oKTeuN3A97Z9EZeBEhJSQnv3r1L3x4bG6OkpMTGiuxVXFwMQFFREUeP\nHmVgYMDmiuzldrsZHx8HZo//btq0yeaK7FNUVJT+J3ny5EmePXtmc0W58+3bN5qammhpaaGxsRFw\nbm9kWgsrvZEXAVJVVcWbN2+Ix+N8/fqVrq4uGhoa7C7LFtPT00xNTQHw+fNnHjx4kB7UdApjzLxj\nuQ0NDQSDQQA6OzvTbxgnWLgWiUQi/fOPQ7xOkGmo2am98bMB7zlZ98aSZ9f/UdFo1JSVlZnt27eb\nS5cu2V2ObUZHR43H4zEVFRVm165djluLQCBgiouLzZo1a8yWLVvMrVu3zIcPH8z+/ftNWVmZ8fv9\nZnJy0u4ycyLTWrS0tJjy8nLj8XhMY2OjSSQSdpeZE7FYzKxcuTL93qisrDTRaNRMTEw4rjd+thZW\neiMvvsYrIiK5lxeHsEREJPcUICIiYokCRERELFGAiIiIJQoQERGxRAEiIiKWKEBEfiMej1NeXv7P\nP6dIrilARLKwHOdIctJ5lyQ/KUBElmB0dJTdu3fz/Pnzeb8PBAJEo9H07ba2Nrq7u4nH49TV1eHz\n+fD5fPT39y96zs7OTs6cOZO+ffjwYXp7ewF4+PAhNTU1+Hw+mpubmZ6eXqYtE1k6BYhIlkZGRmhq\naiIUCuH1eufd19zcTDgcBiCVSvHkyRMOHTqE2+3m0aNHDA4O0tXVNS8ofpRpb2RiYoKLFy/y+PFj\nBgcH8Xq9XL169e9vmIhFBXYXIPJ/kEwmOXLkCN3d3RmvdHnw4EHa29tJpVJEo1Hq6upwuVx8+vSJ\n06dPMzQ0xKpVq3j9+nXWr9nf38/Lly+pra3FGEMqlWLPnj1/c7NE/ogCRCQL69evZ+vWrTx9+jRj\ngLhcLvbt28f9+/cJh8MEAgEArl27xubNmxkeHmZmZobCwsJFf1tQUDDvCpJfvnwBZs+kW19fz+3b\nt5dpq0T+jA5hiWTB5XJx9+5dQqEQd+7cyfiYY8eO0dHRQSwW48CBAwB8/PgxfX2WUCjEzMxM+vFz\n5zEtLS1laGgIYwzv379PX7+lurqavr4+3r59C8yeqn8pezAiy00BIpKlwsJCenp6uH79Oj09PYvu\nr6+vp7e3F7/fT0HB7M79qVOnCAaDVFZWMjIywtq1a9OPn/vco7a2ltLSUnbu3El7e3v685WNGzcS\nDAYJBAJ4PB5qamp49epVDrZUJDs6nbuIiFiiPRAREbFEASIiIpYoQERExBIFiIiIWKIAERERSxQg\nIiJiiQJEREQsUYCIiIgl/wHd7XNARnQADAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fe5b530f4e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for k,v in k_rmse_results.items():\n",
    "    x = list(v.keys())\n",
    "    y = list(v.values())  \n",
    "    plt.plot(x,y, label=\"{}\".format(k))\n",
    "    \n",
    "plt.xlabel('k value')\n",
    "plt.ylabel('RMSE')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, I attempted to use some k-nearest neighbors algorithms to get the best predictions for car prices. Univariate and Multivariate models and K-fold cross validation has been used with the following results:\n",
    "\n",
    "**Model             RMSE    \tK\tFold\tFeatures**              \n",
    "Univariate          3238    \t5\t2      engine_size          \n",
    "Multivariate        2658    \t2\t2  \t   engine_size, horsepower\n",
    "K-fold cross        2601\t    2\t10     engine_size, horsepower\n",
    "\n",
    "The best result has been achieved by the k-fold cross with keeping 2 as the number of neighbors, using 10 folds and 2 best features model.\n",
    "\n",
    "**Suggested Next Steps:**\n",
    "\n",
    "- Test k-fold cross validation with different numbers of neighbors\n",
    "- Collect more data about high price cars"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}