How To Split The Data Effectively for Your Data Science Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How To Split The Data Effectively for Your Data Science Project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data is one of the most important resources for any data science project. But what good is abundant data if you can't use it effectively? After all, your success as a data scientist hinges on how adeptly you can manipulate and analyze data to produce actionable insights from it. \n",
    "\n",
    "One of the common steps in your data science project after collecting the data is to split the data into train and test. Although this step might seem simple, if it is done in an effective way, it may affect your results and lead to unrealistic models. With this in mind, this article will walk you through the steps of splitting your data effectively as a data scientist. These tips will help you understand how to split your data for different types of analysis. We'll also look at some potential issues with splitting your data and give you some general rules to follow when undertaking this process. Read on to discover more!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Table of content:\n",
    "\n",
    "1. Why should you split your data?\n",
    "2. What Should Be The Splitting Percentage?\n",
    "3. Split at the early  beginning\n",
    "4. Consistent Split \n",
    "5. Avoid Sampling Bias"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import tarfile\n",
    "import urllib.request\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Why should you split your data?\n",
    "\n",
    "Splitting data into train and test or splitting it into train, validation, and test is a common step in supervised machine learning projects. The train set is used to fit and train the model while the test set is used to evaluate the trained model so as to get a better idea of how good the model is on new data and how it will act in the production environment. Therefore the test data need to be similar to what is expected to be seen in the production. \n",
    "\n",
    "Another common splitting technique is to split the data into three datasets: train, validation, and test dataset. The validation dataset will be used to choose the best hyperparameters for your project."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. What Should Be The Splitting Percentage?\n",
    "\n",
    "The percentage of the test data depends on many factors that's why there is no optimal splitting percentage. You must choose a split percentage that meets your project's objectives with considerations that include:\n",
    "\n",
    "* **Computational cost in training the model:** If the cost of training the model is high this might affect our ability to evaluate multiple models especially if we will use a validation dataset. \n",
    "\n",
    "* **Size of the data:** This is a very important factor if we have a small dataset so probably we will not be able to split the data another good option will be using k-fold cross-validation for evaluation. If the size of the data is so big then it will be more than enough to split a small portion of the data and use it as a test dataset. \n",
    "\n",
    "However, there are some common splitting percentages:\n",
    "* Train: 80%, Test: 20%\n",
    "* Train: 67%, Test: 33%\n",
    "* Train: 50%, Test: 50%\n",
    "* Train 90%, Test: 10 %\n",
    "* Train 95% , Test 5%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Split At The Early Beginning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the important practical tips is to split your data in an early stage of your project directly after collecting the data. Although it may sound strange to voluntarily set aside part of the data at the early stage of your data science project before even data exploration. The reason for this is that your brain is an amazing pattern detection system, which means that it is highly prone to overfitting: if you look at the test set, you may stumble upon some seemingly interesting pattern in the test data that leads you to select a particular kind of Machine Learning model. Therefore when you estimate the generalization error using the test set after training your model, your estimate will be too optimistic, and you will launch a system into production that will not perform as well as expected. This is called snooping data bias or data leakage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Consistent Splitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although creating a test set is theoretically quite simple: you can just pick some instances randomly, typically 20% of the dataset, and set them aside as shown in the code below, but before that let's load the data we will be using. We will be using the California housing prices dataset throughout this article:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>longitude</th>\n",
       "      <th>latitude</th>\n",
       "      <th>housing_median_age</th>\n",
       "      <th>total_rooms</th>\n",
       "      <th>total_bedrooms</th>\n",
       "      <th>population</th>\n",
       "      <th>households</th>\n",
       "      <th>median_income</th>\n",
       "      <th>median_house_value</th>\n",
       "      <th>ocean_proximity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-122.23</td>\n",
       "      <td>37.88</td>\n",
       "      <td>41.0</td>\n",
       "      <td>880.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>322.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>8.3252</td>\n",
       "      <td>452600.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-122.22</td>\n",
       "      <td>37.86</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7099.0</td>\n",
       "      <td>1106.0</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>1138.0</td>\n",
       "      <td>8.3014</td>\n",
       "      <td>358500.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-122.24</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1467.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>177.0</td>\n",
       "      <td>7.2574</td>\n",
       "      <td>352100.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1274.0</td>\n",
       "      <td>235.0</td>\n",
       "      <td>558.0</td>\n",
       "      <td>219.0</td>\n",
       "      <td>5.6431</td>\n",
       "      <td>341300.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-122.25</td>\n",
       "      <td>37.85</td>\n",
       "      <td>52.0</td>\n",
       "      <td>1627.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>565.0</td>\n",
       "      <td>259.0</td>\n",
       "      <td>3.8462</td>\n",
       "      <td>342200.0</td>\n",
       "      <td>NEAR BAY</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   longitude  latitude  housing_median_age  total_rooms  total_bedrooms  \\\n",
       "0    -122.23     37.88                41.0        880.0           129.0   \n",
       "1    -122.22     37.86                21.0       7099.0          1106.0   \n",
       "2    -122.24     37.85                52.0       1467.0           190.0   \n",
       "3    -122.25     37.85                52.0       1274.0           235.0   \n",
       "4    -122.25     37.85                52.0       1627.0           280.0   \n",
       "\n",
       "   population  households  median_income  median_house_value ocean_proximity  \n",
       "0       322.0       126.0         8.3252            452600.0        NEAR BAY  \n",
       "1      2401.0      1138.0         8.3014            358500.0        NEAR BAY  \n",
       "2       496.0       177.0         7.2574            352100.0        NEAR BAY  \n",
       "3       558.0       219.0         5.6431            341300.0        NEAR BAY  \n",
       "4       565.0       259.0         3.8462            342200.0        NEAR BAY  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DOWNLOAD_ROOT = \"https://raw.githubusercontent.com/ageron/handson-ml2/master/\"\n",
    "HOUSING_PATH = os.path.join(\"datasets\", \"housing\")\n",
    "HOUSING_URL = DOWNLOAD_ROOT + \"datasets/housing/housing.tgz\"\n",
    "\n",
    "def fetch_housing_data(housing_url=HOUSING_URL, housing_path=HOUSING_PATH):\n",
    "    if not os.path.isdir(housing_path):\n",
    "        os.makedirs(housing_path)\n",
    "    tgz_path = os.path.join(housing_path, \"housing.tgz\")\n",
    "    urllib.request.urlretrieve(housing_url, tgz_path)\n",
    "    housing_tgz = tarfile.open(tgz_path)\n",
    "    housing_tgz.extractall(path=housing_path)\n",
    "    housing_tgz.close()\n",
    "    \n",
    "fetch_housing_data()\n",
    "\n",
    "def load_housing_data(housing_path=HOUSING_PATH):\n",
    "    csv_path = os.path.join(housing_path, \"housing.csv\")\n",
    "    return pd.read_csv(csv_path)\n",
    "\n",
    "housing = load_housing_data()\n",
    "housing.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's define the splitting function to split the data into training and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of the training data 16512\n",
      "The size of the testing data 4128\n"
     ]
    }
   ],
   "source": [
    "#define a splitting fucntion \n",
    "def split_train_test(data, test_ratio):\n",
    "    shuffled_indices = np.random.permutation(len(data))\n",
    "    test_set_size = int(len(data) * test_ratio)\n",
    "    test_indices = shuffled_indices[:test_set_size]\n",
    "    train_indices = shuffled_indices[test_set_size:]\n",
    "    return data.iloc[train_indices], data.iloc[test_indices]\n",
    "\n",
    "# split the data into training and testing data \n",
    "train_set, test_set = split_train_test(housing, 0.2)\n",
    "print('The size of the training data', len(train_set))\n",
    "print('The size of the testing data', len(test_set))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method works,  but it has a major artifact. If you run the code again, it will generate a different test set! Over time, you (or your Machine Learning algorithms) will get to see the whole dataset, which is what you want to avoid. There are some potential solutions that can be done to avoid this:\n",
    "* One solution is to save the test set on the first run in a separate file and then load it in the subsequent runs.\n",
    "* Another option is to set the random number generator's seed (e.g., np.random.seed(42)) before calling np.random.permutation(), so that it always generates the same shuffled indices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**However, both of these solutions will break the next time you fetch an updated dataset.** A common better, and more reliable solution is to use each instance's identifier to decide whether or not it should go in the test set (assuming instances have a unique and immutable identifier). For example, you could compute a hash of each instance's identifier and put that instance in the test set if the hash is lower or equal to 20% of the maximum hash value. This ensures that the test set will remain consistent across multiple runs, even if you refresh the dataset. The new test set will contain 20% of the new instances, but it will not contain any instance that was previously in the training set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a possible implementation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zlib import crc32\n",
    "def test_set_check(identifier, test_ratio):\n",
    "    return crc32(np.int64(identifier)) & 0xffffffff < test_ratio * 2**32\n",
    "\n",
    "def split_train_test_by_id(data, test_ratio, id_column):\n",
    "    ids = data[id_column]\n",
    "    in_test_set = ids.apply(lambda id_: test_set_check(id_, test_ratio))\n",
    "    return data.loc[~in_test_set], data.loc[in_test_set]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, our housing dataset does not have an identifier column. The simplest solution is to use the row index as the ID:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "housing_with_id = housing.reset_index() # adds an `index` column\n",
    "train_set, test_set = split_train_test_by_id(housing_with_id, 0.2, \"index\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, it is important to note that if you use the row index as a unique identifier, you need to make sure that new data gets appended to the end of the dataset and that no row ever gets deleted. If this is not possible, then you can try to use the most stable features to build a unique identifier. For example, in the housing data set, the district's latitude and longitude are very good choices since they are guaranteed to be stable for a few million years so you could combine them into an ID like the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "housing_with_id[\"id\"] = housing[\"longitude\"] * 1000 + housing[\"latitude\"]\n",
    "train_set, test_set = split_train_test_by_id(housing_with_id, 0.2, \"id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Scikit-Learn** provides a few functions to split datasets into multiple subsets in various ways. The simplest function is **train_test_split**, which does pretty much the same thing as the function split_train_test defined earlier, with a couple of additional features. First, there is a random_state parameter that allows you to set the random generator seed as explained previously, and second, you can pass it through multiple datasets with an identical number of rows, and it will split them on the same indices (this is very useful, for example, if you have a separate DataFrame for labels):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "train_set, test_set = train_test_split(housing, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Avoid Sampling Bias"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far, we have considered purely random sampling methods for getting and splitting test data. This is generally fine if your dataset is large enough (especially relative to the number of attributes), but if it is not, you run the risk of introducing a significant sampling bias."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's consider a survey company that decided to call 1,000 people to ask them a few questions, they don't just pick 1,000 people randomly in a phone book. They try to ensure that these 1,000 people are representative of the whole population. For example, the US population is composed of 51.3% female and 48.7% male, so a well-conducted survey in the US would try to maintain this ratio in the sample: 513 females and 487 males. We should try to do the same when splitting our dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is called stratified sampling, in which the population is divided into homogeneous subgroups called strata, and the right number of instances is sampled from each stratum to guarantee that the test set is representative of the overall population. So in the previous example, if they used purely random sampling, there would be about a 12% chance of sampling a skewed test set with either less than 49% female or more than 54% female. Either way, the survey results would be significantly biased."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's go back to the housing prices dataset used in this blog. We can assume that median income is a very important attribute in predicting median housing prices. Therefore we should  ensure that the test set is representative of the various categories of incomes in the whole dataset.\n",
    "\n",
    "Since the median income is a continuous numerical attribute, you first need to create an income category attribute. Let's plot a histogram for the median income feature and have a close look at it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Histogram for the median income')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABJgAAANsCAYAAAAX8BIxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA9RElEQVR4nO3de5xlZ13n++/PtECgJQlGekIS7agRhcRbWsTr6QwoGYOEOQc1HNBEGfMaZBCdoHTGmcHRiSczI14YBCdHkDBwaCKiRFqQDNowFwISRZoAGTKkJTcTuSTSwIAJv/PHXtFKpbvTu5+q2lXd7/fr1a/ae+21nnq66klX59NrrV3dHQAAAAA4XF+06AkAAAAAsLEJTAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQC4n6q6rqq2L3oeq62q/m1Vfayq/mqVxt9eVTevxtgrraq6qr56evybVfWvVuFzfFdVXb/S4wIAiycwAcBRpqr2VtUTl227sKr+273Pu/ux3b37AcbZOkWJTas01VVVVacmuTjJY7r7H6zQmH8XaTay7v6n3f2LqzDuf+3uR6/0uADA4glMAMC6tAbh6iuSfLy775j3wI0a1QAAVovABADcz9KznKrqcVX1nqr6m6q6vap+ZdrtHdPHO6tqX1V9W1V9UVX9y6r6y6q6o6peVVXHLRn3R6bXPl5V/2rZ5/n5qnp9Vb26qv4myYXT535nVd1ZVbdV1Uuq6kFLxuuq+omq+nBVfaqqfrGqvmo65m+q6sql+y857olJrk7yqGnur5y2P2W6PPDOqtpdVV+37Gvygqp6X5JPL49MVXXv1+MvpjF/aMlrF09fj9uq6keXbH9wVf1yVX10+tr+ZlUde4DvyYVV9d+r6len+X2kqr592n7TNP4Fhzp2Vf3MNJ9bq+rHln2uV1bVv50en1BVb6qqv66qT06PT1my7+7p6/7fp+/BW6vqxAP8Hu5zyeD0NX1+Vb2vqu6qqtdV1UOWvH5eVb13+l7+r6o6Z9r+qKq6qqo+UVU3VNWPLznm56vqd6Z19Kmq2lNVX1NVl0xfo5uq6nuX7H9cVb18+lrcUrPLJo/Z3/wBgAMTmACAB/LrSX69ux+e5KuSXDlt/+7p4/Hdvbm735nkwunX2Um+MsnmJC9Jkqp6TJKXJnlGkpOSHJfk5GWf67wkr09yfJLXJLknyU8nOTHJtyV5QpKfWHbMOUnOSvL4JD+b5PLpc5ya5IwkT1/+G+ru/5LkHyW5dZr7hVX1NUlem+SnknxZkj9M8gfLAtXTk5w7/Z7vXjbmvV+Pb5jGfN30/B8s+b0+K8lvVNUJ02v/LsnXJPnGJF897fOvl893iW9N8r4kX5rk/0uyM8m3TMc+M8lLqmrzA409hZrnJ/meJKcnuc8lk8t8UZLfzuyMry9P8tlM39Ml/u8kP5rkkUkeNI19qH4ws+/haUm+PrP1k6p6XJJXJfmZzNbDdyfZOx3z2iQ3J3lUkqcl+aWqesKSMb8/yX9OckKSP0/yR9Pv4+Qkv5DkPy3Z94okd2f2NfqmJN+b5J/MMX8AIAITABytfn86C+bOqrozs/BzIH+b5Kur6sTu3tfd1xxk32ck+ZXu/kh370tySZLzp7N9npbkD7r7v3X35zOLHb3s+Hd29+939xe6+7PdfW13X9Pdd3f33szCwP+x7Jh/191/093XJXl/krdOn/+uJG/OLBocih9Ksqu7r+7uv03yy0mOTfLtS/Z5cXff1N2fPcQxk9nX7xe6+2+7+w+T7Evy6KqqJD+e5Ke7+xPd/akkv5Tk/IOMdWN3/3Z335PkdZlFtF/o7s9191uTfD6z79UDjf2DSX67u9/f3Z9O8vMH+oTd/fHu/t3u/sw0zqW5//fgt7v7f05flyszi1qH6sXdfWt3fyLJHyw59llJXjF9P77Q3bd094dqdu+s70zygu7+39393iS/leSHl4z5X7v7j6YI+DuZBcPLpu/rziRbq+r4qtqSWWj8qe7+9HS55K/m4N8DAGA/3D8AAI5OT53O4kkyu/wqBz5r41mZnfXxoaq6Mcm/6e43HWDfRyX5yyXP/zKzv29smV676d4XuvszVfXxZcfftPTJdFbRryTZluSh01jXLjvm9iWPP7uf54d6A+/7zL27v1BVN+W+Z1nddL+jHtjHl53t9JnMzuz6ssx+T9fOelCSpJIc7PKs5b+3dPfybYcy9qNy36/j0u/ZfVTVQzOLLudkdkZQknxJVR0zha4kWfoufPf+/g7V8mMfNT0+NbOzyJZ7VJJ7o9m9/jKzNXKv5V+Tjy2Z671xcPM01hcnuW3J1+mLcnjfZwA4qjmDCQA4qO7+cHc/PbPLn/5dktdX1cNy/7OPkuTWzC6luteXZ3b50e1Jbkuy9N49x2Z2qdd9Pt2y5y9L8qEkp0+X6P2LzELJarjP3KezgE5NcstB5jfiY5nFjsd29/HTr+O6e544c7hj35bZ7+1eX36QsS5O8ugk3zp9D+69FHC1vg/3uimzSzKXuzXJI6rqS5Zs+/Lc9/s0z+f4XJITl3ydHt7djz2MsQDgqCYwAQAHVVXPrKov6+4vJLlz2nxPkr9O8oXM7rV0r9cm+emqOm26F9AvJXnddAbP65N8/3Rj6gcl+Td54EjxJUn+Jsm+qvraJM9eqd/XflyZ5NyqekJVfXFmYeVzSf7HHGPcnvt+PQ5o+nr+v0l+taoemSRVdXJVPWm+aR/W2FdmdhP1x0xnKL3wIMN9SWax6s6qesQD7LuSXp7kR6fvxxdN8//a7r4ps+/J/1NVD6mqr8/sLLvXzPsJuvu2JG9N8qKqevj0eb6qqpZfAggAPACBCQB4IOckua6q9mV2w+/zp3vffCaz+/H89+leTo9P8orMbq78jiQ3JvnfSZ6bJNM9kp6b2T1wbkvyqSR3ZBZxDuT5md1A+lOZBZPXHWTfId19fWY3yv6PmZ0B9P1Jvn+6X9Sh+vkkV0xfjx88hP1fkOSGJNfU7J3z/ktmZwuthAOO3d1vTvJrSf542uePDzLOr2V2L6qPJbkmyVtWaH4H1d3vzuzG4b+a5K4kb8/fn2H29CRbMzub6feSvLC7rz7MT/Ujmd2Y/ANJPplZCD3psCcOAEep6l7JM70BAA7NdIbTnZld/nbjgqcDAMAAZzABAGumqr6/qh463cPpl5Psyd+/9TwAABuUwAQArKXzMrus6dYkp2d2uZ3TqQEANjiXyAEAAAAwxBlMAAAAAAzZtOgJrJYTTzyxt27deljHfvrTn87DHvawlZ0QRyzrhXlYL8zDemEe1gvzsF6Yh/XCPKyXI9+11177se7+suXbj9jAtHXr1rznPe85rGN3796d7du3r+yEOGJZL8zDemEe1gvzsF6Yh/XCPKwX5mG9HPmq6i/3t90lcgAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwJBNi54AcHTaumPXio6397JzV3Q8AAAADp0zmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAkFULTFX1iqq6o6rev2z7c6vq+qq6rqr+/ZLtl1TVDdNrT1qy/ayq2jO99uKqqtWaMwAAAADzW80zmF6Z5JylG6rq7CTnJfn67n5skl+etj8myflJHjsd89KqOmY67GVJLkpy+vTrPmMCAAAAsFirFpi6+x1JPrFs87OTXNbdn5v2uWPafl6Snd39ue6+MckNSR5XVScleXh3v7O7O8mrkjx1teYMAAAAwPxq1m1WafCqrUne1N1nTM/fm+SNmZ2F9L+TPL+7/7SqXpLkmu5+9bTfy5O8OcnezILUE6ft35XkBd395AN8vosyO9spW7ZsOWvnzp2HNe99+/Zl8+bNh3UsRx/r5fDsueWuFR3vzJOPW9HxVov1wjysF+ZhvTAP64V5WC/Mw3o58p199tnXdve25ds3rfE8NiU5Icnjk3xLkiur6iuT7O++Sn2Q7fvV3ZcnuTxJtm3b1tu3bz+sSe7evTuHeyxHH+vl8Fy4Y9eKjrf3GdtXdLzVYr0wD+uFeVgvzMN6YR7WC/OwXo5ea/0ucjcneUPPvDvJF5KcOG0/dcl+pyS5ddp+yn62AwAAALBOrHVg+v0k/zBJquprkjwoyceSXJXk/Kp6cFWdltnNvN/d3bcl+VRVPX5697gfyewSOwAAAADWiVW7RK6qXptke5ITq+rmJC9M8ookr6iq9yf5fJILppt3X1dVVyb5QJK7kzynu++Zhnp2Zu9Id2xm92V682rNGY4UW1f48rMk2XvZuSs+JgAAAEeGVQtM3f30A7z0zAPsf2mSS/ez/T1JzljBqQEAAACwgtb6EjkAAAAAjjACEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMGTToicAbAxbd+xa9BQAAABYp5zBBAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADPEucsARYTXe5W7vZeeu+JgAAABHImcwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgyKoFpqp6RVXdUVXv389rz6+qrqoTl2y7pKpuqKrrq+pJS7afVVV7ptdeXFW1WnMGAAAAYH6reQbTK5Ocs3xjVZ2a5HuSfHTJtsckOT/JY6djXlpVx0wvvyzJRUlOn37db0wAAAAAFmfVAlN3vyPJJ/bz0q8m+dkkvWTbeUl2dvfnuvvGJDckeVxVnZTk4d39zu7uJK9K8tTVmjMAAAAA86tZt1mlwau2JnlTd58xPX9Kkid09/Oqam+Sbd39sap6SZJruvvV034vT/LmJHuTXNbdT5y2f1eSF3T3kw/w+S7K7GynbNmy5aydO3ce1rz37duXzZs3H9axHH3W43rZc8tdi57CEeHMk49b0fH23HJXthyb3P7ZlRlvpefH+rMe/3xh/bJemIf1wjysF+ZhvRz5zj777Gu7e9vy7ZvWagJV9dAkP5fke/f38n629UG271d3X57k8iTZtm1bb9++ff6JJtm9e3cO91iOPutxvVy4Y9eip3BE2PuM7Ss63oU7duXiM+/Oi/aszB+9Kz0/1p/1+OcL65f1wjysF+ZhvTAP6+XotWaBKclXJTktyV9M9+k+JcmfVdXjktyc5NQl+56S5NZp+yn72Q4AAADAOrGaN/m+j+7e092P7O6t3b01s3j0zd39V0muSnJ+VT24qk7L7Gbe7+7u25J8qqoeP7173I8keeNazRkAAACAB7ZqgamqXpvknUkeXVU3V9WzDrRvd1+X5MokH0jyliTP6e57ppefneS3Mrvx9//K7N5MAAAAAKwTq3aJXHc//QFe37rs+aVJLt3Pfu9JcsaKTg4AAACAFbNml8gBAAAAcGQSmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGDIpkVPAEi27ti16CkAAADAYXMGEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDNi16ArARbd2x6+8eX3zm3blwyXMAAAA42jiDCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgyKZFTwBgvdq6Y9eipwAAALAhOIMJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAkFULTFX1iqq6o6rev2Tbf6iqD1XV+6rq96rq+CWvXVJVN1TV9VX1pCXbz6qqPdNrL66qWq05AwAAADC/1TyD6ZVJzlm27eokZ3T31yf5n0kuSZKqekyS85M8djrmpVV1zHTMy5JclOT06dfyMQEAAABYoFULTN39jiSfWLbtrd199/T0miSnTI/PS7Kzuz/X3TcmuSHJ46rqpCQP7+53dncneVWSp67WnAEAAACYX826zSoNXrU1yZu6+4z9vPYHSV7X3a+uqpckuaa7Xz299vIkb06yN8ll3f3Eaft3JXlBdz/5AJ/voszOdsqWLVvO2rlz52HNe9++fdm8efNhHcvRYc8td/3d4y3HJrd/doGTYUNZyfVy5snHrcxArFt+HjEP64V5WC/Mw3phHtbLke/ss8++tru3Ld++aRGTqaqfS3J3ktfcu2k/u/VBtu9Xd1+e5PIk2bZtW2/fvv2w5rd79+4c7rEcHS7csevvHl985t150Z6F/KfEBrSS62XvM7avyDisX34eMQ/rhXlYL8zDemEe1svRa83/r7iqLkjy5CRP6L8/fermJKcu2e2UJLdO20/Zz3YAAAAA1onVvMn3/VTVOUlekOQp3f2ZJS9dleT8qnpwVZ2W2c28393dtyX5VFU9fnr3uB9J8sa1nDMAAAAAB7dqZzBV1WuTbE9yYlXdnOSFmb1r3IOTXD3rRbmmu/9pd19XVVcm+UBml849p7vvmYZ6dmbvSHdsZvdlevNqzRkAAACA+a1aYOrup+9n88sPsv+lSS7dz/b3JLnfTcIBAAAAWB/W9BI5AAAAAI48AhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAM2bToCcBa2Lpj16KnAAAAAEcsZzABAAAAMERgAgAAAGDIqgWmqnpFVd1RVe9fsu0RVXV1VX14+njCktcuqaobqur6qnrSku1nVdWe6bUXV1Wt1pwBAAAAmN9qnsH0yiTnLNu2I8nbuvv0JG+bnqeqHpPk/CSPnY55aVUdMx3zsiQXJTl9+rV8TAAAAAAWaNUCU3e/I8knlm0+L8kV0+Mrkjx1yfad3f257r4xyQ1JHldVJyV5eHe/s7s7yauWHAMAAADAOlCzbrNKg1dtTfKm7j5jen5ndx+/5PVPdvcJVfWSJNd096un7S9P8uYke5Nc1t1PnLZ/V5IXdPeTD/D5LsrsbKds2bLlrJ07dx7WvPft25fNmzcf1rGsT3tuuWvVxt5ybHL7Z1dteI4wK7lezjz5uJUZiHXLzyPmYb0wD+uFeVgvzMN6OfKdffbZ13b3tuXbNy1iMvuxv/sq9UG271d3X57k8iTZtm1bb9++/bAms3v37hzusaxPF+7YtWpjX3zm3XnRnvXynxLr3Uqul73P2L4i47B++XnEPKwX5mG9MA/rhXlYL0evtX4Xuduny94yfbxj2n5zklOX7HdKklun7afsZzsAAAAA68RaB6arklwwPb4gyRuXbD+/qh5cVadldjPvd3f3bUk+VVWPn9497keWHAMAAADAOrBq1/VU1WuTbE9yYlXdnOSFSS5LcmVVPSvJR5P8QJJ093VVdWWSDyS5O8lzuvueaahnZ/aOdMdmdl+mN6/WnAEAAACY36oFpu5++gFeesIB9r80yaX72f6eJGes4NQAjghbV+HeYnsvO3fFxwQAAI58a32JHAAAAABHGIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAkE2LngAA68fWHbtWdLy9l527ouMBAADrkzOYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIYcUmKrqOw5lGwAAAABHn0M9g+k/HuI2AAAAAI4ymw72YlV9W5JvT/JlVfXPl7z08CTHrObEAAAAANgYDhqYkjwoyeZpvy9Zsv1vkjxttSYFAAAAwMZx0MDU3W9P8vaqemV3/+UazQkAAACADeSBzmC614Or6vIkW5ce093/cDUmBQAAAMDGcaiB6XeS/GaS30pyz+pNBwAAAICN5lAD093d/bJVnQkAAAAAG9IXHeJ+f1BVP1FVJ1XVI+79taozAwAAAGBDONQzmC6YPv7Mkm2d5CtXdjoAAAAAbDSHFJi6+7TVnggAAAAAG9MhBaaq+pH9be/uV63sdAAAAADYaA71ErlvWfL4IUmekOTPkghMAAAAAEe5Q71E7rlLn1fVcUn+86rMCAAAAIAN5VDfRW65zyQ5fSUnAgAAAMDGdKj3YPqDzN41LkmOSfJ1Sa5crUkBAAAAsHEc6j2YfnnJ47uT/GV337wK8wEAAABggzmkS+S6++1JPpTkS5KckOTzqzkpAAAAADaOQwpMVfWDSd6d5AeS/GCSd1XV01ZzYgAAAABsDId6idzPJfmW7r4jSarqy5L8lySvX62JAQAAALAxHOq7yH3RvXFp8vE5jgUAAADgCHaoZzC9par+KMlrp+c/lOQPV2dKAAAAAGwkBw1MVfXVSbZ0989U1f+Z5DuTVJJ3JnnNGswPAAAAgHXugS5z+7Ukn0qS7n5Dd//z7v7pzM5e+rXVnRoAAAAAG8EDBaat3f2+5Ru7+z1Jtq7KjAAAAADYUB4oMD3kIK8du5ITAQAAAGBjeqDA9KdV9ePLN1bVs5JcuzpTAgAAAGAjeaB3kfupJL9XVc/I3welbUkelOQfr+K8AAAAANggDhqYuvv2JN9eVWcnOWPavKu7/3jVZwYAAADAhvBAZzAlSbr7T5L8ySrPBQAAAIAN6JACEwAcjq07dq3oeHsvO3dFxwMAAFbGA93kGwAAAAAOaiGBqap+uqquq6r3V9Vrq+ohVfWIqrq6qj48fTxhyf6XVNUNVXV9VT1pEXMGAAAAYP/WPDBV1clJfjLJtu4+I8kxSc5PsiPJ27r79CRvm56nqh4zvf7YJOckeWlVHbPW8wYAAABg/xZ1idymJMdW1aYkD01ya5LzklwxvX5FkqdOj89LsrO7P9fdNya5Icnj1na6AAAAABxIdffaf9Kq5yW5NMlnk7y1u59RVXd29/FL9vlkd59QVS9Jck13v3ra/vIkb+7u1+9n3IuSXJQkW7ZsOWvnzp2HNb99+/Zl8+bNh3Us69OeW+5atbG3HJvc/tlVG54jjPUy5syTj1v0FNaUn0fMw3phHtYL87BemIf1cuQ7++yzr+3ubcu3r/m7yE33VjovyWlJ7kzyO1X1zIMdsp9t+61i3X15ksuTZNu2bb19+/bDmuPu3btzuMeyPl24wu9ktdTFZ96dF+3xhowcGutlzN5nbF/0FNaUn0fMw3phHtYL87BemIf1cvRaxCVyT0xyY3f/dXf/bZI3JPn2JLdX1UlJMn28Y9r/5iSnLjn+lMwuqQMAAABgHVhEYPpoksdX1UOrqpI8IckHk1yV5IJpnwuSvHF6fFWS86vqwVV1WpLTk7x7jecMAAAAwAGs+XUa3f2uqnp9kj9LcneSP8/ssrbNSa6sqmdlFqF+YNr/uqq6MskHpv2f0933rPW8AQAAANi/hdwIpLtfmOSFyzZ/LrOzmfa3/6WZ3RQcAAAAgHVmEZfIAQAAAHAEEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBk06InAACHauuOXSs63t7Lzl3R8QAA4GjlDCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAzZtOgJwHJbd+xa9BQAAACAOTiDCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwJCFBKaqOr6qXl9VH6qqD1bVt1XVI6rq6qr68PTxhCX7X1JVN1TV9VX1pEXMGQAAAID9W9QZTL+e5C3d/bVJviHJB5PsSPK27j49ydum56mqxyQ5P8ljk5yT5KVVdcxCZg0AAADA/ax5YKqqhyf57iQvT5Lu/nx335nkvCRXTLtdkeSp0+Pzkuzs7s91941JbkjyuLWcMwAAAAAHVt29tp+w6huTXJ7kA5mdvXRtkucluaW7j1+y3ye7+4SqekmSa7r71dP2lyd5c3e/fj9jX5TkoiTZsmXLWTt37jysOe7bty+bN28+rGMZt+eWuxY9hblsOTa5/bOLngUbhfWyvpx58nGLnsJB+XnEPKwX5mG9MA/rhXlYL0e+s88++9ru3rZ8+6YFzGVTkm9O8tzufldV/Xqmy+EOoPazbb9VrLsvzyxeZdu2bb19+/bDmuDu3btzuMcy7sIduxY9hblcfObdedGeRfynxEZkvawve5+xfdFTOCg/j5iH9cI8rBfmYb0wD+vl6LWIezDdnOTm7n7X9Pz1mQWn26vqpCSZPt6xZP9Tlxx/SpJb12iuAAAAADyANQ9M3f1XSW6qqkdPm56Q2eVyVyW5YNp2QZI3To+vSnJ+VT24qk5LcnqSd6/hlAEAAAA4iEVdp/HcJK+pqgcl+UiSH80sdl1ZVc9K8tEkP5Ak3X1dVV2ZWYS6O8lzuvuexUwbAAAAgOUWEpi6+71J7ndDqMzOZtrf/pcmuXQ15wQAAADA4VnEPZgAAAAAOIIITAAAAAAMEZgAAAAAGLKom3wDwMJt3bFrxcfce9m5Kz4mAACsd85gAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYsmnREwAADmzrjl0rOt7ey85d0fEAACBxBhMAAAAAgwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIYITAAAAAAMWVhgqqpjqurPq+pN0/NHVNXVVfXh6eMJS/a9pKpuqKrrq+pJi5ozAAAAAPe3yDOYnpfkg0ue70jytu4+PcnbpuepqsckOT/JY5Ock+SlVXXMGs8VAAAAgANYSGCqqlOSnJvkt5ZsPi/JFdPjK5I8dcn2nd39ue6+MckNSR63RlMFAAAA4AEs6gymX0vys0m+sGTblu6+LUmmj4+ctp+c5KYl+908bQMAAABgHajuXttPWPXkJN/X3T9RVduTPL+7n1xVd3b38Uv2+2R3n1BVv5Hknd396mn7y5P8YXf/7n7GvijJRUmyZcuWs3bu3HlYc9y3b182b958WMcybs8tdy16CnPZcmxy+2cXPQs2CuvlyHfmycet2Fj79u3LjXfds2LjJSs7P9YXf39hHtYL87BemIf1cuQ7++yzr+3ubcu3b1rAXL4jyVOq6vuSPCTJw6vq1Ulur6qTuvu2qjopyR3T/jcnOXXJ8ackuXV/A3f35UkuT5Jt27b19u3bD2uCu3fvzuEey7gLd+xa9BTmcvGZd+dFexbxnxIbkfVyFNjz6RUb6uIz71nx9bL3GdtXdDzWD39/YR7WC/OwXpiH9XL0WvNL5Lr7ku4+pbu3Znbz7j/u7mcmuSrJBdNuFyR54/T4qiTnV9WDq+q0JKcnefcaTxsAAACAA1hP/4x+WZIrq+pZST6a5AeSpLuvq6ork3wgyd1JntPdK3u9AAAAAACHbaGBqbt3J9k9Pf54kiccYL9Lk1y6ZhMDAAAA4JAt6l3kAAAAADhCCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwZNOiJ8DGt3XHrkVPAQAAAFggZzABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMGTToicAAKydrTt2rfiYey87d8XHBABgY3EGEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMGTToicAAGxsW3fsWtHx9l527oqOBwDA6nMGEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhqx5YKqqU6vqT6rqg1V1XVU9b9r+iKq6uqo+PH08Yckxl1TVDVV1fVU9aa3nDAAAAMCBLeIMpruTXNzdX5fk8UmeU1WPSbIjydu6+/Qkb5ueZ3rt/CSPTXJOkpdW1TELmDcAAAAA+7Hmgam7b+vuP5sefyrJB5OcnOS8JFdMu12R5KnT4/OS7Ozuz3X3jUluSPK4NZ00AAAAAAe00HswVdXWJN+U5F1JtnT3bcksQiV55LTbyUluWnLYzdM2AAAAANaB6u7FfOKqzUnenuTS7n5DVd3Z3ccvef2T3X1CVf1Gknd296un7S9P8ofd/bv7GfOiJBclyZYtW87auXPnYc1t37592bx582EdezTac8tdi57CQm05Nrn9s4ueBRuF9cI8jtb1cubJxy16ChuSv78wD+uFeVgvzMN6OfKdffbZ13b3tuXbNy1iMlX1xUl+N8lruvsN0+bbq+qk7r6tqk5Kcse0/eYkpy45/JQkt+5v3O6+PMnlSbJt27bevn37Yc1v9+7dOdxjj0YX7ti16Cks1MVn3p0X7VnIf0psQNYL8zha18veZ2xf9BQ2JH9/YR7WC/OwXpiH9XL0WsS7yFWSlyf5YHf/ypKXrkpywfT4giRvXLL9/Kp6cFWdluT0JO9eq/kCAAAAcHCL+GfR70jyw0n2VNV7p23/IsllSa6sqmcl+WiSH0iS7r6uqq5M8oHM3oHuOd19z5rPGgAAAID9WvPA1N3/LUkd4OUnHOCYS5NcumqTAgAAAOCwLfRd5AAAAADY+AQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhAhMAAAAAQwQmAAAAAIZsWvQEAACW2rpj14qOt/eyc1d0PAAA7s8ZTAAAAAAMcQbTUWil/2UYAAAAOLo5gwkAAACAIQITAAAAAEMEJgAAAACGuAcTAMAcVuNeht7pDgDY6JzBBAAAAMAQgQkAAACAIQITAAAAAEPcgwkAOKKtxj2TAAC4L2cwAQAAADBEYAIAAABgiMAEAAAAwBCBCQAAAIAhbvINALBgK3Ej8ovPvDsXTuPsvezc4fEAAObhDCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMGTToicAAMDK2rpj14qOt/eyc1d0PADgyOMMJgAAAACGCEwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGCEwAAAAADNm06AkAALC+bd2xa8XH3HvZuSs+JgCwOM5gAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDBCYAAAAAhghMAAAAAAwRmAAAAAAYIjABAAAAMERgAgAAAGCIwAQAAADAEIEJAAAAgCECEwAAAABDNi16AgAAHH227ti1ouPtvezcFR0PAJiPwAQAAIh+AAxxiRwAAAAAQwQmAAAAAIa4RA4AAABgCZcNz09gWudWelEDAAAArDSXyAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEMEJgAAAACGbFr0BAAAYNTWHbtWdLy9l527ouMdjVb6e7IafJ8BVo7ABAAAG9BIwLn4zLtz4QYIQABsHC6RAwAAAGCIM5gAAGAZl9wBwHwEJgAA4KgkJAKsHJfIAQAAADDEGUwAALDKNsI7qgHACGcwAQAAADBEYAIAAABgiMAEAAAAwBD3YAIAAFiHVuPeXd7pDlgtAhMAAACHZaUjmAAGG5fABAAAsAK8W+C4o/GsraPx98yRSWACAAA4ShxOzLj4zLtz4QaOZ8IfrI0Nc5Pvqjqnqq6vqhuqasei5wMAAADAzIY4g6mqjknyG0m+J8nNSf60qq7q7g8sdmYAAACwvqz3e2Ot9/lxeDZEYEryuCQ3dPdHkqSqdiY5L4nABAAAAKtoniC0iEsqXQa5PlR3L3oOD6iqnpbknO7+J9PzH07yrd39z5btd1GSi6anj05y/WF+yhOTfOwwj+XoY70wD+uFeVgvzMN6YR7WC/OwXpiH9XLk+4ru/rLlGzfKGUy1n233K2PdfXmSy4c/WdV7unvb6DgcHawX5mG9MA/rhXlYL8zDemEe1gvzsF6OXhvlJt83Jzl1yfNTkty6oLkAAAAAsMRGCUx/muT0qjqtqh6U5PwkVy14TgAAAABkg1wi1913V9U/S/JHSY5J8oruvm4VP+XwZXYcVawX5mG9MA/rhXlYL8zDemEe1gvzsF6OUhviJt8AAAAArF8b5RI5AAAAANYpgQkAAACAIQLTMlV1TlVdX1U3VNWORc+H9auqTq2qP6mqD1bVdVX1vEXPifWvqo6pqj+vqjctei6sb1V1fFW9vqo+NP05822LnhPrV1X99PSz6P1V9dqqesii58T6UVWvqKo7qur9S7Y9oqqurqoPTx9PWOQcWT8OsF7+w/Tz6H1V9XtVdfwCp8g6sr/1suS151dVV9WJi5gba09gWqKqjknyG0n+UZLHJHl6VT1msbNiHbs7ycXd/XVJHp/kOdYLh+B5ST646EmwIfx6krd099cm+YZYNxxAVZ2c5CeTbOvuMzJ7Q5TzFzsr1plXJjln2bYdSd7W3acnedv0HJL9r5erk5zR3V+f5H8muWStJ8W69crcf72kqk5N8j1JPrrWE2JxBKb7elySG7r7I939+SQ7k5y34DmxTnX3bd39Z9PjT2X2P38nL3ZWrGdVdUqSc5P81qLnwvpWVQ9P8t1JXp4k3f357r5zoZNivduU5Niq2pTkoUluXfB8WEe6+x1JPrFs83lJrpgeX5HkqWs5J9av/a2X7n5rd989Pb0mySlrPjHWpQP8+ZIkv5rkZ5N4V7GjiMB0XycnuWnJ85sjGHAIqmprkm9K8q4FT4X17dcy+0H7hQXPg/XvK5P8dZLfni6p/K2qetiiJ8X61N23JPnlzP6V+LYkd3X3Wxc7KzaALd19WzL7R7Mkj1zwfNg4fizJmxc9CdavqnpKklu6+y8WPRfWlsB0X7WfbYorB1VVm5P8bpKf6u6/WfR8WJ+q6slJ7ujuaxc9FzaETUm+OcnLuvubknw6Ll/hAKZ755yX5LQkj0rysKp65mJnBRyJqurnMrtNxGsWPRfWp6p6aJKfS/KvFz0X1p7AdF83Jzl1yfNT4hRzDqKqvjizuPSa7n7DoufDuvYdSZ5SVXszu/z2H1bVqxc7Jdaxm5Pc3N33nhX5+syCE+zPE5Pc2N1/3d1/m+QNSb59wXNi/bu9qk5KkunjHQueD+tcVV2Q5MlJntHd/hGeA/mqzP7B4y+mv/eekuTPquofLHRWrAmB6b7+NMnpVXVaVT0osxtkXrXgObFOVVVldn+UD3b3ryx6Pqxv3X1Jd5/S3Vsz+7Plj7vbGQbsV3f/VZKbqurR06YnJPnAAqfE+vbRJI+vqodOP5ueEDeF54FdleSC6fEFSd64wLmwzlXVOUlekOQp3f2ZRc+H9au793T3I7t76/T33puTfPP0dxuOcALTEtON6/5Zkj/K7C9mV3b3dYudFevYdyT54czORHnv9Ov7Fj0p4Ijx3CSvqar3JfnGJL+02OmwXk1nur0+yZ8l2ZPZ3+8uX+ikWFeq6rVJ3pnk0VV1c1U9K8llSb6nqj6c2Ts9XbbIObJ+HGC9vCTJlyS5evo7728udJKsGwdYLxylytmNAAAAAIxwBhMAAAAAQwQmAAAAAIYITAAAAAAMEZgAAAAAGCIwAQAAADBEYAIAjipV1VX1n5c831RVf11Vb5pznN1VtW16/IdVdfwKzO3CqnrJ6DgAAGtt06InAACwxj6d5IyqOra7P5vke5LcMjJgd3/fiswMAGCDcgYTAHA0enOSc6fHT0/y2ntfqKqHVdUrqupPq+rPq+q8afuxVbWzqt5XVa9LcuySY/ZW1YnT49+vqmur6rqqumjJPvuq6tKq+ouquqaqthxsglX1yqp6cVX9j6r6SFU9bclrP1tVe6axLpu2feM07vuq6veq6oRp++6q+tWqekdVfbCqvqWq3lBVH66qf7tkzGdW1bur6r1V9Z+q6pjD//ICAEcbgQkAOBrtTHJ+VT0kydcnedeS134uyR9397ckOTvJf6iqhyV5dpLPdPfXJ7k0yVkHGPvHuvusJNuS/GRVfem0/WFJrunub0jyjiQ/fgjzPCnJdyZ5cpJ7Q9I/SvLUJN86jfXvp31fleQF0/z2JHnhknE+393fneQ3k7wxyXOSnJHkwqr60qr6uiQ/lOQ7uvsbk9yT5BmHMD8AgCQukQMAjkLd/b6q2prZ2Ut/uOzl703ylKp6/vT8IUm+PMl3J3nxkuPfd4Dhf7Kq/vH0+NQkpyf5eJLPJ7n3Pk/XZnZp3gP5/e7+QpIPLDnj6YlJfru7PzPN5RNVdVyS47v77dM+VyT5nSXjXDV93JPkuu6+LUmq6iPTHL8zs2D2p1WVzM7OuuMQ5gcAkERgAgCOXlcl+eUk25N86ZLtleT/6u7rl+48hZc+2IBVtT2zAPRt3f2ZqtqdWaBKkr/t7nuPvyeH9vewzy2b170fDzqPg4zzhWVjfmGaRyW5orsvmXNcAIAkLpEDAI5er0jyC929Z9n2P0ry3JqKUlV907T9HZkuG6uqMzK7tG6545J8copLX5vk8asw77cm+bGqeug0l0d0911JPllV3zXt88NJ3n6gAfbjbUmeVlWPvHfMqvqKlZw0AHBkE5gAgKNSd9/c3b++n5d+MckXJ3lfVb1/ep4kL0uyebo07meTvHs/x74lyaZpn19Mcs0qzPstmZ199Z6qem+Sey/luyCz+0W9L8k3JvmFOcb8QJJ/meSt0/FXZ3b/JwCAQ1J/f6Y2AAAAAMzPGUwAAAAADBGYAAAAABgiMAEAAAAwRGACAAAAYIjABAAAAMAQgQkAAACAIQITAAAAAEP+f0hgrbN8OweHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x1080 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "housing['median_income'].hist(bins=50, figsize=(20,15))\n",
    "plt.xlabel('Median Income')\n",
    "plt.ylabel('Count')\n",
    "plt.title('Histogram for the median income')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that most of the median income values are clustered around 2 to 5 (i.e.,$20,000–$50,000), but some median incomes go far beyond 6 (i.e., $60,000). It is important to have a sufficient number of instances in your dataset for each stratum, or else the estimate of the stratum's importance may be biased. This means that you should not have too many strata, and each stratum should be large enough."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code creates an income category attribute by dividing the median income by 1.5 (to limit the number of income categories) and rounding up using ceil (to have discrete categories), and then keeping only the categories lower than 5 and merging the other categories into category 5:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Histogram for the median income')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjY0lEQVR4nO3df5xcdX3v8debhB+RJT8Q3IYkGq7GH4EomjWNRbybkkqqaOittOGiJEpNi9Rqi5XQX2o119xeUQQETQs3QZA1pWJSNNQYWbhtwZhYZA2YS2oChKSJkh9khcYmfPrH+W49mczumZ3dM7PJvp+PxzzmnO853+/3M9+Znc+eH3OOIgIzM7O+HNfsAMzMbOhzsjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhSNooqb3ZcZRN0qck/VTSv5XUfrukbWW0PdgkhaRXpOkvSvrzEvo4T9KmwW7XmsPJ4hgnaauk2RVlCyT9Y898RJwVEZ0F7UxOXzAjSwq1VJImAVcBUyPilwapzf/6wj2aRcTvRcQnS2j3/0XEqwa7XWsOJwsbEhqQhF4GPBMRu/pb8WhNkGaDycnCDtv6kDRD0npJz0raKemzabUH0vNeSd2S3iTpOEl/JukJSbsk3SZpTK7dy9KyZyT9eUU/H5d0l6TbJT0LLEh9Pyhpr6Qdkm6UdEKuvZD0AUmPS9ov6ZOSXp7qPCtpRX79XL3ZwBrgjBT7slT+zrQLbq+kTkmvqRiTqyU9AvysMmFI6hmPH6Q2fzu37Ko0HjskvTdXfqKkz0h6Mo3tFyWN6uU9WSDpnyR9LsX3Y0m/ksqfSu3Pr7VtSX+c4tku6X0VfS2T9Kk0PU7SPZJ+ImlPmp6YW7czjfs/pffgW5JO6+U1HLZbLo3pRyQ9ImmfpK9KOim3fK6kh9N7+a+S5qTyMyStkrRb0mZJ78/V+bikv02fo/2SuiS9UtI1aYyekvTW3PpjJN2SxuJpZbsmR1SL3ypEhB/H8APYCsyuKFsA/GO1dYAHgfek6RZgZpqeDAQwMlfvfcBm4L+ldb8GfDktmwp0A28GTgA+A/xHrp+Pp/mLyP5pGQVMB2YCI1N/jwEfzvUXwCpgNHAWcABYm/ofAzwKzO9lHNqBbbn5VwI/A34NOB74aHotJ+TG5GFgEjCqlzYDeEVFHweBv0xtvg14DhiXll+X4j8VOAX4e+DTvbS9ILX1XmAE8CngSeALwInAW4H9QEtR28AcYCdwNnAy8JV87MAy4FNp+sXAbwIvSu38LfD1XFydwL+m8RuV5pfUOOZbgXXAGSnOx4DfS8tmAPvS+3EcMAF4dVp2P3ATcBJwDvAT4Pzc5+jfgQvIPje3AVuAP03vwfuBLbkYvg58KY3DS1I8v9vsv9Oj4dH0APwo+Q3O/kC7gb25x3P0niweAD4BnFbRzmSOTBZrgQ/k5l9FlgBGAn8B3Jlb9iLg5xyeLB4oiP3DwN25+QDOzc1vAK7OzV8LXNdLW5VfXH8OrMjNHwc8DbTnxuR9BfFVSxbPV4zRLrIEKLLk9PLcsjflv8gq2l4APJ6bn5b6a82VPZO+PPtsG7iV3Bc62Rd91WRRJY5zgD25+U7gz3LzHwDurXHMtwLvzs3/FfDFNP0l4HNV2pgEHAJOyZV9GliW+xytyS17B9nnfUSaPyW91rFAK9k/GKNy618C3NfIv8mj9eF9scPDRRHx7Z4ZSQuA3+ll3cvJ/jP+kaQtwCci4p5e1j0DeCI3/wRZomhNy57qWRARz0l6pqL+U/kZSa8EPgu0kSWXkWQJIW9nbvr5KvO1Hrw+LPaIeEHSU2T/0VaNr0bPRMTB3PxzZFtdp5O9pg2SepaJbKuhN5WvjYioLKul7TM4fBzz79lhJL0I+BzZ1si4VHyKpBERcSjN588m63l9taqse0aangR8s8r6ZwC7I2J/ruwJss9Ij8ox+Wku1ufTc0tq63hgR26cjqO+93nY8TELO0xEPB4Rl5Btov9v4C5JJ5P9d1ZpO9mB4x4vJdt1shPYAeT3dY8i28VxWHcV8zcDPwKmRMRo4E/IvvTKcFjsyr49JpFtXfQW30D8lOyL66yIGJseYyKiP1+09ba9g+y19XhpH21dRbaF+MvpPXhLKi/rfejxFPDyKuXbgVMlnZIreymHv0/96eMA2VZzzziNjoiz6mhr2HGysMNIerek0yPiBbJdVpDtBvgJ8ALZ8YEedwJ/KOlMSS3A/wK+mv6zvgt4RzooewLZrq2iL5xTgGeBbkmvBq4YrNdVxQrg7ZLOl3Q82ZfkAeCf+9HGTg4fj16l8fxr4HOSXgIgaYKkC/oXdl1tryA7gWBq2nL4WB/NnUKWePZKOrVg3cF0C/De9H4cl+J/dUQ8RfaefFrSSZJeS7b1e0d/O4iIHcC3gGsljU79vFzSfx/UV3KMcrKwSnOAjZK6gc8D8yLi3yPiOWAx8E/p7JyZZPvCv0x2nGML2YHGDwJExMY03UH2n+1+sv33B/ro+yPA/0zr/jXw1cF/eZmI2AS8G7iB7D/zdwDviIif96OZjwPL03j8Vg3rX012EP0hZWeAfZvsv/jB0GvbEbGa7AD4d9I63+mjnevIDlz/FHgIuHeQ4utTRKwjO5j/ObID3ffziy2/S8iOmW0H7gY+FhFr6uzqMrITLh4F9pD9UzO+7sCHEaWDPGalSlsee8l2MW1pcjhm1k/esrDSSHqHpBelYx6fAbrIzogxs6OMk4WVaS7ZroPtwBSyXVrelDU7Cnk3lJmZFfKWhZmZFTpmf5R32mmnxeTJk+uq+7Of/YyTTz55cAMaBI6rfxxX/ziu/jlW49qwYcNPI+L0IxY0+yfkZT2mT58e9brvvvvqrlsmx9U/jqt/HFf/HKtxAeujyneqd0OZmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwKOVmYmVmhY/ZyH2ZDVdfT+1iw6BtN6Xvrkrc3pV87+nnLwszMCjlZmJlZodKShaRXSXo493hW0oclnSppjaTH0/O4XJ1rJG2WtCl/I3tJ0yV1pWXXS1JZcZuZ2ZFKSxYRsSkizomIc4DpwHNkN1tfBKyNiCnA2jSPpKnAPOAsYA5wk6QRqbmbgYVkd1ubkpabmVmDNGo31PnAv0bEE2S32lyeypcDF6XpuUBHRByIiC3AZmCGpPHA6Ih4MF0+97ZcHTMza4CG3FZV0q3A9yPiRkl7I2JsbtmeiBgn6UbgoYi4PZXfAqwGtgJLImJ2Kj8PuDoiLqzSz0KyLRBaW1und3R01BVvd3c3LS0tddUtk+Pqn6Ea167d+9j5fHP6njZhTK/Lhup4Oa7+GWhcs2bN2hARbZXlpZ86K+kE4J3ANUWrVimLPsqPLIxYCiwFaGtri/b29toDzens7KTeumVyXP0zVOO64Y6VXNvVnLPWt17a3uuyoTpejqt/yoqrEbuhfp1sq2Jnmt+Zdi2Rnnel8m3ApFy9icD2VD6xSrmZmTVII5LFJcCduflVwPw0PR9YmSufJ+lESWeSHcheFxE7gP2SZqazoC7L1TEzswYodVtY0ouAXwN+N1e8BFgh6XLgSeBigIjYKGkF8ChwELgyIg6lOlcAy4BRZMcxVpcZt5mZHa7UZBERzwEvrih7huzsqGrrLwYWVylfD5xdRoxmZlbMv+A2M7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhXynPANg8gDu3HbVtIN13/nNd24zOzp4y8LMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMysUKnJQtJYSXdJ+pGkxyS9SdKpktZIejw9j8utf42kzZI2SbogVz5dUldadr0klRm3mZkdruwti88D90bEq4HXAY8Bi4C1ETEFWJvmkTQVmAecBcwBbpI0IrVzM7AQmJIec0qO28zMckpLFpJGA28BbgGIiJ9HxF5gLrA8rbYcuChNzwU6IuJARGwBNgMzJI0HRkfEgxERwG25OmZm1gDKvn9LaFg6B1gKPEq2VbEB+BDwdESMza23JyLGSboReCgibk/ltwCrga3AkoiYncrPA66OiAur9LmQbAuE1tbW6R0dHXXF3t3dTUtLS111y1RmXF1P76u7buso2Pl8fXWnTRhTd79Fhur7uGv3vrrHa6D6Gu+hOl6Oq38GGtesWbM2RERbZXmZd8obCbwB+GBEfFfS50m7nHpR7ThE9FF+ZGHEUrIERVtbW7S3t/cr4B6dnZ3UW7dMZcZV753uILtT3rVd9X2Utl7aXne/RYbq+3jDHSvrHq+B6mu8h+p4Oa7+KSuuMo9ZbAO2RcR30/xdZMljZ9q1RHrelVt/Uq7+RGB7Kp9YpdzMzBqktGQREf8GPCXpVanofLJdUquA+alsPrAyTa8C5kk6UdKZZAey10XEDmC/pJnpLKjLcnXMzKwByt4W/iBwh6QTgB8D7yVLUCskXQ48CVwMEBEbJa0gSygHgSsj4lBq5wpgGTCK7DjG6pLjNjOznFKTRUQ8DBxxoIRsK6Pa+ouBxVXK1wNnD2pwZmZWM/+C28zMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFSk0WkrZK6pL0sKT1qexUSWskPZ6ex+XWv0bSZkmbJF2QK5+e2tks6XpJKjNuMzM7XCO2LGZFxDkR0ZbmFwFrI2IKsDbNI2kqMA84C5gD3CRpRKpzM7AQmJIecxoQt5mZJc3YDTUXWJ6mlwMX5co7IuJARGwBNgMzJI0HRkfEgxERwG25OmZm1gDKvn9LalzaAuwBAvhSRCyVtDcixubW2RMR4yTdCDwUEben8luA1cBWYElEzE7l5wFXR8SFVfpbSLYFQmtr6/SOjo664u7u7qalpaWuumUqM66up/fVXbd1FOx8vr660yaMqbvfIkP1fdy1e1/d4zVQfY33UB0vx9U/A41r1qxZG3J7gv7LyAFFVezciNgu6SXAGkk/6mPdaschoo/yIwsjlgJLAdra2qK9vb2f4WY6Ozupt26ZyoxrwaJv1F33qmkHubarvo/S1kvb6+63yFB9H2+4Y2Xd4zVQfY33UB0vx9U/ZcVV6m6oiNienncBdwMzgJ1p1xLpeVdafRswKVd9IrA9lU+sUm5mZg1SWrKQdLKkU3qmgbcCPwRWAfPTavOBlWl6FTBP0omSziQ7kL0uInYA+yXNTGdBXZarY2ZmDVDmtnArcHc6y3Uk8JWIuFfS94AVki4HngQuBoiIjZJWAI8CB4ErI+JQausKYBkwiuw4xuoS4zYzswqlJYuI+DHwuirlzwDn91JnMbC4Svl64OzBjtHMzGrjX3CbmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwK1ZQsJJ1bS5mZmR2bat2yuKHGMjMzOwb1eVtVSW8CfgU4XdIf5RaNBkaUGZiZmQ0dRVsWJwAtZEnllNzjWeBdtXQgaYSkf5F0T5o/VdIaSY+n53G5da+RtFnSJkkX5MqnS+pKy66XpP69TDMzG4g+tywi4n7gfknLIuKJOvv4EPAY2dYIwCJgbUQskbQozV8taSowDzgLOAP4tqRXRsQh4GZgIfAQ8E1gDrC6znjMzKyfaj1mcaKkpZK+Jek7PY+iSpImAm8H/iZXPBdYnqaXAxflyjsi4kBEbAE2AzMkjQdGR8SDERHAbbk6ZmbWAMq+fwtWkn4AfBHYABzqKY+IDQX17gI+Tbbr6iMRcaGkvRExNrfOnogYJ+lG4KGIuD2V30K29bAVWBIRs1P5ecDVEXFhlf4Wkm2B0NraOr2jo6PwtVXT3d1NS0tLXXXLVGZcXU/vq7tu6yjY+Xx9dadNGFN3v0WG6vu4a/e+usdroPoa76E6Xo6rfwYa16xZszZERFtleZ+7oXIORsTN/elQ0oXArojYIKm9lipVyqKP8iMLI5YCSwHa2tqivb2Wbo/U2dlJvXXLVGZcCxZ9o+66V007yLVdtX6UDrf10va6+y0yVN/HG+5YWfd4DVRf4z1Ux8tx9U9ZcdX6if17SR8A7gYO9BRGxO4+6pwLvFPS24CTgNGSbgd2ShofETvSLqZdaf1twKRc/YnA9lQ+sUq5mZk1SK3HLOYDfwz8M9muqA3A+r4qRMQ1ETExIiaTHbj+TkS8G1iV2utpd2WaXgXMk3SipDOBKcC6iNgB7Jc0M50FdVmujpmZNUBNWxYRceYg9rkEWCHpcuBJ4OLUx0ZJK4BHgYPAlelMKIArgGXAKLLjGD4TysysgWpKFpIuq1YeEbfVUj8iOoHONP0McH4v6y0GFlcpXw+cXUtfZmY2+Go9ZvHG3PRJZF/23yc7jdXMzI5xte6G+mB+XtIY4MulRGRmZkNOvZcof47sALSZmQ0DtR6z+Ht+8duGEcBrgBVlBWVmZkNLrccsPpObPgg8ERHbSojHzMyGoFqPWdwvqZVfHOh+vLyQzOxYM3mAVwio9woDW5e8ve5+7XC13invt4B1ZL+J+C3gu5JqukS5mZkd/WrdDfWnwBsjYheApNOBbwN3lRWYmZkNHbWeDXVcT6JInulHXTMzO8rVumVxr6R/AO5M879NdhMiMzMbBoruwf0KoDUi/ljS/wDeTHbJ8AeBOxoQn5mZDQFFu5KuA/YDRMTXIuKPIuIPybYqris3NDMzGyqKksXkiHiksjBd2G9yKRGZmdmQU5QsTupj2ajBDMTMzIauomTxPUnvryxM96Lo8/7bZmZ27Cg6G+rDwN2SLuUXyaENOAH4jRLjMjOzIaTPZBERO4FfkTSLX9x86BsR8Z3SIzMzsyGj1mtD3QfcV3IsZmY2RPlX2GZmVsjJwszMCpWWLCSdJGmdpB9I2ijpE6n8VElrJD2ensfl6lwjabOkTZIuyJVPl9SVll0vSWXFbWZmRypzy+IA8KsR8TrgHGCOpJnAImBtREwB1qZ5JE0F5gFnAXOAmySNSG3dDCwku5XrlLTczMwapLRkEZnuNHt8egQwF1ieypcDF6XpuUBHRByIiC3AZmCGpPHA6Ih4MCICuC1Xx8zMGkDZ929JjWdbBhuAVwBfiIirJe2NiLG5dfZExDhJNwIPRcTtqfwWYDWwFVgSEbNT+XnA1RFxYZX+FpJtgdDa2jq9o6Ojrri7u7tpaWmpq26Zyoyr6+l9dddtHQU7n6+v7rQJY+rut8hQfR937d5X93gNVF/j7c9X/wzVz9dA45o1a9aGiGirLK/1EuV1iYhDwDmSxpL9uO/sPlavdhwi+iiv1t9SYClAW1tbtLe39yveHp2dndRbt0xlxlXvbSshu+3ltV31fZS2Xtped79Fhur7eMMdK+ser4Hqa7z9+eqfofr5KiuuhpwNFRF7gU6yYw07064l0nPPTZW2AZNy1SYC21P5xCrlZmbWIGWeDXV62qJA0ihgNvAjYBUwP602H1iZplcB8ySdKOlMsgPZ6yJiB7Bf0sx0FtRluTpmZtYAZW4LjweWp+MWxwErIuIeSQ8CK9LFCJ8ELgaIiI2SVgCPAgeBK9NuLIArgGVkV7pdnR5mZtYgpSWLdB+M11cpfwY4v5c6i4HFVcrX84trU5mZWYP5F9xmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK9ScO7CYmR3jJg/ghk8DsWzOyaW06y0LMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVqi0ZCFpkqT7JD0maaOkD6XyUyWtkfR4eh6Xq3ONpM2SNkm6IFc+XVJXWna9JJUVt5mZHanMLYuDwFUR8RpgJnClpKnAImBtREwB1qZ50rJ5wFnAHOAmSSNSWzcDC4Ep6TGnxLjNzKxCackiInZExPfT9H7gMWACMBdYnlZbDlyUpucCHRFxICK2AJuBGZLGA6Mj4sGICOC2XB0zM2uAhhyzkDQZeD3wXaA1InZAllCAl6TVJgBP5aptS2UT0nRluZmZNYiyf9ZL7EBqAe4HFkfE1yTtjYixueV7ImKcpC8AD0bE7an8FuCbwJPApyNidio/D/hoRLyjSl8LyXZX0draOr2jo6OumLu7u2lpaamrbpnKjKvr6X11120dBTufr6/utAlj6u63yFB9H3ft3lf3eA1UX+Ptz1f/FI3XQF7zQJw5ZsSA3sdZs2ZtiIi2yvJSrw0l6Xjg74A7IuJrqXinpPERsSPtYtqVyrcBk3LVJwLbU/nEKuVHiIilwFKAtra2aG9vryvuzs5O6q1bpjLjWjCA69hcNe0g13bV91Haeml73f0WGarv4w13rKx7vAaqr/H256t/isZrIK95IJbNObmU97HMs6EE3AI8FhGfzS1aBcxP0/OBlbnyeZJOlHQm2YHsdWlX1X5JM1Obl+XqmJlZA5T57825wHuALkkPp7I/AZYAKyRdTraL6WKAiNgoaQXwKNmZVFdGxKFU7wpgGTAKWJ0eZmbWIKUli4j4R6C330Oc30udxcDiKuXrgbMHLzozM+sP/4LbzMwKOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyvkZGFmZoVKSxaSbpW0S9IPc2WnSloj6fH0PC637BpJmyVtknRBrny6pK607HpJKitmMzOrrswti2XAnIqyRcDaiJgCrE3zSJoKzAPOSnVukjQi1bkZWAhMSY/KNs3MrGSlJYuIeADYXVE8F1ieppcDF+XKOyLiQERsATYDMySNB0ZHxIMREcBtuTpmZtYgyr6DS2pcmgzcExFnp/m9ETE2t3xPRIyTdCPwUETcnspvAVYDW4ElETE7lZ8HXB0RF/bS30KyrRBaW1und3R01BV3d3c3LS0tddUtU5lxdT29r+66raNg5/P11Z02YUzd/RYZqu/jrt376h6vgeprvP356p+i8RrIax6IM8eMGND7OGvWrA0R0VZZPnJAUQ2easchoo/yqiJiKbAUoK2tLdrb2+sKprOzk3rrlqnMuBYs+kbdda+adpBru+r7KG29tL3ufosM1ffxhjtW1j1eA9XXePvz1T9F4zWQ1zwQy+acXMr72OizoXamXUuk512pfBswKbfeRGB7Kp9YpdzMzBqo0cliFTA/Tc8HVubK50k6UdKZZAey10XEDmC/pJnpLKjLcnXMzKxBStsWlnQn0A6cJmkb8DFgCbBC0uXAk8DFABGxUdIK4FHgIHBlRBxKTV1BdmbVKLLjGKvLitnMzKorLVlExCW9LDq/l/UXA4urlK8Hzh7E0Ap1Pb2vKfsbty55e8P7NDOrhX/BbWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCR02ykDRH0iZJmyUtanY8ZmbDyVGRLCSNAL4A/DowFbhE0tTmRmVmNnwcFckCmAFsjogfR8TPgQ5gbpNjMjMbNhQRzY6hkKR3AXMi4nfS/HuAX46I369YbyGwMM2+CthUZ5enAT+ts26ZHFf/OK7+cVz9c6zG9bKIOL2ycOQAGmwkVSk7IstFxFJg6YA7k9ZHRNtA2xlsjqt/HFf/OK7+GW5xHS27obYBk3LzE4HtTYrFzGzYOVqSxfeAKZLOlHQCMA9Y1eSYzMyGjaNiN1REHJT0+8A/ACOAWyNiY4ldDnhXVkkcV/84rv5xXP0zrOI6Kg5wm5lZcx0tu6HMzKyJnCzMzKzQsE0Wkm6VtEvSD3tZLknXp8uLPCLpDUMkrnZJ+yQ9nB5/0aC4Jkm6T9JjkjZK+lCVdRo+ZjXG1fAxk3SSpHWSfpDi+kSVdZoxXrXE1ZTPWOp7hKR/kXRPlWVN+ZusIa5m/U1uldSV+lxfZfngjldEDMsH8BbgDcAPe1n+NmA12W88ZgLfHSJxtQP3NGG8xgNvSNOnAP8fmNrsMasxroaPWRqDljR9PPBdYOYQGK9a4mrKZyz1/UfAV6r136y/yRriatbf5FbgtD6WD+p4Ddsti4h4ANjdxypzgdsi8xAwVtL4IRBXU0TEjoj4fpreDzwGTKhYreFjVmNcDZfGoDvNHp8elWeTNGO8aomrKSRNBN4O/E0vqzTlb7KGuIaqQR2vYZssajABeCo3v40h8CWUvCntRlgt6axGdy5pMvB6sv9K85o6Zn3EBU0Ys7Tr4mFgF7AmIobEeNUQFzTnM3Yd8FHghV6WN+vzdR19xwXNGa8AviVpg7JLHVUa1PFysuhdTZcYaYLvk1275XXADcDXG9m5pBbg74APR8SzlYurVGnImBXE1ZQxi4hDEXEO2RUHZkg6u2KVpoxXDXE1fLwkXQjsiogNfa1WpazU8aoxrmb9TZ4bEW8guxr3lZLeUrF8UMfLyaJ3Q/ISIxHxbM9uhIj4JnC8pNMa0bek48m+kO+IiK9VWaUpY1YUVzPHLPW5F+gE5lQsaupnrLe4mjRe5wLvlLSV7KrSvyrp9op1mjFehXE16/MVEdvT8y7gbrKrc+cN6ng5WfRuFXBZOqNgJrAvInY0OyhJvyRJaXoG2Xv4TAP6FXAL8FhEfLaX1Ro+ZrXE1Ywxk3S6pLFpehQwG/hRxWrNGK/CuJoxXhFxTURMjIjJZJfz+U5EvLtitYaPVy1xNenzdbKkU3qmgbcClWdQDup4HRWX+yiDpDvJzmI4TdI24GNkB/uIiC8C3yQ7m2Az8Bzw3iES17uAKyQdBJ4H5kU69aFk5wLvAbrS/m6APwFemoutGWNWS1zNGLPxwHJlN+46DlgREfdI+r1cXM0Yr1riatZn7AhDYLxqiasZ49UK3J1y1EjgKxFxb5nj5ct9mJlZIe+GMjOzQk4WZmZWyMnCzMwKOVmYmVkhJwszMyvkZGHHNEkh6cu5+ZGSfqIqVw8taKdTUlua/mbPbxUGGNsCSTcOtB2zRhi2v7OwYeNnwNmSRkXE88CvAU8PpMGIeNugRGZ2FPGWhQ0Hq8muGgpwCXBnz4L0S9hbJX1P2f0K5qbyUZI6lN0H4KvAqFydrT2Xc5D09XQht435i7lJ6pa0OF1c7iFJrX0FKGmZsnsP/LOkH0t6V27ZR5Xdt+AHkpaksnNSu49IulvSuFTeKelzkh5Qdo+PN0r6mqTHJX0q1+a7ld3X4mFJX0o/0jPrlZOFDQcdwDxJJwGv5fCr0v4p2SUc3gjMAv5PunzCFcBzEfFaYDEwvZe23xcR04E24A8kvTiVnww8lC4u9wDw/hriHA+8GbgQ6EkKvw5cBPxyauuv0rq3AVen+LrIfunf4+cR8Rbgi8BK4ErgbGCBpBdLeg3w22QXojsHOARcWkN8Nox5N5Qd8yLiEWWXL7+E7BIIeW8lu1DcR9L8SWSXCnkLcH2u/iO9NP8Hkn4jTU8CppBdF+jnQM9xkQ1ku7+KfD0iXgAezW2JzAb+b0Q8l2LZLWkMMDYi7k/rLAf+NtfOqvTcBWzsuR6QpB+nGN9Mlvy+ly4XMYrscuVmvXKysOFiFfAZsutuvThXLuA3I2JTfuX0JdrntXAktZN9mb8pIp6T1EmWbAD+I3d9oEPU9rd2oCKunuf+XpOnp50XKtp8IcUhYHlEXNPPdm0Y824oGy5uBf4yIroqyv8B+GDuqqGvT+UPkHbNKLvfw2urtDkG2JMSxavJbl052L4FvE/Si1Isp0bEPmCPpPPSOu8B7u+tgSrWAu+S9JKeNiW9bDCDtmOPk4UNCxGxLSI+X2XRJ8mu6vuIpB+meYCbgZa0++mjwLoqde8FRqZ1Pgk8VELc95JtFa1PV9Xt2V02n+z4yiPAOcBf9qPNR4E/I7vL2iPAGrLjJWa98lVnzcyskLcszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK/SfVnKB+FJe+roAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "housing[\"income_cat\"] = np.ceil(housing[\"median_income\"] / 1.5)\n",
    "housing[\"income_cat\"].where(housing[\"income_cat\"] < 5, 5.0, inplace=True)\n",
    "housing[\"income_cat\"].hist()\n",
    "plt.xlabel('Median Income')\n",
    "plt.ylabel('Count')\n",
    "plt.title('Histogram for the median income')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you are ready to do stratified sampling based on the income category. For this, we will use Scikit-Learn's **StratifiedShuffleSplit** class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import StratifiedShuffleSplit\n",
    "split = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=42)\n",
    "\n",
    "for train_index, test_index in split.split(housing, housing[\"income_cat\"]):\n",
    "    strat_train_set = housing.loc[train_index]\n",
    "    strat_test_set = housing.loc[test_index]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if this worked as expected. You can start by looking at the income category proportions in the stratified test set and compare it with the proportions in the overall dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0    0.350533\n",
       "2.0    0.318798\n",
       "4.0    0.176357\n",
       "5.0    0.114583\n",
       "1.0    0.039729\n",
       "Name: income_cat, dtype: float64"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strat_test_set[\"income_cat\"].value_counts() / len(strat_test_set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also measure the income category proportions in the full dataset as the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0    0.350581\n",
       "2.0    0.318847\n",
       "4.0    0.176308\n",
       "5.0    0.114438\n",
       "1.0    0.039826\n",
       "Name: income_cat, dtype: float64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "housing[\"income_cat\"].value_counts() / len(housing)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}