pandas_5_transform_finished.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cleaning and transforming data\n",
    "\n",
    "Files needed = ('atussum_2017.dat', movie lens small data set)\n",
    "\n",
    "We now know how to create some complicated datasets. In this notebook we will continue practicing techniques for manipulating data into forms that are amenable to analysis.  \n",
    "\n",
    "We will also learn about the American Time Use Survey. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd                 # pandas for data handling\n",
    "import matplotlib.pyplot as plt     # matplotlib for plots\n",
    "import numpy as np                  # numpy for numerical methods "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## American Time Use Survey (ATUS)\n",
    "The Bureau of Labor Statistics oversees the [American Time Use Survey](https://www.bls.gov/tus/), which asks a sample of Americans to complete detailed diaries keeping track of each minute of their day. \n",
    "\n",
    "Follow this link [https://www.bls.gov/tus/datafiles_2017.htm](https://www.bls.gov/tus/datafiles_2017.htm) to the page for the 2017 survey. Download the **ATUS 2017 Activity summary file (zip)** file located in the **2017 Basic ATUS Data Files** section of the page. Alternatively, download it directly [https://www.bls.gov/tus/special.requests/atussum_2017.zip](https://www.bls.gov/tus/special.requests/atussum_2017.zip). \n",
    "\n",
    "Unzip the file. We are looking for `atussum_2017.dat`. It is a coma separated file. Let's get it loaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "atus = pd.read_csv('atussum_2017/atussum_2017.dat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TUCASEID</th>\n",
       "      <th>TUFINLWGT</th>\n",
       "      <th>TRYHHCHILD</th>\n",
       "      <th>TEAGE</th>\n",
       "      <th>TESEX</th>\n",
       "      <th>PEEDUCA</th>\n",
       "      <th>PTDTRACE</th>\n",
       "      <th>PEHSPNON</th>\n",
       "      <th>GTMETSTA</th>\n",
       "      <th>TELFS</th>\n",
       "      <th>...</th>\n",
       "      <th>t181601</th>\n",
       "      <th>t181699</th>\n",
       "      <th>t181801</th>\n",
       "      <th>t181899</th>\n",
       "      <th>t189999</th>\n",
       "      <th>t500101</th>\n",
       "      <th>t500103</th>\n",
       "      <th>t500105</th>\n",
       "      <th>t500106</th>\n",
       "      <th>t500107</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20170101170002</td>\n",
       "      <td>5.217551e+06</td>\n",
       "      <td>15</td>\n",
       "      <td>34</td>\n",
       "      <td>2</td>\n",
       "      <td>39</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20170101170012</td>\n",
       "      <td>3.106847e+06</td>\n",
       "      <td>7</td>\n",
       "      <td>28</td>\n",
       "      <td>2</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20170101170027</td>\n",
       "      <td>1.653409e+07</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>35</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>20170101170030</td>\n",
       "      <td>1.757071e+07</td>\n",
       "      <td>-1</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20170101170033</td>\n",
       "      <td>1.022275e+06</td>\n",
       "      <td>-1</td>\n",
       "      <td>85</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 421 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         TUCASEID     TUFINLWGT  TRYHHCHILD  TEAGE  TESEX  PEEDUCA  PTDTRACE  \\\n",
       "0  20170101170002  5.217551e+06          15     34      2       39         2   \n",
       "1  20170101170012  3.106847e+06           7     28      2       40         1   \n",
       "2  20170101170027  1.653409e+07          15     15      1       35         3   \n",
       "3  20170101170030  1.757071e+07          -1     46      1       39         1   \n",
       "4  20170101170033  1.022275e+06          -1     85      1       44         1   \n",
       "\n",
       "   PEHSPNON  GTMETSTA  TELFS   ...     t181601  t181699  t181801  t181899  \\\n",
       "0         2         1      1   ...           0        0        0        0   \n",
       "1         2         1      5   ...           0        0        0        0   \n",
       "2         2         1      5   ...           0        0        0        0   \n",
       "3         2         1      1   ...           0        0        0        0   \n",
       "4         1         1      1   ...           0        0        0        0   \n",
       "\n",
       "   t189999  t500101  t500103  t500105  t500106  t500107  \n",
       "0        0        0        0        0        0        0  \n",
       "1        0        0        0        0        0        0  \n",
       "2        0        0        0        0        0        0  \n",
       "3        0        0        0        0        0        0  \n",
       "4        0        0        0        0        0        0  \n",
       "\n",
       "[5 rows x 421 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "421 variables! That's too many for us today. Let's just keep a few. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['TUCASEID', 'TUFINLWGT', 'TRYHHCHILD', 'TEAGE', 'TESEX', 'PEEDUCA', 'PTDTRACE', 'PEHSPNON', 'GTMETSTA', 'TELFS', 'TEMJOT', 'TRDPFTPT', 'TESCHENR', 'TESCHLVL', 'TRSPPRES', 'TESPEMPNOT', 'TRERNWA', 'TRCHILDNUM', 'TRSPFTPT', 'TEHRUSLT'] \n",
      "\n",
      "['t181204', 't181205', 't181299', 't181301', 't181302', 't181399', 't181401', 't181499', 't181501', 't181599', 't181601', 't181699', 't181801', 't181899', 't189999', 't500101', 't500103', 't500105', 't500106', 't500107']\n"
     ]
    }
   ],
   "source": [
    "col_list=atus.columns.tolist()   # columns returns a list object. Turn it into a list for easier printing.\n",
    "\n",
    "# Print the first and last 20 variables. \n",
    "print(col_list[0:20],'\\n')\n",
    "print(col_list[-20:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The demographic variables are all uppercase letters. The time variables are of the form 'txxyyzz' where xx is the major category code, yy is the second-tier code, and zz is the third-tier code. ([docs](https://www.bls.gov/tus/lexiconnoex2017.pdf))\n",
    "\n",
    "Let's keep some demo data and some data about working and sleeping. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "vars_to_keep = ['TEAGE', 'TESEX', 'PTDTRACE', 'PEEDUCA', 'GTMETSTA', 'TELFS', \n",
    "                'TUDIARYDAY', 't050101', 't050102', 't050201', 't010101', 't010102']\n",
    "\n",
    "atus_small = atus[vars_to_keep]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TEAGE</th>\n",
       "      <th>TESEX</th>\n",
       "      <th>PTDTRACE</th>\n",
       "      <th>PEEDUCA</th>\n",
       "      <th>GTMETSTA</th>\n",
       "      <th>TELFS</th>\n",
       "      <th>TUDIARYDAY</th>\n",
       "      <th>t050101</th>\n",
       "      <th>t050102</th>\n",
       "      <th>t050201</th>\n",
       "      <th>t010101</th>\n",
       "      <th>t010102</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>450</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>728</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>385</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>525</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   TEAGE  TESEX  PTDTRACE  PEEDUCA  GTMETSTA  TELFS  TUDIARYDAY  t050101  \\\n",
       "0     34      2         2       39         1      1           1      450   \n",
       "1     28      2         1       40         1      5           7        0   \n",
       "2     15      1         3       35         1      5           4        0   \n",
       "3     46      1         1       39         1      1           2      480   \n",
       "4     85      1         1       44         1      1           7        0   \n",
       "\n",
       "   t050102  t050201  t010101  t010102  \n",
       "0        0        0      728        0  \n",
       "1        0        0      385        0  \n",
       "2        0        0      570        0  \n",
       "3        0        0      525        0  \n",
       "4        0        0      756        0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's give the columns some more reasonable names. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>t050101</th>\n",
       "      <th>t050102</th>\n",
       "      <th>t050201</th>\n",
       "      <th>t010101</th>\n",
       "      <th>t010102</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>450</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>728</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>385</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>525</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender  race  edu  metro  employ  day  t050101  t050102  t050201  \\\n",
       "0   34       2     2   39      1       1    1      450        0        0   \n",
       "1   28       2     1   40      1       5    7        0        0        0   \n",
       "2   15       1     3   35      1       5    4        0        0        0   \n",
       "3   46       1     1   39      1       1    2      480        0        0   \n",
       "4   85       1     1   44      1       1    7        0        0        0   \n",
       "\n",
       "   t010101  t010102  \n",
       "0      728        0  \n",
       "1      385        0  \n",
       "2      570        0  \n",
       "3      525        0  \n",
       "4      756        0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# See https://www.bls.gov/tus/freqvariables.pdf for frequently used variables\n",
    "atus_small = atus_small.rename(columns={'TEAGE':'age', 'TESEX':'gender', 'PTDTRACE':'race', \n",
    "                           'PEEDUCA':'edu', 'GTMETSTA':'metro', 'TELFS':'employ', 'TUDIARYDAY':'day'})\n",
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# See https://www.bls.gov/tus/lexiconnoex2017.pdf for definitions\n",
    "atus_small = atus_small.rename(columns={'t050101':'work_main', 't050102':'work_other', 't010101':'sleep', \n",
    "                                        't050201':'work_soc', 't010102':'no_sleep'})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, we have variable names that we can live with. Let's start transforming the data to get it ready for analysis. atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, we have variable names that we can live with. Let's start transforming the data to get it ready for analysis. \n",
    "\n",
    "### A word on loops\n",
    "We have seen many places where a loop saved us time and effort. It might seem natural to use a loop to perform an operation on each element of a column or a DataFrame. \n",
    "\n",
    "In general, however, we want to avoid this. Instead, we have used pandas vectorized operations such as \n",
    "```python\n",
    "x['new_var'] = x['var_1'] / x['var_2']\n",
    "```\n",
    "to perform element-wise division. Using pandas' native operations is much (much much) faster than looping over the rows of a DataFrame. A lot of optimization has been written into the native functions that is not there when we loop ourselves. \n",
    "\n",
    "Fortunately, pandas provides methods that let us to very complex and very general operations to a DataFrame without resorting to a loop. We consider a few of these below. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### replace( ) \n",
    "\n",
    "The gender variable is coded 1 for male and 2 for female. I do not want to have to remember that!\n",
    "\n",
    "The `replace( )` method replaces one value for another (duh). One syntax is \n",
    "```\n",
    "atus_small['gender'] = atus_small['gender'].replace(1, 'male')\n",
    "```\n",
    "but a more powerful one passes a dict or a list.\n",
    "```\n",
    "atus_small['gender'] = atus_small['gender'].replace({1:'male', 2:'female'})\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>450</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>728</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>385</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>525</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender  race  edu  metro  employ  day  work_main  work_other  \\\n",
       "0   34  female     2   39      1       1    1        450           0   \n",
       "1   28  female     1   40      1       5    7          0           0   \n",
       "2   15    male     3   35      1       5    4          0           0   \n",
       "3   46    male     1   39      1       1    2        480           0   \n",
       "4   85    male     1   44      1       1    7          0           0   \n",
       "\n",
       "   work_soc  sleep  no_sleep  \n",
       "0         0    728         0  \n",
       "1         0    385         0  \n",
       "2         0    570         0  \n",
       "3         0    525         0  \n",
       "4         0    756         0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gender_codes = {1:'male', 2:'female'}\n",
    "atus_small['gender'] = atus_small['gender'].replace(gender_codes)\n",
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### unique( )\n",
    "Let's code race, too. What codes are in our data? The method `unique( )` returns the unique values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1  2  3  4  5  6  7  8  9 10 11 13 16 18]\n"
     ]
    }
   ],
   "source": [
    "races = atus_small['race'].unique()\n",
    "races.sort()\n",
    "print(races)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I looked up the codes in the documentation...\n",
    "\n",
    "race_codes = {1:'white', 2:'black', 3:'native_am', 4:'asian', 5:'hawaiian', 6:'wh_bl', \n",
    "              7:'wh_na', 8:'wh_as', 9:'wh_ha', 10:'bl_na', 11:'bl_as', 13:'na_as', 16:'wh_bl_as', 18:'wh_as_ha'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "atus_small['race'] = atus_small['race'].replace(race_codes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>450</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>728</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>385</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>480</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>525</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race  edu  metro  employ  day  work_main  work_other  \\\n",
       "0   34  female      black   39      1       1    1        450           0   \n",
       "1   28  female      white   40      1       5    7          0           0   \n",
       "2   15    male  native_am   35      1       5    4          0           0   \n",
       "3   46    male      white   39      1       1    2        480           0   \n",
       "4   85    male      white   44      1       1    7          0           0   \n",
       "\n",
       "   work_soc  sleep  no_sleep  \n",
       "0         0    728         0  \n",
       "1         0    385         0  \n",
       "2         0    570         0  \n",
       "3         0    525         0  \n",
       "4         0    756         0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply a function to a column: map( )\n",
    "\n",
    "We can apply functions to columns, too. The `map()` functions handles this for us. It applies the given function to each element of the column. These can be built in functions, or user defined functions. This quite powerful. We can write a function that performs a complicated transformation and apply to each element of a column in one simple line.  \n",
    "\n",
    "Let's define a function that converts minutes to hours. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def minutes_to_hours(x):\n",
    "    return x/60"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now apply `map( )` to the work column. \n",
    "\n",
    "\\[This example is a bit contrived. We could have just as easily done `atus_small['work_main']/60`. But that wouldn't have given us practice with `map( )`.\\] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>728</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>385</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>525</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race  edu  metro  employ  day  work_main  work_other  \\\n",
       "0   34  female      black   39      1       1    1        7.5           0   \n",
       "1   28  female      white   40      1       5    7        0.0           0   \n",
       "2   15    male  native_am   35      1       5    4        0.0           0   \n",
       "3   46    male      white   39      1       1    2        8.0           0   \n",
       "4   85    male      white   44      1       1    7        0.0           0   \n",
       "\n",
       "   work_soc  sleep  no_sleep  \n",
       "0         0    728         0  \n",
       "1         0    385         0  \n",
       "2         0    570         0  \n",
       "3         0    525         0  \n",
       "4         0    756         0  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small['work_main'] = atus_small['work_main'].map(minutes_to_hours)\n",
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply a function to a DataFrame: applymap( )\n",
    "If we want to apply the same function to several columns of a DataFrame (rather than a Series) we use `applymap( )`. It works the same way, applying the function to each element. \n",
    "\n",
    "\\[I am not sure why the developers of pandas created separate method for a Series vs a DataFrame. My best guess is that each method takes advantage of Series- or Dataframe-specific functionality for better performance.\\]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>39</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race  edu  metro  employ  day  work_main  work_other  \\\n",
       "0   34  female      black   39      1       1    1        7.5         0.0   \n",
       "1   28  female      white   40      1       5    7        0.0         0.0   \n",
       "2   15    male  native_am   35      1       5    4        0.0         0.0   \n",
       "3   46    male      white   39      1       1    2        8.0         0.0   \n",
       "4   85    male      white   44      1       1    7        0.0         0.0   \n",
       "\n",
       "   work_soc      sleep  no_sleep  \n",
       "0         0  12.133333       0.0  \n",
       "1         0   6.416667       0.0  \n",
       "2         0   9.500000       0.0  \n",
       "3         0   8.750000       0.0  \n",
       "4         0  12.600000       0.0  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can map to several columns at once.\n",
    "atus_small[['work_other', 'sleep', 'no_sleep']] = atus_small[['work_other', 'sleep', 'no_sleep']].applymap(minutes_to_hours)\n",
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice\n",
    "Take a few minutes and try the following. Feel free to chat with those around you if you get stuck. The TA and I are here, too. \n",
    "\n",
    "1. `educ` holds the highest level of education obtained. It can take values between 31 and 46. Covert the numeric values to labels. \n",
    "\n",
    "   * codes < 39 are 'less than high'\n",
    "   * 39 = 'high school'\n",
    "   * 40 = 'some college'\n",
    "   * 41 & 42 are 'associate'\n",
    "   * 43 = 'bachelor'\n",
    "   * 44 = 'master'\n",
    "   * 45 = 'prof'\n",
    "   * 46 = 'phd'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "educ_codes = {39:'high school', 40:'some college', 41:'associate', 42:'associate', \n",
    "              43: 'bachelor', 44: 'master', 45:'prof', 46:'phd'}\n",
    "\n",
    "atus_small.loc[ atus_small['edu']<39, 'edu'] = 'less than high'\n",
    "atus_small['edu'] = atus_small['edu'].replace(educ_codes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Make sure you covered all your bases. Print out a list of the unique values in the column 'edu'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['high school', 'some college', 'less than high', 'master',\n",
       "       'bachelor', 'prof', 'associate', 'phd'], dtype=object)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small['edu'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Apply the `minutes_to_hours` function to the `work_soc` variable. This variable is the time spent 'Socializing, relaxing, and leisure as part of job'. I have no idea what kind of jobs make you relax and take leisure. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1        7.5   \n",
       "1   28  female      white    some college      1       5    7        0.0   \n",
       "2   15    male  native_am  less than high      1       5    4        0.0   \n",
       "3   46    male      white     high school      1       1    2        8.0   \n",
       "4   85    male      white          master      1       1    7        0.0   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep  \n",
       "0         0.0       0.0  12.133333       0.0  \n",
       "1         0.0       0.0   6.416667       0.0  \n",
       "2         0.0       0.0   9.500000       0.0  \n",
       "3         0.0       0.0   8.750000       0.0  \n",
       "4         0.0       0.0  12.600000       0.0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small['work_soc'] = atus_small['work_soc'].map(minutes_to_hours)\n",
    "atus_small.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Create a column named 'work' that is the sum of 'work_main' and 'work_other'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "atus_small['work'] = atus_small['work_main'] + atus_small['work_other']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Create a histogram of 'work' for only those observations with work>0. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAGDCAYAAAAYtQWTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XvcrfWc//HXp70rUUoHpnZbW4cfyYyYkHFK4VcYZYhMUomNMYPBEGNG/PjJOBS/cdpE5VQ5lrNUuyIiipDDnqS9tbX37qQQqs/vj+/3rmuv1rrvde+9vvs+vZ6Px/24r9P6Xt91ndZ7fa9rXVdkJpIkSRqtjaa6ApIkSbORIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRNYxHx/oj4jxGVde+IuCki5tX+pRHxvFGUXcv7SkQcPqryJjHfN0XEmoj4bZ9x+0TEig1dp6kSEUdExDdHVNZIt4/1FRGLIiIjYn7tH3d7G+W+M0G9roiIx7WeTyvjreep2qcn0rstaG31OL/zVNdDhRvpFImIK4B7AbcAtwI/BU4GlmTmbQCZ+cJJlPW8zPzGoGky80pg8/Wr9e3zOwbYNTOf3Sn/gFGUPcl6LAReAeyUmas29Pw1dbrbW0QcQdn+H9kZP9S+o8GmYp+GEvyAj2Xmh6Zi/rUOx9BzjJspMnNUx/kj6NmvNHm2ZE2tv8/MLYCdgGOBVwMnjHoms/gb307ANVMdsKKY0n1pFq9jNTAdtllpTshM/6bgD7gCeFzPsIcCtwEPqP0nAm+q3dsCXwSuB64FzqeE5I/W1/wRuAl4FbAISOAo4ErgvM6w+bW8pcBbgO8CNwCnA1vXcfsAK/rVF9gf+DPwlzq/H3bKe17t3gh4HfBrYBWlhW7LOm6sHofXuq0B/n2c5bRlff3qWt7ravmPq+/5tlqPE/u8dh9gBaW1axWwEjhyorLruGMo36bpqXd3+b0Z+Fatx67AEcDlwI3Ar4BD+9TpLnX6bWv/6yitmXev/W8Cjh+ifkfUeR9H2R7eVId9szOvtwHf7Cz75wKXAdcBX6O0AI5N+3jgZ3Vb+G/g3LH1OcS2vDXwEeCqWvbnO+OeDyyrdTwD2KEzLoEXAr+sr3sPEHXcPODtdfu4HHhxn+X/PGB34GZKa/BNwPW9+8561mMX4GzgmlqXjwNbjbcfD7l9HVHXzdvrPH8FHDCgnCOBL3T6lwGndfqXA3vW7r8DvlfX4/eAv+tMt5Q7b7NLuWO/3R74EfDKPvv0uPUF7kM5ztwIfKMuw48NeD/3oBzLVteyvgjsWMe9ua7Lm+v6/O8+r1/Usy1sSflyuhL4DWVfmDfk+nt1fc2NwM+B/RhwjOtTj93rMroe+AnwlM64E+sy+FIt+0Jgl874+wFnUrbHnwPPGGf/Wlrf0wW1Pl8Atqnv5Xd1PS/q2Z53nagevctxyP1q07oNXAlcDbwf2GyY48Rc/ZvyCszVPwYcnOvG+6LafSJ3hKy31A164/r3KO74IFirrM7OczJwN2CzPgempfXg8oA6zWeoB0XGCVm1+xh6DqCsfUB+LuWDYGfKKcrPAh/tqdsHa70eCPwJ2H3AcjqZEgC3qK/9BXDUoHr2vHYfSoB5Y11mTwT+ANxjiLLXeo8Dlt+VwB6U0+5bUg54963jtwf2GFCv84Cn1e6vA/9D/cCq4546RP2OqO/tX+r8N+OOD8KN6vL9GnDXOv1BdZ3sXqd/HXBBHbdtrfvT63L611r2sCHrS8CplA/PjYHH1OH7Uj7YHkw5OP8/4LzO65LyAbsVcG/Kh+7+ddwLKaFvISXEndNn+a8VAHrqdCJ37DvrU49dKQF0U2C7un6On2g/HnL9/YUS/uYBL6KE1OhTzs6UD/KNKNvVr4HfdMZdV8dtXbsPq+v4WbV/mwHb7Mbc8aE6Vr/FA/bpcesLfJvy4bsJ8EjK9jQoZG0DPA24a102n2LtYH77fAe8flHPtvB54AOU49g9KV8cXzDR+gPuSwmoO3TKHQsgxwyqfx2/MWV/em19z/tSQszY/n8iJUA9tC7rjwOn1HF3q/M9so57MGX7HHS8WFrntQvlOPPTuq4eV19/MvCRnu151yHqsdZyHHK/Op7yJWXruu6+ALxlmOPEXP2b8grM1T8Gh6zvUFt2WPuD4o2UA/auE5XV2Xl27jOs+yF1bGf8/Snf3uax/iHrLOCfOuPuSzlAz+/UY8fO+O8Ch/R5X/MoAez+nWEvAJbW7jvVs+f1+1C+sXcPIquAvYcoe633OGD5vbEz/m6UD8KnMcE3O+D/AO+uy+O3wEspp4tvb+Uaon5HAFf2lHsE5ZvqqZTQvEln3FeoH/C1fyNK4NwJeA7wnc64oLQAThiyKB/6t1GDa8+4E4D/6vRvXreDRbU/gUd2xp8GHF27zwZe2Bn3hD7Lf9iQtc716POeDgIuHmI/Hmb9LeuMu2utx18NmO9yyofxIcASyj5zP8oH9Rl1msOA7/a87tvAEf222c6wd9b38axx9umB9aUE01uogb6O/xjjhJSe+ewJXNdvvgOmXzS2LVCua/0TnX2OEi7PmWj9UQLYKkpY2bhnumPGqz/lS+5vqS2TddgngWM629+HOuOeCPysdj8TOL+nvA8Arx8wr6V0WvuBdwBf6fT/PXBJp783ZA2qx+3LcZx13m0ZD+D3rN0i93DgV8Os57n65zn56WcB5ZtHr7dRvs18PSIuj4ijhyhr+STG/5ry7WzboWo5vh1qed2yxw6IY7q/BvwD/S/K35byLbG3rAWTqMs1mXlLn3mNouzbl19m/p5y8HwhsDIivhQR9xvwunMpAfDBwKWU0waPoYS/ZZm5Zsj69Vu/uwIHAm/IzD93hu8EvCsiro+IsVPOUcvboee95ICy+1kIXJuZ1/UZt9Z2kJk3UU7bdN/DoO1grTqx9nKYrHWuR0TcMyJOiYjfRMTvKOFhmH1kmPV3+zwz8w+1c9BFy2PbzKNr91LKNvOY2g933u/6zbPfej2U0qr96QHznqi+O1C2gT90ph24/UTEXSPiAxHx67pMzwO2Gvvl8yTtRDlurexs2x+gtGiNu/4ycxnwMkqgWlWn22HI+e4ALM/6I6Vq4Ppl7W17J+BhY/WtdT6UElgHubrT/cc+/eNd7D7MsXYY21HC9fc79f5qHa4BDFnTSEQ8hLKT3uln+Jl5Y2a+IjN3pnxzeXlE7Dc2ekCRg4aPWdjpvjfl2/0ayreVu3bqNY+1d6SJyr2KciDpln0Lax8YhrGm1qm3rN9Mspx1KXutZUD/A+BayyEzv5aZj6e07vyMcsqunwsorXtPBc7NzJ/WeT+JOz4wh3nv/dbDZZTWja9ExH07w5dTTqFs1fnbLDMvoFzLcvu2EBHB2tvGeJYDW0fEVn3GrbUdRMTdKKeKhll/a9WJ8t4HmdT2OMl6vKWW/zeZeXfg2ZRwOpFRb7tjIetRtftc7hyyeve7fvPst6yOqfX9xDoGnZWUbaC7v4y3/byCsv0/rC7TR9fhY8t1ovXZtZzSkrVtZ7u+e2buUcePu/4y8xNZfj23U53urUPW4SpgYc+PB4Zdv8sp+313X9w8M180xGtH6ff1/6DjXO8yWEMJdHt06r1ljujXjLOVIWsaiIi7R8STgVMoTdSX9pnmyRGxa/0A/B3lgsRb6+irKddmTNazI+L+9eD4RuDTmXkr5Xz/XSLiSRGxMeX6nU07r7saWDTOr5M+CfxrRNwnIjYH/i9wak+L0oRqXU4D3hwRW0TETsDLKd9G18sQZV8CPLreX2xL4DXjlRcR94qIp9QP8D9RLha9td+09Rv/9ykXc499QF5AOZ107pD1G++9fZJyrcg3ImKXOvj9wGsiYo9a3y0j4uA67kvAHhHxD/VXii+hc7Dt3JdoUZ95raScinxvRNwjIjaOiLEPzU8AR0bEnhGxKWU7uDAzr5joPdT3/pKI2DEi7gGM13J7NbBjRGwyYPz61GML6oW/EbEA+LchXtNi2z0XeCzltNgKyg9f9qeExYvrNF8G/ldE/GNEzI+IZ1IuA/jiBGX/BTiYcsr7o5P91WFm/hq4CDgmIjaJiIdTvggOsgXlw/r6iNgaeH3P+KGPZ3X7+zrwjnoc3SgidomIx3Tm1Xf9RcR9I2Lfuk3cXOvUPaaOd4y7kBJSXlW3+X3qez5liGp/kbKeDquv3TgiHhIRuw/znkclM1dTQuGzI2JeRDyXct3XmLX2q9pq90HguIgYaylcEBH/e0PWe6YxZE2tL0TEjZRvNv9OuTbiyAHT7kb51c5NlOss3puZS+u4twCvq024r5zE/D9KOWf/W8r1QC8ByMwbgH8CPkTZCX9PuUZnzKfq/2si4gd9yv1wLfs8yq+QbqZcoL0u/qXO/3JKC98navmjMLDszDyTcm3TjyiBaKIPqo0o39CvopyKewxlGQ5yLuU0x3c7/VtQltmE9ZtIZp5ECc5nR8SizPwc5Vv6KfW0yY+BA+q0aygfssdSTqPtRvkF2piF1IutB8zuMMoH9c8o17i8rJZ7FvAflOvDVlIO4IcMU3/uuHD/h8APKD+eGORsyq+7fhsRa3pHrmc93kA5rXsDJYyOV49eI9t2M/MXlH3//Nr/u1rut2qgIzOvAZ5M2Q6vofzS+Ml1/U5U/p+Bf6CcZvvwOtze4VDK9TnXUH4Jdyrly0Y/x1N+qLGGcg3qV3vGvwt4ekRcFxHvHmLez6Gcmv0p5UL/T1Nak2H89bcpZZtfQzkG3pPy5QQmOMbV5fUUyj60Bngv8JzM/NlElc3MGynXGB5COV78lrJvbjre6xp5PiV4XkP5QcQFnXH99qtXUy5b+U49jnyD0iqpAcZ+GSJJfUXE64DVmfmBqa6LZoaIOJVygXVvK5U0pxiyJEnrpV5Pei2l5foJlNsqPDwzLx73hdIs512iJUnr668op+K2oVxa8CIDlmRLliRJUhNe+C5JktSAIUuSJKmBaXFN1v77759f/Wrvr3glSZKmpWFuSjw9WrLWrJnwNi6SJEkzyrQIWZIkSbONIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDcyf6gpIamzJkslNv3hxm3pI0hxjS5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAW9GKs0kk72xqCRpytiSJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDQ4esiJgXERdHxBdr/30i4sKI+GVEnBoRm9Thm9b+ZXX8ojZVlyRJmr4m05L1UuCyTv9bgeMyczfgOuCoOvwo4LrM3BU4rk4nSZI0pwwVsiJiR+BJwIdqfwD7Ap+uk5wEHFS7D6z91PH71eklSZLmjGFbso4HXgXcVvu3Aa7PzFtq/wpgQe1eACwHqONvqNNLkiTNGROGrIh4MrAqM7/fHdxn0hxiXLfcxRFxUURctHr16qEqK0mSNFMM05L1COApEXEFcArlNOHxwFYRMb9OsyNwVe1eASwEqOO3BK7tLTQzl2TmXpm513bbbbdeb0KSJGm6mTBkZeZrMnPHzFwEHAKcnZmHAucAT6+THQ6cXrvPqP3U8Wdn5p1asiRJkmaz9blP1quBl0fEMso1VyfU4ScA29ThLweOXr8qSpIkzTzzJ57kDpm5FFhauy8HHtpnmpuBg0dQN0mSpBnLO75LkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ1MGLIi4i4R8d2I+GFE/CQi3lCH3yciLoyIX0bEqRGxSR2+ae1fVscvavsWJEmSpp9hWrL+BOybmQ8E9gT2j4i9gbcCx2XmbsB1wFF1+qOA6zJzV+C4Op0kSdKcMmHIyuKm2rtx/UtgX+DTdfhJwEG1+8DaTx2/X0TEyGosSZI0Awx1TVZEzIuIS4BVwJnA/wDXZ+YtdZIVwILavQBYDlDH3wBs06fMxRFxUURctHr16vV7F5IkSdPMUCErM2/NzD2BHYGHArv3m6z+79dqlXcakLkkM/fKzL222267YesrSZI0I8yfzMSZeX1ELAX2BraKiPm1tWpH4Ko62QpgIbAiIuYDWwLXjq7K0iyxZMlU10CS1NAwvy7cLiK2qt2bAY8DLgPOAZ5eJzscOL12n1H7qePPzsw7tWRJkiTNZsO0ZG0PnBQR8yih7LTM/GJE/BQ4JSLeBFwMnFCnPwH4aEQso7RgHdKg3pIkSdPahCErM38EPKjP8Msp12f1Dr8ZOHgktZMkSZqhvOO7JElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqYFJPbtQ0hywLs9UXLx49PWQpBnOlixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJamD+VFdA0iywZMnkX7N48ejrIUnTiC1ZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDE4asiFgYEedExGUR8ZOIeGkdvnVEnBkRv6z/71GHR0S8OyKWRcSPIuLBrd+EJEnSdDNMS9YtwCsyc3dgb+DFEXF/4GjgrMzcDTir9gMcAOxW/xYD7xt5rSVJkqa5CUNWZq7MzB/U7huBy4AFwIHASXWyk4CDaveBwMlZfAfYKiK2H3nNJUmSprFJXZMVEYuABwEXAvfKzJVQghhwzzrZAmB552Ur6jBJkqQ5Y+iQFRGbA58BXpaZvxtv0j7Dsk95iyPiooi4aPXq1cNWQ5IkaUYYKmRFxMaUgPXxzPxsHXz12GnA+n9VHb4CWNh5+Y7AVb1lZuaSzNwrM/fabrvt1rX+kiRJ09Iwvy4M4ATgssx8Z2fUGcDhtftw4PTO8OfUXxnuDdwwdlpRkiRprpg/xDSPAA4DLo2IS+qw1wLHAqdFxFHAlcDBddyXgScCy4A/AEeOtMaSJEkzwIQhKzO/Sf/rrAD26zN9Ai9ez3pJkiTNaN7xXZIkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqYP5UV2CDWbJk8q9ZvHj09ZAkSXOCLVmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDcydO75Lml4m+xQGn8AgaYaxJUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IAhS5IkqQFDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEmSJDVgyJIkSWpg/lRXQJo1liyZ6hpIkqYRW7IkSZIaMGRJkiQ1YMiSJElqwGuyJM0M63LN2+LFo6+HJA3JlixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAe+TNR7vyzN3+RxCSdJ6siVLkiSpgQlbsiLiw8CTgVWZ+YA6bGvgVGARcAXwjMy8LiICeBfwROAPwBGZ+YM2Vde0Y8ufJEm3G6Yl60Rg/55hRwNnZeZuwFm1H+AAYLf6txh432iqKUmSNLNMGLIy8zzg2p7BBwIn1e6TgIM6w0/O4jvAVhGx/agqK0mSNFOs6zVZ98rMlQD1/z3r8AXA8s50K+qwO4mIxRFxUURctHr16nWshiRJ0vQ06l8XRp9h2W/CzFwCLAHYa6+9+k4jSevF6wQlTaF1bcm6euw0YP2/qg5fASzsTLcjcNW6V0+SJGlmWteQdQZweO0+HDi9M/w5UewN3DB2WlGSJGkuGeYWDp8E9gG2jYgVwOuBY4HTIuIo4Erg4Dr5lym3b1hGuYXDkQ3qLE2ONxaVJE2BCUNWZj5rwKj9+kybwIvXt1KSJEkznXd8lyRJasBnF2pq+esvSdIsZUuWJElSA4YsSZKkBjxdqJnHXwtKkmYAQ9ZU85okSZJmJU8XSpIkNWDIkiRJasCQJUmS1IDXZEnS+vC6SkkD2JIlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGvAWDpLU5bMxJY2ILVmSJEkNGLIkSZIa8HThXOFdqSVJ2qAMWRrMa1MkSVpnhqxRM5hIkiS8JkuSJKkJQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ14C0cJGlD8+bA0pxgS5YkSVIDhixJkqQGPF04E3lXeUmSpj1bsiRJkhowZEmSJDVgyJIkSWrAkCVJktSAIUuSJKkBQ5YkSVIDhixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgA+IlqSZYLIPhl+8uE09JA3NlixJkqQGDFmSJEkNGLIkSZIaMGRJkiQ1YMiSJElqwJAlSZLUgLdwkKTZaLK3fABv+yCNmC1ZkiRJDRiyJEmSGjBkSZIkNWDIkiRJasCQJUmS1IC/LpQkFf4iURopW7IkSZIasCVLkrTu1qX1a7JsLdMMZUuWJElSA4YsSZKkBpqcLoyI/YF3AfOAD2XmsS3mI0maAzbUBfmTnY+nMTWBkYesiJgHvAd4PLAC+F5EnJGZPx31vCRJ6mtDXCsmTaBFS9ZDgWWZeTlARJwCHAgYsiRJs8dsuuXFbHov00iLkLUAWN7pXwE8rMF8JEmaWWZTC9uGOL06w8Nfi5AVfYblnSaKWAyMLYmbIuLnDerStS2wpvE8tOG5Xmcv1+3s5HqdnSZery94wYapyYaZz1czc/+JJmoRslYACzv9OwJX9U6UmUuADRbpI+KizNxrQ81PG4brdfZy3c5OrtfZyfXaX4tbOHwP2C0i7hMRmwCHAGc0mI8kSdK0NfKWrMy8JSL+Gfga5RYOH87Mn4x6PpIkSdNZk/tkZeaXgS+3KHs9zKKrDdXhep29XLezk+t1dnK99hGZd7omXZIkSevJx+pIkiQ1MCdCVkTsHxE/j4hlEXH0VNdHoxERV0TEpRFxSURcNNX10bqJiA9HxKqI+HFn2NYRcWZE/LL+v8dU1lHrZsC6PSYiflP320si4olTWUdNTkQsjIhzIuKyiPhJRLy0Dnef7WPWh6zOY34OAO4PPCsi7j+1tdIIPTYz9/SnwzPaiUDv/WaOBs7KzN2As2q/Zp4TufO6BTiu7rd71mt4NXPcArwiM3cH9gZeXD9T3Wf7mPUhi85jfjLzz8DYY34kTQOZeR5wbc/gA4GTavdJwEEbtFIaiQHrVjNYZq7MzB/U7huByyhPenGf7WMuhKx+j/lZMEV10Wgl8PWI+H59goBmj3tl5kooB3XgnlNcH43WP0fEj+rpRE8rzVARsQh4EHAh7rN9zYWQNdRjfjQjPSIzH0w5FfziiHj0VFdI0oTeB+wC7AmsBN4xtdXRuoiIzYHPAC/LzN9NdX2mq7kQsoZ6zI9mnsy8qv5fBXyOcmpYs8PVEbE9QP2/aorroxHJzKsz89bMvA34IO63M05EbEwJWB/PzM/Wwe6zfcyFkOVjfmahiLhbRGwx1g08Afjx+K/SDHIGcHjtPhw4fQrrohEa+yCunor77YwSEQGcAFyWme/sjHKf7WNO3Iy0/kT4eO54zM+bp7hKWk8RsTOl9QrKkws+4XqdmSLik8A+wLbA1cDrgc8DpwH3Bq4EDs5ML6CeYQas230opwoTuAJ4wdi1PJr+IuKRwPnApcBtdfBrKddluc/2mBMhS5IkaUObC6cLJUmSNjhDliRJUgOGLEmSpAYMWZIkSQ0YsiRJkhowZEkauYhYFBHT7v5HEfGoiPhJRFwSEZs1mscVEbFti7IlzSyGLEkzRkTMX88iDgXenpl7ZuYfR1GnroiYN+oyJc1chixJrcyLiA/WlqOvj7UcRcSeEfGd+oDgz409IDgilkbEXrV724i4onYfERGfiogvUB4Ivn1EnFdbo34cEY/qnXFE7BcRF0fEpfUhxJtGxPOAZwD/GREf75n+VRHxktp9XESc3SnnY7X7WbW8H0fEWzuvvSki3hgRFwIP7wzfLCK+GhHPH+EylTSDGLIktbIb8J7M3AO4HnhaHX4y8OrM/BvKXaNfP0RZDwcOz8x9gX8EvpaZewIPBC7pThgRdwFOBJ6ZmX9NeSLAizLzQ5RHf/xbZh5EGJkFAAAB4klEQVTaU/55wFhY2wvYvD6f7ZHA+RGxA/BWYF/K3cofEhEH1envBvw4Mx+Wmd+swzYHvkB5EsEHh3h/kmYhQ5akVn6VmWMB6PvAoojYEtgqM8+tw08CHj1EWWd2HtHxPeDIiDgG+OvMvLFn2vvWef9iEvP4PvC39XmYfwK+TQlbj6I8QuQhwNLMXJ2ZtwAf75R5K+VhuV2nAx/JzJOHeG+SZilDlqRW/tTpvpXSojSeW7jjmHSXnnG/H+vIzPMoAec3wEcj4jk908ZkK5qZf6E8R+9I4AJKsHossAtw2QRl3pyZt/YM+xZwQH2YrqQ5ypAlaYPJzBuA6zrXUR0GjLVqXQH8be1++qAyImInYFU9DXcC8OCeSX5GaTXbtc88xnMe8Mr6/3zghcAlWR7weiHwmHqt2DzgWROU+Z/ANcB7h5ivpFnKkCVpQzsceFtE/IhyfdMb6/C3Ay+KiAuA8W6BsA9wSURcTLnO613dkZl5M6VF6lMRcSlwG/D+Iep1PrA98O3MvBq4uQ4jM1cCrwHOAX4I/CAzT5+gvJcBd4mI/xpi3pJmoShf0iRJkjRKtmRJkiQ1YMiSJElqwJAlSZLUgCFLkiSpAUOWJElSA4YsSZKkBgxZkiRJDRiyJEmSGvj/apLkuPKJesAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "\n",
    "ax.hist(atus_small.loc[ atus_small['work']>0, 'work'], bins=40, alpha = 0.4, color = 'red')\n",
    "\n",
    "ax.set_xlabel('hours of work')\n",
    "ax.set_title('Distribution of hours worked, conditional on working at least one minute')\n",
    "\n",
    "ax.spines['top'].set_visible(False)\n",
    "ax.spines['right'].set_visible(False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binning \n",
    "\n",
    "Often we would like to take a continuous variable and make it discrete. One way to do this is to place the data into 'bins' or 'buckets'. (This is implicitly being done when we create a histogram.) Pandas has a nice facility for this. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10223.000000\n",
       "mean        49.876846\n",
       "std         18.008395\n",
       "min         15.000000\n",
       "25%         35.000000\n",
       "50%         50.000000\n",
       "75%         64.000000\n",
       "max         85.000000\n",
       "Name: age, dtype: float64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# What range are the ages?\n",
    "atus_small['age'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 15 to 85 years \n",
    "\n",
    "bins = [15, 25, 35, 45, 55, 65, 75]  # These are the endpoints for the bins\n",
    "\n",
    "atus_small['age_bin'] = pd.cut(atus_small['age'], bins)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "      <th>work</th>\n",
       "      <th>age_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>(25, 35]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(25, 35]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>(45, 55]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(55, 65]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>44</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.250000</td>\n",
       "      <td>(35, 45]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>bachelor</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>(45, 55]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>63</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>(55, 65]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>70</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(65, 75]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1       7.50   \n",
       "1   28  female      white    some college      1       5    7       0.00   \n",
       "2   15    male  native_am  less than high      1       5    4       0.00   \n",
       "3   46    male      white     high school      1       1    2       8.00   \n",
       "4   85    male      white          master      1       1    7       0.00   \n",
       "5   60    male      white  less than high      1       1    7       0.00   \n",
       "6   44    male      white     high school      2       1    2       8.25   \n",
       "7   49  female      black        bachelor      1       1    3       0.00   \n",
       "8   63    male      white    some college      1       1    1       1.50   \n",
       "9   70  female      white          master      1       5    6       0.00   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep      work   age_bin  \n",
       "0    0.000000       0.0  12.133333       0.0  7.500000  (25, 35]  \n",
       "1    0.000000       0.0   6.416667       0.0  0.000000  (25, 35]  \n",
       "2    0.000000       0.0   9.500000       0.0  0.000000       NaN  \n",
       "3    0.000000       0.0   8.750000       0.0  8.000000  (45, 55]  \n",
       "4    0.000000       0.0  12.600000       0.0  0.000000       NaN  \n",
       "5    0.000000       0.0   9.750000       0.0  0.000000  (55, 65]  \n",
       "6    0.000000       0.0   5.750000       0.0  8.250000  (35, 45]  \n",
       "7    7.433333       0.0   6.500000       0.0  7.433333  (45, 55]  \n",
       "8    0.000000       0.0   7.666667       0.0  1.500000  (55, 65]  \n",
       "9    0.000000       0.0   8.000000       0.0  0.000000  (65, 75]  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the NaN for observation 4. Our lowest bin starts at 15, but it is open on the left end point --- 15 is not included in the bin. We could set the bottom interval to 14 to catch this observation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "      <th>work</th>\n",
       "      <th>age_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>(25, 35]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(25, 35]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(14, 25]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>(45, 55]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(55, 65]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>44</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.250000</td>\n",
       "      <td>(35, 45]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>bachelor</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>(45, 55]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>63</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>(55, 65]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>70</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(65, 75]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1       7.50   \n",
       "1   28  female      white    some college      1       5    7       0.00   \n",
       "2   15    male  native_am  less than high      1       5    4       0.00   \n",
       "3   46    male      white     high school      1       1    2       8.00   \n",
       "4   85    male      white          master      1       1    7       0.00   \n",
       "5   60    male      white  less than high      1       1    7       0.00   \n",
       "6   44    male      white     high school      2       1    2       8.25   \n",
       "7   49  female      black        bachelor      1       1    3       0.00   \n",
       "8   63    male      white    some college      1       1    1       1.50   \n",
       "9   70  female      white          master      1       5    6       0.00   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep      work   age_bin  \n",
       "0    0.000000       0.0  12.133333       0.0  7.500000  (25, 35]  \n",
       "1    0.000000       0.0   6.416667       0.0  0.000000  (25, 35]  \n",
       "2    0.000000       0.0   9.500000       0.0  0.000000  (14, 25]  \n",
       "3    0.000000       0.0   8.750000       0.0  8.000000  (45, 55]  \n",
       "4    0.000000       0.0  12.600000       0.0  0.000000       NaN  \n",
       "5    0.000000       0.0   9.750000       0.0  0.000000  (55, 65]  \n",
       "6    0.000000       0.0   5.750000       0.0  8.250000  (35, 45]  \n",
       "7    7.433333       0.0   6.500000       0.0  7.433333  (45, 55]  \n",
       "8    0.000000       0.0   7.666667       0.0  1.500000  (55, 65]  \n",
       "9    0.000000       0.0   8.000000       0.0  0.000000  (65, 75]  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bins = [14, 25, 35, 45, 55, 65, 75]  # These are the endpoints for the bins\n",
    "atus_small['age_bin'] = pd.cut(atus_small['age'], bins)\n",
    "atus_small.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could also make the bins closed on the left and opened on the right by passing `'right=False'` parameter to `cut( )`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "      <th>work</th>\n",
       "      <th>age_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>[25, 35)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>[25, 35)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>[15, 25)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>[45, 55)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>[55, 65)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>44</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.250000</td>\n",
       "      <td>[35, 45)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>bachelor</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>[45, 55)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>63</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>[55, 65)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>70</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>[65, 75)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1       7.50   \n",
       "1   28  female      white    some college      1       5    7       0.00   \n",
       "2   15    male  native_am  less than high      1       5    4       0.00   \n",
       "3   46    male      white     high school      1       1    2       8.00   \n",
       "4   85    male      white          master      1       1    7       0.00   \n",
       "5   60    male      white  less than high      1       1    7       0.00   \n",
       "6   44    male      white     high school      2       1    2       8.25   \n",
       "7   49  female      black        bachelor      1       1    3       0.00   \n",
       "8   63    male      white    some college      1       1    1       1.50   \n",
       "9   70  female      white          master      1       5    6       0.00   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep      work   age_bin  \n",
       "0    0.000000       0.0  12.133333       0.0  7.500000  [25, 35)  \n",
       "1    0.000000       0.0   6.416667       0.0  0.000000  [25, 35)  \n",
       "2    0.000000       0.0   9.500000       0.0  0.000000  [15, 25)  \n",
       "3    0.000000       0.0   8.750000       0.0  8.000000  [45, 55)  \n",
       "4    0.000000       0.0  12.600000       0.0  0.000000       NaN  \n",
       "5    0.000000       0.0   9.750000       0.0  0.000000  [55, 65)  \n",
       "6    0.000000       0.0   5.750000       0.0  8.250000  [35, 45)  \n",
       "7    7.433333       0.0   6.500000       0.0  7.433333  [45, 55)  \n",
       "8    0.000000       0.0   7.666667       0.0  1.500000  [55, 65)  \n",
       "9    0.000000       0.0   8.000000       0.0  0.000000  [65, 75)  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bins = [15, 25, 35, 45, 55, 65, 75]  # These are the endpoints for the bins\n",
    "atus_small['age_bin'] = pd.cut(atus_small['age'], bins, right=False)\n",
    "atus_small.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`cut( )` can create bins for us, too. Pass cut an integer and get back that many equally-sized bins. The `precision` parameter determines how many decimal places are used in the bin edges. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "      <th>work</th>\n",
       "      <th>age_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>(15.0, 38.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(15.0, 38.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(15.0, 38.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>(38.0, 62.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(62.0, 85.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(38.0, 62.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>44</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.250000</td>\n",
       "      <td>(38.0, 62.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>bachelor</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>(38.0, 62.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>63</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>(62.0, 85.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>70</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(62.0, 85.0]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1       7.50   \n",
       "1   28  female      white    some college      1       5    7       0.00   \n",
       "2   15    male  native_am  less than high      1       5    4       0.00   \n",
       "3   46    male      white     high school      1       1    2       8.00   \n",
       "4   85    male      white          master      1       1    7       0.00   \n",
       "5   60    male      white  less than high      1       1    7       0.00   \n",
       "6   44    male      white     high school      2       1    2       8.25   \n",
       "7   49  female      black        bachelor      1       1    3       0.00   \n",
       "8   63    male      white    some college      1       1    1       1.50   \n",
       "9   70  female      white          master      1       5    6       0.00   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep      work       age_bin  \n",
       "0    0.000000       0.0  12.133333       0.0  7.500000  (15.0, 38.0]  \n",
       "1    0.000000       0.0   6.416667       0.0  0.000000  (15.0, 38.0]  \n",
       "2    0.000000       0.0   9.500000       0.0  0.000000  (15.0, 38.0]  \n",
       "3    0.000000       0.0   8.750000       0.0  8.000000  (38.0, 62.0]  \n",
       "4    0.000000       0.0  12.600000       0.0  0.000000  (62.0, 85.0]  \n",
       "5    0.000000       0.0   9.750000       0.0  0.000000  (38.0, 62.0]  \n",
       "6    0.000000       0.0   5.750000       0.0  8.250000  (38.0, 62.0]  \n",
       "7    7.433333       0.0   6.500000       0.0  7.433333  (38.0, 62.0]  \n",
       "8    0.000000       0.0   7.666667       0.0  1.500000  (62.0, 85.0]  \n",
       "9    0.000000       0.0   8.000000       0.0  0.000000  (62.0, 85.0]  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small['age_bin'] = pd.cut(atus_small['age'], 3, precision = 0)\n",
    "atus_small.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(38.0, 62.0]    4033\n",
       "(15.0, 38.0]    3176\n",
       "(62.0, 85.0]    3014\n",
       "Name: age_bin, dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.value_counts(atus_small['age_bin'])  # How many observations in each bin?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quantiles\n",
    "Notice that the bins are equally spaced from the smallest to the largest values of age. The number of observations in each is not equal. \n",
    "\n",
    "The method `qcut( )` splits the data according to the sample quantiles. This should get you bins of approximately equal numbers of observations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>race</th>\n",
       "      <th>edu</th>\n",
       "      <th>metro</th>\n",
       "      <th>employ</th>\n",
       "      <th>day</th>\n",
       "      <th>work_main</th>\n",
       "      <th>work_other</th>\n",
       "      <th>work_soc</th>\n",
       "      <th>sleep</th>\n",
       "      <th>no_sleep</th>\n",
       "      <th>work</th>\n",
       "      <th>age_bin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.133333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>(14.999, 35.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.416667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(14.999, 35.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15</td>\n",
       "      <td>male</td>\n",
       "      <td>native_am</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(14.999, 35.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>46</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>(35.0, 50.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>85</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.600000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(64.0, 85.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>less than high</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(50.0, 64.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>44</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>high school</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8.25</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.750000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.250000</td>\n",
       "      <td>(35.0, 50.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>49</td>\n",
       "      <td>female</td>\n",
       "      <td>black</td>\n",
       "      <td>bachelor</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.433333</td>\n",
       "      <td>(35.0, 50.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>63</td>\n",
       "      <td>male</td>\n",
       "      <td>white</td>\n",
       "      <td>some college</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.50</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.666667</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>(50.0, 64.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>70</td>\n",
       "      <td>female</td>\n",
       "      <td>white</td>\n",
       "      <td>master</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>(64.0, 85.0]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  gender       race             edu  metro  employ  day  work_main  \\\n",
       "0   34  female      black     high school      1       1    1       7.50   \n",
       "1   28  female      white    some college      1       5    7       0.00   \n",
       "2   15    male  native_am  less than high      1       5    4       0.00   \n",
       "3   46    male      white     high school      1       1    2       8.00   \n",
       "4   85    male      white          master      1       1    7       0.00   \n",
       "5   60    male      white  less than high      1       1    7       0.00   \n",
       "6   44    male      white     high school      2       1    2       8.25   \n",
       "7   49  female      black        bachelor      1       1    3       0.00   \n",
       "8   63    male      white    some college      1       1    1       1.50   \n",
       "9   70  female      white          master      1       5    6       0.00   \n",
       "\n",
       "   work_other  work_soc      sleep  no_sleep      work         age_bin  \n",
       "0    0.000000       0.0  12.133333       0.0  7.500000  (14.999, 35.0]  \n",
       "1    0.000000       0.0   6.416667       0.0  0.000000  (14.999, 35.0]  \n",
       "2    0.000000       0.0   9.500000       0.0  0.000000  (14.999, 35.0]  \n",
       "3    0.000000       0.0   8.750000       0.0  8.000000    (35.0, 50.0]  \n",
       "4    0.000000       0.0  12.600000       0.0  0.000000    (64.0, 85.0]  \n",
       "5    0.000000       0.0   9.750000       0.0  0.000000    (50.0, 64.0]  \n",
       "6    0.000000       0.0   5.750000       0.0  8.250000    (35.0, 50.0]  \n",
       "7    7.433333       0.0   6.500000       0.0  7.433333    (35.0, 50.0]  \n",
       "8    0.000000       0.0   7.666667       0.0  1.500000    (50.0, 64.0]  \n",
       "9    0.000000       0.0   8.000000       0.0  0.000000    (64.0, 85.0]  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "atus_small['age_bin'] = pd.qcut(atus_small['age'], 4)  # create quartiles based on age\n",
    "atus_small.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(35.0, 50.0]      2644\n",
       "(14.999, 35.0]    2560\n",
       "(50.0, 64.0]      2517\n",
       "(64.0, 85.0]      2502\n",
       "Name: age_bin, dtype: int64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.value_counts(atus_small['age_bin'])  # How many observations in each bin?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String methods\n",
    "We have seen some of these before. When we learned about strings. These are analogous to the string methods, but set up for DataFrames. These *vectorized string methods*, are optimized to work over an entire column. The method call looks like\n",
    "\n",
    "```python\n",
    "data['var'].str.method()\n",
    "```\n",
    "\n",
    "where `method( )` is the method we are applying. A list of vectorized string methods is on page 218 in McKinney. Below, we try a few out. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       "1                   Adventure|Children|Fantasy  \n",
       "2                               Comedy|Romance  \n",
       "3                         Comedy|Drama|Romance  \n",
       "4                                       Comedy  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies = pd.read_csv('ml-latest-small/movies.csv')\n",
    "movies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### str.contains( )\n",
    "The genres are mixed together. Let's get all the comedies. The `contains( )` method returns a bool Series with True for observations in which the string contains the search term."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        True\n",
       "1       False\n",
       "2        True\n",
       "3        True\n",
       "4        True\n",
       "5       False\n",
       "6        True\n",
       "7       False\n",
       "8       False\n",
       "9       False\n",
       "10       True\n",
       "11       True\n",
       "12      False\n",
       "13      False\n",
       "14      False\n",
       "15      False\n",
       "16      False\n",
       "17       True\n",
       "18       True\n",
       "19       True\n",
       "20       True\n",
       "21      False\n",
       "22      False\n",
       "23      False\n",
       "24      False\n",
       "25      False\n",
       "26      False\n",
       "27      False\n",
       "28      False\n",
       "29      False\n",
       "        ...  \n",
       "9712     True\n",
       "9713     True\n",
       "9714    False\n",
       "9715     True\n",
       "9716     True\n",
       "9717     True\n",
       "9718     True\n",
       "9719    False\n",
       "9720    False\n",
       "9721    False\n",
       "9722    False\n",
       "9723     True\n",
       "9724    False\n",
       "9725    False\n",
       "9726     True\n",
       "9727    False\n",
       "9728    False\n",
       "9729    False\n",
       "9730    False\n",
       "9731     True\n",
       "9732     True\n",
       "9733    False\n",
       "9734     True\n",
       "9735    False\n",
       "9736    False\n",
       "9737     True\n",
       "9738     True\n",
       "9739    False\n",
       "9740    False\n",
       "9741     True\n",
       "Name: genres, Length: 9742, dtype: bool"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies['genres'].str.contains('Comedy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Sabrina (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "6        7                      Sabrina (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       "2                               Comedy|Romance  \n",
       "3                         Comedy|Drama|Romance  \n",
       "4                                       Comedy  \n",
       "6                               Comedy|Romance  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comedies = movies[movies['genres'].str.contains('Comedy')]\n",
    "comedies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### str.split( )\n",
    "This method splits the string up at the delimiter that is passed to split( ). It returns a list of each chunk that falls between the delimiter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       [Adventure, Animation, Children, Comedy, Fantasy]\n",
       "1                          [Adventure, Children, Fantasy]\n",
       "2                                       [Comedy, Romance]\n",
       "3                                [Comedy, Drama, Romance]\n",
       "4                                                [Comedy]\n",
       "5                               [Action, Crime, Thriller]\n",
       "6                                       [Comedy, Romance]\n",
       "7                                   [Adventure, Children]\n",
       "8                                                [Action]\n",
       "9                           [Action, Adventure, Thriller]\n",
       "10                               [Comedy, Drama, Romance]\n",
       "11                                       [Comedy, Horror]\n",
       "12                       [Adventure, Animation, Children]\n",
       "13                                                [Drama]\n",
       "14                           [Action, Adventure, Romance]\n",
       "15                                         [Crime, Drama]\n",
       "16                                       [Drama, Romance]\n",
       "17                                               [Comedy]\n",
       "18                                               [Comedy]\n",
       "19               [Action, Comedy, Crime, Drama, Thriller]\n",
       "20                              [Comedy, Crime, Thriller]\n",
       "21              [Crime, Drama, Horror, Mystery, Thriller]\n",
       "22                              [Action, Crime, Thriller]\n",
       "23                                        [Drama, Sci-Fi]\n",
       "24                                       [Drama, Romance]\n",
       "25                                                [Drama]\n",
       "26                                      [Children, Drama]\n",
       "27                                       [Drama, Romance]\n",
       "28           [Adventure, Drama, Fantasy, Mystery, Sci-Fi]\n",
       "29                                         [Crime, Drama]\n",
       "                              ...                        \n",
       "9712                            [Comedy, Fantasy, Sci-Fi]\n",
       "9713         [Action, Adventure, Comedy, Fantasy, Sci-Fi]\n",
       "9714                                       [Crime, Drama]\n",
       "9715                                    [Comedy, Romance]\n",
       "9716                                             [Comedy]\n",
       "9717                         [Adventure, Comedy, Fantasy]\n",
       "9718                                      [Comedy, Drama]\n",
       "9719                                        [Documentary]\n",
       "9720                        [Action, Adventure, Thriller]\n",
       "9721                            [Action, Crime, Thriller]\n",
       "9722                                     [Action, Sci-Fi]\n",
       "9723                               [Comedy, Crime, Drama]\n",
       "9724                                   [Sci-Fi, Thriller]\n",
       "9725                                     [Drama, Romance]\n",
       "9726                                             [Comedy]\n",
       "9727                                              [Drama]\n",
       "9728                                              [Drama]\n",
       "9729                                          [Animation]\n",
       "9730                                        [Documentary]\n",
       "9731                  [Action, Adventure, Comedy, Sci-Fi]\n",
       "9732                  [Action, Animation, Comedy, Sci-Fi]\n",
       "9733                                   [Animation, Drama]\n",
       "9734                                      [Comedy, Drama]\n",
       "9735                                          [Animation]\n",
       "9736                                        [Documentary]\n",
       "9737                 [Action, Animation, Comedy, Fantasy]\n",
       "9738                         [Animation, Comedy, Fantasy]\n",
       "9739                                              [Drama]\n",
       "9740                                  [Action, Animation]\n",
       "9741                                             [Comedy]\n",
       "Name: genres, Length: 9742, dtype: object"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The movie genres are separated with the '|' character. \n",
    "\n",
    "movies['genres'].str.split('|')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This could be useful processing name data that come in the form: last,first or city,state. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>genre_split</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>[Adventure, Children, Fantasy]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>[Comedy, Romance]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>[Comedy, Drama, Romance]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>[Comedy]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Heat (1995)</td>\n",
       "      <td>Action|Crime|Thriller</td>\n",
       "      <td>[Action, Crime, Thriller]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Sabrina (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>[Comedy, Romance]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Tom and Huck (1995)</td>\n",
       "      <td>Adventure|Children</td>\n",
       "      <td>[Adventure, Children]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Sudden Death (1995)</td>\n",
       "      <td>Action</td>\n",
       "      <td>[Action]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>GoldenEye (1995)</td>\n",
       "      <td>Action|Adventure|Thriller</td>\n",
       "      <td>[Action, Adventure, Thriller]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "5        6                         Heat (1995)   \n",
       "6        7                      Sabrina (1995)   \n",
       "7        8                 Tom and Huck (1995)   \n",
       "8        9                 Sudden Death (1995)   \n",
       "9       10                    GoldenEye (1995)   \n",
       "\n",
       "                                        genres  \\\n",
       "0  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "1                   Adventure|Children|Fantasy   \n",
       "2                               Comedy|Romance   \n",
       "3                         Comedy|Drama|Romance   \n",
       "4                                       Comedy   \n",
       "5                        Action|Crime|Thriller   \n",
       "6                               Comedy|Romance   \n",
       "7                           Adventure|Children   \n",
       "8                                       Action   \n",
       "9                    Action|Adventure|Thriller   \n",
       "\n",
       "                                         genre_split  \n",
       "0  [Adventure, Animation, Children, Comedy, Fantasy]  \n",
       "1                     [Adventure, Children, Fantasy]  \n",
       "2                                  [Comedy, Romance]  \n",
       "3                           [Comedy, Drama, Romance]  \n",
       "4                                           [Comedy]  \n",
       "5                          [Action, Crime, Thriller]  \n",
       "6                                  [Comedy, Romance]  \n",
       "7                              [Adventure, Children]  \n",
       "8                                           [Action]  \n",
       "9                      [Action, Adventure, Thriller]  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remember, DataFrames can have columns of lists...\n",
    "\n",
    "movies['genre_split'] = movies['genres'].str.split('|')\n",
    "movies.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### str.join ( )\n",
    "Put strings together. Separate the pieces with a delimiter of your choosing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       Adventure::Animation::Children::Comedy::Fantasy\n",
       "1                          Adventure::Children::Fantasy\n",
       "2                                       Comedy::Romance\n",
       "3                                Comedy::Drama::Romance\n",
       "4                                                Comedy\n",
       "5                               Action::Crime::Thriller\n",
       "6                                       Comedy::Romance\n",
       "7                                   Adventure::Children\n",
       "8                                                Action\n",
       "9                           Action::Adventure::Thriller\n",
       "10                               Comedy::Drama::Romance\n",
       "11                                       Comedy::Horror\n",
       "12                       Adventure::Animation::Children\n",
       "13                                                Drama\n",
       "14                           Action::Adventure::Romance\n",
       "15                                         Crime::Drama\n",
       "16                                       Drama::Romance\n",
       "17                                               Comedy\n",
       "18                                               Comedy\n",
       "19               Action::Comedy::Crime::Drama::Thriller\n",
       "20                              Comedy::Crime::Thriller\n",
       "21              Crime::Drama::Horror::Mystery::Thriller\n",
       "22                              Action::Crime::Thriller\n",
       "23                                        Drama::Sci-Fi\n",
       "24                                       Drama::Romance\n",
       "25                                                Drama\n",
       "26                                      Children::Drama\n",
       "27                                       Drama::Romance\n",
       "28           Adventure::Drama::Fantasy::Mystery::Sci-Fi\n",
       "29                                         Crime::Drama\n",
       "                             ...                       \n",
       "9712                            Comedy::Fantasy::Sci-Fi\n",
       "9713         Action::Adventure::Comedy::Fantasy::Sci-Fi\n",
       "9714                                       Crime::Drama\n",
       "9715                                    Comedy::Romance\n",
       "9716                                             Comedy\n",
       "9717                         Adventure::Comedy::Fantasy\n",
       "9718                                      Comedy::Drama\n",
       "9719                                        Documentary\n",
       "9720                        Action::Adventure::Thriller\n",
       "9721                            Action::Crime::Thriller\n",
       "9722                                     Action::Sci-Fi\n",
       "9723                               Comedy::Crime::Drama\n",
       "9724                                   Sci-Fi::Thriller\n",
       "9725                                     Drama::Romance\n",
       "9726                                             Comedy\n",
       "9727                                              Drama\n",
       "9728                                              Drama\n",
       "9729                                          Animation\n",
       "9730                                        Documentary\n",
       "9731                  Action::Adventure::Comedy::Sci-Fi\n",
       "9732                  Action::Animation::Comedy::Sci-Fi\n",
       "9733                                   Animation::Drama\n",
       "9734                                      Comedy::Drama\n",
       "9735                                          Animation\n",
       "9736                                        Documentary\n",
       "9737                 Action::Animation::Comedy::Fantasy\n",
       "9738                         Animation::Comedy::Fantasy\n",
       "9739                                              Drama\n",
       "9740                                  Action::Animation\n",
       "9741                                             Comedy\n",
       "Name: genre_split, Length: 9742, dtype: object"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies['genre_split'].str.join('::')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice: What era had the best movies?\n",
    "Take a few minutes and try the following. Feel free to chat with those around you if you get stuck. The TA and I are here, too. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "0. Create a copy of the 'movies' DataFrame named 'movies_prac'. Work with 'movies_prac' for these exercises.\n",
    "1. Extract the four-digit year from the titles and put them into a new column named 'year'.  You might try `str.slice()`. \n",
    "2. There are 12 movies that do not have a year. Assign -1 to those. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The negative notation counts backwards. I am relying on the fact that each title ends in '(year)'. Not the most\n",
    "# robust method.\n",
    "\n",
    "movies_prac = movies\n",
    "\n",
    "movies_prac['title'] = movies_prac['title'].str.strip()\n",
    "movies_prac['year'] = movies_prac['title'].str.slice(-5, -1)\n",
    "\n",
    "movies_prac.loc[~movies_prac['year'].str.isdigit(),'year'] = -1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Drop the movies without years.\n",
    "4. Convert the year data to int.\n",
    "5. 'Bin' the years into decades. Create a column called 'decade' to store the binned values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  \n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:5: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  \"\"\"\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>genre_split</th>\n",
       "      <th>year</th>\n",
       "      <th>decade</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>[Adventure, Children, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>[Comedy, Romance]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>[Comedy, Drama, Romance]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>[Comedy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                        genres  \\\n",
       "0  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "1                   Adventure|Children|Fantasy   \n",
       "2                               Comedy|Romance   \n",
       "3                         Comedy|Drama|Romance   \n",
       "4                                       Comedy   \n",
       "\n",
       "                                         genre_split  year        decade  \n",
       "0  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]  \n",
       "1                     [Adventure, Children, Fantasy]  1995  (1990, 2000]  \n",
       "2                                  [Comedy, Romance]  1995  (1990, 2000]  \n",
       "3                           [Comedy, Drama, Romance]  1995  (1990, 2000]  \n",
       "4                                           [Comedy]  1995  (1990, 2000]  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_prac = movies_prac[ movies_prac['year']!=-1 ] \n",
    "movies_prac['year'] = movies_prac['year'].astype(int)\n",
    "\n",
    "decades = [1899, 1910, 1920, 1930, 1940, 1950, 1960, 1970, 1980, 1990, 2000, 2010, 2020]\n",
    "movies_prac['decade'] = pd.cut(movies_prac['year'], decades)\n",
    "movies_prac.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Load 'ratings.csv' from the MovieLens data and merge it to `movies_prac` to create a new DataFrame named `movies_2`. Use an inner merge and merge on 'movieId'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>genre_split</th>\n",
       "      <th>year</th>\n",
       "      <th>decade</th>\n",
       "      <th>userId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964982703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "      <td>5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>847434962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "      <td>7</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1106635946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "      <td>15</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1510577970</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>[Adventure, Animation, Children, Comedy, Fantasy]</td>\n",
       "      <td>1995</td>\n",
       "      <td>(1990, 2000]</td>\n",
       "      <td>17</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1305696483</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId             title                                       genres  \\\n",
       "0        1  Toy Story (1995)  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "1        1  Toy Story (1995)  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "2        1  Toy Story (1995)  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "3        1  Toy Story (1995)  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "4        1  Toy Story (1995)  Adventure|Animation|Children|Comedy|Fantasy   \n",
       "\n",
       "                                         genre_split  year        decade  \\\n",
       "0  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]   \n",
       "1  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]   \n",
       "2  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]   \n",
       "3  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]   \n",
       "4  [Adventure, Animation, Children, Comedy, Fantasy]  1995  (1990, 2000]   \n",
       "\n",
       "   userId  rating   timestamp  \n",
       "0       1     4.0   964982703  \n",
       "1       5     4.0   847434962  \n",
       "2       7     4.5  1106635946  \n",
       "3      15     2.5  1510577970  \n",
       "4      17     4.5  1305696483  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rate = pd.read_csv('ml-latest-small/ratings.csv')\n",
    "\n",
    "movies_2 = pd.merge(left=movies_prac, right=rate, on='movieId', how='inner')\n",
    "movies_2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. Run the following code. What does it do? We will learn more about groupby in the next class. \n",
    "```python\n",
    "means = movies_2.groupby('decade')['rating'].mean()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "decade\n",
       "(1899, 1910]    3.312500\n",
       "(1910, 1920]    3.562500\n",
       "(1920, 1930]    3.794776\n",
       "(1930, 1940]    3.745801\n",
       "(1940, 1950]    3.872955\n",
       "(1950, 1960]    3.831110\n",
       "(1960, 1970]    3.811119\n",
       "(1970, 1980]    3.769622\n",
       "(1980, 1990]    3.491111\n",
       "(1990, 2000]    3.431104\n",
       "(2000, 2010]    3.482803\n",
       "(2010, 2020]    3.471050\n",
       "Name: rating, dtype: float64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "means = movies_2.groupby('decade')['rating'].mean()\n",
    "means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8. Create a horizontal bar chart with the average movie ratings. The index of `means` is an interval object. To assign it to the y-axis labels, convert it to a string \n",
    "```python\n",
    "ax.barh(means.index.astype(str), means)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAocAAAE/CAYAAAA0ZWlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XucXVV99/HPlxBuIrdMUAQqICiNXFIyBtuHaIQCQbERgRKKEARKa/VpH2jzQK/UC49SqrQWhVKNgoriBSQikoRAIJYgTIBgACFRA0RQiCSACthhvs8fe53DyZkzcw7DZYaZ7/v1Oq/svdbaa/3OGvry17X3Olu2iYiIiIgA2Gi4A4iIiIiIkSPJYURERETUJTmMiIiIiLokhxERERFRl+QwIiIiIuqSHEZEREREXZLDiIgXiaQLJf3jMI7/PUmzX4J+p0ta82L3W/o+UdL3X4q+R9KYEa8kGw93ABEx8khaDOwLvNb2M8McziuG7T9/ucaS9M/A7rbf1zD+YS/X+BExemXlMCI2IGkXYBpg4I9eojHy/5gOIvMTEcMpyWFENDsBuBn4IlC/RSnprZJ+LmlcQ9kRku4sxxtJOlPSjyX9UtLXJW1X6naRZEknS3oAuK6Uf6P0+bikGyW9uaHvCZK+I+kJSbdK+ljjrUBJe0paKOkxSfdK+uOBvpCkxeX6myT9qvQ7QdJXGvrfpaH9H5Syx8u/f1DKZ0nqaer7NEnzyvEXJX2soe5wSXdIWl/G3meQGC3pg5JWAitL2b9LerDEuEzStFI+A/g74JjyfZY3fM9TyvGJkr4v6V8lrZP0U0mHNYy3a5nzJyVdK+kzkr48UHzlmr+TtFbSaknHlbK3SPpFY0Ir6UhJdwzQxwRJ88p3ugV4Q1P9gH9XSZtL+qSk+8vf5vuSNi917f5bGtKYEWNRksOIaHYC8JXyOVTSawBs3wz8Gjiwoe2fAJeW478E3gO8HXgdsA74TFPfbwd+Fzi0nH8P2APYHritjFnzmTLea6mS1MZE9VXAwjL29sCxwGcbE4IWZgHHAztSJQdLgS8A2wH3AGeVvrcDvgt8GpgAfAr4rqQJwDzgTZL2GGAO6iTtB8wF/qz085/APEmbDhLje4D9gUnl/FZgconxUuAbkjazfQ3w/4DLbG9pe98B+tsfuBfoAv4F+LwklbpLgVtKbP9c5mYwry397Ej1t7hI0pts3wr8Eji4oe37gC8N0M9ngKeBHYCTygfo6O/6r8AU4A/KnPxfoK/UtftvaahjRow9tvPJJ598sA1wAPA/QFc5/xFwWkP9x4C55fjVVMnb68v5PcBBDW13KH1tDOxCdZt6t0HG3qa02RoYV659U9PY3y/HxwBLmq7/T+CsAfpeDPx9w/knge81nL8buKMcHw/c0nT9UuDEcvxl4J/K8R7Ak8AW5fyLwMfK8QXAR5v6uRd4+wAxGjiwzd9nHbBvOf5n4Mstvucp5fhEYFVD3RZljNcCvwP01uJu+F5fHmDc6aX9qxrKvg78Yzk+A/hKOd4O+A2wQ4t+an/XPRvK/l8nf1eqxYynat+/zTy1+m/peY85nP+3mE8+w/nJymFENJoNLLC9tpxfSsOKXTl/b1n9ei9wm+37S93rgSvKLdT1VMnis8BrGq5/sHYgaZykT6i6Df0EsLpUdQETqZLKB1tdW8bavzZWGe84qsRnIL9oOH6qxfmW5fh1wP1s6H6qFbPaHBxbjv8E+Lbt37QY7/XAXzfFuHPpfyCN3xFJfy3pnnKrdD1VstM1yPXNfl47aIhxyxLDY01xbzB2C+ts/7rh/H6e+y5fBt4taUvgj6mSrYdb9NHq79o414P9XbuAzYAfN3c6hP+WOh0zYkzKQ88RAVTPc1H9D/s4SbWkYlNgG0n72l5u+25J9wOH0f926oPASbb/u0Xfu5RDNxT/CTAT+EOq/zHfmmplTMCjVCtVOwH3lfY7N411g+3GW5kvloeoEoZGvwNcU44XAF2SJlMliacN0M+DwNm2z34eY9fnpzxfeAZwEHCX7T5JtfnZoO0QPAxsJ2mLhgRx58EuALaV9KqGBPF3gBUAtn8maSlwBNXK6wUD9FH7u+5MtSpd66dmwL+rpI2obg2/AVjeVN3Jf0vPe8yIsSorhxFR8x6qlb5JVM+5TaZ6PnAJ1XOINZdSPV/4NuAbDeUXAmdLej2ApImSZg4y3quBZ6ieV9uC6lYfALafBS4H/lnSFpL2bIrhKuCNko6XNL583iLpd4fwvZtdXfr+E0kbSzqGak6uKrH1At8EzqW6hbpwgH7+C/hzSfur8ipJ75L06g7jeDVVUvMosLGkfwK2aqj/BbBLSZqel7La20M1v5tI+n2qW+vtfLi0nwYczoZ//0uongHcG7higHGb/66T2HBlesC/q+0+qmc4PyXpdWW18PfLKvbz+W+p4zE7mI+IUSnJYUTUzAa+YPsB2z+vfYDzgeP03G7Ur1I9g3Zdw+1ngH+n2rCxQNKTVDue9x9kvEuobu/9DLi7tG/0IaoVoJ9TbW74KlUCgO0ngUOoNpk8VNqcQ7XS+YLY/iVV4vPXVMnG/wUOb/qul1KtUn2jJIut+ukB/pRq/tYBq6ieA+zUfKpNFvdRzdPTbHhrtJaY/VLSbc+j35rjgN+n+o4fAy6jzO8Afk71PR6i2uzx57Z/1FB/BeXRgqbbz80+RHVr++dUz2h+oVbRwd/1b4AfUm3UeazUbURn/y0NdcyIMUf2C7kzERHx8pB0DtWPcr/obwAJkHQZ8CPbZ72APn4M/Jnta1+8yCLi5ZaVw4gYkcpvz+1TbslOBU5mgNuV8fyVW6dvUPX7lDOontn79gvo70iq5yCve7FijIjhkQ0pETFSvZrqVvLrgEeofn7mymGNaHR5LdWzeBOANcAHbN8+lI5UvW5xEnB8eTYwIl7Bcls5IiIiIupyWzkiIiIi6pIcRkRERERdnjl8AWbMmOFrrrmmfcOIiIiI4af2TbJy+IKsXbu2faOIiIiIV5AkhxERERFRl+QwIiIiIuqSHEZEREREXZLDiIiIiKhLchgRERERdUkOIyIiIqIuyWFERERE1CU5jIiIiIi6JIcRERERUZfkMCIiIiLq8m7lF6KvDxYvHu4oRpYpU2DZsuGOYuTIfPSXOdlQ5qO/zEl/Y31Opk8f7gjGlKwcRkRERERdksOIiIiIqEtyGBERERF1SQ4jIiIioi7JYURERETUtU0OJW0u6QZJ4yRNlrRU0l2S7pR0TEO7XSX9QNJKSZdJ2qSUv03SbZJ6JR3V1Pfs0n6lpNkdxHKupB+Vsa+QtE1D3d9KWiXpXkmHlrKdJV0v6Z4S8181tN9O0sIy9kJJ25byY0o/V3UygRERERGjSScrhycBl9t+FvgNcILtNwMzgH9rSNDOAc6zvQewDji5lD8AnAhc2tippO2As4D9ganAWbUEbRALgb1s7wPcB/xt6WsSMAuoxfVZSeOAXuCvbf8u8Fbgg6UtwJnAohLvonKO7cuAUzqYl4iIiIhRp5Pk8DjgSgDb99leWY4fAh4BJkoScCDwzXLNxcB7SrvVtu8E+pr6PRRYaPsx2+uoEr8ZgwVie4Ht3nJ6M7BTOZ4JfM32M7Z/CqwCptp+2PZt5dongXuAHRuuubg53oiIiIixbNDksNwa3s326hZ1U4FNgB8DE4D1DYnbGp5LwgayI/Bgw3kn1zQ6Cfhep31J2gX4PeAHpeg1th8GKP9u38mgkk6V1COp59G1a59HuBEREREjX7uVwy5gfXOhpB2ALwHvt90HqMW1btP3UK6pjf/3VLeMv9JJX5K2BL4F/B/bT3QyxkBsX2S723b3xK6uF9JVRERExIjTLjl8CtissUDSVsB3gX+wfXMpXgtsI6n2Or6dgIfa9L0G2LnhvJNrKBtXDgeOs11LAAfsS9J4qsTwK7Yvb2jzi5Lk1pLdR9qNHRERETHaDZoclmcBx0naDOq3ma8ALrH9jYZ2Bq4HaruRZ1OeUxzEfOAQSduWjSiHlDIkXVJuW29A0gzgDOCPbP+moWoeMEvSppJ2BfYAbinPQn4euMf2p5q6m1fi7DTeiIiIiFGvkw0pC4ADyvEfA28DTpR0R/lMLnVnAKdLWkX1DOLnASS9RdIa4GjgPyXdBWD7MeCjwK3l85FSBrAP8HCLWM4HXg0sLGNfWPq6C/g6cDdwDfDBsrv6fwHHAwc2xPvO0tcngIMlrQQOLucRERERY9rG7ZtwPnA6cK3tLwNfbtXI9k+ofpKmufxWnttV3Fw3F5jbWFZuW6+0/WCL9rsPFKTts4Gzm8q+T+vnEbH9S+CggfqLiIiIGIvarhzavh24vvxu4EvO9hO2j345xmql/LD3Z6l+qzEiIiJiTOlk5bC2wjcmlB/Bvmy444iIiIgYDnpuw288X93d3e7p6RnuMCIiIiI60fJRu2adbEiJiIiIiDEiyWFERERE1CU5jIiIiIi6jjakxAD6+mDx4uGOYmSZMgWWLRvuKEaOzEd/mZMNZT76y5z0lzmpTJ8+3BGMCVk5jIiIiIi6JIcRERERUZfkMCIiIiLqkhxGRERERF2Sw4iIiIio6yg5lLS5pBskjZM0WdJSSXdJurO8i7jWbldJP5C0UtJlkjYp5ZuW81WlfpeGa/62lN8r6dAOYvlKabtC0lxJ40u5JH269HWnpP0arrlG0npJVzX1NVC8p0l6QNL5ncxPRERExGjR6crhScDltp8FfgOcYPvNwAzg3yRtU9qdA5xnew9gHXByKT8ZWGd7d+C80g5Jk4BZQK2vz0oa1yaWrwB7AnsDmwOnlPLDgD3K51TggoZrzgWOb9FXy3htnwf8U5s4IiIiIkadTpPD44ArAWzfZ3tlOX4IeASYKEnAgcA3yzUXA+8pxzPLOaX+oNJ+JvA128/Y/imwCpg6WCC2r3YB3ALs1DDGJaXqZmAbSTuUaxYBTzb20ybeiIiIiDGpbXJYbrXuZnt1i7qpwCbAj4EJwHrbvaV6DbBjOd4ReBCg1D9e2tfLW1zTLq7xVKuB1zSP0WFfg8U72LinSuqR1PPo2rWdhBoRERHxitHJymEXsL65sKzKfQl4v+0+QC2uda35AHWDXdPOZ4EbbS9pM8ZAhjS27Ytsd9vuntjV1UGYEREREa8cnSSHTwGbNRZI2gr4LvAP5RYuwFqqW7m1V/LtBDxUjtcAO5drNwa2Bh5rLG9xzYAknQVMBE5vKH6+fQ0Wb0RERMSY1DY5tL0OGCdpM6jfZr6C6vm+bzS0M3A9cFQpmk15ThGYV84p9deV9vOAWWU3865Um0luKeMsktTvNq+kU4BDgWPLimXNPOCEsmv5rcDjth8e5HsNFm9ERETEmNTphpQFwAHl+I+BtwEnSrqjfCaXujOA0yWtonqm7/Ol/PPAhFJ+OnAmgO27gK8Dd1M9O/hB289K2gjYnWp1sdmFwGuApWXs2q7iq4GfUG1q+S/gL2oXSFoCfINqI8yahp/MGSjeiIiIiDFp4/ZNADifKqm71vaXgS+3amT7J7TYbWz7aeDoAa45Gzi7qXgS8C3bT7Vo3zLmshL4wQHqpj2feCMiIiLGqo5WDm3fDlzfwW8Qvihsr7B9evuWLw1JpwF/CzwxXDFEREREDIdOVw6xPfelDGQkKT+Cfd5wxxERERHxclN1NzaGoru72z09PcMdRkREREQnWv2MXz+dbkiJiIiIiDEgyWFERERE1CU5jIiIiIi6jjekRAt9fbB48XBHMbJMmQLLlg13FCNH5qO/zMmGMh/9ZU76G0tzMn36cEcw5mXlMCIiIiLqkhxGRERERF2Sw4iIiIioS3IYEREREXVJDiMiIiKirm1yKGlzSTfU3qss6RpJ6yVd1dTuQEm3SVoh6WJJG5fybSVdIelOSbdI2qvhmhmS7pW0StKZHcRyuqS7S1+LJL2+oW62pJXlM7uhfIqkH5YxPi1JpXw7SQtL+4WSti3lx5S2V/WPICIiImJ062Tl8CTgctvPlvNzgeMbG0jaCLgYmGV7L+B+oJag/R1wh+19gBOAfy/XjAM+AxwGTAKOlTSpTSy3A92lr28C/1L62g44C9gfmAqcVUv2gAuAU4E9ymdGKT8TWGR7D2BROcf2ZcApHcxLRERExKjTSXJ4HHBl7cT2IuDJpjYTgGds31fOFwJHluNJVMkXtn8E7CLpNVRJ3CrbP7H9W+BrwMzBArF9ve3flNObgZ3K8aHAQtuP2V5Xxp8haQdgK9tLXb1E+hLgPeWamVQJLeXfWnlERETEmDVocihpE2A326vb9LMWGC+pu5wfBexcjpcD7y39TQVeT5XU7Qg82NDHmlLWqZOB75XjgfrasRy3GuM1th8GKP9u38mgkk6V1COp59G1a59HuBEREREjX7uVwy5gfbtOyqrcLOA8SbdQrSz2lupPANtKugP431S3hnsBteqqk6AlvQ/oprrFzSB9DXmMgdi+yHa37e6JXV0vpKuIiIiIEafd6/OeAjbrpCPbS4FpAJIOAd5Yyp8A3l/KBfy0fLbgudVFqFYTH2o3jqQ/BP4eeLvtZ0rxGmB6U1+LS/lOTeW1MX4haQfbD5fbz4908j0jIiIiRrNBVw7L83vjJLVNECVtX/7dFDgDuLCcb1NuT0O10ePGkjDeCuwhaddSPwuYV675uKQjWozxe8B/An9kuzGZmw8cUnZGbwscAswvt4uflPTWkpiewHPPT87juU0zsxvKIyIiIsasTjakLAAOqJ1IWgJ8AzhI0hpJh5aqOZLuAe4EvmP7ulL+u8Bdkn5EtTP5rwBs9wIfokrs7gG+bvuucs3ewM9bxHIusCXwDUl3SJpX+noM+ChVwnkr8JFSBvAB4HPAKuDHPPec4ieAgyWtBA4u5xERERFjWrvbygDnA6cD1wLYntaqke05wJwW5UupfkKm1TVXA1e3qBpfrmtu/4cDBWl7LjC3RXkPsFeL8l8CBw3UX0RERMRY1Hbl0PbtwPW1H8F+Odg+tH2rl4akY4DPAuuGK4aIiIiI4dLJymFtVW5MKD+CfdlwxxERERExHFT9Ck0MRXd3t3t6eoY7jIiIiIhOtPqJv3462ZASEREREWNEksOIiIiIqEtyGBERERF1HW1IiQH09cHixcMdxcgyZQosWzbcUYwcmY/+Micbynz0lznpL3PynOnThzuCUS8rhxERERFRl+QwIiIiIuqSHEZEREREXZLDiIiIiKhLchgRERERdW2TQ0mbS7qh9m5lSddIWi/pqqZ2B0q6TdIKSRdL2riUby3pO5KWS7pL0vsbrpktaWX5zO4glqNLH32SuhvKN5H0BUk/LONMb6g7RtKd5bp/aSjfVNJlklZJ+oGkXUr5NEl3S1rRLp6IiIiI0aaTlcOTgMttP1vOzwWOb2wgaSPgYmCW7b2A+4FasvdB4G7b+wLTgU+WZG474Cxgf2AqcJakbdvEsgJ4L3BjU/mfAtjeGzi4jLGRpAkl3oNsvxl4jaSDyjUnA+ts7w6cB5xT+lgCvLP9tERERESMPp0kh8cBV9ZObC8CnmxqMwF4xvZ95XwhcGTtEuDVkgRsCTwG9AKHAgttP2Z7XblmxmCB2L7H9r0tqiYBi0qbR4D1QDewG3Cf7UdLu2sb4ppJldACfBM4qMQYERERMWYNmhxK2gTYzfbqNv2sBcY33Oo9Cti5HJ8P/C7wEPBD4K9s9wE7Ag829LGmlA3FcmCmpI0l7QpMKeOvAvaUtEu5zf2ehrjq49vuBR6nSnIHJelUST2Seh5du3aI4UZERESMTO1WDruoVuEGZdvALOA8SbdQrSz2lupDgTuA1wGTgfMlbQW0WqVzh3E3m0uVXPYA/wbcBPSWFckPAJcBS4DVDXENaXzbF9nutt09satriOFGREREjEztXp/3FLBZJx3ZXgpMA5B0CPDGUvV+4BMlgVwl6afAnlTJ3PSGLnYCFncaeNPYvcBptXNJNwErS913gO+U8lOB2rOTa6hWEdeUVcWtqW55R0RERIxZg64clpW3cZLaJoiSti//bgqcAVxYqh4ADip1rwHeBPwEmA8cImnbshHlkFKGpEskTe30S0jaQtKryvHBVKuGdzfFtS3wF8DnymXzeG7TzFHAdSWBjYiIiBiz2q0cAiwADqDazIGkJVQrf1tKWgOcbHs+MEfS4VQJ5wW2ryvXfxT4oqQfUt3KPcP22tLXR4FbS7uP2K6t3O0DPNwciKQjgP8AJgLflXSH7UOB7YH5kvqAn7Hhbup/l7Rvwxi1TTOfB74kaRXViuGsDuYiIiIiYlTrJDk8Hzidkhzantaqke05wJwW5Q9RrQq2umYu1fOCdeV5xJW2H2zR/grgihblq6lWJFuNcewA5U8DR7eqi4iIiBir2v6Uje3bgetrP4L9UrP9hO1hS9okTaN6RjFbkSMiImLM6WTlsLbCNyaUH8Hee7jjiIiIiBgOyh6Moevu7nZPT89whxERERHRiY5e9tHJG1IiIiIiYoxIchgRERERdUkOIyIiIqKuow0pMYC+Pli8eLijGFmmTIFly4Y7ipEj89Ff5mRDmY/+Mif9ZU42NBLnY/r04Y7gRZOVw4iIiIioS3IYEREREXVJDiMiIiKiLslhRERERNQlOYyIiIiIurbJoaTNJd1Qe7eypGskrZd0VVO7AyXdJmmFpIslbVzK50i6o3xWSHpW0nalboakeyWtknRmB7G8rYzRK+moprpzSv8rJB3TUH5QueYOSd+XtHsp31TSZWXsH0japZRPk3S3pBXt4omIiIgYbTpZOTwJuNz2s+X8XOD4xgaSNgIuBmbZ3gu4H5gNYPtc25NtTwb+FrjB9mMl2fwMcBgwCThW0qQ2sTwAnAhc2jT+u4D9gMnA/sAcSVuV6guA48r4lwL/UMpPBtbZ3h04DzinxLsEeGcH8xIREREx6nSSHB4HXFk7sb0IeLKpzQTgGdv3lfOFwJEt+joW+Go5ngqssv0T278FvgbMHCwQ26tt3wn0NVVNoko6e23/GlgOzKhdBtQSxa2Bh8rxTKqEFuCbwEGS2r5zUNKpknok9Ty6dm275hERERGvKIMmh5I2AXazvbpNP2uB8ZK6y/lRwM5NfW1BlbB9qxTtCDzY0GRNKRuK5cBhkraQ1AW8o2H8U4CrJa2hWvH8RPP4tnuBx6mS3EHZvsh2t+3uiV1dQww3IiIiYmRqt3LYBaxv14ltA7OA8yTdQrWy2NvU7N3Af9t+rJy3WqVzu7EGGH8BcDVwE9XK5NKG8U8D3ml7J+ALwKde7PEjIiIiRot2yeFTwGaddGR7qe1ptqcCNwIrm5rM4rlbylCtFDauLu7Ec7d8nzfbZ5dnGw+mSvxWSpoI7Gv7B6XZZcAfNI9fNs9sDTxGRERExBg2aHJoex0wTlLbBFHS9uXfTYEzgAsb6rYG3k7Ds4vArcAeknYtt69nAfNK+49LOqLTLyFpnKQJ5XgfYB9gAbAO2FrSG0vTg4F7yvE8yqYZqtvg15UV0IiIiIgxa+MO2iwADgCuBZC0BNgT2LI8x3ey7flUO4QPp0o4L7B9XUMfRwALymYRoHrOT9KHgPnAOGCu7btK9d6URLGRpLcAVwDbAu+W9GHbbwbGA0vKfpIngPeV5wiR9KfAtyT1USWLJ5XuPg98SdIqqhXDWR3MRURERMSo1klyeD5wOiU5tD2tVSPbc4A5A9R9Efhii/KrqZ4VbDbe9tIW7W+luv3cXP401Y7lVmNfQZVQtrrm6FbXRERERIxVbX/KxvbtwPW1H8F+Odg+9OUaq5mkacB3qHZgR0RERIwpnawcYnvuSx3ISFF+BHvv4Y4jIiIiYjgoezCGrru72z09PcMdRkREREQn2r7sAzp7Q0pEREREjBFJDiMiIiKiLslhRERERNR1tCElBtDXB4sXD3cUI8uUKbBs2XBHMXJkPvrLnGwo89Ff5qS/zMmGXknzMX36cEfwvGXlMCIiIiLqkhxGRERERF2Sw4iIiIioS3IYEREREXVJDiMiIiKirm1yKGlzSTfU3q0s6RpJ6yVd1dTuQEm3SVoh6WJJGzfUTZd0h6S7JN3QUD5D0r2SVkk6s4NY3lbG6JV0VFPdOWXsFZKOaShfUsa+Q9JDkr5dyiXp02XsOyXtV8rfUNr+ql08EREREaNNJyuHJwGX2362nJ8LHN/YQNJGwMXALNt7AfcDs0vdNsBngT+y/Wbg6FI+DvgMcBgwCThW0qQ2sTwAnAhc2jT+u4D9gMnA/sAcSVsB2J5me7LtycBS4PJy2WHAHuVzKnBBaf/j0jYiIiJizOkkOTwOuLJ2YnsR8GRTmwnAM7bvK+cLgSPL8Z9QJZcPlOsfKeVTgVW2f2L7t8DXgJmDBWJ7te07gb6mqknADbZ7bf8aWA7MaGwg6dXAgcC3S9FM4BJXbga2kbTDYOOXfk6V1COp59G1a9s1j4iIiHhFGTQ5lLQJsJvt1W36WQuMl9Rdzo8Cdi7HbwS2lbRY0jJJJ5TyHYEHG/pYU8qGYjlwmKQtJHUB72gYv+YIYJHtJ17I+LYvst1tu3tiV9cQw42IiIgYmdq9IaULWN+uE9uWNAs4T9KmwAKgt2GMKcBBwObAUkk3A2rVVaeBN42/QNJbgJuAR6luH/c2NTsW+FzD+Ys2fkRERMRo0S45fArYrJOObC8FpgFIOoRqxRCqFbm15XbvryXdCOxbyhtX93YCHuo89H7jnw2cXca/FFhZq5M0geo29hENl7yo40dERESMBoPeVra9DhgnqW2CKGn78u+mwBnAhaXqSmCapI0lbUG1YeQe4FZgD0m7ltvXs4B5pY+PSzqieYxBxh5XEkAk7QPsQ7V6WXM0cJXtpxvK5gEnlF3LbwUet/1wp2NGREREjEbtVg6hSrIOAK6F6qdhgD2BLSWtAU62PZ9qh/DhVAnnBbavA7B9j6RrgNpGks/ZXlH6+hAwHxgHzLV9Vxlzb0qi2KjcOr4C2BZ4t6QPlx3Q44ElkgCeAN5nu/G28izgE03dXQ28E1gF/AZ4fwdzERERETGqdZIcng+cTkkObU9r1cj2HGDOAHXnUv0ETnP51VRJWrPx5TZ1c/tbqW7/Npc/TbVjuSXb01uUGfjgQNdEREREjEVtf8rG9u1NxISGAAAdSUlEQVTA9bUfwX452D705RqrWe1HsIFfDFcMEREREcOlk5VDbM99qQMZKWz/mOrHtCMiIiLGHFV3V2Mouru73dPTM9xhRERERHSi1c/49dPJG1IiIiIiYoxIchgRERERdUkOIyIiIqKuow0pMYC+Pli8eLijGFmmTIFly4Y7ipEj89Ff5mRDmY/+Mif9ZU429Eqdj+nThzuCjmTlMCIiIiLqkhxGRERERF2Sw4iIiIioS3IYEREREXVtk0NJm0u6ofb6PEnXSFov6aqmdgdKuk3SCkkXS9q4lE+X9LikO8rnnxqumSHpXkmrJJ3ZQSxvK2P0Sjqqqe6cMvYKScc0lEvS2ZLuk3SPpL9sKP90GftOSfuV8jeUOH/VLp6IiIiI0aaT3conAZfbfracnwtsAfxZrYGkjYCLgYNs3yfpI8Bs4POlyRLbhzd2WpLNzwAHA2uAWyXNs333ILE8AJwI/E1TX+8C9qN67d2mwA2Svmf7idJ+Z2BP232Sti+XHQbsUT77AxcA+9den5fkMCIiIsaiTm4rHwdcWTuxvQh4sqnNBOAZ2/eV84XAkW36nQqssv0T278FvgbMHOwC26tt3wn0NVVNAm6w3Wv718ByYEap+wDwEdt9pY9HSvlM4BJXbga2kbRDm5gjIiIiRrVBk0NJmwC72V7dpp+1wHhJ3eX8KKrVuprfl7Rc0vckvbmU7Qg82NBmTSkbiuXAYZK2kNQFvKNh/DcAx0jqKePv8ULGl3Rq6avn0bVrhxhuRERExMjUbuWwC1jfrhPbBmYB50m6hWplsbdU3wa83va+wH8A3y7lrV7+7E6CbjH+AuBq4Cbgq8DShvE3BZ623Q38FzD3hYxv+yLb3ba7J3Z1DSXciIiIiBGrXXL4FLBZJx3ZXmp7mu2pwI3AylL+hO1fleOrqVYYu6hW6hpXF3cCHnqe8TeOf7btybYPpkr8VpaqNcC3yvEVwD4N5S/a+BERERGjwaDJoe11wDhJbRPE2kYPSZsCZwAXlvPXSlI5nlrG/CVwK7CHpF3L7etZwLzS7uOSjuj0S0gaJ2lCOd6HKgFcUKq/DRxYjt8O1J6LnAecUHYtvxV43PbDnY4ZERERMRp1slt5AXAAcC2ApCXAnsCWktYAJ9ueD8yRdDhV8neB7evK9UcBH5DUS7USOavchu6V9CFgPjAOmGv7rnLN3pREsZGkt1Ct/m0LvFvSh22/GRgPLCk56BPA+2zXbit/AviKpNOAXwGnlPKrgXcCq4DfAO/vYC4iIiIiRrVOksPzgdMpyaHtaa0a2Z4DzGlRfn7po9U1V1Mlac3G217aov2tVLd/m8ufptqx3GqM9cC7WpQb+GCrayIiIiLGqrY/ZWP7duD62o9gvxxsH/pyjdWs9iPYwC+GK4aIiIiI4dLJyiG257ZvNTrUfgR7uOOIiIiIGA55t3JERERE1Kl69C6Goru72z09PcMdRkREREQnWv3Gcz9ZOYyIiIiIuiSHEREREVGX5DAiIiIi6jrarRwD6OuDxYuHO4qRZcoUWLZsuKMYOTIf/WVONpT56C9z0l/mZEOjaT6mTx/uCPrJymFERERE1CU5jIiIiIi6JIcRERERUZfkMCIiIiLq2iaHkjaXdEPt3cqSrpG0XtJVTe0OlHSbpBWSLpa0cVP9WyQ9K+mohrLZklaWz+wOYjla0l2S+iR1N5RvIukLkn4oabmk6Q11iyXdK+mO8tm+lG8q6TJJqyT9QNIupXyapLslrWgXT0RERMRo08nK4UnA5bafLefnAsc3NpC0EXAxMMv2XsD9wOyG+nHAOcD8hrLtgLOA/YGpwFmStm0TywrgvcCNTeV/CmB7b+Bg4JMlpprjbE8un0dK2cnAOtu7A+eV+LC9BHhnmzgiIiIiRqVOksPjgCtrJ7YXAU82tZkAPGP7vnK+EDiyof5/A98CHmkoOxRYaPsx2+vKNTMGC8T2PbbvbVE1CVhU2jwCrAe6W7RrNJMqoQX4JnCQpI5eKxMRERExWg2aHEraBNjN9uo2/awFxjfc6j0K2Ln0sSNwBHBh0zU7Ag82nK8pZUOxHJgpaWNJuwJTauMXXyi3lP+xIQGsj2+7F3icKskdlKRTJfVI6nl07dohhhsRERExMrVbOeyiWoUblG0Ds4DzJN1CtbLYW6r/DTij4bZ0TatVOrcbawBzqZLLnjLeTQ3jH1duN08rn9ot8SGNb/si2922uyd2dQ0x3IiIiIiRqd0bUp4CNuukI9tLqZIvJB0CvLFUdQNfKwt2XcA7JfVSJXPTG7rYCVjcYdzNY/cCp9XOJd0ErCx1Pyv/PinpUqrnGy8p4+8MrCmbZ7YGHhvK+BERERGjxaArh+VZwHGS2iaIjbuAgTMot5Ft72p7F9u7UD3b9xe2v021OeUQSduWjSiHlDIkXSJpaqdfQtIWkl5Vjg8Gem3fXW4zd5Xy8cDhVJtaAObx3KaZo4DrygpoRERExJjVybuVFwAHANcCSFoC7AlsKWkNcLLt+cAcSYdTJZwX2L5usE5tPybpo8Ctpegjtmsrd/sADzdfI+kI4D+AicB3Jd1h+1Bge2C+pD7gZzx363jTUj4eGFe+w3+Vus8DX5K0imrFcFYHcxERERExqnWSHJ4PnE5JDm1Pa9XI9hxgzmAd2T6x6Xwu1fOCdZK2AlbabtysUmt/BXBFi/LVwJtalP+aanNKq1ieBo4eLN6IiIiIsabtT9nYvh24vvYj2C8120/YHrakTdI04DtUO7AjIiIixpROVg5rK3xjQvkR7L2HO46IiIiI4ZB3K0dEREREnbJBd+i6u7vd09Mz3GFEREREdKKjN8Fl5TAiIiIi6pIcRkRERERdksOIiIiIqOtot3IMoK8PFi8e7ihGlilTYNmy4Y5i5Mh89Jc52VDmo7/MSX+Zkw2NlPmYPn24I3hJZOUwIiIiIuqSHEZEREREXZLDiIiIiKhLchgRERERdW2TQ0mbS7qh9m5lSddIWi/pqqZ2B0q6TdIKSRdL2riUz5R0p6Q7JPVIOqDhmtmSVpbP7A5iOVrSXZL6JHU3lG8i6QuSfihpuaTpLa6dJ2lFw/l2khaWsRdK2raUHyNpVfP3i4iIiBgLOlk5PAm43Paz5fxc4PjGBpI2Ai4GZtneC7gfqCV7i4B9bU8ufX2uXLMdcBawPzAVOKuWoA1iBfBe4Mam8j8FsL03cDDwyRJTLb73Ar9quuZMYJHtPUqMZ5Y+LgNOaRNHRERExKjUSXJ4HHBl7cT2IuDJpjYTgGds31fOFwJHlva/8nPv6HsVUDs+FFho+zHb68o1MwYLxPY9tu9tUTWJKsHD9iPAeqAbQNKWwOnAx5qumUmV0FL+fc9gY0dERESMBYMmh5I2AXazvbpNP2uB8Q23eo8Cdm7o5whJPwK+S7V6CLAj8GBDH2tK2VAsB2ZK2ljSrsCUhvE/CnwS+E3TNa+x/TBA+Xf7TgaSdGq5Pd7z6Nq1Qww3IiIiYmRqt3LYRbUKN6iyMjgLOE/SLVQri70N9VfY3pNqde6jpbjVy5/doqwTc6mSyx7g34CbgF5Jk4HdbV8xxH77sX2R7W7b3RO7ul6sbiMiIiJGhHZvSHkK2KyTjmwvBaYBSDoEeGOLNjdKeoOkLqpkbnpD9U7A4k7GatFvL3Ba7VzSTcBK4O3AFEmrqb7r9pIW254O/ELSDrYflrQD8MhQxo6IiIgYTQZdOSzPAo6T1DZBlLR9+XdT4AzgwnK+uySV4/2ATYBfAvOBQyRtWzaiHFLKkHSJpKmdfglJW0h6VTk+GOi1fbftC2y/zvYuwAHAfSUxBJjHc5tmZtPwXGVERETEWNXJu5UXUCVW1wJIWgLsCWwpaQ1wsu35wBxJh1MlnBfYvq5cfyRwgqT/oVqJPKbchn5M0keBW0u7j9h+rBzvAzzcHIikI4D/ACYC35V0h+1DqZ4XnC+pD/gZTbupB/AJ4OuSTgYeAI7u4JqIiIiIUa2T5PB8qt2+1wLYntaqke05wJwW5ecA5wxwzVyq5wXrJG0FrLT9YIv2VwD9nh8sG2beNNiXKG32ajj/JXDQYNdEREREjDVtf8rG9u3A9bUfwX6p2X7C9rCt4kk6BvgssG64YoiIiIgYLp2sHNZW+MaE8iPYlw13HBERERHDIe9WjoiIiIg6Pffykni+uru73dPTM9xhRERERHSi1W9M95OVw4iIiIioS3IYEREREXVJDiMiIiKirqPdyjGAvj5YvHi4oxhZpkyBZcuGO4qRI/PRX+ZkQ5mP/jIn/WVONvRKmI/p04c7giHLymFERERE1CU5jIiIiIi6JIcRERERUZfkMCIiIiLq2iaHkjaXdEPt3cqSrpG0XtJVTe0OlHSbpBWSLpa0cSk/TtKd5XOTpH0brpkh6V5JqySd2UEsbytj9Eo6qqnunDL2ivJ+5Fr55yUtL+N/U9KWpXxTSZeVsX8gaZdSPk3S3ZJWtIsnIiIiYrTpZOXwJOBy28+W83OB4xsbSNoIuBiYZXsv4H5gdqn+KfB22/sAHwUuKteMAz4DHAZMAo6VNKlNLA8AJwKXNo3/LmA/YDKwPzBH0lal+jTb+5bxHwA+VMpPBtbZ3h04DzgHwPYS4J1t4oiIiIgYlTpJDo8Drqyd2F4EPNnUZgLwjO37yvlC4MjS/ibb60r5zcBO5XgqsMr2T2z/FvgaMHOwQGyvtn0n0NdUNQm4wXav7V8Dy4EZ5ZonACQJ2ByovS9wJlVCC/BN4KDSJiIiImLMGjQ5lLQJsJvt1W36WQuMl9Rdzo8Cdm7R7mTge+V4R+DBhro1pWwolgOHSdpCUhfwjsbxJX0B+DmwJ/AfzePb7gUep0pyByXpVEk9knoeXbt2iOFGREREjEztVg67gPXtOrFtYBZwnqRbqFYWexvbSHoHVXJ4Rq2oVVftxhpg/AXA1cBNwFeBpY3j234/8DrgHqD2POKQxrd9ke1u290Tu7qGEm5ERETEiNUuOXwK2KyTjmwvtT3N9lTgRmBlrU7SPsDngJm2f1mK17Dh6uJOwEOdBt5i/LNtT7Z9MFXit7Kp/lngMsrt7sbxy+aZrYHHhjp+RERExGgwaHJYnhUcJ6ltgihp+/LvplSrgxeW898BLgeOb3gmEeBWYA9Ju5bb17OAeeWaj0s6otMvIWmcpAnleB9gH2CBKruXcgHvBn5ULpvHc5tmjgKuKyugEREREWNWJ+9WXgAcAFwLIGkJ1bN7W0paA5xsez7VDuHDqRLOC2xfV67/J6pn+T5b9nv0ltuyvZI+BMwHxgFzbd9Vrtmbkig2kvQW4ApgW+Ddkj5s+83AeGBJ6f8J4H2l/42Ai8vOZVE9m/iB0t3ngS9JWkW1Yjirg7mIiIiIGNU6SQ7PB06nJIe2p7VqZHsOMKdF+SnAKQNcczXVs4LNxtte2qL9rTy327mx/GmqHcvN5X3A/xpg7KeBo1vVRURERIxVbX/KxvbtwPW1H8F+Odg+9OUaq5mkacB3qHZgR0RERIwpnawcYnvuSx3ISFF+BHvv4Y4jIiIiYjjk3coRERERUads0B267u5u9/T0DHcYEREREZ3o6E1wWTmMiIiIiLokhxERERFRl+QwIiIiIuo62q0cA+jrg8WLhzuKkWXKFFi2bLijGDkyH/1lTjaU+egvc9Jf5mRDnczH9OkvSyijUVYOIyIiIqIuyWFERERE1CU5jIiIiIi6JIcRERERUdc2OZS0uaQbau9WlnSNpPWSrmpqd6Ck2yStkHSxpI1L+Z6Slkp6RtLfNF0zQ9K9klZJOrODWN5WxuiVdFRT3Tll7BWSjmko/0oZY4WkuZLGl3JJ+nQZ+05J+5XyN0i6Q9Kv2sUTERERMdp0snJ4EnC57WfL+bnA8Y0NJG0EXAzMsr0XcD8wu1Q/Bvwl8K9N14wDPgMcBkwCjpU0qU0sDwAnApc29fUuYD9gMrA/MEfSVqX6K8CeVO9L3hw4pZQfBuxRPqcCFwDY/rHtyW3iiIiIiBiVOkkOjwOurJ3YXgQ82dRmAvCM7fvK+ULgyNL+Edu3Av/TdM1UYJXtn9j+LfA1YOZggdhebftOoK+pahJwg+1e278GlgMzyjVXuwBuAXYq18wELilVNwPbSNph0JmIiIiIGOUGTQ4lbQLsZnt1m37WAuMldZfzo4Cd21yzI/Bgw/maUjYUy4HDJG0hqQt4R/P45Xby8cA1L2R8SadK6pHU8+jatUMMNyIiImJkardy2AWsb9dJWZWbBZwn6RaqlcXeNpe1evmz2401wPgLgKuBm4CvAktbjP9Z4EbbS17I+LYvst1tu3tiV9dQwo2IiIgYsdolh08Bm3XSke2ltqfZngrcCKxsc8kaNlzd2wl4qJOxBhj/bNuTbR9MlfjVx5d0FjAROP2lGj8iIiJiNBg0ObS9DhgnqW2CKGn78u+mwBnAhW0uuRXYQ9Ku5fb1LGBe6ePjko7oIP7a2OMkTSjH+wD7AAvK+SnAocCxthufVZwHnFB2Lb8VeNz2w52OGRERETEadfJu5QXAAcC1AJKWUO3+3VLSGuBk2/OpdggfTpVwXmD7utL+tUAPsBXQJ+n/AJNsPyHpQ8B8YBww1/ZdZcy9KYliI0lvAa4AtgXeLenDtt8MjAeWSAJ4Anif7dpt5Qupdk8vLfWX2/4I1W3odwKrgN8A7+9kwiIiIiJGs06Sw/OpbsdeC2B7WqtGtucAc1qU/5zndgg3111NlaQ1G297aYv2t7bqy/bTVDuWW43R8juW5yQ/2KouIiIiYqxq+1M2tm8Hrq/9CPbLwfahL9dYzWo/gg38YrhiiIiIiBgunawcYnvuSx3ISGH7x1Q/ph0REREx5uTdyhERERFRp+rRuxiK7u5u9/T0DHcYEREREZ1o9RvP/WTlMCIiIiLqkhxGRERERF2Sw4iIiIio62i3cgygrw8WLx7uKEaWKVNg2bLhjmLkyHz0lznZUOajv8xJf6+UOZk+fbgjiBdBVg4jIiIioi7JYURERETUJTmMiIiIiLokhxERERFR1zY5lLS5pBtq71aWdI2k9ZKuamp3kKTbJN0h6fuSdi/lr5e0SNKdkhZL2qnhmnMkrSifYzqI5W1ljF5JRzXVtexL0ockrZJkSV0N5ZL06VJ3p6T9Svkbynf4Vbt4IiIiIkabTlYOTwIut/1sOT8XOL5FuwuA42xPBi4F/qGU/ytwie19gI8AHweQ9C5gP6r3GO8PzJG0VZtYHgBOLP3Xtenrv4E/BO5v6uswYI/yObXEj+0fl+8QERERMeZ0khweB1xZO7G9CHiyRTsDtYRsa+ChcjwJWFSOrwdmNpTfYLvX9q+B5cCMwQKxvdr2nUBfU9WAfdm+3fbqFt3NpEpabftmYBtJOww2fkRERMRoN2hyKGkTYLcBkqtmpwBXS1pDtbL4iVK+HDiyHB8BvFrShFJ+mKQtyu3edwA7P/+vUB/j+fa1I/Bgw/maUjYoSadK6pHU8+jatUMMNyIiImJkardy2AWs77Cv04B32t4J+ALwqVL+N8DbJd0OvB34GdBrewFwNXAT8FVgKdD7/MKvDLGvVi+fdgdjXWS723b3xK6uds0jIiIiXlHaJYdPAZu160TSRGBf2z8oRZcBfwBg+yHb77X9e8Dfl7LHy79n255s+2CqZG3l0L7GkPpaw4arizvx3K3wiIiIiDFp0OTQ9jpgnKR2CeI6YGtJ/7+9u4uRq67DOP59LCuaoGKsCYSiNRFjgta3UjFq0qBERGxjJLEXiCB4YSRgJCHihQZviDfGVIzGlwZQo5hWzEpKeAkYaiLQpSCKtclqTGxsrLy1EBRTebw4Z093Z2Z3jjp7zrr/55NsmJnz39nfPvw7/9/seZk31PfPBfYDSForae7nXAvsqB9fU+9eRtIGYANwZ33/ekkfaftLLPVcS5gGLq7PWj4bOGL7UNufGREREbEatfls5TuB9wB3A0jaA7wROKk+vvAy23dI+hSwS9ILVM3iJ+vv3wxcL8nAfcBn6sengD2SAI4CF9me2xX8ZqrmbQFJZwG3Aq8EPizpOttnLvVckq4ErgFOAR6VtNv25VS7oc8HZoHngEtbZBERERGxqrVpDm8APkfdHNp+76hBtm+latwGH98J7Bzx+D+ozjIeZcr2r0Z8z16q3b+tn8v2dmD7iMfN8UY1IiIiImhxKRvbDwP3zl0Euwu2P9DVzxo0dxFs4K991RARERHRlzZ/OcT2juUuZKWw/Qeqi2lHREREFEfV3tX4b2zcuNEzMzN9lxERERHRxqjL+A1p8wkpEREREVGINIcRERER0UhzGBERERGNNIcRERER0UhzGBERERGNNIcRERER0UhzGBERERGNNIcRERER0UhzGBERERGNNIcRERER0UhzGBERERGNNIcRERER0ZDtvmv4vyXpGeBA33WsMGuBx/suYgVJHsOSyULJY1gyGZZMFkoew9pk8rjt88Y90QmTqadYB2xv7LuIlUTSTDI5LnkMSyYLJY9hyWRYMlkoeQybZCbZrRwRERERjTSHEREREdFIc/i/+XbfBaxAyWSh5DEsmSyUPIYlk2HJZKHkMWximeSElIiIiIho5C+HEREREdFIc9iCpPMkHZA0K+nzI7afKOmWevsDktZ3X2V3WuRxiaS/SXqk/rq8jzq7ImmHpMOSfrvIdknaXuf1qKS3d11j11pkslnSkXlz5Itd19glSadLulfSfkmPSbpqxJii5knLTIqZJ5JeIulBSb+u87huxJjS1po2mRS13gBIWiPpYUm3jdg2kTmSS9mMIWkN8A3gXOAgsFfStO3fzRt2GfCU7ddL2gZ8BfhY99Uuv5Z5ANxi+4rOC+zHjcANwM2LbP8gcEb99U7gm/V/V7MbWToTgD22L+imnN4dA662vU/Sy4CHJN018O+mtHnSJhMoZ548D5xj+1lJU8AvJd1u+/55Y4pZa2ptMoGy1huAq4D9wMtHbJvIHMlfDsfbBMza/qPtfwI/BrYOjNkK3FTf3gm8T5I6rLFLbfIoiu37gCeXGLIVuNmV+4GTJZ3aTXX9aJFJUWwfsr2vvv0M1Qv7aQPDiponLTMpRv3//dn67lT9NXhSQElrTdtMiiJpHfAh4LuLDJnIHElzON5pwJ/n3T/I8AtYM8b2MeAI8KpOqutemzwAPlrvGtsp6fRuSlux2mZWmnfVu4tul3Rm38V0pd7N8zbggYFNxc6TJTKBguZJvbvwEeAwcJftRedIAWsN0CoTKGu9+RpwDfDCItsnMkfSHI43quMefOfSZsxq0eZ3/Tmw3vYG4G6Ov4spVUnzo619wGttvwX4OvCznuvphKSTgF3AZ20fHdw84ltW/TwZk0lR88T2v2y/FVgHbJL0poEhxc2RFpkUs95IugA4bPuhpYaNeOw/niNpDsc7CMx/J7IO+MtiYySdALyC1btLbWwetp+w/Xx99zvAOzqqbaVqM4eKYvvo3O4i27uBKUlrey5rWdXHTO0Cfmj7pyOGFDdPxmVS4jwBsP008Atg8DNwS1prFlgsk8LWm3cDWyT9ieqQrnMk/WBgzETmSJrD8fYCZ0h6naQXA9uA6YEx08An6tsXAvd49V5AcmweA8dJbaE6lqhk08DF9dmoZwNHbB/qu6g+STpl7jgYSZuoXoue6Leq5VP/rt8D9tv+6iLDiponbTIpaZ5IerWkk+vbLwXeD/x+YFhJa02rTEpab2xfa3ud7fVUa+89ti8aGDaROZKzlcewfUzSFcAdwBpgh+3HJH0ZmLE9TfUC931Js1Qd+rb+Kl5eLfO4UtIWqrMRnwQu6a3gDkj6EbAZWCvpIPAlqgOnsf0tYDdwPjALPAdc2k+l3WmRyYXApyUdA/4ObFvNixzVO/6PA7+pj58C+ALwGih2nrTJpKR5cipwU31FiBcBP7F9W6lrTa1NJkWtN6MsxxzJJ6RERERERCO7lSMiIiKikeYwIiIiIhppDiMiIiKikeYwIiIiIhppDiMiIiKikeYwIiIiIhppDiMiIiKikeYwIiIiIhr/Bn389Sa4qF9VAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "\n",
    "ax.barh(means.index.astype(str), means, color = 'red', alpha = 0.25 )\n",
    "\n",
    "ax.spines['top'].set_visible(False)\n",
    "ax.spines['right'].set_visible(False)\n",
    "\n",
    "ax.set_title('Average movie rating by decade')\n",
    "\n",
    "ax.grid(axis = 'x', color='white')\n",
    "\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Attachments",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}