Bus Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bus Reliability\n",
    "==========\n",
    "I take the bus home from work, and the schedule can't always be trusted. I figured I would start collecting some data on my bus at about the time I take it home from work. I've targetted it pretty narrowly because the live transit feed data discourages mass data gathering.\n",
    "\n",
    "I made a script to gather data from [OC Transpo's live transit feed](http://www.octranspo.com/developers) and store it in an SQLite database (ocbusdata.db). I've been running this as a scheduled task, it starts up at about 4pm every day and runs for an hour, checking the estimated arrival time more frequently as the bus gets closer.\n",
    "\n",
    "I then need to link this with the schedule data, as the live feed data doesn't actually include what time the trip was scheduled to arrive.\n",
    "\n",
    "So far this is based on weekdays, between 4 and 5pm, from October 4th 2016 to October 12th 2016."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from Tables import Trip, Estimate, Base\n",
    "\n",
    "from sqlalchemy.orm import sessionmaker\n",
    "from sqlalchemy.ext.declarative import declarative_base\n",
    "from sqlalchemy import create_engine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step one is to connect to the database of live transit data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "engine = create_engine('sqlite:///ocbusdata.db')\n",
    "Base.metadata.bind = engine\n",
    "DBSession = sessionmaker(bind=engine)\n",
    "session = DBSession()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['id', 'route', 'route_label', 'start_time', 'estimate_id',\n",
       "       'trip_id', 'stop_id', 'estimate_processing_time',\n",
       "       'adjusted_schedule_time', 'adjustment_age', 'latitude', 'longitude'], dtype=object)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = session.query(Trip, Estimate).join(Estimate)\n",
    "bus_live_data = pd.read_sql(query.statement, query.session.bind)\n",
    "\n",
    "bus_live_data.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basically what I have is a series of estimates for different trips. Each time I call the API I get back estimates for the next 3 trips, so I have lots of estimates per trip. \"adjusted_schedule_time\" is the estimated number of minutes until the bus arrives, and \"estimate_processing_time\" is the time the request was processed on the server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>70</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>2</td>\n",
       "      <td>0.41</td>\n",
       "      <td>45.407509</td>\n",
       "      <td>-75.713688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>71</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>18</td>\n",
       "      <td>0.74</td>\n",
       "      <td>45.424884</td>\n",
       "      <td>-75.683978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>72</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>33</td>\n",
       "      <td>3.84</td>\n",
       "      <td>45.416420</td>\n",
       "      <td>-75.669496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>73</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>1</td>\n",
       "      <td>0.50</td>\n",
       "      <td>45.405422</td>\n",
       "      <td>-75.712427</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>74</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>17</td>\n",
       "      <td>0.90</td>\n",
       "      <td>45.424734</td>\n",
       "      <td>-75.684338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>75</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>32</td>\n",
       "      <td>5.05</td>\n",
       "      <td>45.416420</td>\n",
       "      <td>-75.669496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>76</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:00:40</td>\n",
       "      <td>17</td>\n",
       "      <td>0.78</td>\n",
       "      <td>45.423725</td>\n",
       "      <td>-75.686666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>77</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:00:40</td>\n",
       "      <td>31</td>\n",
       "      <td>6.48</td>\n",
       "      <td>45.416420</td>\n",
       "      <td>-75.669496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:16:00</td>\n",
       "      <td>78</td>\n",
       "      <td>10</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:00:40</td>\n",
       "      <td>42</td>\n",
       "      <td>-1.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>79</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:06:01</td>\n",
       "      <td>12</td>\n",
       "      <td>0.59</td>\n",
       "      <td>45.421634</td>\n",
       "      <td>-75.696356</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id route route_label start_time  estimate_id  trip_id  stop_id  \\\n",
       "0   7    85    Bayshore   15:31:00           70        7     6653   \n",
       "1   8    85    Bayshore   15:46:00           71        8     6653   \n",
       "2   9    85    Bayshore   16:01:00           72        9     6653   \n",
       "3   7    85    Bayshore   15:31:00           73        7     6653   \n",
       "4   8    85    Bayshore   15:46:00           74        8     6653   \n",
       "5   9    85    Bayshore   16:01:00           75        9     6653   \n",
       "6   8    85    Bayshore   15:46:00           76        8     6653   \n",
       "7   9    85    Bayshore   16:01:00           77        9     6653   \n",
       "8  10    85    Bayshore   16:16:00           78       10     6653   \n",
       "9   8    85    Bayshore   15:46:00           79        8     6653   \n",
       "\n",
       "  estimate_processing_time  adjusted_schedule_time  adjustment_age   latitude  \\\n",
       "0      2016-10-04 15:58:02                       2            0.41  45.407509   \n",
       "1      2016-10-04 15:58:02                      18            0.74  45.424884   \n",
       "2      2016-10-04 15:58:02                      33            3.84  45.416420   \n",
       "3      2016-10-04 15:59:15                       1            0.50  45.405422   \n",
       "4      2016-10-04 15:59:15                      17            0.90  45.424734   \n",
       "5      2016-10-04 15:59:15                      32            5.05  45.416420   \n",
       "6      2016-10-04 16:00:40                      17            0.78  45.423725   \n",
       "7      2016-10-04 16:00:40                      31            6.48  45.416420   \n",
       "8      2016-10-04 16:00:40                      42           -1.00        NaN   \n",
       "9      2016-10-04 16:06:01                      12            0.59  45.421634   \n",
       "\n",
       "   longitude  \n",
       "0 -75.713688  \n",
       "1 -75.683978  \n",
       "2 -75.669496  \n",
       "3 -75.712427  \n",
       "4 -75.684338  \n",
       "5 -75.669496  \n",
       "6 -75.686666  \n",
       "7 -75.669496  \n",
       "8        NaN  \n",
       "9 -75.696356  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bus_live_data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I created a database of the google transit feed using [pygtfs](http://pygtfs.readthedocs.io/en/latest/index.html), but I decided it was easier to query that database directly than via the library.\n",
    "I also created an aditional view: \n",
    "\n",
    "`CREATE VIEW trip_start_times AS SELECT trip_id, arrival_time as start_time FROM stop_times WHERE stop_sequence= 1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "\n",
    "conn = sqlite3.connect('gtfs2.db') \n",
    "c = conn.cursor();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The GTFS database uses route and stop ids that aren't the ones posted on the bus and the stop, so look up these internal ids from the route number (85) and stop number (6653)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "route_id = c.execute(\"SELECT route_id FROM routes WHERE route_short_name = '85'\").fetchone()[0]\n",
    "stop_id = c.execute(\"SELECT stop_id FROM stops WHERE stop_code = '6653'\").fetchone()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I want all of the start times for regular weekday service, and the time that trip is scheduled to arrive at my stop. Unfortunately start_time is the only way I have to link the GTFS schedule with the live data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time</th>\n",
       "      <th>arrival_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>43028550-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 12:18:00.000000</td>\n",
       "      <td>1970-01-01 12:41:00.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>43028608-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 18:16:00.000000</td>\n",
       "      <td>1970-01-01 18:38:00.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>43029106-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 17:46:00.000000</td>\n",
       "      <td>1970-01-01 18:11:00.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>43029132-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 20:48:00.000000</td>\n",
       "      <td>1970-01-01 21:09:00.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>43029134-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 22:32:00.000000</td>\n",
       "      <td>1970-01-01 22:53:00.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   stop_times.trip_id                  start_time  \\\n",
       "0  43028550-SEPT16-SEPDA16-Weekday-36  1970-01-01 12:18:00.000000   \n",
       "1  43028608-SEPT16-SEPDA16-Weekday-36  1970-01-01 18:16:00.000000   \n",
       "2  43029106-SEPT16-SEPDA16-Weekday-36  1970-01-01 17:46:00.000000   \n",
       "3  43029132-SEPT16-SEPDA16-Weekday-36  1970-01-01 20:48:00.000000   \n",
       "4  43029134-SEPT16-SEPDA16-Weekday-36  1970-01-01 22:32:00.000000   \n",
       "\n",
       "                 arrival_time  \n",
       "0  1970-01-01 12:41:00.000000  \n",
       "1  1970-01-01 18:38:00.000000  \n",
       "2  1970-01-01 18:11:00.000000  \n",
       "3  1970-01-01 21:09:00.000000  \n",
       "4  1970-01-01 22:53:00.000000  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "service_id = \"SEPT16-SEPDA16-Weekday-36\" # Regular weekday service\n",
    "start_times_query = \"SELECT stop_times.trip_id, start_time, arrival_time \" \\\n",
    "\"FROM stop_times JOIN trip_start_times on stop_times.trip_id = trip_start_times.trip_id \"\\\n",
    "\"WHERE stop_id = ? and stop_times.trip_id in (SELECT trip_id from trips where route_id = ? and service_id=?)\"\n",
    "schedule_start_times = pd.read_sql(start_times_query, conn, params=(stop_id,route_id,service_id))\n",
    "schedule_start_times.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "So now we need to link two tables based on a time! And one of our tables as this as a time, and one as a formatted string including a garbage date. This is probably easier as a string comparison"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    15:31:00\n",
       "1    15:46:00\n",
       "2    16:01:00\n",
       "Name: start_time_string, dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bus_live_data[\"start_time_string\"] = bus_live_data[\"start_time\"].astype(str)\n",
    "bus_live_data[\"start_time_string\"].head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This one has the date as a formatted string, with a useless date. So I parse it as a date and then format it as a time-only string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    12:18:00\n",
       "1    18:16:00\n",
       "2    17:46:00\n",
       "Name: start_time_string, dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "schedule_start_times[\"start_time_date\"] = pd.to_datetime(schedule_start_times[\"start_time\"], format='%Y-%m-%d %H:%M:%S.%f')\n",
    "schedule_start_times[\"start_time_string\"] = schedule_start_times[\"start_time_date\"].apply(lambda x: x.strftime(\"%H:%M:%S\"))\n",
    "schedule_start_times[\"start_time_string\"].head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can merge the schedule with the live data on the time string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with_scheduled_arrival_time = pd.merge(bus_live_data, schedule_start_times, how=\"left\", on=\"start_time_string\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time_x</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>start_time_string</th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time_y</th>\n",
       "      <th>arrival_time</th>\n",
       "      <th>start_time_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>70</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>2</td>\n",
       "      <td>0.41</td>\n",
       "      <td>45.407509</td>\n",
       "      <td>-75.713688</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>71</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>18</td>\n",
       "      <td>0.74</td>\n",
       "      <td>45.424884</td>\n",
       "      <td>-75.683978</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>72</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>33</td>\n",
       "      <td>3.84</td>\n",
       "      <td>45.416420</td>\n",
       "      <td>-75.669496</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>43031864-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:01:00.000000</td>\n",
       "      <td>1970-01-01 16:31:00.000000</td>\n",
       "      <td>1970-01-01 16:01:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>73</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>1</td>\n",
       "      <td>0.50</td>\n",
       "      <td>45.405422</td>\n",
       "      <td>-75.712427</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>74</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>17</td>\n",
       "      <td>0.90</td>\n",
       "      <td>45.424734</td>\n",
       "      <td>-75.684338</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id route route_label start_time_x  estimate_id  trip_id  stop_id  \\\n",
       "0   7    85    Bayshore     15:31:00           70        7     6653   \n",
       "1   8    85    Bayshore     15:46:00           71        8     6653   \n",
       "2   9    85    Bayshore     16:01:00           72        9     6653   \n",
       "3   7    85    Bayshore     15:31:00           73        7     6653   \n",
       "4   8    85    Bayshore     15:46:00           74        8     6653   \n",
       "\n",
       "  estimate_processing_time  adjusted_schedule_time  adjustment_age   latitude  \\\n",
       "0      2016-10-04 15:58:02                       2            0.41  45.407509   \n",
       "1      2016-10-04 15:58:02                      18            0.74  45.424884   \n",
       "2      2016-10-04 15:58:02                      33            3.84  45.416420   \n",
       "3      2016-10-04 15:59:15                       1            0.50  45.405422   \n",
       "4      2016-10-04 15:59:15                      17            0.90  45.424734   \n",
       "\n",
       "   longitude start_time_string                  stop_times.trip_id  \\\n",
       "0 -75.713688          15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "1 -75.683978          15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "2 -75.669496          16:01:00  43031864-SEPT16-SEPDA16-Weekday-36   \n",
       "3 -75.712427          15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "4 -75.684338          15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "\n",
       "                 start_time_y                arrival_time     start_time_date  \n",
       "0  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00  \n",
       "1  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00  \n",
       "2  1970-01-01 16:01:00.000000  1970-01-01 16:31:00.000000 1970-01-01 16:01:00  \n",
       "3  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00  \n",
       "4  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_scheduled_arrival_time.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lgaud/anaconda/lib/python3.5/site-packages/pandas/core/ops.py:533: PerformanceWarning: Adding/subtracting array of DateOffsets to Series not vectorized\n",
      "  \"Series not vectorized\", PerformanceWarning)\n"
     ]
    }
   ],
   "source": [
    "with_scheduled_arrival_time[\"estimate_processing_time\"] = \\\n",
    "pd.to_datetime(with_scheduled_arrival_time[\"estimate_processing_time\"], format='%Y-%m-%d %H:%M:%S')\n",
    "with_scheduled_arrival_time[\"estimated_arrival_time\"] = with_scheduled_arrival_time[\"estimate_processing_time\"] +\\\n",
    "with_scheduled_arrival_time[\"adjusted_schedule_time\"].apply(pd.offsets.Minute)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The live feed data gives us an estimated number of minutes until arrival, not the estimated arrival time. So we calculate the estimated arrival time using the `adjusted_schedule_time` and `estimate_processing_time`.\n",
    "\n",
    "And then we need the scheduled arrival time as a full date so we can compare them. This code assumes that the estimated time and scheduled time are the same day; good thing I don't leave work at midnight!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time_x</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>start_time_string</th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time_y</th>\n",
       "      <th>arrival_time</th>\n",
       "      <th>start_time_date</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>70</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>2</td>\n",
       "      <td>0.41</td>\n",
       "      <td>45.407509</td>\n",
       "      <td>-75.713688</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "      <td>2016-10-04 16:00:02</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>71</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>18</td>\n",
       "      <td>0.74</td>\n",
       "      <td>45.424884</td>\n",
       "      <td>-75.683978</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "      <td>2016-10-04 16:16:02</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>72</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>33</td>\n",
       "      <td>3.84</td>\n",
       "      <td>45.416420</td>\n",
       "      <td>-75.669496</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>43031864-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:01:00.000000</td>\n",
       "      <td>1970-01-01 16:31:00.000000</td>\n",
       "      <td>1970-01-01 16:01:00</td>\n",
       "      <td>2016-10-04 16:31:02</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>73</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>1</td>\n",
       "      <td>0.50</td>\n",
       "      <td>45.405422</td>\n",
       "      <td>-75.712427</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "      <td>2016-10-04 16:00:15</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>74</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>17</td>\n",
       "      <td>0.90</td>\n",
       "      <td>45.424734</td>\n",
       "      <td>-75.684338</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "      <td>2016-10-04 16:16:15</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id route route_label start_time_x  estimate_id  trip_id  stop_id  \\\n",
       "0   7    85    Bayshore     15:31:00           70        7     6653   \n",
       "1   8    85    Bayshore     15:46:00           71        8     6653   \n",
       "2   9    85    Bayshore     16:01:00           72        9     6653   \n",
       "3   7    85    Bayshore     15:31:00           73        7     6653   \n",
       "4   8    85    Bayshore     15:46:00           74        8     6653   \n",
       "\n",
       "  estimate_processing_time  adjusted_schedule_time  adjustment_age   latitude  \\\n",
       "0      2016-10-04 15:58:02                       2            0.41  45.407509   \n",
       "1      2016-10-04 15:58:02                      18            0.74  45.424884   \n",
       "2      2016-10-04 15:58:02                      33            3.84  45.416420   \n",
       "3      2016-10-04 15:59:15                       1            0.50  45.405422   \n",
       "4      2016-10-04 15:59:15                      17            0.90  45.424734   \n",
       "\n",
       "   longitude start_time_string                  stop_times.trip_id  \\\n",
       "0 -75.713688          15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "1 -75.683978          15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "2 -75.669496          16:01:00  43031864-SEPT16-SEPDA16-Weekday-36   \n",
       "3 -75.712427          15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "4 -75.684338          15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "\n",
       "                 start_time_y                arrival_time     start_time_date  \\\n",
       "0  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00   \n",
       "1  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00   \n",
       "2  1970-01-01 16:01:00.000000  1970-01-01 16:31:00.000000 1970-01-01 16:01:00   \n",
       "3  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00   \n",
       "4  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00   \n",
       "\n",
       "  estimated_arrival_time scheduled_arrival_time  \n",
       "0    2016-10-04 16:00:02               15:57:00  \n",
       "1    2016-10-04 16:16:02               16:12:00  \n",
       "2    2016-10-04 16:31:02               16:31:00  \n",
       "3    2016-10-04 16:00:15               15:57:00  \n",
       "4    2016-10-04 16:16:15               16:12:00  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_scheduled_arrival_time[\"scheduled_arrival_time\"] = \\\n",
    "    pd.to_datetime(with_scheduled_arrival_time[\"arrival_time\"], format='%Y-%m-%d %H:%M:%S.%f')\n",
    "with_scheduled_arrival_time[\"scheduled_arrival_time\"] = with_scheduled_arrival_time[\"scheduled_arrival_time\"].dt.time\n",
    "with_scheduled_arrival_time.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2016-10-04 15:57:00\n",
       "1   2016-10-04 16:12:00\n",
       "2   2016-10-04 16:31:00\n",
       "Name: scheduled_date_time, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_scheduled_arrival_time[\"estimated_date\"] = with_scheduled_arrival_time[\"estimated_arrival_time\"].dt.date\n",
    "with_scheduled_arrival_time[\"scheduled_date_time\"] = \\\n",
    "with_scheduled_arrival_time.apply(lambda r : pd.datetime.combine(r['estimated_date'],r['scheduled_arrival_time']),1)\n",
    "with_scheduled_arrival_time[\"scheduled_date_time\"].head(3)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to figure out how far off the estimate is from the scheduled time. Datetime math is fun!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with_scheduled_arrival_time[\"minutes_off_sched\"] = (with_scheduled_arrival_time[\"estimated_arrival_time\"]\\\n",
    "- with_scheduled_arrival_time[\"scheduled_date_time\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time_x</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>...</th>\n",
       "      <th>start_time_string</th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time_y</th>\n",
       "      <th>arrival_time</th>\n",
       "      <th>start_time_date</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "      <th>estimated_date</th>\n",
       "      <th>scheduled_date_time</th>\n",
       "      <th>minutes_off_sched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>70</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>2</td>\n",
       "      <td>0.41</td>\n",
       "      <td>...</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "      <td>2016-10-04 16:00:02</td>\n",
       "      <td>15:57:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 15:57:00</td>\n",
       "      <td>3.033333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>71</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>18</td>\n",
       "      <td>0.74</td>\n",
       "      <td>...</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "      <td>2016-10-04 16:16:02</td>\n",
       "      <td>16:12:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:12:00</td>\n",
       "      <td>4.033333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>72</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>33</td>\n",
       "      <td>3.84</td>\n",
       "      <td>...</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>43031864-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:01:00.000000</td>\n",
       "      <td>1970-01-01 16:31:00.000000</td>\n",
       "      <td>1970-01-01 16:01:00</td>\n",
       "      <td>2016-10-04 16:31:02</td>\n",
       "      <td>16:31:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:31:00</td>\n",
       "      <td>0.033333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>73</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>1</td>\n",
       "      <td>0.50</td>\n",
       "      <td>...</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "      <td>2016-10-04 16:00:15</td>\n",
       "      <td>15:57:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 15:57:00</td>\n",
       "      <td>3.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>74</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>17</td>\n",
       "      <td>0.90</td>\n",
       "      <td>...</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "      <td>2016-10-04 16:16:15</td>\n",
       "      <td>16:12:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:12:00</td>\n",
       "      <td>4.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id route route_label start_time_x  estimate_id  trip_id  stop_id  \\\n",
       "0   7    85    Bayshore     15:31:00           70        7     6653   \n",
       "1   8    85    Bayshore     15:46:00           71        8     6653   \n",
       "2   9    85    Bayshore     16:01:00           72        9     6653   \n",
       "3   7    85    Bayshore     15:31:00           73        7     6653   \n",
       "4   8    85    Bayshore     15:46:00           74        8     6653   \n",
       "\n",
       "  estimate_processing_time  adjusted_schedule_time  adjustment_age  \\\n",
       "0      2016-10-04 15:58:02                       2            0.41   \n",
       "1      2016-10-04 15:58:02                      18            0.74   \n",
       "2      2016-10-04 15:58:02                      33            3.84   \n",
       "3      2016-10-04 15:59:15                       1            0.50   \n",
       "4      2016-10-04 15:59:15                      17            0.90   \n",
       "\n",
       "         ...         start_time_string                  stop_times.trip_id  \\\n",
       "0        ...                  15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "1        ...                  15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "2        ...                  16:01:00  43031864-SEPT16-SEPDA16-Weekday-36   \n",
       "3        ...                  15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "4        ...                  15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "\n",
       "                 start_time_y                arrival_time     start_time_date  \\\n",
       "0  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00   \n",
       "1  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00   \n",
       "2  1970-01-01 16:01:00.000000  1970-01-01 16:31:00.000000 1970-01-01 16:01:00   \n",
       "3  1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000 1970-01-01 15:31:00   \n",
       "4  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000 1970-01-01 15:46:00   \n",
       "\n",
       "  estimated_arrival_time scheduled_arrival_time estimated_date  \\\n",
       "0    2016-10-04 16:00:02               15:57:00     2016-10-04   \n",
       "1    2016-10-04 16:16:02               16:12:00     2016-10-04   \n",
       "2    2016-10-04 16:31:02               16:31:00     2016-10-04   \n",
       "3    2016-10-04 16:00:15               15:57:00     2016-10-04   \n",
       "4    2016-10-04 16:16:15               16:12:00     2016-10-04   \n",
       "\n",
       "  scheduled_date_time minutes_off_sched  \n",
       "0 2016-10-04 15:57:00          3.033333  \n",
       "1 2016-10-04 16:12:00          4.033333  \n",
       "2 2016-10-04 16:31:00          0.033333  \n",
       "3 2016-10-04 15:57:00          3.250000  \n",
       "4 2016-10-04 16:12:00          4.250000  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_scheduled_arrival_time[\"minutes_off_sched\"] = pd.to_timedelta(with_scheduled_arrival_time[\"minutes_off_sched\"])\\\n",
    ".dt.total_seconds() / 60 \n",
    "with_scheduled_arrival_time.head(5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How close to the scheduled time do buses actually arrive?\n",
    "------------------------------------------------------\n",
    "For my first analysis, I want to see how close to the scheduled time the buses are actually arriving. For that, I want to look at the last estimate I have for each trip."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time_x</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>...</th>\n",
       "      <th>start_time_string</th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time_y</th>\n",
       "      <th>arrival_time</th>\n",
       "      <th>start_time_date</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "      <th>estimated_date</th>\n",
       "      <th>scheduled_date_time</th>\n",
       "      <th>minutes_off_sched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>73</td>\n",
       "      <td>7</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 15:59:15</td>\n",
       "      <td>1</td>\n",
       "      <td>0.50</td>\n",
       "      <td>...</td>\n",
       "      <td>15:31:00</td>\n",
       "      <td>43029734-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:31:00.000000</td>\n",
       "      <td>1970-01-01 15:57:00.000000</td>\n",
       "      <td>1970-01-01 15:31:00</td>\n",
       "      <td>2016-10-04 16:00:15</td>\n",
       "      <td>15:57:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 15:57:00</td>\n",
       "      <td>3.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>8</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>91</td>\n",
       "      <td>8</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:18:45</td>\n",
       "      <td>1</td>\n",
       "      <td>0.76</td>\n",
       "      <td>...</td>\n",
       "      <td>15:46:00</td>\n",
       "      <td>43032887-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:46:00.000000</td>\n",
       "      <td>1970-01-01 16:12:00.000000</td>\n",
       "      <td>1970-01-01 15:46:00</td>\n",
       "      <td>2016-10-04 16:19:45</td>\n",
       "      <td>16:12:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:12:00</td>\n",
       "      <td>7.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>9</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>106</td>\n",
       "      <td>9</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:31:16</td>\n",
       "      <td>2</td>\n",
       "      <td>0.78</td>\n",
       "      <td>...</td>\n",
       "      <td>16:01:00</td>\n",
       "      <td>43031864-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:01:00.000000</td>\n",
       "      <td>1970-01-01 16:31:00.000000</td>\n",
       "      <td>1970-01-01 16:01:00</td>\n",
       "      <td>2016-10-04 16:33:16</td>\n",
       "      <td>16:31:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:31:00</td>\n",
       "      <td>2.266667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>63</th>\n",
       "      <td>10</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:16:00</td>\n",
       "      <td>133</td>\n",
       "      <td>10</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:52:14</td>\n",
       "      <td>1</td>\n",
       "      <td>0.52</td>\n",
       "      <td>...</td>\n",
       "      <td>16:16:00</td>\n",
       "      <td>43032313-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:16:00.000000</td>\n",
       "      <td>1970-01-01 16:43:00.000000</td>\n",
       "      <td>1970-01-01 16:16:00</td>\n",
       "      <td>2016-10-04 16:53:14</td>\n",
       "      <td>16:43:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:43:00</td>\n",
       "      <td>10.233333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>11</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>16:31:00</td>\n",
       "      <td>139</td>\n",
       "      <td>11</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-04 16:58:53</td>\n",
       "      <td>14</td>\n",
       "      <td>0.51</td>\n",
       "      <td>...</td>\n",
       "      <td>16:31:00</td>\n",
       "      <td>43032859-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 16:31:00.000000</td>\n",
       "      <td>1970-01-01 16:58:00.000000</td>\n",
       "      <td>1970-01-01 16:31:00</td>\n",
       "      <td>2016-10-04 17:12:53</td>\n",
       "      <td>16:58:00</td>\n",
       "      <td>2016-10-04</td>\n",
       "      <td>2016-10-04 16:58:00</td>\n",
       "      <td>14.883333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    id route route_label start_time_x  estimate_id  trip_id  stop_id  \\\n",
       "3    7    85    Bayshore     15:31:00           73        7     6653   \n",
       "21   8    85    Bayshore     15:46:00           91        8     6653   \n",
       "36   9    85    Bayshore     16:01:00          106        9     6653   \n",
       "63  10    85    Bayshore     16:16:00          133       10     6653   \n",
       "68  11    85    Bayshore     16:31:00          139       11     6653   \n",
       "\n",
       "   estimate_processing_time  adjusted_schedule_time  adjustment_age  \\\n",
       "3       2016-10-04 15:59:15                       1            0.50   \n",
       "21      2016-10-04 16:18:45                       1            0.76   \n",
       "36      2016-10-04 16:31:16                       2            0.78   \n",
       "63      2016-10-04 16:52:14                       1            0.52   \n",
       "68      2016-10-04 16:58:53                      14            0.51   \n",
       "\n",
       "          ...         start_time_string                  stop_times.trip_id  \\\n",
       "3         ...                  15:31:00  43029734-SEPT16-SEPDA16-Weekday-36   \n",
       "21        ...                  15:46:00  43032887-SEPT16-SEPDA16-Weekday-36   \n",
       "36        ...                  16:01:00  43031864-SEPT16-SEPDA16-Weekday-36   \n",
       "63        ...                  16:16:00  43032313-SEPT16-SEPDA16-Weekday-36   \n",
       "68        ...                  16:31:00  43032859-SEPT16-SEPDA16-Weekday-36   \n",
       "\n",
       "                  start_time_y                arrival_time  \\\n",
       "3   1970-01-01 15:31:00.000000  1970-01-01 15:57:00.000000   \n",
       "21  1970-01-01 15:46:00.000000  1970-01-01 16:12:00.000000   \n",
       "36  1970-01-01 16:01:00.000000  1970-01-01 16:31:00.000000   \n",
       "63  1970-01-01 16:16:00.000000  1970-01-01 16:43:00.000000   \n",
       "68  1970-01-01 16:31:00.000000  1970-01-01 16:58:00.000000   \n",
       "\n",
       "       start_time_date estimated_arrival_time scheduled_arrival_time  \\\n",
       "3  1970-01-01 15:31:00    2016-10-04 16:00:15               15:57:00   \n",
       "21 1970-01-01 15:46:00    2016-10-04 16:19:45               16:12:00   \n",
       "36 1970-01-01 16:01:00    2016-10-04 16:33:16               16:31:00   \n",
       "63 1970-01-01 16:16:00    2016-10-04 16:53:14               16:43:00   \n",
       "68 1970-01-01 16:31:00    2016-10-04 17:12:53               16:58:00   \n",
       "\n",
       "   estimated_date scheduled_date_time minutes_off_sched  \n",
       "3      2016-10-04 2016-10-04 15:57:00          3.250000  \n",
       "21     2016-10-04 2016-10-04 16:12:00          7.750000  \n",
       "36     2016-10-04 2016-10-04 16:31:00          2.266667  \n",
       "63     2016-10-04 2016-10-04 16:43:00         10.233333  \n",
       "68     2016-10-04 2016-10-04 16:58:00         14.883333  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grouped = with_scheduled_arrival_time.groupby([\"trip_id\", \"estimated_date\"])\n",
    "last_estimates = with_scheduled_arrival_time[with_scheduled_arrival_time['estimate_processing_time'] ==  grouped['estimate_processing_time'].transform(max)]\n",
    "last_estimates.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    37.000000\n",
       "mean     10.701351\n",
       "std       8.435172\n",
       "min      -0.050000\n",
       "25%       4.916667\n",
       "50%       9.016667\n",
       "75%      13.566667\n",
       "max      31.783333\n",
       "Name: minutes_off_sched, dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_estimates[\"minutes_off_sched\"].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[<matplotlib.axes._subplots.AxesSubplot object at 0x112e6f080>]], dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAEKCAYAAAA2Mm/+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG21JREFUeJzt3XuQZGd93vHvI60uLJIZkIgkhMQoCBKHkBqMgjGY+Kgs\nHGEuuVUAVWI8pBK5KsFLkWAbqASp4hRJjMGqOKnYXoRGJBhsC7DNRSCp0JFFMJLAaiFAYIO16LKS\nEOiC1jKL0P7yR59ZjYaZ7rd7zum33z7Pp2pq+3T3nPP02z2/7Xm6Z0YRgZmZzb8jcgcwM7M0Hthm\nZoXwwDYzK4QHtplZITywzcwK4YFtZlYID2wzs0J4YFsySadLekiScmfZKUknSfoTSd+V9M7mvEsk\n3Sfpcy0fa1XStS3ta1nSIUn+2u2hXbkDWDki4jbg+Db2JelC4JkR8XNt7G8K5wPfiogfafK8BDgH\neFpEfC9TJrOR/L+09dUzgFs2be/zsLZ55oFtSNon6c2SbpJ0QNJ7msrg8qYyuFLS0uZvxyXVkv6z\npM801/uUpBOayypJt29xnJ+WdC7wVuA1TcVyY3P5kyRdLGm/pDsk/eqGY50p6RpJD0i6V9IHE27X\niyTd0HzO9ZJ+ojl/DXgd8MvN8c8H9gI/0WxfMGKfJ0r6mKT7JX2nqVXUXHaapA9L+pakb0v6zU2f\n+86mcvnLZg3Wzx91u4+Q9OvNbf4G8PJxt9sWlysRAwjgnzKsBI4CbgSeB/wr4KvAJ4A9wPu2+Nzz\ngJcBdwCXA29mOIy3O05ExCclvYNhJfK6DZevAXcDzwSOAz4G3A78DvCrwCcj4qckHQ2cNeoGSXoK\n8HHgDcAHgFcDH5f0zIhYlRTA7RHx9ub6B4F/HREvGbVf4D80mU5stl8YESHpyCbvVcC/AA4Bz9/w\neT/e3L4TgF8ALgZOTbjd5zMc0ivAw8CHGa6j9ZCfYdu634yIeyNiP3At8LmIuCkiDgIfYTjANw+K\nAC6JiK83VcLvMxwsKdR8DDekkxgO/jdFxF9HxL3ARcBrm6t8H1iWdGpEfD8iPjtm/y8HvhYR74+I\nQxHxQYb/+bxqU4atTo/yfeAUYDkiHo2I/9ec/4Lm/F9q8h/clPGbEXFxDH/b2vuAUyT9jYTb/Wrg\nNyLizoi4H3jHBFltwXhg27p7Npz+603b32P4zG8rd2/6vO2uN84zGD67v6upG+4Hfgt4anP5LzMc\nVNdL+pKk14/Z39OA2zad983m/J14J/B14ApJ35D0K835pzEcyoe2+bzD6xQRDzcnj2P87T6F4bPt\ndZtvk/WIKxHbzk6fxf0VsPvwzoaVwVM3XL752frtwEHghK2GXkTcw7AeQNKLgaskXRMRf7nN8e9k\nWPNs9AyGtc3UIuIAw9rnzZKeA3xa0g0MB+npko6MiEcn2OXI2w3cBZy+Yfv0La5jPeFn2LZT2w32\nPweOlfSzko4C/iNwzIbL72ZYcQggIu4CrgDeLen45sW2Z0r6BwCS/rmkpzef+wDDgb/ds1kY9u7P\nlnSepF2SXgP8bYb98KjcI0l6efMCqIDvAo82H9czHK7/TdJuScdKetG4/Y273Qxrpj2STpX0ZOAt\n0+S2xTB2YEt6U/Mt6M2SflfSMeM+xxZCbDodW5y/7fUi4kHg3wLvYfiC5AEe/639HzT/fkfS55vT\nrwOOBr4C3Ndc5+TmsrOAz0l6CPgjYE9E7Ns2fMR9wCsYvkj4bYbPil/RnL/5Nm21vZ1nAVcCDwGf\nBf5XRFzTPDt+JXAmw2fbtzPsn7fb98btUbd7L/Ap4Cbg88CHEnPaAtKovzgj6VSGL0D9aEQclPR7\nwCci4tJZBTQzs6GUDnsXsFvSoww7yTu7jWRmZlsZWYlExJ3Auxh+i7cfeCAirppFMLNxJL2k+UGX\nzR/f3eF+37bNfj/eVnazaYyrRJ4MXMawi3uQYbd2WUS8fzbxzMxs3bhK5Bzg1oj4DoCkDwMvAg4P\n7OYnxszMbEIRMdG7lca9S+SbwAslPaF5G9M5DF/J3nzQuf+44IILZnasZlWm/LhgguvmW/tZrqdz\nzs9HCTlLyBgx3fPccR329QwrkT8Dvtic/TtTHSmzffv25Y6QaF/uAElKWU/nbFcJOUvIOK2x7xKJ\niAuBCztPYmZmI/XmJx1XV1dzR0i0mjtAklLW0znbVULOEjJOa+S7RJJ2IMVO97FohnX/LNZEU3dh\nZpaXJKLlFx0XRl3XuSMkqnMHSFLKejpnu0rIWULGafVmYJuZlc6VSAdciZjZOK5EzMwWWG8Gdjm9\nVp07QJJS1tM521VCzhIyTqs3A9vMrHTusDvgDtvMxnGHbWa2wHozsMvptercAZKUsp7O2a4ScpaQ\ncVq9GdhmZqVzh90Bd9hmNo47bDOzBdabgV1Or1XnDpCklPV0znaVkLOEjNPqzcA2MyudO+wOuMM2\ns3HcYZuZLbDeDOxyeq06d4Akpaync7arhJwlZJzW2IEt6W9JunHDx4OS9swinJmZPWaiDlvSEcCd\nwAsi4vbmPHfYm7jDNrNxZtFhnwN8Y31Ym5nZ7Ew6sF8L/G4XQbpWTq9V5w6QpJT1dM52lZCzhIzT\nSh7Yko4GXgn8QXdxzMxsO7smuO7LgC9ExL2bL1hdXWV5eRmApaUlVlZWqKoKeOx/u75tP2Z9u0rc\nXj8v7fqzuj1nn302s7Teze80//p5uR8Pi7K9ft685Bn39Tcveaqqoq5r1tbWAA7Py0klv+go6YPA\n5RFx6abz/aLjJov4ouPsbhP4xVTrg85edJT0RIYvOH54mmDz4Ief+c6rOneARHXuAElKud+dsz0l\nZJxWUiUSEX8FnNhxFjMzG8G/S6QDrkR2fDRXIrbw/LtEzMwWWG8Gdjm9Vp07QKI6d4Akpdzvztme\nEjJOqzcD28ysdO6wO+AOe8dHc4dtC88dtpnZAuvNwC6n16pzB0hU5w6QpJT73TnbU0LGafVmYJuZ\nlc4ddgfcYe/4aO6wbeG5wzYzW2C9Gdjl9Fp17gCJ6twBkpRyvztne0rIOK3eDGwzs9K5w+6AO+wd\nH80dti08d9hmZgusNwO7nF6rzh0gUZ07QJJS7nfnbE8JGafVm4FtZlY6d9gdcIe946O5w7aF5w7b\nzGyB9WZgl9Nr1bkDJKpzB0hSyv3unO0pIeO0xg5sSUuSLpN0i6SvSHrhLIKZmdnjje2wJV0KXBMR\n75W0C3hiRDy44XJ32Ju4w97x0dxh28KbpsMeObAlPQm4MSL+5ojreGBv4oG946N5YNvC6+JFxzOA\neyVdIunPJO2VtHv6iPmU02vVuQMkqnMHSFLK/e6c7Skh47R2JVz+Y8AbIuIGSRcBbwHevvFKq6ur\nLC8vA7C0tMTKygpVVQGPLV7u7XWzPt5jg61K3B5MdP3hM99ZqrfZrlrebrZ2eH8MBoMdff6iPj4X\neT0Hg8Fc5VnfruuatbU1gMPzclLjKpGTgT+NiDOa7Z8E3hIRr9hwHVcim8yyEpllTeFKxKw9rVci\nEXE3cLukZzdnnQN8ecp8Zma2Aynvw/5F4P2SbgL+HvCObiN1o5xeq84dIFGdO0CSUu5352xPCRmn\nNa7DJiJuAv7+DLKYmdkI/l0iHXCHvfNj+TFli86/S8TMbIH1ZmCX02vVuQMkqnMHSFLK/e6c7Skh\n47R6M7DNzErnDrsD7rB3fiw/pmzRucM2M1tgvRnY5fRade4AiercAZKUcr87Z3tKyDit3gxsM7PS\nucPugDvsnR/LjylbdO6wzcwWWG8Gdjm9Vp07QKI6d4AkpdzvztmeEjJOqzcD28ysdO6wO+AOe+fH\n8mPKFp07bDOzBdabgV1Or1XnDpCozh0gSSn3u3O2p4SM0+rNwDYzK5077A64w975sfyYskXnDtvM\nbIH1ZmCX02vVuQMkqnMHSFLK/e6c7Skh47TG/k1HAEn7gO8CjwKPRMQLugxlZmY/LKnDlnQr8PyI\nuG+Ly9xhb+IOe+fH8mPKFl3XHfZEOzYzs3YlVSIMn1pdJelR4LcjYm+HmVp32223cfnll3PWWWfl\njpKgBqrMGVLUlJCzrmuqqsodYyznbE8JGaeVOrBfHBF3SXoqcKWkr0bEtesXrq6usry8DMDS0hIr\nKyuHF2z9BYCc2+9+9//gk5+8lt27T+cHP3hoeMN3HQ/Q+vaDD964aenq5t8qcXsw4fVntc2Y7W6O\nt9P7fzAY7OjzZ7Xd1u3teruE9RwMBnOVZ327rmvW1tYADs/LSU38PmxJFwAHIuJdzfbcd9jnn7+H\nvXvPBPZ0fqxjjjmBgwfvwx32zo41748ps53qpMOWtFvS8c3pJwI/A9w8XUQzM5tWyouOJwHXShoA\n1wEfi4gruo3Vhb/IHSBRnTtAojp3gCSlvCfXOdtTQsZpje2wI+JWYGUGWczMbIRe/C4Rd9jlHWve\nH1NmO+XfJWJmtsB6NLDdYberzh0gSSl9pnO2p4SM0+rRwDYzK5s77Ja5w27nWPP+mDLbKXfYZmYL\nrEcD2x12u+rcAZKU0mc6Z3tKyDitHg1sM7OyucNumTvsdo41748ps51yh21mtsB6NLDdYberzh0g\nSSl9pnO2p4SM0+rRwDYzK5s77Ja5w27nWPP+mDLbKXfYZmYLrEcD2x12u+rcAZKU0mc6Z3tKyDit\nHg1sM7OyucNumTvsdo41748ps51yh21mtsB6NLDdYberzh0gSSl9pnO2p4SM00oa2JKOlHSjpI92\nHcjMzLaW1GFL+vfA84HjI+JVmy5zh72BO+x2jjXvjymzneqkw5b0dOBngfcw/Ko1M7MMUiqR3wB+\nCTjUcZaOucNuV507QJJS+kznbE8JGae1a9SFkl4BfCsibpRUbXe91dVVlpeXAVhaWmJlZYWqGl59\nffFybu/ffwdwapO2bv6tOtk+dOgRHm/S/Q06zTf9NmO22z2eNJtv5q6++urh0TM/XtfNw9fLqO3B\nYDBXebbaHgwGc5Vnfbuua9bW1gAOz8tJjeywJb0D+DngB8CxwI8AH4qI1224jjvsDdxhl3Qsd+WW\nT+sddkS8LSJOi4gzgNcCn944rM3MbHYmfR92wU9H3GG3q84dIFGdO0CSUnrXEnKWkHFaIzvsjSLi\nGuCaDrOYmdkI/l0iLXOHXdKx3GFbPv5dImZmC6xHA9sddrvq3AES1bkDJCmldy0hZwkZp9WjgW1m\nVjZ32C1zh13SsdxhWz7usM3MFliPBrY77HbVuQMkqnMHSFJK71pCzhIyTqtHA9vMrGzusFvmDruk\nY7nDtnzcYZuZLbAeDWx32O2qcwdIVOcOkKSU3rWEnCVknFaPBraZWdncYbfMHXZJx3KHbfm4wzYz\nW2A9GtjusNtV5w6QqM4dIEkpvWsJOUvIOK0eDWwzs7K5w26ZO+ySjuUO2/Jxh21mtsB6NLDdYber\nzh0gUZ07QJJSetcScpaQcVpjB7akYyVdJ2kg6UuSLpxBLjMz2ySpw5a0OyIelrQL+Azwxoi4rrnM\nHfYG7rBLOpY7bMunsw47Ih5uTh4NHAUcmjCbmZntUNLAlnSEpAFwD3BFRNzQbawuuMNuV507QKI6\nd4AkpfSuJeQsIeO0dqVcKSIOASuSngR8RNJzIuLL65evrq6yvLwMwNLSEisrK1RVBTy2eDm39++/\nAzi1SVs3/1adbB869AiPN+n+Bp3mm36bMdtdH2/a/Y1ez1k9Hs8++2xm5eqrr+7s9gwGg1b316f1\nq+uatbU1gMPzclITvw9b0n8CHo6IdzXb7rA3cIdd0rFm12FLi3ebZmkR16+TDlvSiZKWmtNPAF4K\n3DJdRDMzm1ZKh30K8GlJNwHXM+ywP9FtrC64w25XnTtAojp3gER17gBJyuiH69wBOjO2w46Im4Ef\nm0EWMzMboUc/6fis3AESVbkDJKpyB0hU5Q6QqModIMn6i2nzrcodoDM9GthmZmXr0cB2h92uOneA\nRHXuAInq3AGSuMPOq0cD28ysbD0a2O6w21XlDpCoyh0gUZU7QBJ32Hn1aGCbmZWtRwPbHXa76twB\nEtW5AySqcwdI4g47rx4NbDOzsvVoYLvDbleVO0CiKneARFXuAEncYefVo4FtZla2Hg1sd9jtqnMH\nSFTnDpCozh0giTvsvHo0sM3Mytajge0Ou11V7gCJqtwBElW5AyRxh51Xjwa2mVnZejSw3WG3q84d\nIFGdO0CiOneAJO6w8+rRwDYzK1uPBrY77HZVuQMkqnIHSFTlDpDEHXZePRrYZmZl69HAdofdrjp3\ngER17gCJ6twBkrjDzivlr6afJulqSV+W9CVJe2YRzMzMHm/sH+EFHgHeFBEDSccBX5B0ZUTc0nG2\nlrnDbleVO0CiKneARFXuAEncYec19hl2RNwdEYPm9AHgFuBpXQczM7PHm6jDlrQMPA+4rosw3XKH\n3a46d4BEde4AiercAZK4w84rpRIBoKlDLgPe2DzTPmx1dZXl5WUAlpaWWFlZOfyt0/odnHN7//47\ngFObtHXzb9XJ9qFDj/B4k+5v0Gm+6bcZs9318abd3+j1nNXj8THb5R13eer28Jhd3Z7BYNDq/tpb\nv43bgzGXj9rubv3qumZtbQ3g8LyclCJi/JWko4CPAZdHxEWbLouUfeR0/vl72Lv3TKD710uPOeYE\nDh68D5jFmmhGx1nUY4lZPXalxbtNs7SI6yeJiNAkn5PyLhEBFwNf2TyszcxsdlI67BcD/xI4W9KN\nzce5HefqgDvsdtW5AySqcwdIVOcOkMQddl5jO+yI+Ay9+gEbM7P51KNB7Pdht6vKHSBRlTtAoip3\ngCR+H3ZePRrYZmZl69HAdofdrjp3gER17gCJ6twBkrjDzqtHA9vMrGw9GtjusNtV5Q6QqModIFGV\nO0ASd9h59Whgm5mVrUcD2x12u+rcARLVuQMkqnMHSOIOO68eDWwzs7L1aGC7w25XlTtAoip3gERV\n7gBJ3GHn1aOBbWZWth4NbHfY7apzB0hU5w6QqM4dIIk77Lx6NLDNzMrWo4HtDrtdVe4AiarcARJV\nuQMkcYedV48GtplZ2Xo0sN1ht6vOHSBRnTtAojp3gCTusPPq0cA2Mytbjwa2O+x2VbkDJKpyB0hU\n5Q6QxB12Xj0a2GZmZUv5I7zvlXSPpJtnEag77rDbVecOkKjOHSBRnTtAEnfYeaU8w74EKPCP7pqZ\nLZaxAzsirgXun0GWjrnDbleVO0CiKneARFXuAEncYeflDtvMrBA9GtjusNtV5w6QqM4dIFGdO0AS\nd9h57WpjJ6urqywvLwOwtLTEysrK4W+d1u/gnNv7998BnNqkrZt/q062Dx16hMebdH+DTvNNv82Y\n7a6PN+3+Rq+nJGbr8cdv//YOH/NdfT0NBoNW95e6/ZjNt3er7cGYy0dtd7d+dV2ztrYGcHheTkoR\nMf5K0jLw0Yh47haXRco+cjr//D3s3XsmsKfzYx1zzAkcPHgfMIs10YyOs6jHWszbNO9fj9MY/se6\nWOsniYiY6BlDytv6PgB8Fni2pNslvX7agGZmNr2Ud4mcFxFPi4hjIuK0iLhkFsHa5w67XXXuAInq\n3AES1bkDJHGHnVePXnQ0Mytbjwa234fdrip3gERV7gCJqtwBkvh92Hn1aGCbmZWtRwPbHXa76twB\nEtW5AySqcwdI4g47rx4NbDOzsvVoYLvDbleVO0CiKneARFXuAEncYefVo4FtZla2Hg1sd9jtqnMH\nSFTnDpCozh0giTvsvHo0sM3Mytajge0Ou11V7gCJqtwBElW5AyRxh51Xjwa2mVnZejSw3WG3q84d\nIFGdO0CiOneAJO6w8+rRwDYzK1uPBrY77HZVuQMkqnIHSFTlDpDEHXZePRrYZmZl69HAdofdrjp3\ngER17gCJ6twBkrjDzqtHA9vMrGw9GtjusNtV5Q6QqModIFGVO0ASd9h59Whgm5mVLeWP8J4r6auS\n/kLSr8wiVDfcYberzh0gUZ07QKI6d4Ak7rDzGjmwJR0J/E/gXODvAOdJ+tFZBGvfHbkDJBrkDpDI\nOdtVRs7BoIScJWSczrhn2C8Avh4R+yLiEeCDwD/qPlYXvpc7QKIHcgdI5JztKiPnAw+UkLOEjNMZ\nN7BPBW7fsH1Hc56Zmc3YrjGXx0xSdOyII+CII/6U4457ZefHOnDguzvcw742YszAvtwBEu3LHSDR\nvtwBkuzbty93hAT7cgfojCK2n8mSXghcGBHnNttvBQ5FxH/fcJ2FGOpmZrMWEZrk+uMG9i7ga8BP\nA/uB64HzIuKWnYQ0M7PJjaxEIuIHkt4AfAo4ErjYw9rMLI+Rz7DNzGx+tPKTjpIulHSHpBubj3Pb\n2G9bSvnhH0n7JH2xWcPrc+dZJ+m9ku6RdPOG854i6UpJfy7pCklLOTM2mbbKOVePTUmnSbpa0pcl\nfUnSnub8uVrPETnnbT2PlXSdpEGT88Lm/Hlbz+1yTrSerTzDlnQB8FBEvHvHO2tZ88M/XwPOAe4E\nbmBOe3hJtwLPj4j7cmfZSNJLgAPA+yLiuc15vwZ8OyJ+rflP8MkR8ZY5zDlXj01JJwMnR8RA0nHA\nF4B/DLyeOVrPETlfzRytJ4Ck3RHxcPOa22eANwL/jDlazxE5z2WC9Wzzd4lM9GrnDJX2wz9zt44R\ncS1w/6azXwVc2py+lOEXc1bb5IQ5WtOIuDsiBs3pA8AtDH+2Ya7Wc0ROmKP1BIiIh5uTRwNHMXw7\n8lytJ2ybEyZYzzYH9i9KuknSxbm//dikpB/+CeAqSZ+X9G9yhxnjpIi4pzl9D3BSzjBjzOVjU9Iy\n8DzgOuZ4PTfk/Fxz1lytp6QjJA0YrtsVEXE9c7ie2+SECdYzeWA3fdDNW3y8CvjfwBnACnAX8K7p\nblInSnpV9cUR8TzgZcC/a77Fn3sx7NXmdZ3n8rHZ1AwfAt4YEQ9tvGye1rPJeRnDnAeYw/WMiEMR\nsQI8HfhxSX930+VzsZ5b5HwOE67nuJ903Hiwl6ZcT9J7gI+m7ncG7gRO27B9GnP6m6Ai4q7m33sl\nfYRhnXNt3lTbukfSyRFxt6RTgG/lDrSViDica14em5KOYjis/09E/GFz9tyt54ac/3c95zyu57qI\neFDS1cA/ZA7Xc92GnOdGxOEBnbKebb1L5JQNm/8EuHm762bweeBZkpYlHQ28BvjjzJl+iKTdko5v\nTj8R+Bnmax03+2Pg55vTPw/84YjrZjNvj01JAi4GvhIRF224aK7Wc7ucc7ieJ67XCJKeALyUYd8+\nb+u5Zc7mxd11Y9ezrXeJvI/hU/oAbgV+YUN/lJ2klwEX8dgP//zXzJF+iKQzgI80m7uA989LTkkf\nAH4KOJFh//Z24I+A3wdOZ/jLG14dEVl/TdoWOS9g+OdH5uaxKekngT8Bvshj36a/leFPEc/Nem6T\n823AeczXej6X4YuKRzJ8Avp7EfFfJD2F+VrP7XJONDv9gzNmZoXwnwgzMyuEB7aZWSE8sM3MCuGB\nbWZWCA9sM7NCeGCbmRXCA9vMrBAe2GZmhfj/8Ns5UDCQ6XkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112033d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "last_estimates.hist(\"minutes_off_sched\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mean arrival time is over 10 minutes late, for a bus that is scheduled to come every 15 minutes!\n",
    "\n",
    "Which trip is furthest off schedule?\n",
    "----------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>route</th>\n",
       "      <th>route_label</th>\n",
       "      <th>start_time_x</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>...</th>\n",
       "      <th>start_time_string</th>\n",
       "      <th>stop_times.trip_id</th>\n",
       "      <th>start_time_y</th>\n",
       "      <th>arrival_time</th>\n",
       "      <th>start_time_date</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "      <th>estimated_date</th>\n",
       "      <th>scheduled_date_time</th>\n",
       "      <th>minutes_off_sched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>15</td>\n",
       "      <td>85</td>\n",
       "      <td>Bayshore</td>\n",
       "      <td>15:16:00</td>\n",
       "      <td>244</td>\n",
       "      <td>15</td>\n",
       "      <td>6653</td>\n",
       "      <td>2016-10-06 16:11:47</td>\n",
       "      <td>1</td>\n",
       "      <td>0.29</td>\n",
       "      <td>...</td>\n",
       "      <td>15:16:00</td>\n",
       "      <td>43034068-SEPT16-SEPDA16-Weekday-36</td>\n",
       "      <td>1970-01-01 15:16:00.000000</td>\n",
       "      <td>1970-01-01 15:41:00.000000</td>\n",
       "      <td>1970-01-01 15:16:00</td>\n",
       "      <td>2016-10-06 16:12:47</td>\n",
       "      <td>15:41:00</td>\n",
       "      <td>2016-10-06</td>\n",
       "      <td>2016-10-06 15:41:00</td>\n",
       "      <td>31.783333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     id route route_label start_time_x  estimate_id  trip_id  stop_id  \\\n",
       "166  15    85    Bayshore     15:16:00          244       15     6653   \n",
       "\n",
       "    estimate_processing_time  adjusted_schedule_time  adjustment_age  \\\n",
       "166      2016-10-06 16:11:47                       1            0.29   \n",
       "\n",
       "           ...         start_time_string                  stop_times.trip_id  \\\n",
       "166        ...                  15:16:00  43034068-SEPT16-SEPDA16-Weekday-36   \n",
       "\n",
       "                   start_time_y                arrival_time  \\\n",
       "166  1970-01-01 15:16:00.000000  1970-01-01 15:41:00.000000   \n",
       "\n",
       "        start_time_date estimated_arrival_time scheduled_arrival_time  \\\n",
       "166 1970-01-01 15:16:00    2016-10-06 16:12:47               15:41:00   \n",
       "\n",
       "    estimated_date scheduled_date_time minutes_off_sched  \n",
       "166     2016-10-06 2016-10-06 15:41:00         31.783333  \n",
       "\n",
       "[1 rows x 22 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_estimates[last_estimates[\"minutes_off_sched\"] == last_estimates[\"minutes_off_sched\"].max()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Are some trips consistently late?\n",
    "--------------------------------\n",
    "\n",
    "Note: Due to how this data is gathered, the trip that is scheduled to arrive at 15:41 will only be captured if it is quite late, and the 17:13 trip (and sometimes a couple earlier trips if they're running late!) is an estimate quite far out from when it actually arrives.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>minutes_off_sched</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">15:41:00</th>\n",
       "      <th>count</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>28.222222</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>5.726409</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>21.616667</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>26.441667</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>31.266667</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>31.525000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>31.783333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">15:57:00</th>\n",
       "      <th>count</th>\n",
       "      <td>5.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.383333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.266305</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>3.250000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.400000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.883333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.200000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>11.183333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">16:12:00</th>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>8.405556</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>5.091733</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.783333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>5.787500</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>8.383333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>12.429167</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>14.183333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">16:31:00</th>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.361111</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.513398</td>\n",
       "      <td>0.547723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.266667</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>4.945833</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>5.425000</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.066667</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>12.650000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">16:43:00</th>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>14.186111</td>\n",
       "      <td>6.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>10.429181</td>\n",
       "      <td>8.869423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>3.283333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.395833</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>10.541667</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>18.650000</td>\n",
       "      <td>10.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>31.716667</td>\n",
       "      <td>22.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">16:58:00</th>\n",
       "      <th>count</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>13.530556</td>\n",
       "      <td>11.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>5.970254</td>\n",
       "      <td>5.750362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>4.500000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>10.804167</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>13.800000</td>\n",
       "      <td>11.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>16.433333</td>\n",
       "      <td>13.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>21.966667</td>\n",
       "      <td>20.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">17:13:00</th>\n",
       "      <th>count</th>\n",
       "      <td>5.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4.893333</td>\n",
       "      <td>18.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>4.781164</td>\n",
       "      <td>5.787918</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-0.050000</td>\n",
       "      <td>11.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.966667</td>\n",
       "      <td>15.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2.500000</td>\n",
       "      <td>16.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>9.883333</td>\n",
       "      <td>24.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>10.166667</td>\n",
       "      <td>24.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              minutes_off_sched  adjusted_schedule_time\n",
       "scheduled_arrival_time                                                 \n",
       "15:41:00               count           3.000000                3.000000\n",
       "                       mean           28.222222                1.000000\n",
       "                       std             5.726409                0.000000\n",
       "                       min            21.616667                1.000000\n",
       "                       25%            26.441667                1.000000\n",
       "                       50%            31.266667                1.000000\n",
       "                       75%            31.525000                1.000000\n",
       "                       max            31.783333                1.000000\n",
       "15:57:00               count           5.000000                5.000000\n",
       "                       mean            6.383333                1.000000\n",
       "                       std             3.266305                0.000000\n",
       "                       min             3.250000                1.000000\n",
       "                       25%             3.400000                1.000000\n",
       "                       50%             6.883333                1.000000\n",
       "                       75%             7.200000                1.000000\n",
       "                       max            11.183333                1.000000\n",
       "16:12:00               count           6.000000                6.000000\n",
       "                       mean            8.405556                1.000000\n",
       "                       std             5.091733                0.000000\n",
       "                       min             0.783333                1.000000\n",
       "                       25%             5.787500                1.000000\n",
       "                       50%             8.383333                1.000000\n",
       "                       75%            12.429167                1.000000\n",
       "                       max            14.183333                1.000000\n",
       "16:31:00               count           6.000000                6.000000\n",
       "                       mean            6.361111                1.500000\n",
       "                       std             3.513398                0.547723\n",
       "                       min             2.266667                1.000000\n",
       "                       25%             4.945833                1.000000\n",
       "                       50%             5.425000                1.500000\n",
       "                       75%             7.066667                2.000000\n",
       "                       max            12.650000                2.000000\n",
       "16:43:00               count           6.000000                6.000000\n",
       "                       mean           14.186111                6.666667\n",
       "                       std            10.429181                8.869423\n",
       "                       min             3.283333                1.000000\n",
       "                       25%             8.395833                1.000000\n",
       "                       50%            10.541667                1.500000\n",
       "                       75%            18.650000               10.250000\n",
       "                       max            31.716667               22.000000\n",
       "16:58:00               count           6.000000                6.000000\n",
       "                       mean           13.530556               11.333333\n",
       "                       std             5.970254                5.750362\n",
       "                       min             4.500000                3.000000\n",
       "                       25%            10.804167                9.000000\n",
       "                       50%            13.800000               11.000000\n",
       "                       75%            16.433333               13.750000\n",
       "                       max            21.966667               20.000000\n",
       "17:13:00               count           5.000000                5.000000\n",
       "                       mean            4.893333               18.000000\n",
       "                       std             4.781164                5.787918\n",
       "                       min            -0.050000               11.000000\n",
       "                       25%             1.966667               15.000000\n",
       "                       50%             2.500000               16.000000\n",
       "                       75%             9.883333               24.000000\n",
       "                       max            10.166667               24.000000"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_estimates.groupby(\"scheduled_arrival_time\")[[\"minutes_off_sched\", \"adjusted_schedule_time\"]].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Only the 16:12 bus is ever on time, and 16:43 and 16:58 buses are typically 10 minutes late.\n",
    "\n",
    "The 16:58 bus is, as of about it's scheduled arrival time, predicted to arrive over 13 minutes late half the time. Based on experience, I would say that it's not usually that late in practice, as the buses frequently do gain on their predicted time that far out."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How accurate are the API's estimated arrival times?\n",
    "----------------------------------\n",
    "\n",
    "It's been my experience that many days, the predictions seem to get later and later, until somewhere around 10 minutes out, when the bus starts catching up.\n",
    "\n",
    "Compare the last estimated time for a trip to the different estimates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>minutes_off_sched</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>scheduled_arrival_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>trip_id</th>\n",
       "      <th>scheduled_date_time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">7</th>\n",
       "      <th>2016-10-04 15:57:00</th>\n",
       "      <td>3.250000</td>\n",
       "      <td>2016-10-04 16:00:15</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-05 15:57:00</th>\n",
       "      <td>12.100000</td>\n",
       "      <td>2016-10-05 16:09:06</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 15:57:00</th>\n",
       "      <td>7.200000</td>\n",
       "      <td>2016-10-06 16:04:12</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 15:57:00</th>\n",
       "      <td>6.883333</td>\n",
       "      <td>2016-10-07 16:03:53</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 15:57:00</th>\n",
       "      <td>3.400000</td>\n",
       "      <td>2016-10-11 16:00:24</td>\n",
       "      <td>15:57:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">8</th>\n",
       "      <th>2016-10-04 16:12:00</th>\n",
       "      <td>8.283333</td>\n",
       "      <td>2016-10-04 16:20:17</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-05 16:12:00</th>\n",
       "      <td>5.383333</td>\n",
       "      <td>2016-10-05 16:17:23</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 16:12:00</th>\n",
       "      <td>0.883333</td>\n",
       "      <td>2016-10-06 16:12:53</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 16:12:00</th>\n",
       "      <td>14.450000</td>\n",
       "      <td>2016-10-07 16:26:27</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 16:12:00</th>\n",
       "      <td>10.516667</td>\n",
       "      <td>2016-10-11 16:22:31</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-12 16:12:00</th>\n",
       "      <td>14.183333</td>\n",
       "      <td>2016-10-12 16:26:11</td>\n",
       "      <td>16:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">9</th>\n",
       "      <th>2016-10-04 16:31:00</th>\n",
       "      <td>4.850000</td>\n",
       "      <td>2016-10-04 16:35:51</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-05 16:31:00</th>\n",
       "      <td>6.900000</td>\n",
       "      <td>2016-10-05 16:37:54</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 16:31:00</th>\n",
       "      <td>12.650000</td>\n",
       "      <td>2016-10-06 16:43:39</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 16:31:00</th>\n",
       "      <td>10.566667</td>\n",
       "      <td>2016-10-07 16:41:34</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 16:31:00</th>\n",
       "      <td>7.133333</td>\n",
       "      <td>2016-10-11 16:38:08</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-12 16:31:00</th>\n",
       "      <td>10.616667</td>\n",
       "      <td>2016-10-12 16:41:37</td>\n",
       "      <td>16:31:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">10</th>\n",
       "      <th>2016-10-04 16:43:00</th>\n",
       "      <td>10.233333</td>\n",
       "      <td>2016-10-04 16:53:14</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-05 16:43:00</th>\n",
       "      <td>31.716667</td>\n",
       "      <td>2016-10-05 17:14:43</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 16:43:00</th>\n",
       "      <td>3.283333</td>\n",
       "      <td>2016-10-06 16:46:17</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 16:43:00</th>\n",
       "      <td>11.733333</td>\n",
       "      <td>2016-10-07 16:54:44</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 16:43:00</th>\n",
       "      <td>21.250000</td>\n",
       "      <td>2016-10-11 17:04:15</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-12 16:43:00</th>\n",
       "      <td>8.733333</td>\n",
       "      <td>2016-10-12 16:51:44</td>\n",
       "      <td>16:43:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"6\" valign=\"top\">11</th>\n",
       "      <th>2016-10-04 16:58:00</th>\n",
       "      <td>14.883333</td>\n",
       "      <td>2016-10-04 17:12:53</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-05 16:58:00</th>\n",
       "      <td>12.716667</td>\n",
       "      <td>2016-10-05 17:10:43</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 16:58:00</th>\n",
       "      <td>21.966667</td>\n",
       "      <td>2016-10-06 17:19:58</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 16:58:00</th>\n",
       "      <td>10.166667</td>\n",
       "      <td>2016-10-07 17:08:10</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 16:58:00</th>\n",
       "      <td>5.633333</td>\n",
       "      <td>2016-10-11 17:03:38</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-12 16:58:00</th>\n",
       "      <td>16.966667</td>\n",
       "      <td>2016-10-12 17:14:58</td>\n",
       "      <td>16:58:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">12</th>\n",
       "      <th>2016-10-04 17:13:00</th>\n",
       "      <td>9.883333</td>\n",
       "      <td>2016-10-04 17:22:53</td>\n",
       "      <td>17:13:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-06 17:13:00</th>\n",
       "      <td>1.966667</td>\n",
       "      <td>2016-10-06 17:14:58</td>\n",
       "      <td>17:13:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 17:13:00</th>\n",
       "      <td>10.166667</td>\n",
       "      <td>2016-10-07 17:23:10</td>\n",
       "      <td>17:13:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 17:13:00</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>2016-10-11 17:16:00</td>\n",
       "      <td>17:13:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-12 17:13:00</th>\n",
       "      <td>0.566667</td>\n",
       "      <td>2016-10-12 17:13:34</td>\n",
       "      <td>17:13:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">15</th>\n",
       "      <th>2016-10-06 15:41:00</th>\n",
       "      <td>31.783333</td>\n",
       "      <td>2016-10-06 16:12:47</td>\n",
       "      <td>15:41:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-07 15:41:00</th>\n",
       "      <td>21.616667</td>\n",
       "      <td>2016-10-07 16:02:37</td>\n",
       "      <td>15:41:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2016-10-11 15:41:00</th>\n",
       "      <td>31.266667</td>\n",
       "      <td>2016-10-11 16:12:16</td>\n",
       "      <td>15:41:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             minutes_off_sched estimated_arrival_time  \\\n",
       "trip_id scheduled_date_time                                             \n",
       "7       2016-10-04 15:57:00           3.250000    2016-10-04 16:00:15   \n",
       "        2016-10-05 15:57:00          12.100000    2016-10-05 16:09:06   \n",
       "        2016-10-06 15:57:00           7.200000    2016-10-06 16:04:12   \n",
       "        2016-10-07 15:57:00           6.883333    2016-10-07 16:03:53   \n",
       "        2016-10-11 15:57:00           3.400000    2016-10-11 16:00:24   \n",
       "8       2016-10-04 16:12:00           8.283333    2016-10-04 16:20:17   \n",
       "        2016-10-05 16:12:00           5.383333    2016-10-05 16:17:23   \n",
       "        2016-10-06 16:12:00           0.883333    2016-10-06 16:12:53   \n",
       "        2016-10-07 16:12:00          14.450000    2016-10-07 16:26:27   \n",
       "        2016-10-11 16:12:00          10.516667    2016-10-11 16:22:31   \n",
       "        2016-10-12 16:12:00          14.183333    2016-10-12 16:26:11   \n",
       "9       2016-10-04 16:31:00           4.850000    2016-10-04 16:35:51   \n",
       "        2016-10-05 16:31:00           6.900000    2016-10-05 16:37:54   \n",
       "        2016-10-06 16:31:00          12.650000    2016-10-06 16:43:39   \n",
       "        2016-10-07 16:31:00          10.566667    2016-10-07 16:41:34   \n",
       "        2016-10-11 16:31:00           7.133333    2016-10-11 16:38:08   \n",
       "        2016-10-12 16:31:00          10.616667    2016-10-12 16:41:37   \n",
       "10      2016-10-04 16:43:00          10.233333    2016-10-04 16:53:14   \n",
       "        2016-10-05 16:43:00          31.716667    2016-10-05 17:14:43   \n",
       "        2016-10-06 16:43:00           3.283333    2016-10-06 16:46:17   \n",
       "        2016-10-07 16:43:00          11.733333    2016-10-07 16:54:44   \n",
       "        2016-10-11 16:43:00          21.250000    2016-10-11 17:04:15   \n",
       "        2016-10-12 16:43:00           8.733333    2016-10-12 16:51:44   \n",
       "11      2016-10-04 16:58:00          14.883333    2016-10-04 17:12:53   \n",
       "        2016-10-05 16:58:00          12.716667    2016-10-05 17:10:43   \n",
       "        2016-10-06 16:58:00          21.966667    2016-10-06 17:19:58   \n",
       "        2016-10-07 16:58:00          10.166667    2016-10-07 17:08:10   \n",
       "        2016-10-11 16:58:00           5.633333    2016-10-11 17:03:38   \n",
       "        2016-10-12 16:58:00          16.966667    2016-10-12 17:14:58   \n",
       "12      2016-10-04 17:13:00           9.883333    2016-10-04 17:22:53   \n",
       "        2016-10-06 17:13:00           1.966667    2016-10-06 17:14:58   \n",
       "        2016-10-07 17:13:00          10.166667    2016-10-07 17:23:10   \n",
       "        2016-10-11 17:13:00           3.000000    2016-10-11 17:16:00   \n",
       "        2016-10-12 17:13:00           0.566667    2016-10-12 17:13:34   \n",
       "15      2016-10-06 15:41:00          31.783333    2016-10-06 16:12:47   \n",
       "        2016-10-07 15:41:00          21.616667    2016-10-07 16:02:37   \n",
       "        2016-10-11 15:41:00          31.266667    2016-10-11 16:12:16   \n",
       "\n",
       "                            scheduled_arrival_time  \n",
       "trip_id scheduled_date_time                         \n",
       "7       2016-10-04 15:57:00               15:57:00  \n",
       "        2016-10-05 15:57:00               15:57:00  \n",
       "        2016-10-06 15:57:00               15:57:00  \n",
       "        2016-10-07 15:57:00               15:57:00  \n",
       "        2016-10-11 15:57:00               15:57:00  \n",
       "8       2016-10-04 16:12:00               16:12:00  \n",
       "        2016-10-05 16:12:00               16:12:00  \n",
       "        2016-10-06 16:12:00               16:12:00  \n",
       "        2016-10-07 16:12:00               16:12:00  \n",
       "        2016-10-11 16:12:00               16:12:00  \n",
       "        2016-10-12 16:12:00               16:12:00  \n",
       "9       2016-10-04 16:31:00               16:31:00  \n",
       "        2016-10-05 16:31:00               16:31:00  \n",
       "        2016-10-06 16:31:00               16:31:00  \n",
       "        2016-10-07 16:31:00               16:31:00  \n",
       "        2016-10-11 16:31:00               16:31:00  \n",
       "        2016-10-12 16:31:00               16:31:00  \n",
       "10      2016-10-04 16:43:00               16:43:00  \n",
       "        2016-10-05 16:43:00               16:43:00  \n",
       "        2016-10-06 16:43:00               16:43:00  \n",
       "        2016-10-07 16:43:00               16:43:00  \n",
       "        2016-10-11 16:43:00               16:43:00  \n",
       "        2016-10-12 16:43:00               16:43:00  \n",
       "11      2016-10-04 16:58:00               16:58:00  \n",
       "        2016-10-05 16:58:00               16:58:00  \n",
       "        2016-10-06 16:58:00               16:58:00  \n",
       "        2016-10-07 16:58:00               16:58:00  \n",
       "        2016-10-11 16:58:00               16:58:00  \n",
       "        2016-10-12 16:58:00               16:58:00  \n",
       "12      2016-10-04 17:13:00               17:13:00  \n",
       "        2016-10-06 17:13:00               17:13:00  \n",
       "        2016-10-07 17:13:00               17:13:00  \n",
       "        2016-10-11 17:13:00               17:13:00  \n",
       "        2016-10-12 17:13:00               17:13:00  \n",
       "15      2016-10-06 15:41:00               15:41:00  \n",
       "        2016-10-07 15:41:00               15:41:00  \n",
       "        2016-10-11 15:41:00               15:41:00  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_scheduled_arrival_time.groupby([\"trip_id\", \"scheduled_date_time\"])[\"minutes_off_sched\", \"estimated_arrival_time\", \"scheduled_arrival_time\"].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "I am calling a last estimate \"reliable\" if it was estimating an arrival within 3 minutes. This removes the trips that actually arrived well outside of the window I was gathering data for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>estimate_id</th>\n",
       "      <th>trip_id</th>\n",
       "      <th>stop_id</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>adjustment_age</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>minutes_off_sched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>25.000000</td>\n",
       "      <td>25.00000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>25.0</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>25.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>9.320000</td>\n",
       "      <td>365.60000</td>\n",
       "      <td>9.320000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>1.240000</td>\n",
       "      <td>0.684000</td>\n",
       "      <td>45.406314</td>\n",
       "      <td>-75.712534</td>\n",
       "      <td>9.673333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.410394</td>\n",
       "      <td>222.07206</td>\n",
       "      <td>2.410394</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.522813</td>\n",
       "      <td>1.259176</td>\n",
       "      <td>0.002439</td>\n",
       "      <td>0.001100</td>\n",
       "      <td>7.995430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>7.000000</td>\n",
       "      <td>73.00000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>45.404645</td>\n",
       "      <td>-75.715161</td>\n",
       "      <td>0.783333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.000000</td>\n",
       "      <td>196.00000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.370000</td>\n",
       "      <td>45.405266</td>\n",
       "      <td>-75.713254</td>\n",
       "      <td>4.916667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>9.000000</td>\n",
       "      <td>301.00000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>45.405323</td>\n",
       "      <td>-75.712345</td>\n",
       "      <td>7.483333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>10.000000</td>\n",
       "      <td>616.00000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.650000</td>\n",
       "      <td>45.406794</td>\n",
       "      <td>-75.712252</td>\n",
       "      <td>11.183333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>15.000000</td>\n",
       "      <td>727.00000</td>\n",
       "      <td>15.000000</td>\n",
       "      <td>6653.0</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>6.490000</td>\n",
       "      <td>45.416249</td>\n",
       "      <td>-75.708698</td>\n",
       "      <td>31.783333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              id  estimate_id    trip_id  stop_id  adjusted_schedule_time  \\\n",
       "count  25.000000     25.00000  25.000000     25.0               25.000000   \n",
       "mean    9.320000    365.60000   9.320000   6653.0                1.240000   \n",
       "std     2.410394    222.07206   2.410394      0.0                0.522813   \n",
       "min     7.000000     73.00000   7.000000   6653.0                1.000000   \n",
       "25%     8.000000    196.00000   8.000000   6653.0                1.000000   \n",
       "50%     9.000000    301.00000   9.000000   6653.0                1.000000   \n",
       "75%    10.000000    616.00000  10.000000   6653.0                1.000000   \n",
       "max    15.000000    727.00000  15.000000   6653.0                3.000000   \n",
       "\n",
       "       adjustment_age   latitude  longitude  minutes_off_sched  \n",
       "count       25.000000  24.000000  24.000000          25.000000  \n",
       "mean         0.684000  45.406314 -75.712534           9.673333  \n",
       "std          1.259176   0.002439   0.001100           7.995430  \n",
       "min         -1.000000  45.404645 -75.715161           0.783333  \n",
       "25%          0.370000  45.405266 -75.713254           4.916667  \n",
       "50%          0.500000  45.405323 -75.712345           7.483333  \n",
       "75%          0.650000  45.406794 -75.712252          11.183333  \n",
       "max          6.490000  45.416249 -75.708698          31.783333  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reliable_last_estimates = last_estimates[last_estimates[\"adjusted_schedule_time\"] <= 3]\n",
    "reliable_last_estimates.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "actual_arrival_times = reliable_last_estimates[[\"scheduled_date_time\", \"estimated_arrival_time\", \"minutes_off_sched\"]]\n",
    "actual_arrival_times.columns = [\"scheduled_date_time\", \"actual_arrival_time\", \"actual_minutes_off_sched\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with_actual_time = pd.merge(with_scheduled_arrival_time, actual_arrival_times, on=\"scheduled_date_time\", how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>estimate_processing_time</th>\n",
       "      <th>adjusted_schedule_time</th>\n",
       "      <th>estimated_arrival_time</th>\n",
       "      <th>actual_arrival_time</th>\n",
       "      <th>actual_minutes_wait</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>2</td>\n",
       "      <td>2016-10-04 16:00:02</td>\n",
       "      <td>2016-10-04 16:00:15</td>\n",
       "      <td>2.216667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>18</td>\n",
       "      <td>2016-10-04 16:16:02</td>\n",
       "      <td>2016-10-04 16:19:45</td>\n",
       "      <td>21.716667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016-10-04 15:58:02</td>\n",
       "      <td>33</td>\n",
       "      <td>2016-10-04 16:31:02</td>\n",
       "      <td>2016-10-04 16:33:16</td>\n",
       "      <td>35.233333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  estimate_processing_time  adjusted_schedule_time estimated_arrival_time  \\\n",
       "0      2016-10-04 15:58:02                       2    2016-10-04 16:00:02   \n",
       "1      2016-10-04 15:58:02                      18    2016-10-04 16:16:02   \n",
       "2      2016-10-04 15:58:02                      33    2016-10-04 16:31:02   \n",
       "\n",
       "  actual_arrival_time  actual_minutes_wait  \n",
       "0 2016-10-04 16:00:15             2.216667  \n",
       "1 2016-10-04 16:19:45            21.716667  \n",
       "2 2016-10-04 16:33:16            35.233333  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_differences = with_actual_time.loc[:, \\\n",
    "    [\"estimate_processing_time\", \"adjusted_schedule_time\", \"estimated_arrival_time\", \"actual_arrival_time\"]]\n",
    "time_differences[\"actual_minutes_wait\"] = pd.to_timedelta(time_differences[\"actual_arrival_time\"] \\\n",
    "    - time_differences[\"estimate_processing_time\"]).dt.total_seconds() / 60\n",
    "time_differences.dropna()\n",
    "time_differences.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x112fd1588>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/lgaud/anaconda/lib/python3.5/site-packages/matplotlib/collections.py:590: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
      "  if self._edgecolors == str('face'):\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEQCAYAAABFtIg2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X18XGWZ//HP1U5DA7SU0NKCYMGA1AeUlAoorgkrIYBQ\nhSo+IJuy+wN3QYs2pVBbl7JNLQ+2uqD+VtBtI4vPbNnqQtLgkiororXhSeyiBapIWywFoftrTdNc\nvz/OmWQyOSedSWYyD/m+X6955cydc87cJ4Vzzbmv+8HcHRERkShjCl0BEREpXgoSIiISS0FCRERi\nKUiIiEgsBQkREYmlICEiIrEKHiTMbJKZ/cDMfmNmT5nZ6WZWZWbtZva0ma03s0mFrqeIyGhU8CAB\n/DNwn7u/CXgbsBm4Hmh39zcCPw7fi4jICLNCDqYzs8OATnd/Q1r5ZqDW3XeY2TSgw91nFKSSIiKj\nWKGfJI4H/mRmq81sk5ndaWaHAFPdfUe4zw5gauGqKCIyehU6SCSAmcBX3X0m8L+kNS158KijuUNE\nRAogUeDPfx543t1/Gb7/AbAI2G5m09x9u5kdBbyYfqCZKXCIiAyBu1um+xb0ScLdtwN/MLM3hkVn\nA78Gfgg0hmWNwL0xx5ft64Ybbih4HXR9ur7ReH3lfG3u2X+3LvSTBMCngLvNrALYAlwOjAW+Z2Z/\nBzwHXFK46omIjF4FDxLu/hjwjohfnT3SdRERkf4KnbiWGHV1dYWuQl7p+kpbOV9fOV/bUBR0nMRw\nmJmXat1FRArFzPBSSVyLiEhxU5AQEZFYChIiIhJLQUJERGIpSIiISCwFCRERiaUgISIisRQkREQk\nloKEiIjEUpAQEZFYChIiIhJLQUJERGIpSIiISCwFCRERiaUgISIyBG1tbZxzzhzOOWcObW1tI378\nSNF6EiIiWWpra+OiixrZs+dmACorr2Pt2hYaGhpG5PjhyHY9CQUJEZEsnXPOHNrbZwONYUkL9fXr\nWL/+nhE5fji06JCIiORMotAVEBEpNU1NV/LQQ43s2RO8r6y8jqamlhE7fiSpuUlEZAja2tpYufIO\nILjpZ5tPGO7xQ6WchIiIxMo2SBS8ucnMngNeBfYD+9z9NDOrAr4LTAeeAy5x91cKVkkRkVGqGBLX\nDtS5e427nxaWXQ+0u/sbgR+H70VEZIQVQ5AASH/0mQ0kszgtwAdGtjoiUs5KZSBbMSh4TsLMngH+\nTNDc9DV3v9PMXnb3w8PfG7Ar+T7lOOUkRCRrhRzIVgxKLicBnOnu28xsCtBuZptTf+nubmaKBiKS\nEytX3hEGiGAg2549QdloCRLZKniQcPdt4c8/mdla4DRgh5lNc/ftZnYU8GLUsUuXLu3drquro66u\nLv8VFhEpIR0dHXR0dAz5+II2N5nZwcBYd3/NzA4B1gM3AmcDL7n7zWZ2PTDJ3a9PO1bNTSKSNTU3\nldA4CTM7Hlgbvk0Ad7v7irAL7PeA1xPTBVZBQkSGqlAD2YpBSQWJ4VCQEBHJnib4ExEZAnWLjaYg\nISJlYTg3+WSeor19Nu3ts7nookYFipCam0Sk5A03GV3I9R1GWimOkxARGRaNfcgfBQkRGfVKaX2H\nkabmJhEpebkY+zBausWqC6yIjEqj5SY/XAoSIiISS+MkREQkZxQkREQkloKEiBScRjsXL+UkRKSg\nRvusrCNNiWsRKSmjabRzMVDiWkREckYjrkWkoDTaubipuUlECk4D4UaOchIiIhJLOQkREckZBQkR\nyYtsxj5onETxUnOTiORcNmMfNE5iZKm5SUQKrv8iQEEASCamh7PvYPQ0kh/qAisiJS/9aeShhxr1\nNJIjChIiknPZjH3IxTgJLV+aP0XR3GRmY82s08x+GL6vMrN2M3vazNab2aRC11FEMtfQ0MDatcH0\nGvX163q/1Uc1CcXtK8WhKBLXZjYfOBWY4O6zzewWYKe732Jm1wGHu/v1accocS1SQvKZoFbyO3Ml\nN5jOzI4B1gDLgfnufqGZbQZq3X2HmU0DOtx9RtpxChIiJSTfE/lp1HZmsg0SGeUkzOwN7v7MgcqG\n6IvAtcDElLKp7r4j3N4BTM3B54hIGWtoaFBgyINME9f3ADVpZd8naCIaMjO7AHjR3TvNrC5qH3d3\nM4t8ZFi6dGnvdl1dHXV1kacQkRhx377z8a1cE/kVRkdHBx0dHUM+ftDmJjN7E/Bm4FZgAWCAE3zr\nv9bd3zLkTw7O/3ngMqAbGB+e99+BdwB17r7dzI4CHlRzk8jwpN/4gch2/LjyXOUO1CRUWDnNSZjZ\nB4APABcC61J+9RrwHXf/2VArGvFZtcCCMCdxC/CSu99sZtcDk5S4Fhm6qMTujBkz6Oy8nPQcAaBF\ngMpYTnMS7n4vcK+ZvdPdHx527Q4sede/Cfiemf0d8BxwyQh8tkjZihpHsHXrssJWSkrCoEHCzK5z\n95uBj5nZx9J+7e4+L1cVcfcNwIZwexdwdq7OLSIDTZ8+jT17rovMEeQrd6DmptJzoMT1U+HPXxF8\ny099RFFbj0iJiEoar1gR3Pj7btp9eYe1a1siy4dDU2eUpoKPkxgq5SREslPob/H5HichmcnXOIkj\ngYUEPZ0qw2J397/OvooiUgjZjCModECR4pHpOIm7ge8CFwCfAOYCf8pTnUSkgPLVLKRxEqUpo+Ym\nM9vk7jPN7HF3f1tYttHdZ+W9hvF1UnOTSB5k2yyUzVOHnlAKLy/NTUBX+HN7OEr6BeDwbCsnIvm3\nfPlyVq1aDcD8+ZezePHivH1Wtk8dmjqj9GT6JHEh8FPgWOB2gpHRS9193aAH5pGeJEQGWr58OUuW\n3ALcFpbMo7l5YVaBoq2tjdmzL6Or61YAKiquZd26uyJv7sFTx/HAs2HJ8dTXP6tkdBHL15NEu7vv\nBV4B6oZSMRHJv+AJ4jb6mopg1aplQ3ia2Af8S8p2tJ07dwA/Ab4Qlixg586TsvwsKWaZLjr0pJn9\nzMxuNrP3mdlhea2ViBTMypV30NX1JeBh4GG6ur7Um0cYuGhQgiBANIavLzDYd0+tQ116MnqScPcT\nzGw68G6CHk5fNbOX3f2UvNZORLIyf/7lLFmSOhHCPObPX5iTc6c3Q23YcBlvecsbB+w3efIRscdr\nMF0JcvcDvoBjgI8RPH/+HLgPWJTJsfl6BVUXGR1aW1u9vv5ir6+/2FtbWwfdt7m52auqqr2qqtqb\nm5uH9FmVlVMd1jis8crKqd7a2uo1NbVhmYevNV5dfUrkvlHq6y8ecHx9/cVZ10+GJ7x3Zn6vzWgn\n6AEeIZgR1rL5gHy9FCRktIi7aQ/lPJkGmqh9q6qqB9zkq6qqMz6vgkRxyFeQeDvwSYIBdQ8D3wT+\nTzYflOuXgoSMFtneXKNu2rkINDU1ZzpU9Z4Dqrym5syM65GrYCfDk22QyDQn8ZiZPQP8DngP8HGC\nXk5fH15jl4jkUly7f9RU4StX3pFVPmDOnPPo7Oykr9fTXubMOS+reuRj4kDJr0znbtpIsHLczwj6\nu/2Vu2/NZ8VERqv0UcnZTGcRFwxyYcOGTcBXSR2JvWHDOqJ618bVY/36exQYSkym4yTOd/cX435p\nZo3urklYRIYp7hv44sWfYtWqYJGg+fM/lfWNNtt5k+Knz3gCmBNuH59VHaREZdM2FfcCOnNxniw/\nMxfNcyJFJSr/UFNzZsZt+c3NzQ4TU/IGE3t7OGWaYG5tbfWKiim956iomOKtra2x585XDkTyg3wk\nrg94EgUJkZyIChJRvYriEtfB8U0OF4evpqx7EEV1da2pqY0JYLWxwSCb3lQycrINEpk2N4nICIhq\nFpo+fQa7dg3/3JnOwLp16/ORZVGD5LZufT42Ia7J/MqDgoRIEWloaBiQf5g1a1aYpwj2GSyfUFs7\nk/b2/hP81dYuzGq08/Tp09i1a0FKyQKmTz8pJoCdkJMAJkUsm8eOuBfw5VycJ8vPzNnjl0ixiGvL\nH+6AtWzGWgQ5iUkOZzic4RUVk2KbkJR7KD3ko7nJzD4NrAZeJRgbMRO43t3bwrv1J/MRwERGm3x1\nHd2586WMyiB4mlm37jspTVNLez87qglJYx/KW6bNTX/r7l8yswagCrgMuAvQNI4iRSSuq+uiRcuA\n/k1IED+ldzb5BOUeylumU4UnF6h4H3CXuz+Ziw83s/Fm9oiZPWpmT5rZ0rC8yszazexpM1tvZpNy\n8XkixSRq2uympiupqPg08E7gnVRUfJqmpiszPkcyp1FVtYyqqmUsXhyMqZg8eSrB08m68NUYlokc\nQCZtUsAaYD3BtBwHE6xM96ts2rUGOffB4c8EwQyzpwO3AAvD8uuAmyKOy31jncgIGSz3EDVGIdNz\nNDc3x55XuQNxzz4nkenypWMJJvl7xt1fMbMjgNe5++O5ClZmdjDBEqn/QDCBYK277zCzaUCHu89I\n298zqbtIMQqW/ZxN6hQX9fXBasBR5VHLgUado6pqGbt2fYCo5UQz7QIr5S1fy5c68BaCBYf+CTiE\nYC6nYTOzMcAmoJqgl9QvzGyqu+8Id9kB6LlYJAN79rwKtJC6nOgzzxwFKHcgQ5NpkPgqsB94L0GQ\n2B2WzRpuBdy9BzglXBJ1rZm9Ne33bmaRjwxLly7t3a6rq6Ourm641REZEU1NV7Jhw0fo6gpmVK2o\n2ExT03cAYsujzpGepN6/H/qWEw288ML1QOaD6aS8dHR00NHRMeTjMw0Sp7t7jZl1Arj7LjMbN+RP\njeDufzazB4EGYIeZTXP37WZ2FBA5uWBqkBApPeOAvw+3r82gvL+GhoYB3U8vuOBvBuy3b1+Plg4d\nxdK/QN94441ZHZ9pTuIR4F3AxjBYTAHWu3tNVp828LyTge4wz1FJ0KX2JoK1Kl5y95vN7Hpgkrtf\nn3aschJSsnKRk4hy9NHVbNv2KqnNTUcdNZG3vvWUYZ1Xyke2OYlMu8DeDqwFjjSzzwP/DawYQv3S\nHQX8l5k9BvyCIPDcRxAo6s3saeCvw/ciBRHVVbVYTZt2FNBFsDDQvwBdYZnIEGXaDQp4E8ESpp8E\n3pRNF6p8vFAXWBkB2XYdHe503INNiRGlsbHRE4kjPZE40hsbG2NngVUXWEkiT9Ny3OXulwG/iSgT\nKVuDrfSWngTOvt1/H31Lge5LKc8sJzF37lxaWtaSnMyvpWUeZ599GpWV/9Zbh+SI66j8hfIRkpFM\nIglp60UQJLyfyiYa5fqFniRkBGSzhkI2k+jlYiK+sWOnDNh37Ngp3tzc7FVV1V5VVd274JBIErl8\nkjCzzwKLgEozey3lV/uA3CycK1LEorqZwgk5Wkd6eEuB7t/fE1G2n+XLb+99kli+/DpmzZqlpwYZ\nskET1+7+eXefAHzB3SekvKo8rbeRSDlKNtPU16+jvn4da9e2xM551NR0ZRhEWoCWsKknet6l2tqZ\nwJ3A7PB1J7W1M2lqupJEoqn3HIlEU+w5jjrqEGBe774wj3Hjutiz5+Mk52jas+fjQwxgIoFMx0nc\nb2bvSS9095/kuD4iRSdqpHLUgLds2v03bNhEkEtoTCkLusB2d+8hmavo7t7Dxo0bI8+zevUdnH/+\n++npCWZ3HTPmL7z+9dVs2dJ/xPXOnfGzvYocSKZB4lqCqTkgmI7jNOBXBN1TRUah6OTycKe+WLVq\nNcFkBn3jGVatWsbixYsH7NvQ0MB99/1HvwT6okUrgIWkBp9gKRiRoclonIS7X+DuF4aveuCtwCv5\nrZpIcVq58g66um4luBE30tV1a++NOtMxFdk0TQ2moaGB9evv6V2UKGod6qgykUxlOpgu3fME4yZE\nJJTsAtvePpv29tlcdFFjbKBoaGjgkkvOJZFYSCKxkEsuOZeGhgbmz78cuIrkehJwVViWmVwFH5Gk\nTMdJ3J7ydgxwCkFzk0jZS58YL271t7gxFVHNT8uXLx8wxuHEE09k1qxZJBKVdHcHTVmJRBOzZmU+\nj6bGQ0iuZTp309yUt93Ac+7+UL4qlQnN3SQjoa2tjdmzLwubl6Ci4lrWrbsLGDiYLm4+pqamKwfs\ne8QRJ7Br1+dIXwvi1FPfrjmWJK/ysp6Eu68Zco1EStiiRStS8g/Q1RWUbdrUMeAbetQTRm3tp/oF\nmQ0bLusNMiKlINPmpncDNwDHpRzj7v6GPNVLpChs3fo86YPegrKBopp64oLM/PmXs2TJVfRNy/E4\n8+d/llmzZkU2ZYkUSqZdYL8BfJpgBbn9+auOSHE5/PCD2LXrTpK5A5jH4YcfE7t/ehfYj33s6gH7\nbN36fGzuQTkFKTaZBolX3P3+vNZEpChVkD7oDb6U8dHTp09j1675KSXzmT79TaxceQfd3St7z9vd\n3Zfk1jKjUkwy7QL7oJndambvNLOZyVdeayYywqLGOLz88msD9osqizNnznnAXvrWd9gblomUhkyf\nJM4gGHGd3hfvrNxWRyT/otZ6jpvmO+5JIFPB9Bv9R1Bv2LAuthutSLHJtHdTXZ7rITIi4oJB3BiH\nOXPOo7Ozk74Ec/AkEBVosqHcg5SKA00Vfpm732VmTfTN3QRgBL2bVuW1diI5NtgiQlFWr/4B6U8C\nX/nK53nlldsjFxfKdOCdSKk40JPEweHPCUQEibzUSKQA4m7mF1zwNwP23bbtJaAv6ZwaaKKeUqKe\nGLJfxU6kMAYNEu7+tfDn0hGpjUie1dbOpL19XkrJPGprF9LQ0MDixZ9i1aplAMyf/ykaGhoYP34s\nu3f3z0mMGbOfnp6BCwbFPaUkJ99Llc0UHiKFlOlgujcAn2LgYLrZeaqXSF4EieQrCBblAbiCDRs2\nMWtWW+SKbmec8RYeeOCnwJJw///l+OOPYcuW/mMnamsXhufOxvBWphMZCZn2broX+DrwQyC5ZqKa\nm6REnUzfojwtwLOx3+x//vNfE7S2Nof7L+C5514hasGgbPIPwRPNLaQHGpFik2mQ2OPutx14N5Hi\nNtgMrlH27t1PEFD6AsL+/fMH7Ldz50uxTVZR4lami1hbSKSgMg0St5nZDcB64C/JQnfP9vm6HzM7\nFvgmcCTBk8kd7n6bmVUB3wWmA88Bl7i7FjmSYRvsRv7ggx+muzvo6ppI/Iampu/yzDPPsGXLD4Fl\n4RlOoaICuroWpJx1AXASbW3RTVbKM0gpyzRIvBW4jGC50p6U8uEOptsHfMbdHzWzQ4FfmVk7cDnQ\n7u63mNl1wPXhS2RY4m7kGzdupLvbSS5J2t09j40bN/Lud7+dLVv61n2AeRxxxCS2bfsQfXmNRiZP\njm+yigoS6horJcPdD/gCtgAVmew7nBdB7uNsYDMwNSybBmyO2NdFslVff7HDGgcPX2u8vv5ir6qq\nHlBeVVXtEya8fkB5ZeXRXlExyeEMhzO8omKSt7a2xp47TvKY+vqLvbW1dQT/CjKahffOjO/LmT5J\nPAEcDuzIaYRKYWbHATXAIwQBIvlZO4Cp+fpckcHs2bN3QFlX1z7M+vpt9PQE29k+HWgiPykFmQaJ\nw4HNZvZL+nIS7jnqAhs2Nd0DXOPur5n1LZrk7m6p/0emWLp0ae92XV0ddXV1uaiOFLnhTInR1HQl\nGzZ8hK6uIPdQUbGZpqbvsHHjRpYs6T9+Yv78haxe/S22bOlfPmZMD/v2jaevaWoBixYtY9OmhzJO\nXIuMlI6ODjo6OoZ+gkweN4C6iFdtNo8sg5x7HNAGfDqlbDMwLdw+CjU3Sai1tdUrK6eGzTprvLJy\nalZNNa2trV5RMaX3+IqKKd7a2uqtra0+ZszBDsc4HONjxhzcW55IHNJbnkgc4hMmHBvZNDXcuomM\nBLJsbspVLuHhIR5nBL2bvphWfgtwXbh9PXBTxLG5/ttJCci23T/T42tqznSY3HuDh8leU3Omu7s3\nNzd7VVW1V1VVe3Nzs9fU1A44R01N7bDrJjISsg0Sma4ncSDjh3jcmcDHgbPMrDN8nQvcBNSb2dME\nPapuylE9RSJt3bqdoFfSuvDVyNat23t7Q+3a9Tl27focy5ffzpw59SQS1wDvBN5JInENK1YsKmT1\nRfInm4gS9wI6c3GeLD8zZ5FVSkcumpuijq+uPnnAk0R19cmRTwc1NWfGNllF9XoSKSYU6ElCJOei\nVopLrsNQX7+O+vp1Wc+cmhxMV1W1jKqqZSxeHCSXJ06som9kdSPwhbBsoK1bt9PVdWvvvl1dt6aM\n2B5HkND++3BbpLRl2rtJZEQNNpV2Nl1H03tCAZGD6SZPPmLAsZMnHxHZrXX69Bns2jXws1auvCMl\neEBXl2Z2ldKXqyAxcNJ9kWHIxVTaUYFmxowZkecNusZeRldXcGxFxbU0Nd0VuYIcJNeNCPY90PxP\nIqXsQCvT7SZ+tld394nhxhO5rpjIcEUFmq1blxE/Rfc++pYp3ddbGvXkErf0qKbakHJjQR6j9JiZ\nl2rd5cDSnwIqK6/LOv9wzjlzaG8/Hng2LDme6ur72bLleVLnYmpuDtaCaG+fTeoypfX161i//p6s\n6z2cta9F8s3McHc78J6BrJqbzOxIUrq7uvvvszleJFPZTLsdJ2rNBngDUVN054qm2pByk1HvJjOb\nbWa/JfhKtoFg+u7781gvGeWixickezhlqv8qdOuAK3j55dfoa26aE24HAQWuIjn2Aa4Ky0RGt0y7\nwDYT/J/ztLsfD7yXYCI+kbzon08Imp2SzThRXWOj7Ny5g2Dludnhq4WDDtoH3JlSdie1tTO55577\nCR6Sk91Xx4dlIqNbps1N+9x9p5mNMbOx7v6gmf1zXmsmEmGwrrEDJUhfVW737n8kqrkpGHG9ql95\nkOSOr4dyDzIaZPok8bKZTQB+CtxtZrcBu/NXLRntmpquJJG4GjgWOJZE4mqamq6MfcKYO3cu48ZN\nZdy4qcydOxcgcuxDlJ07X2L69GMGlEeVQV+gam+fTXv7bC66qDHrpjCRUpHpk8QHgD3AZ4BLgYnA\njfmqlEiwUtxYgpbOvpXiojz55Eba218hmaBuaQmm9g4S1/2n+T700Em89trApUdXrPgcs2f3Hyex\nYsVdkZ+XizEcIqUioyDh7smnhv3AmrzVRiS0atVq0puFVq1axre+9ZUBYxFefLFnwL53372Qs856\njb7ENcAV7N79ffom8iPc3khDQwPr1t2V0oR0l276ImQYJNIG1VUQTEqzOzmYTmS4oqbPiBI1Avp9\n77t0wH7u+8OtkwnyEhAksb8X/kyWBU8SyXNnEhjiFi4SKUeZPkkcmtw2szEE3ULOyFelZHRpa2tj\n9uyP0NU1A4ANGz7Ce94ziwce6N9UdOGFFwFBU9SvfvVY7/aRRx7Gtm39m5COPPKwyHmXTjjhBDo7\npwHJpHQtkycPZVBmciI/gGuHcLxIaRjyiGsze9TdT8lxfbL5fI24LhMzZ76bzs7/IfXb/aGHjmP3\n7ncBj4Zlp1Bf79TWzmTJks8DbwvLH6e6upotW84hdWR1ff2zrF9/D3PnzuXuu4OurJdeeh4nnngi\nS5b0H2DX3LyQxYsXZ1zfYCT38EdnixRCXkZcm9mclLdjgFMJEtkiwxZ0P+3fVXXv3oXAcfS1ch4H\nPMtNN30FOJi+b/ELeOGFbVRW/lu/KTyamlpYvnw5LS1rSU1oV1cfQ1QX2CxihMiokmkX2AuBC8LX\nOcBrwPvzVSkpfXED3qLKo7qaTplyKFGD3vbu3U/6ug/79o2JXGOif/K7EbiNrVt3Zl3ndE1NV1JZ\neR1BbqMlDErxeRSRkpbJykTAuyPKzsxmdaNcv9DKdEUrbvW3uJXbWltbPZE4rLc8kTgsch3p+vqL\nvbr6lAHl1dWnRNajqqp6wL4TJrw+tm7ZrHjX2trq9fUXe339xVp9TkoKWa5Ml+kNeVNE2YgvWZr2\n+bn7q0lORS35WV9/sdfUnDlgidCamjMjg0QQDJocLg5fTV5TU+vNzc0OB/fuCwd7c3NzZD3i9o26\nwcfVWaTcZBskDrSexDuBdwFHmtl8IJnsmEDmTVUiQHTuYevWZSxatIzu7r7eQt3dC/j9738HPENq\ngvnVV49Jm2MJYD733HN/ZOJ51qxZJBKVdHcH+yYSTcyaNUsztYpk4UCJ6wqCgDA2/Jn0KvDBfFVK\nSltU19OmphYWLVoxYNnP6dOPYevW50kPHvv2NQEr+5W9/PIyXn458zmWVq68g+7uvnN0d8ePjNbY\nB5FogwYJd98AbDCz1e6+dYTqJCVusLUgzj//Q/T0BDfiMWN+zYoV3w+DR//V4iorD+oNMknJBHdU\noMkNjX0QGSCTNimgHZiU8r4KaMumXSvXL5STKFpxSeAgRzAxJScx0Zubm72xsXFA+dlnn+0VFVN6\nyyoqpqQkvweWZ1OPKMpJyGhBLnMSKaa4+yspgWWXmU0dboAys38F3ge86O4nh2VVwHeB6QSLG12S\n+tlSfNKn1IibAC8YJT1wPqZA//JNm5bFzqWU6RxLUVN4KBchkp1Mg8R+M5vuYZOTmR0H9OTg81cD\ntwPfTCm7Hmh391vM7Lrw/fU5+CzJg6j1HWbMmDGEM/VvboL4uZSiyuPWd8hmPqaoPIrIqJfJ4wZw\nLvB74C7g38Ltc7N5ZBnk3McBT6S83wxMDbenAZtjjsvpI5gMTVQzTXX1mx0OcjgmfB2U0qzUv0tq\nY2NjbHmmsh3jMNh5NPZByh35aG5y91YzmwVcSTCZzr3A/8tVoEoz1d13hNs7gGE3a0luZLoa2x//\nuB04iORaEDCP1au/xcSJh4flyeTwZ3j88d+F2/2n2ugrP7Bcre+grrEiA2U6d9MVwDyCZcI6CWaA\nfRj46/xVLQh3ZhY7i9/SpUt7t+vq6qirq8tndUa1qJla1637TmTX0X37xjGwm+pCJk78C/BForuv\nDhw/ISLD19HRQUdHx5CPzzQncQ3wDuBhdz/LzGYAK4b8qYPbYWbT3H27mR0FvBi3Y2qQkPxatGgZ\nXV0Jkt/2u7oWsGjRMlas+BzpXUcrKhjQfbWycjzTpx8T2311ON1alU8QiZf+BfrGG7NbVDTTILHX\n3feYGWY23t03m9lJWX1S5tYRfKVMth/cm6fPkSzEjZZeufIOurpu7S3v6oKqqhvYs6f/WhCnn34a\nCxYsiF0FKzcmAAAPlklEQVQiNNOlQ6OoF5NI/mQaJP5gZocT3LDbzexlgu6pw2Jm3wZqgclm9gfg\nH4GbgO+Z2d+Fn3HJcD9Hhm+wp4D0nkmvvroHqKdvYZ96Nm16dNAlQoe7dKjyCSL5kfWiQ2ZWB0wE\nWt29Kx+VyrAenm3dZeiWL18euVjPb3/7235rNsA8xo3rYd++T5C6CFBV1b289FLmyWgRyY+8LDqU\nyt07sj1GSk96T6YNGzaR/nSwYcMmfv7zjaQPhHO/lmAtiL7AkVx6VERKS9ZBQsrL8uXLw8V5YP78\ny1m8eHHkALmjjz6CoA9B3xKjO3eexJ49ewecs7u7h/TA8cIL6/J6HSKSHwoSo1h6E9KSJUGyecOG\nTQPGHbz44j8CJwELw6NPAhJMnz6ZLVv6J6nHj0+wd2DsEJESpCAxivVf3jNZtoxTT337gH337t0F\nvEJqE9L27ZNYvfoOLrhgDt3dSwBIJPazZMlCli+/Tl1SRcqAgsQoET9aeuCcSVHjDv7yl/Gkd4F9\n8cVraWho4Ec/umfAuWfNmqUuqSJlIOveTcVCvZsy19bWFn7bPxyAROJlfvSje/j2t79NS8v3gbeF\nez5OY+OHWLNmzYCgcsEFf0N39y30BYkWEomF7Nu3I/3jRKSI5b13k5Seq6+eT3f3WJJzKXV3z+Pq\nq+cTLDjYf86khx7aFHmOSy89j5aW/rmHSy9VjyWRcqcgMQo8++yLpOcenn12AWPGjGHgKOqFkb2b\n1q4Ncgp33x0kri+99CLWrFkzMhcgIgWjIFGG0puKenoGLv3R09PDIYdU8tpr/csrKw+KnVV1/fp7\nUFwQGV0UJMpMW1sb559/MT09VQD8+MetjB3bw/79/ZuKxo3r4YQTptPZeQWwICz/Myec8NaRrrKI\nFDElrsvM0Ucfz7ZtL5KajIa/EHwfmBKW/Ynx4yvYu7eLYH2Hvm6t8BdaW/+jX3NTZeV1rF2rHkoi\n5SDbxLWCRJkxm0hw4+8bGQ27gUPTyvYCFfRf96EFmI/7SxkvMCQipUW9m0aRqCk1grUd+iej4TMR\nZfMHPbdmVRURUJAoWcGUGjcChwGE23DUUZPZti19bwd+SN/kfKcwdqxz1lmn8sAD/XMVZ599Wl7r\nLSKlRc1NJWrcuEPo7k6Qmk9IJLpJJHrYu7eC/nmGZHNTX9mhh47ltdd2UV9fzwMPdAJw9tk1tLe3\nj+BViMhIU05ilAjWgPoS/fMJnwa6gTHAm8PypwgGzX0xbd8m3HeOWH1FpDhkGyTG5LMykk9R/3Rj\nCJLRtwEPh6/bIvaDsWP1Ty8iB6acRImYOXMmnZ1bAaipmU4isZ/u7k8AS8I9/kQiMZ79+2HgA9b/\nEjQ7Jc3jrLOUexCRA1OQKAFBgNhC8qmgs3Me8CrBKrLN4V7zSCT+wsSJE9i1q39ASCQq6O7uv6qc\n2ehtqhORzClIlIDgCaL/3EtBt9Yv9ivbu/cz7N3bA1wBJFeCu4Lu7tXAhcAPwrKWlN+LiMRTkChZ\nUXknY+zYMezffzJ9A+daGDMGDjpIiwCJSPbUu6kETJhQxe7du0mOiYA/A/sImpv6d3VtbLyMlpa1\n/cobGy/iox/9qEZQi4i6wJYjM2NgQHiVoDfTpLDsFWpq3s6mTZuYO3cud999PxCsA6EpvUUkqWyC\nhJmdSzAQYCzwdXe/Oe33oyhIHEHUHEs1NdP79XjatCl6wSARkaSymLvJzMYCXwbOBv4I/NLM1rn7\nbwpbs+KioCAi+VaUQQI4Dfiduz8HYGbfAd4PjNIgsYv0cQ5Bc5OISH4Va5B4HfCHlPfPA6cXqC4F\n5+5hXiI5c+urjJamNhEprGINEhndAZcuXdq7XVdXR11dXZ6qU3gKCiIyFB0dHXR0dAz5+KJMXJvZ\nGcBSdz83fL8I6ElNXo+mxLWISK6UywR/G4ETzew4M6sAPoyGCIuIjLiibG5y924z+yTQRtAF9hvq\n2SQiMvKKsrkpE2puEhHJXrk0N4mISBFQkBARkVgKEiIiEktBQkREYilIiIhILAUJERGJpSAhIiKx\nFCRERCSWgoSIiMRSkBARkVgKEiIiEktBQkREYilIiIhILAUJERGJpSAhIiKxFCRERCSWgoSIiMRS\nkBARkVgKEiIiEktBQkREYilIiIhILAUJERGJVbAgYWYfMrNfm9l+M5uZ9rtFZvZbM9tsZucUqo4i\nIqNdIZ8kngAuAn6SWmhmbwY+DLwZOBf4qpmNuieejo6OQlchr3R9pa2cr6+cr20oCnbzdffN7v50\nxK/eD3zb3fe5+3PA74DTRrRyRaDc/0PV9ZW2cr6+cr62oSjGb+hHA8+nvH8eeF2B6iIiMqol8nly\nM2sHpkX86rPu/sMsTuU5qpKIiGTB3At7/zWzB4Emd98Uvr8ewN1vCt+3Aje4+yNpxylwiIgMgbtb\npvvm9UkiC6kVXgd8y8xWETQznQj8Iv2AbC5SRESGppBdYC8ysz8AZwD/aWb3A7j7U8D3gKeA+4Gr\nvNCPOyIio1TBm5tERKR4FWPvpkGNhkF4ZnZueA2/NbPrCl2f4TKzfzWzHWb2REpZlZm1m9nTZrbe\nzCYVso5DZWbHmtmD4X+TT5rZvLC8XK5vvJk9YmaPhte3NCwvi+tLMrOxZtZpZj8M35fN9ZnZc2b2\neHh9vwjLMr6+kgsSlPkgPDMbC3yZ4BreDHzUzN5U2FoN22qC60l1PdDu7m8Efhy+L0X7gM+4+1sI\nmk6vDv+9yuL63H0vcJa7nwKcApxrZqdTJteX4hqCJu5k00o5XZ8Dde5e4+7JMWcZX1/J3URHwSC8\n04Dfuftz7r4P+A7BtZUsd/8p8HJa8WygJdxuAT4wopXKEXff7u6Phtu7gd8QdLgoi+sDcPf/F25W\nAOMIbjplc31mdgxwPvB1+jrRlM31hdI7+mR8fSUXJAZRLoPwXgf8IeV9qV7HgUx19x3h9g5gaiEr\nkwtmdhxQAzxCGV2fmY0xs0cJrmO9u/+CMro+4IvAtUBPSlk5XZ8DD5jZRjO7IizL+PqKpQtsP6N8\nEF4p1nlY3N1LfdyLmR0K3ANc4+6vmfV9cSv163P3HuAUMzsMWGtmb037fclen5ldALzo7p1mVhe1\nTylfX+hMd99mZlOAdjPbnPrLA11fUQYJd68fwmF/BI5NeX9MWFZq0q/jWPo/IZWLHWY2zd23m9lR\nwIuFrtBQmdk4ggBxl7vfGxaXzfUlufufw8GvDZTP9b0LmG1m5wPjgYlmdhflc324+7bw55/MbC1B\nk3bG11fqzU3pg/A+YmYVZnY8MYPwSsBG4EQzO87MKgiS8esKXKd8WAc0htuNwL2D7Fu0LHhk+Abw\nlLt/KeVX5XJ9k5M9X8ysEqgnyLuUxfW5+2fd/Vh3Px74CPBf7n4ZZXJ9ZnawmU0Itw8BziHo/JP5\n9bl7Sb0Iejb9AdgDbAfuT/ndZwkS1puBhkLXdRjXeB7wP+G1LCp0fXJwPd8GXgC6wn+7y4Eq4AHg\naWA9MKnQ9Rzitb2boC37UaAzfJ1bRtd3MrAJeCy8uSwJy8vi+tKutRZYV07XBxwf/rf5KPBk8n6S\nzfVpMJ2IiMQq9eYmERHJIwUJERGJpSAhIiKxFCRERCSWgoSIiMRSkBARkVgKEiIiEktBQoqGmc01\ns9vD7U+Y2WVDOMdhZvYPQzhuqZk1ZXtczLl2D/P458ys6gD7rDGzOUM4d2M4DUPy/Z1lMBW95JGC\nhBQld/+au981hEMPB64aykcO4Zh8nSuT432InzOXYMbk4CTuV7j7b4ZwHhklFCRkxJjZ2nC64ieT\nUxab2eVm9j9m9gjBZGvJfXu/2ZtZh5mdGm5PNrNnw+23hKumdYYrp50A3ARUh2U3h/tda2a/MLPH\nkiurheWLw8/+KXDSAeo+z4LV5x4zs2+HZYea2epw1a/HzOyilP2bwzo9bGZHhmVTzOwHYV1+YWbv\nCsuPCFcHe9LM7iSckyycvyt1Nb8FZnZDarXC8lPDv9FGM2s1s6gZlDGzDwKzgLvNbJMFq851WLjC\no5ntNrNbwnq0m9lp4e+3mNmF4T5jzezWlL/nlYP93aQMFHpuEb1Gzws4PPxZSTAP0NHAVuAIgsVs\nHgJuC/e5AZgfbj8IzAy3JwPPhtu3Ax8LtxMEs3hOB55I+cxzgK+F22OAHwJ/BZwKPB4eMwH4bfLz\nYur+R2BcuD0x/HkzsCpln0nhzx7gfSn7LA63v0UwbTPA6wkmBQS4jb45kc4Pj68Cjku7libgH8Pt\n1cDF4d/tZ8ARYfmHgW8Mch29f8uIv20P4ZxnwL8DbcBY4G1AZ1h+Zcr1HAT8Ejiu0P9t6ZW/V1FO\nFS5l6xozS66AdSxwGfCgu78EYGbfBd6Yxfl+Biy2YGWxf3f334WzsqY6BzjHzDrD94cQzBA8ITxm\nL7DXzNYxcPWuVI8D3zKze+mbMfO9BDdlANz9lXCzy93/M9z+FcHMqQBnA29KqeKEcGbOvyKYuBJ3\nv8/M0lfxS2Vp2ycBbyFYVAaCm/oLgxyffo5UXe7eFm4/Aex19/1m9iRBwILg73ly+FQCMBE4AXju\nAJ8pJUpBQkaEBQu6vBc4w933WrAuwWaCdbx7d4s5vJu+ptHxyUJ3/7aZ/Ry4ALjPzD4BPBtx/Ap3\nvyOtPtcw8IY7mPcB7wEuJAhMJw9y3L6U7R76/j8z4HR370qrS9x5Uq8bgiewqDzEr939XRHlceJy\nGen17oJg0SEzS71XfNLd27P4PClhyknISJkIvBwGiBnAGQQ3vVozq7Jg4Z4P0f8GlrxxPkfQlg7w\nwd5fmr3B3Z9199uB/yCY1vpVgqeEpDbgb8Nv7JjZ6yxYoesnwAfCdvkJBIEm8uYZPp283t07CBaM\nPww4FGgHrk7Zb9IB/gbrgXkp+7893PwJ8LGw7DyC5DsEy0oeGf59DgrrmMoJppSfYmZnhMePM7M3\nE+81gn+LoWoDrkoGDTN7o5kdPIzzSZFTkJCR0gokzOwpYAXwMEGzyNJw+yHg12nHJG/aXwD+wcw2\nEeQvkuWXhEnWToIml2+6+y7gv83sCTO7OfzG+y3gYTN7HPgecKi7dwLfJVgn4T4GX6BqLHBXePwm\n4J/d/c9AM3B4+FmPAnVp9U5uJ9/PA2aFCd9fA58Iy28E3hM261xEkKfB3fcB/xTWbT3wVHrFwn0+\nCNwc1qETeOcg17IG+Jdk4jr9dIO8T25/PazHpjCp/n9Ri0RZ03oSUpQsGC+x0d1bCl0XkdFMTxJS\ndMxsGfAOynPZVpGSoicJkRRm9mXgzLTiL5XaE025XIcUnoKEiIjEUnOTiIjEUpAQEZFYChIiIhJL\nQUJERGIpSIiISKz/DxQQ5L9nBKD/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112fd1c18>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "time_differences.plot(x=\"adjusted_schedule_time\", y=\"actual_minutes_wait\", kind=\"scatter\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the estimates are generally within a couple minutes up to about 13 minutes out, at which point they start to get less reliable. Overall the correlation is pretty good."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}