labs/lab9.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SQL, and the Grammar of Data\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "pd.set_option('display.width', 500)\n",
    "pd.set_option('display.max_columns', 100)\n",
    "pd.set_option('display.notebook_repr_html', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Populating the Database"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets start with Relational Databases, so called because they contain \"relations\" (tables), which are SETS of \"tuples\" (rows) which map \"attributes\" to atomic values.\n",
    "\n",
    "The available attributes are constrained by a \"header\" tuple of attributes which set the type. We do this below here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ourschema=\"\"\"\n",
    "DROP TABLE IF EXISTS \"candidates\";\n",
    "DROP TABLE IF EXISTS \"contributors\";\n",
    "CREATE TABLE \"candidates\" (\n",
    "    \"id\" INTEGER PRIMARY KEY  NOT NULL ,\n",
    "    \"first_name\" VARCHAR,\n",
    "    \"last_name\" VARCHAR,\n",
    "    \"middle_name\" VARCHAR,\n",
    "    \"party\" VARCHAR NOT NULL\n",
    ");\n",
    "CREATE TABLE \"contributors\" (\n",
    "    \"id\" INTEGER PRIMARY KEY  AUTOINCREMENT  NOT NULL,\n",
    "    \"last_name\" VARCHAR,\n",
    "    \"first_name\" VARCHAR,\n",
    "    \"middle_name\" VARCHAR,\n",
    "    \"street_1\" VARCHAR,\n",
    "    \"street_2\" VARCHAR,\n",
    "    \"city\" VARCHAR,\n",
    "    \"state\" VARCHAR,\n",
    "    \"zip\" VARCHAR,\n",
    "    \"amount\" INTEGER,\n",
    "    \"date\" DATETIME,\n",
    "    \"candidate_id\" INTEGER NOT NULL,\n",
    "    FOREIGN KEY(candidate_id) REFERENCES candidates(id)\n",
    ");\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SQLITE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use sqlite here (and recommend Postgres for production purposes). Still sqlite is great for on-disk large databases which wont fit into memory. \n",
    "\n",
    "Its also built into Python, but to use the [command line tool](https://www.sqlite.org/cli.html), I recommend you install it: https://www.sqlite.org/download.html. I also recommend you download and install the sqlite browser: http://sqlitebrowser.org .\n",
    "\n",
    "Python implements a standard database API over all databases. Its called [DBAPI2](http://cewing.github.io/training.codefellows/lectures/day21/intro_to_dbapi2.html). It works across many SQL databases.\n",
    "\n",
    "There is an even higher level API available, called [SQLAlchemy](http://www.sqlalchemy.org). While we wont use it here, I thoroughly recommend it, either in its direct relational form, or ORM form. Many things in Pandas use it to interface with databases. Here we'll get away with things by using SQLITE."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "Sqlite is a file or memory based database. Connect and get a DBAPI2 connection.\n",
    "\n",
    "DBAPI2, or the Python Database API version 2, allows programmers to easily insert and recieve data from a database. It is used by many other libraries, including Pandas.\n",
    "\n",
    "Sqlite differes from other databases in that the process model for the others (such as mysql and postgresql) is one of a \"server\" which serves out database requests to a client. In postgres, for eg, the server has a networking thingamajammy called a socket which takes a network connection from a client and returns results to the client over the network.\n",
    "\n",
    "Sqlite on the other hand runs within the context of the python process calling the library (or the C process as in the command line client). This means that when multiple processes on the machine use the sqlite library to access a database, what we will see is multiple processes contending for the database file (or parts of the file: as we shall see later, it is possible to co-operatively lock parts of a file). In contrast, in postgres, multiple processes connecting to the database must all funnel through the postgres server which will have its own mechanisms in serving the requests (the default in postgres is to spawn multiple child processes of the server).\n",
    "\n",
    "The DBAPI has a few core concepts:\n",
    "\n",
    "- connection objects: `sqlite_db = sq3.connect(os.path.join(PATHSTART, dbfile))`\n",
    "- cursor objects: for traversing databases, making a query, and returning results: `c=db.cursor().execute(sel)`\n",
    "- queries: sents as strings of SQL\n",
    "- output results: always an array of tuples\n",
    "\n",
    "Sqlite is completely ACID compliant. This means database transactions are:\n",
    "\n",
    "- Atomic: wither a transaction (something upto a `commit`) all happens or not\n",
    "- Consistent: the database is not left in an inconsistent state as defined by certain rules. this does not mean correct.\n",
    "- Isolated: concurrent execution is identical to serial execution: the effect of an in-progress transaction should not be available to another transaction, for example\n",
    "- Durable: a power loff dor example should not violate the state of the database once a transaction has been committed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sqlite3 import dbapi2 as sq3\n",
    "import os\n",
    "PATHSTART=\".\"\n",
    "def get_db(dbfile):\n",
    "    sqlite_db = sq3.connect(os.path.join(PATHSTART, dbfile))\n",
    "    return sqlite_db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop tables if they exist and create them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def init_db(dbfile, schema):\n",
    "    \"\"\"Creates the database tables.\"\"\"\n",
    "    db = get_db(dbfile)\n",
    "    db.cursor().executescript(schema)\n",
    "    db.commit()\n",
    "    return db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use Pandas to read in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>party</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>33</td>\n",
       "      <td>Joseph</td>\n",
       "      <td>Biden</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>36</td>\n",
       "      <td>Samuel</td>\n",
       "      <td>Brownback</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>34</td>\n",
       "      <td>Hillary</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>R.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>39</td>\n",
       "      <td>Christopher</td>\n",
       "      <td>Dodd</td>\n",
       "      <td>J.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>26</td>\n",
       "      <td>John</td>\n",
       "      <td>Edwards</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>22</td>\n",
       "      <td>Rudolph</td>\n",
       "      <td>Giuliani</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>24</td>\n",
       "      <td>Mike</td>\n",
       "      <td>Gravel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>16</td>\n",
       "      <td>Mike</td>\n",
       "      <td>Huckabee</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>30</td>\n",
       "      <td>Duncan</td>\n",
       "      <td>Hunter</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>31</td>\n",
       "      <td>Dennis</td>\n",
       "      <td>Kucinich</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>37</td>\n",
       "      <td>John</td>\n",
       "      <td>McCain</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>20</td>\n",
       "      <td>Barack</td>\n",
       "      <td>Obama</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>32</td>\n",
       "      <td>Ron</td>\n",
       "      <td>Paul</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>29</td>\n",
       "      <td>Bill</td>\n",
       "      <td>Richardson</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>35</td>\n",
       "      <td>Mitt</td>\n",
       "      <td>Romney</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>38</td>\n",
       "      <td>Tom</td>\n",
       "      <td>Tancredo</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>41</td>\n",
       "      <td>Fred</td>\n",
       "      <td>Thompson</td>\n",
       "      <td>D.</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id   first_name   last_name middle_name party\n",
       "0   33       Joseph       Biden         NaN     D\n",
       "1   36       Samuel   Brownback         NaN     R\n",
       "2   34      Hillary     Clinton          R.     D\n",
       "3   39  Christopher        Dodd          J.     D\n",
       "4   26         John     Edwards         NaN     D\n",
       "5   22      Rudolph    Giuliani         NaN     R\n",
       "6   24         Mike      Gravel         NaN     D\n",
       "7   16         Mike    Huckabee         NaN     R\n",
       "8   30       Duncan      Hunter         NaN     R\n",
       "9   31       Dennis    Kucinich         NaN     D\n",
       "10  37         John      McCain         NaN     R\n",
       "11  20       Barack       Obama         NaN     D\n",
       "12  32          Ron        Paul         NaN     R\n",
       "13  29         Bill  Richardson         NaN     D\n",
       "14  35         Mitt      Romney         NaN     R\n",
       "15  38          Tom    Tancredo         NaN     R\n",
       "16  41         Fred    Thompson          D.     R"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcand=pd.read_csv(\"./candidates.txt\", sep='|')\n",
    "dfcand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0 NaN      Agee     Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1 NaN    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   250.0  2007-05-16            16\n",
       "2 NaN    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566    50.0  2007-06-18            16\n",
       "3 NaN    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   100.0  2007-06-21            16\n",
       "4 NaN      Akin    Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci=pd.read_csv(\"./contributors_with_candidate_id.txt\", sep=\"|\")\n",
    "dfcwci.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0      Agee     Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   250.0  2007-05-16            16\n",
       "2    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566    50.0  2007-06-18            16\n",
       "3    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   100.0  2007-06-21            16\n",
       "4      Akin    Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del dfcwci['id']\n",
    "dfcwci.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initializing the database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "db=init_db(\"/tmp/cancont.db\", ourschema)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Populating with Pandas!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dfcand.to_sql(\"candidates\", db, if_exists=\"append\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dfcwci.to_sql(\"contributors\", db, if_exists=\"append\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(175, 11)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sel=\"\"\"\n",
    "SELECT * FROM candidates;\n",
    "\"\"\"\n",
    "c=db.cursor().execute(sel)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(16, 'Mike', 'Huckabee', None, 'R'),\n",
       " (20, 'Barack', 'Obama', None, 'D'),\n",
       " (22, 'Rudolph', 'Giuliani', None, 'R'),\n",
       " (24, 'Mike', 'Gravel', None, 'D'),\n",
       " (26, 'John', 'Edwards', None, 'D'),\n",
       " (29, 'Bill', 'Richardson', None, 'D'),\n",
       " (30, 'Duncan', 'Hunter', None, 'R'),\n",
       " (31, 'Dennis', 'Kucinich', None, 'D'),\n",
       " (32, 'Ron', 'Paul', None, 'R'),\n",
       " (33, 'Joseph', 'Biden', None, 'D'),\n",
       " (34, 'Hillary', 'Clinton', 'R.', 'D'),\n",
       " (35, 'Mitt', 'Romney', None, 'R'),\n",
       " (36, 'Samuel', 'Brownback', None, 'R'),\n",
       " (37, 'John', 'McCain', None, 'R'),\n",
       " (38, 'Tom', 'Tancredo', None, 'R'),\n",
       " (39, 'Christopher', 'Dodd', 'J.', 'D'),\n",
       " (41, 'Fred', 'Thompson', 'D.', 'R')]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c.fetchall()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rem=\"\"\"\n",
    "DELETE FROM candidates;\n",
    "\"\"\"\n",
    "c=db.cursor().execute(rem)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c.fetchall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Or populate with SQL INSERT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![xkcd-sqlinj](http://imgs.xkcd.com/comics/exploits_of_a_mom.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def format_as_sql(qst, t):\n",
    "    return qst.replace('?', '{}').format(*t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "33 Joseph Biden  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (33,Joseph,Biden,,D);\n",
      "\n",
      "36 Samuel Brownback  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (36,Samuel,Brownback,,R);\n",
      "\n",
      "34 Hillary Clinton R. D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (34,Hillary,Clinton,R.,D);\n",
      "\n",
      "39 Christopher Dodd J. D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (39,Christopher,Dodd,J.,D);\n",
      "\n",
      "26 John Edwards  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (26,John,Edwards,,D);\n",
      "\n",
      "22 Rudolph Giuliani  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (22,Rudolph,Giuliani,,R);\n",
      "\n",
      "24 Mike Gravel  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (24,Mike,Gravel,,D);\n",
      "\n",
      "16 Mike Huckabee  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (16,Mike,Huckabee,,R);\n",
      "\n",
      "30 Duncan Hunter  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (30,Duncan,Hunter,,R);\n",
      "\n",
      "31 Dennis Kucinich  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (31,Dennis,Kucinich,,D);\n",
      "\n",
      "37 John McCain  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (37,John,McCain,,R);\n",
      "\n",
      "20 Barack Obama  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (20,Barack,Obama,,D);\n",
      "\n",
      "32 Ron Paul  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (32,Ron,Paul,,R);\n",
      "\n",
      "29 Bill Richardson  D\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (29,Bill,Richardson,,D);\n",
      "\n",
      "35 Mitt Romney  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (35,Mitt,Romney,,R);\n",
      "\n",
      "38 Tom Tancredo  R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (38,Tom,Tancredo,,R);\n",
      "\n",
      "41 Fred Thompson D. R\n",
      "\n",
      "INSERT INTO candidates (id, first_name, last_name, middle_name, party)     VALUES (41,Fred,Thompson,D.,R);\n",
      "\n"
     ]
    }
   ],
   "source": [
    "ins=\"\"\"\n",
    "INSERT INTO candidates (id, first_name, last_name, middle_name, party) \\\n",
    "    VALUES (?,?,?,?,?);\n",
    "\"\"\"\n",
    "with open(\"candidates.txt\") as fd:\n",
    "    slines =[l.strip().split('|') for l in fd.readlines()]\n",
    "    for line in slines[1:]:\n",
    "        theid, first_name, last_name, middle_name, party = line\n",
    "        print(theid, first_name, last_name, middle_name, party)\n",
    "        valstoinsert = (int(theid), first_name, last_name, middle_name, party)\n",
    "        print(format_as_sql(ins, valstoinsert))\n",
    "        db.cursor().execute(ins, valstoinsert)\n",
    "        \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_query(sel):\n",
    "    c=db.cursor().execute(sel)\n",
    "    return c.fetchall()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(16, 'Mike', 'Huckabee', '', 'R'),\n",
       " (20, 'Barack', 'Obama', '', 'D'),\n",
       " (22, 'Rudolph', 'Giuliani', '', 'R'),\n",
       " (24, 'Mike', 'Gravel', '', 'D'),\n",
       " (26, 'John', 'Edwards', '', 'D'),\n",
       " (29, 'Bill', 'Richardson', '', 'D'),\n",
       " (30, 'Duncan', 'Hunter', '', 'R'),\n",
       " (31, 'Dennis', 'Kucinich', '', 'D'),\n",
       " (32, 'Ron', 'Paul', '', 'R'),\n",
       " (33, 'Joseph', 'Biden', '', 'D'),\n",
       " (34, 'Hillary', 'Clinton', 'R.', 'D'),\n",
       " (35, 'Mitt', 'Romney', '', 'R'),\n",
       " (36, 'Samuel', 'Brownback', '', 'R'),\n",
       " (37, 'John', 'McCain', '', 'R'),\n",
       " (38, 'Tom', 'Tancredo', '', 'R'),\n",
       " (39, 'Christopher', 'Dodd', 'J.', 'D'),\n",
       " (41, 'Fred', 'Thompson', 'D.', 'R')]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_query(\"SELECT * FROM candidates;\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rem=\"\"\"\n",
    "DELETE FROM candidates;\n",
    "\"\"\"\n",
    "c=db.cursor().execute(rem)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####Bulk insert"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You almost always have to do this from the command line. Its typically faster, but also different foe every database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "tail -n +2 candidates.txt > candidates_nohead.txt\n",
    "echo \".import candidates_nohead.txt candidates\" | sqlite3 cancont.db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(16, u'Mike', u'Huckabee', u'', u'R'),\n",
       " (20, u'Barack', u'Obama', u'', u'D'),\n",
       " (22, u'Rudolph', u'Giuliani', u'', u'R'),\n",
       " (24, u'Mike', u'Gravel', u'', u'D'),\n",
       " (26, u'John', u'Edwards', u'', u'D'),\n",
       " (29, u'Bill', u'Richardson', u'', u'D'),\n",
       " (30, u'Duncan', u'Hunter', u'', u'R'),\n",
       " (31, u'Dennis', u'Kucinich', u'', u'D'),\n",
       " (32, u'Ron', u'Paul', u'', u'R'),\n",
       " (33, u'Joseph', u'Biden', u'', u'D'),\n",
       " (34, u'Hillary', u'Clinton', u'R.', u'D'),\n",
       " (35, u'Mitt', u'Romney', u'', u'R'),\n",
       " (36, u'Samuel', u'Brownback', u'', u'R'),\n",
       " (37, u'John', u'McCain', u'', u'R'),\n",
       " (38, u'Tom', u'Tancredo', u'', u'R'),\n",
       " (39, u'Christopher', u'Dodd', u'J.', u'D'),\n",
       " (41, u'Fred', u'Thompson', u'D.', u'R')]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_query(\"SELECT * FROM candidates;\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Single Table Verbs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now focus on core data manipulation commands. The reason to do this is that they are *universal across systems, and by identifying them, we can quickly ask how to do these* when we encounter a new system."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See https://gist.github.com/TomAugspurger/6e052140eaa5fdb6e8c0/ which has a comparison of r/dplyr and pandas. I stole and modified this table from there:\n",
    "\n",
    "``dplyr`` has a small set of nicely defined verbs. I've listed their closest pandas verbs.\n",
    "\n",
    "\n",
    "<table>\n",
    "  <tr>\n",
    "    <th><b>VERB</b></th>\n",
    "    <th><b>dplyr</b></th>\n",
    "    <th><b>pandas</b></th>\n",
    "    <th><b>SQL</b></th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>QUERY/SELECTION</td>\n",
    "    <td>filter() (and slice())</td>\n",
    "    <td>query() (and loc[], iloc[])</td>\n",
    "    <td>SELECT WHERE</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>SORT</td>\n",
    "    <td>arrange()</td>\n",
    "    <td>sort()</td>\n",
    "    <td>ORDER BY</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>SELECT-COLUMNS/PROJECTION</td>\n",
    "    <td>select() (and rename())</td>\n",
    "    <td>[](__getitem__) (and rename())</td>\n",
    "    <td>SELECT COLUMN</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>SELECT-DISTINCT</td>\n",
    "    <td>distinct()</td>\n",
    "    <td>unique(),drop_duplicates()</td>\n",
    "    <td>SELECT DISTINCT COLUMN</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>ASSIGN</td>\n",
    "    <td>mutate() (and transmute())</td>\n",
    "    <td>assign</td>\n",
    "    <td>ALTER/UPDATE</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>AGGREGATE</td>\n",
    "    <td>summarise()</td>\n",
    "    <td>describe(), mean(), max()</td>\n",
    "    <td>None, AVG(),MAX()</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>SAMPLE</td>\n",
    "    <td>sample_n() and sample_frac()</td>\n",
    "    <td>sample()</td>\n",
    "    <td>implementation dep, use RAND()</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>GROUP-AGG</td>\n",
    "    <td>group_by/summarize</td>\n",
    "    <td>groupby/agg, count, mean</td>\n",
    "    <td>GROUP BY</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>DELETE</td>\n",
    "    <td>?</td>\n",
    "    <td>drop/masking</td>\n",
    "    <td>DELETE/WHERE</td>\n",
    "  </tr>\n",
    "</table>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0      Agee     Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   250.0  2007-05-16            16\n",
       "2    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566    50.0  2007-06-18            16\n",
       "3    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   100.0  2007-06-21            16\n",
       "4      Akin    Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### QUERY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id\n",
       "27    Buckheit      Bruce         NaN                8904 KAREN DR      NaN          FAIRFAX    VA  220312731  100.00  2007-09-19            20\n",
       "77   Ranganath      Anoop         NaN           2507 Willard Drive      NaN  Charlottesville    VA      22903 -100.00  2008-04-21            32\n",
       "88   Perreault     Louise         NaN    503 Brockridge Hunt Drive      NaN          Hampton    VA      23666  -34.08  2008-04-21            32\n",
       "145    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340      NaN          FAIRFAX    VA  220308440   50.00  2007-09-30            35"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.query(\"state=='VA' & amount < 400\") #in pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id\n",
       "27    Buckheit      Bruce         NaN                8904 KAREN DR      NaN          FAIRFAX    VA  220312731  100.00  2007-09-19            20\n",
       "77   Ranganath      Anoop         NaN           2507 Willard Drive      NaN  Charlottesville    VA      22903 -100.00  2008-04-21            32\n",
       "88   Perreault     Louise         NaN    503 Brockridge Hunt Drive      NaN          Hampton    VA      23666  -34.08  2008-04-21            32\n",
       "145    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340      NaN          FAIRFAX    VA  220308440   50.00  2007-09-30            35"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[(dfcwci.state=='VA') & (dfcwci.amount < 400)] #another way in pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 'id', 'INTEGER', 1, None, 1),\n",
       " (1, 'last_name', 'VARCHAR', 0, None, 0),\n",
       " (2, 'first_name', 'VARCHAR', 0, None, 0),\n",
       " (3, 'middle_name', 'VARCHAR', 0, None, 0),\n",
       " (4, 'street_1', 'VARCHAR', 0, None, 0),\n",
       " (5, 'street_2', 'VARCHAR', 0, None, 0),\n",
       " (6, 'city', 'VARCHAR', 0, None, 0),\n",
       " (7, 'state', 'VARCHAR', 0, None, 0),\n",
       " (8, 'zip', 'VARCHAR', 0, None, 0),\n",
       " (9, 'amount', 'INTEGER', 0, None, 0),\n",
       " (10, 'date', 'DATETIME', 0, None, 0),\n",
       " (11, 'candidate_id', 'INTEGER', 1, None, 0)]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_query(\"PRAGMA table_info(contributors);\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['id',\n",
       " 'last_name',\n",
       " 'first_name',\n",
       " 'middle_name',\n",
       " 'street_1',\n",
       " 'street_2',\n",
       " 'city',\n",
       " 'state',\n",
       " 'zip',\n",
       " 'amount',\n",
       " 'date',\n",
       " 'candidate_id']"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cont_cols = [e[1] for e in make_query(\"PRAGMA table_info(contributors);\")]\n",
    "cont_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_frame(list_of_tuples, legend=cont_cols):\n",
    "    framelist=[]\n",
    "    for i, cname in enumerate(legend):\n",
    "        framelist.append((cname,[e[i] for e in list_of_tuples]))\n",
    "    return pd.DataFrame.from_items(framelist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(28, 'Buckheit', 'Bruce', None, '8904 KAREN DR', None, 'FAIRFAX', 'VA', '220312731', 100, '2007-09-19', 20), (78, 'Ranganath', 'Anoop', None, '2507 Willard Drive', None, 'Charlottesville', 'VA', '22903', -100, '2008-04-21', 32), (89, 'Perreault', 'Louise', None, '503 Brockridge Hunt Drive', None, 'Hampton', 'VA', '23666', -34.08, '2008-04-21', 32), (146, 'ABDELLA', 'THOMAS', 'M.', '4231 MONUMENT WALL WAY #340', None, 'FAIRFAX', 'VA', '220308440', 50, '2007-09-30', 35)]\n"
     ]
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE state='VA' AND amount < 400;\")\n",
    "print(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28</td>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>None</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>78</td>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>None</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>89</td>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>None</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>146</td>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id\n",
       "0   28   Buckheit      Bruce        None                8904 KAREN DR     None          FAIRFAX    VA  220312731  100.00  2007-09-19            20\n",
       "1   78  Ranganath      Anoop        None           2507 Willard Drive     None  Charlottesville    VA      22903 -100.00  2008-04-21            32\n",
       "2   89  Perreault     Louise        None    503 Brockridge Hunt Drive     None          Hampton    VA      23666  -34.08  2008-04-21            32\n",
       "3  146    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340     None          FAIRFAX    VA  220308440   50.00  2007-09-30            35"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_frame(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>126</td>\n",
       "      <td>BOURNE</td>\n",
       "      <td>TRAVIS</td>\n",
       "      <td>None</td>\n",
       "      <td>LAGE KAART 77</td>\n",
       "      <td>None</td>\n",
       "      <td>BRASSCHATT</td>\n",
       "      <td>None</td>\n",
       "      <td>2930</td>\n",
       "      <td>-500</td>\n",
       "      <td>2008-11-20</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id last_name first_name middle_name       street_1 street_2        city state   zip  amount        date  candidate_id\n",
       "0  126    BOURNE     TRAVIS        None  LAGE KAART 77     None  BRASSCHATT  None  2930    -500  2008-11-20            35"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE state IS NULL;\")\n",
    "make_frame(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>BOURNE</td>\n",
       "      <td>TRAVIS</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LAGE KAART 77</td>\n",
       "      <td>NaN</td>\n",
       "      <td>BRASSCHATT</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2930</td>\n",
       "      <td>-500.0</td>\n",
       "      <td>2008-11-20</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    last_name first_name middle_name       street_1 street_2        city state   zip  amount        date  candidate_id\n",
       "125    BOURNE     TRAVIS         NaN  LAGE KAART 77      NaN  BRASSCHATT   NaN  2930  -500.0  2008-11-20            35"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.state.isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(174, 12)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE state IS NOT NULL;\")\n",
    "make_frame(out).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(174, 11)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.state.notnull()].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>None</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.00</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>None</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>63</td>\n",
       "      <td>BURKE</td>\n",
       "      <td>SUZANNE</td>\n",
       "      <td>M.</td>\n",
       "      <td>3401 EVANSTON</td>\n",
       "      <td>None</td>\n",
       "      <td>SEATTLE</td>\n",
       "      <td>WA</td>\n",
       "      <td>981038677</td>\n",
       "      <td>-700.00</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>78</td>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>None</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>89</td>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>None</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>101</td>\n",
       "      <td>Aaronson</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>2000 Village Green Dr Apt 12</td>\n",
       "      <td>None</td>\n",
       "      <td>Mill Creek</td>\n",
       "      <td>WA</td>\n",
       "      <td>980125787</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2008-02-08</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>107</td>\n",
       "      <td>Aaronson</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>2000 Village Green Dr Apt 12</td>\n",
       "      <td>None</td>\n",
       "      <td>Mill Creek</td>\n",
       "      <td>WA</td>\n",
       "      <td>980125787</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2008-02-14</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>146</td>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  last_name first_name middle_name                      street_1 street_2             city state        zip  amount        date  candidate_id\n",
       "0    1       Agee     Steven        None        549 Laurel Branch Road     None            Floyd    VA      24091  500.00  2007-06-30            16\n",
       "1   28   Buckheit      Bruce        None                 8904 KAREN DR     None          FAIRFAX    VA  220312731  100.00  2007-09-19            20\n",
       "2   63      BURKE    SUZANNE          M.                 3401 EVANSTON     None          SEATTLE    WA  981038677 -700.00  2008-03-05            22\n",
       "3   78  Ranganath      Anoop        None            2507 Willard Drive     None  Charlottesville    VA      22903 -100.00  2008-04-21            32\n",
       "4   89  Perreault     Louise        None     503 Brockridge Hunt Drive     None          Hampton    VA      23666  -34.08  2008-04-21            32\n",
       "5  101   Aaronson    Rebecca        None  2000 Village Green Dr Apt 12     None       Mill Creek    WA  980125787  100.00  2008-02-08            34\n",
       "6  107   Aaronson    Rebecca        None  2000 Village Green Dr Apt 12     None       Mill Creek    WA  980125787  100.00  2008-02-14            34\n",
       "7  146    ABDELLA     THOMAS          M.   4231 MONUMENT WALL WAY #340     None          FAIRFAX    VA  220308440   50.00  2007-09-30            35"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE state IN ('VA','WA');\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.00</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>BURKE</td>\n",
       "      <td>SUZANNE</td>\n",
       "      <td>M.</td>\n",
       "      <td>3401 EVANSTON</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SEATTLE</td>\n",
       "      <td>WA</td>\n",
       "      <td>981038677</td>\n",
       "      <td>-700.00</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>Aaronson</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2000 Village Green Dr Apt 12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Mill Creek</td>\n",
       "      <td>WA</td>\n",
       "      <td>980125787</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2008-02-08</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>Aaronson</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2000 Village Green Dr Apt 12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Mill Creek</td>\n",
       "      <td>WA</td>\n",
       "      <td>980125787</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2008-02-14</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name first_name middle_name                      street_1 street_2             city state        zip  amount        date  candidate_id\n",
       "0         Agee     Steven         NaN        549 Laurel Branch Road      NaN            Floyd    VA      24091  500.00  2007-06-30            16\n",
       "27    Buckheit      Bruce         NaN                 8904 KAREN DR      NaN          FAIRFAX    VA  220312731  100.00  2007-09-19            20\n",
       "62       BURKE    SUZANNE          M.                 3401 EVANSTON      NaN          SEATTLE    WA  981038677 -700.00  2008-03-05            22\n",
       "77   Ranganath      Anoop         NaN            2507 Willard Drive      NaN  Charlottesville    VA      22903 -100.00  2008-04-21            32\n",
       "88   Perreault     Louise         NaN     503 Brockridge Hunt Drive      NaN          Hampton    VA      23666  -34.08  2008-04-21            32\n",
       "100   Aaronson    Rebecca         NaN  2000 Village Green Dr Apt 12      NaN       Mill Creek    WA  980125787  100.00  2008-02-08            34\n",
       "106   Aaronson    Rebecca         NaN  2000 Village Green Dr Apt 12      NaN       Mill Creek    WA  980125787  100.00  2008-02-14            34\n",
       "145    ABDELLA     THOMAS          M.   4231 MONUMENT WALL WAY #340      NaN          FAIRFAX    VA  220308440   50.00  2007-09-30            35"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.state.isin(['VA','WA'])].head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>None</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>None</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19</td>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>None</td>\n",
       "      <td>412 Dakota Avenue</td>\n",
       "      <td>None</td>\n",
       "      <td>Springfield</td>\n",
       "      <td>OH</td>\n",
       "      <td>45504</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-06-28</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26</td>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>None</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>None</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-07-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>27</td>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>None</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>None</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>25</td>\n",
       "      <td>2007-08-16</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>Buck</td>\n",
       "      <td>Barbara</td>\n",
       "      <td>None</td>\n",
       "      <td>1780 NE 138th St</td>\n",
       "      <td>None</td>\n",
       "      <td>North Miami</td>\n",
       "      <td>FL</td>\n",
       "      <td>331811316</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-09-13</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>36</td>\n",
       "      <td>Buck</td>\n",
       "      <td>Barbara</td>\n",
       "      <td>None</td>\n",
       "      <td>1780 NE 138th St</td>\n",
       "      <td>None</td>\n",
       "      <td>North Miami</td>\n",
       "      <td>FL</td>\n",
       "      <td>331811316</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-07-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>39</td>\n",
       "      <td>Buchanek</td>\n",
       "      <td>Elizabeth</td>\n",
       "      <td>None</td>\n",
       "      <td>7917 Kentbury Dr</td>\n",
       "      <td>None</td>\n",
       "      <td>Bethesda</td>\n",
       "      <td>MD</td>\n",
       "      <td>208144615</td>\n",
       "      <td>50</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>50</td>\n",
       "      <td>Harrison</td>\n",
       "      <td>Ryan</td>\n",
       "      <td>None</td>\n",
       "      <td>2247 3rd St</td>\n",
       "      <td>None</td>\n",
       "      <td>La Verne</td>\n",
       "      <td>CA</td>\n",
       "      <td>917504918</td>\n",
       "      <td>25</td>\n",
       "      <td>2007-07-26</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>102</td>\n",
       "      <td>Aarons</td>\n",
       "      <td>Elaine</td>\n",
       "      <td>None</td>\n",
       "      <td>481 Buck Island Rd Apt 17A</td>\n",
       "      <td>APT 17A</td>\n",
       "      <td>West Yarmouth</td>\n",
       "      <td>MA</td>\n",
       "      <td>26733300</td>\n",
       "      <td>25</td>\n",
       "      <td>2008-02-26</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>105</td>\n",
       "      <td>Aaron</td>\n",
       "      <td>Shirley</td>\n",
       "      <td>None</td>\n",
       "      <td>101 Cherry Ave</td>\n",
       "      <td>None</td>\n",
       "      <td>Havana</td>\n",
       "      <td>FL</td>\n",
       "      <td>323331311</td>\n",
       "      <td>50</td>\n",
       "      <td>2008-02-29</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id last_name first_name middle_name                    street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "0    3    Ahrens        Don        None        4034 Rennellwood Way     None     Pleasanton    CA      94566      50  2007-06-18            16\n",
       "1   19     Ardle    William        None           412 Dakota Avenue     None    Springfield    OH      45504      50  2007-06-28            16\n",
       "2   26   Buckler      Steve        None             24351 Armada Dr     None     Dana Point    CA  926291306      50  2007-07-30            20\n",
       "3   27   Buckler      Steve        None             24351 Armada Dr     None     Dana Point    CA  926291306      25  2007-08-16            20\n",
       "4   35      Buck    Barbara        None            1780 NE 138th St     None    North Miami    FL  331811316      50  2007-09-13            20\n",
       "5   36      Buck    Barbara        None            1780 NE 138th St     None    North Miami    FL  331811316      50  2007-07-19            20\n",
       "6   39  Buchanek  Elizabeth        None            7917 Kentbury Dr     None       Bethesda    MD  208144615      50  2007-09-30            20\n",
       "7   50  Harrison       Ryan        None                 2247 3rd St     None       La Verne    CA  917504918      25  2007-07-26            20\n",
       "8  102    Aarons     Elaine        None  481 Buck Island Rd Apt 17A  APT 17A  West Yarmouth    MA   26733300      25  2008-02-26            34\n",
       "9  105     Aaron    Shirley        None              101 Cherry Ave     None         Havana    FL  323331311      50  2008-02-29            34"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE amount BETWEEN 10 AND 50;\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>NaN</td>\n",
       "      <td>412 Dakota Avenue</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Springfield</td>\n",
       "      <td>OH</td>\n",
       "      <td>45504</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-28</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-07-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>25.0</td>\n",
       "      <td>2007-08-16</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Barbara</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1780 NE 138th St</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Miami</td>\n",
       "      <td>FL</td>\n",
       "      <td>331811316</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-09-13</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Barbara</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1780 NE 138th St</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Miami</td>\n",
       "      <td>FL</td>\n",
       "      <td>331811316</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-07-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Buchanek</td>\n",
       "      <td>Elizabeth</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7917 Kentbury Dr</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bethesda</td>\n",
       "      <td>MD</td>\n",
       "      <td>208144615</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Harrison</td>\n",
       "      <td>Ryan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2247 3rd St</td>\n",
       "      <td>NaN</td>\n",
       "      <td>La Verne</td>\n",
       "      <td>CA</td>\n",
       "      <td>917504918</td>\n",
       "      <td>25.0</td>\n",
       "      <td>2007-07-26</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>Aarons</td>\n",
       "      <td>Elaine</td>\n",
       "      <td>NaN</td>\n",
       "      <td>481 Buck Island Rd Apt 17A</td>\n",
       "      <td>APT 17A</td>\n",
       "      <td>West Yarmouth</td>\n",
       "      <td>MA</td>\n",
       "      <td>26733300</td>\n",
       "      <td>25.0</td>\n",
       "      <td>2008-02-26</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>Aaron</td>\n",
       "      <td>Shirley</td>\n",
       "      <td>NaN</td>\n",
       "      <td>101 Cherry Ave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Havana</td>\n",
       "      <td>FL</td>\n",
       "      <td>323331311</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2008-02-29</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    last_name first_name middle_name                    street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "2      Ahrens        Don         NaN        4034 Rennellwood Way      NaN     Pleasanton    CA      94566    50.0  2007-06-18            16\n",
       "18      Ardle    William         NaN           412 Dakota Avenue      NaN    Springfield    OH      45504    50.0  2007-06-28            16\n",
       "25    Buckler      Steve         NaN             24351 Armada Dr      NaN     Dana Point    CA  926291306    50.0  2007-07-30            20\n",
       "26    Buckler      Steve         NaN             24351 Armada Dr      NaN     Dana Point    CA  926291306    25.0  2007-08-16            20\n",
       "34       Buck    Barbara         NaN            1780 NE 138th St      NaN    North Miami    FL  331811316    50.0  2007-09-13            20\n",
       "35       Buck    Barbara         NaN            1780 NE 138th St      NaN    North Miami    FL  331811316    50.0  2007-07-19            20\n",
       "38   Buchanek  Elizabeth         NaN            7917 Kentbury Dr      NaN       Bethesda    MD  208144615    50.0  2007-09-30            20\n",
       "49   Harrison       Ryan         NaN                 2247 3rd St      NaN       La Verne    CA  917504918    25.0  2007-07-26            20\n",
       "101    Aarons     Elaine         NaN  481 Buck Island Rd Apt 17A  APT 17A  West Yarmouth    MA   26733300    25.0  2008-02-26            34\n",
       "104     Aaron    Shirley         NaN              101 Cherry Ave      NaN         Havana    FL  323331311    50.0  2008-02-29            34"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.query(\"10 <= amount <= 50\").head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SORT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>Kazor</td>\n",
       "      <td>Christopher</td>\n",
       "      <td>M</td>\n",
       "      <td>707 Spindletree ave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Naperville</td>\n",
       "      <td>IL</td>\n",
       "      <td>60565</td>\n",
       "      <td>-2592.0</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>BRUNO</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10136 WINDERMERE CHASE BLVD.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GOTHA</td>\n",
       "      <td>FL</td>\n",
       "      <td>347344707</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>BURKE</td>\n",
       "      <td>DONALD</td>\n",
       "      <td>J.</td>\n",
       "      <td>12 LOMPOC</td>\n",
       "      <td>NaN</td>\n",
       "      <td>RANCHO SANTA MARGA</td>\n",
       "      <td>CA</td>\n",
       "      <td>926881817</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-11</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>BRUNO</td>\n",
       "      <td>IRENE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10136 WINDERMERE CHASE BLVD.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GOTHA</td>\n",
       "      <td>FL</td>\n",
       "      <td>347344707</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>BROWN</td>\n",
       "      <td>TIMOTHY</td>\n",
       "      <td>J.</td>\n",
       "      <td>26826 MARLOWE COURT</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STEVENSON RANCH</td>\n",
       "      <td>CA</td>\n",
       "      <td>913811020</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>BURTON</td>\n",
       "      <td>GLENN</td>\n",
       "      <td>M.</td>\n",
       "      <td>4404 CHARLESTON COURT</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>336092620</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>BURTON</td>\n",
       "      <td>STEVEN</td>\n",
       "      <td>G.</td>\n",
       "      <td>9938 DEER CREEK DRIVE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>33647</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>Uihlein</td>\n",
       "      <td>Richard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1396 N Waukegan Rd</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lake Forest</td>\n",
       "      <td>IL</td>\n",
       "      <td>600451147</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>BURTON</td>\n",
       "      <td>SUSAN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9338 DEER CREEK DRIVE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>336472286</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>BUSH</td>\n",
       "      <td>ERIC</td>\n",
       "      <td>NaN</td>\n",
       "      <td>P.O. BOX 61046</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DENVER</td>\n",
       "      <td>CO</td>\n",
       "      <td>802061046</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last_name   first_name middle_name                      street_1 street_2                city state        zip  amount        date  candidate_id\n",
       "90     Kazor  Christopher           M           707 Spindletree ave      NaN          Naperville    IL      60565 -2592.0  2008-04-21            32\n",
       "72     BRUNO         JOHN         NaN  10136 WINDERMERE CHASE BLVD.      NaN               GOTHA    FL  347344707 -2300.0  2008-03-06            22\n",
       "64     BURKE       DONALD          J.                     12 LOMPOC      NaN  RANCHO SANTA MARGA    CA  926881817 -2300.0  2008-03-11            22\n",
       "73     BRUNO        IRENE         NaN  10136 WINDERMERE CHASE BLVD.      NaN               GOTHA    FL  347344707 -2300.0  2008-03-06            22\n",
       "74     BROWN      TIMOTHY          J.           26826 MARLOWE COURT      NaN     STEVENSON RANCH    CA  913811020 -2300.0  2008-03-06            22\n",
       "58    BURTON        GLENN          M.         4404 CHARLESTON COURT      NaN               TAMPA    FL  336092620 -2300.0  2008-03-05            22\n",
       "57    BURTON       STEVEN          G.         9938 DEER CREEK DRIVE      NaN               TAMPA    FL      33647 -2300.0  2008-03-05            22\n",
       "84   Uihlein      Richard         NaN            1396 N Waukegan Rd      NaN         Lake Forest    IL  600451147 -2300.0  2008-04-21            32\n",
       "56    BURTON        SUSAN         NaN         9338 DEER CREEK DRIVE      NaN               TAMPA    FL  336472286 -2300.0  2008-03-05            22\n",
       "55      BUSH         ERIC         NaN                P.O. BOX 61046      NaN              DENVER    CO  802061046 -2300.0  2008-03-06            22"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.sort_values(by=\"amount\").head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>ELENA</td>\n",
       "      <td>1291 NIGHTINGALE AVENUE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MIAMI SPRINGS</td>\n",
       "      <td>FL</td>\n",
       "      <td>331663832</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>2008-01-25</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>NaN</td>\n",
       "      <td>211 Long Island Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hot Springs</td>\n",
       "      <td>AR</td>\n",
       "      <td>71913</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-06-12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Blaine</td>\n",
       "      <td>M</td>\n",
       "      <td>45 Eaton Ave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Camden</td>\n",
       "      <td>ME</td>\n",
       "      <td>48431752</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2550 Adamsbrooke Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Conway</td>\n",
       "      <td>AR</td>\n",
       "      <td>72034</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-04-11</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8600 Moody Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fort Smith</td>\n",
       "      <td>AR</td>\n",
       "      <td>72903</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>ABRAMOWITZ</td>\n",
       "      <td>NIRA</td>\n",
       "      <td>NaN</td>\n",
       "      <td>411 HARBOR ROAD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SOUTHPORT</td>\n",
       "      <td>CT</td>\n",
       "      <td>68901376</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-09-14</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>A.</td>\n",
       "      <td>P.O. BOX 7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>CANADIAN</td>\n",
       "      <td>TX</td>\n",
       "      <td>790140007</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2008-01-30</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      last_name first_name middle_name                 street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "30       Buckel      Linda         NaN            PO Box 683130      NaN      Park City    UT  840683130  4600.0  2007-08-14            20\n",
       "159       ABATE      MARIA       ELENA  1291 NIGHTINGALE AVENUE      NaN  MIAMI SPRINGS    FL  331663832  2600.0  2008-01-25            37\n",
       "15      Anthony       John         NaN    211 Long Island Drive      NaN    Hot Springs    AR      71913  2300.0  2007-06-12            16\n",
       "33         Buck     Blaine           M             45 Eaton Ave      NaN         Camden    ME   48431752  2300.0  2007-09-30            20\n",
       "28       Buckel      Linda         NaN            PO Box 683130      NaN      Park City    UT  840683130  2300.0  2007-08-14            20\n",
       "21        Baker      David         NaN   2550 Adamsbrooke Drive      NaN         Conway    AR      72034  2300.0  2007-04-11            16\n",
       "13        Altes       R.D.         NaN          8600 Moody Road      NaN     Fort Smith    AR      72903  2300.0  2007-06-21            16\n",
       "135  ABRAMOWITZ       NIRA         NaN          411 HARBOR ROAD      NaN      SOUTHPORT    CT   68901376  2300.0  2007-09-14            35\n",
       "5          Akin       Mike         NaN         181 Baywood Lane      NaN     Monticello    AR      71655  1500.0  2007-05-18            16\n",
       "174     ABRAHAM      SALEM          A.               P.O. BOX 7      NaN       CANADIAN    TX  790140007  1300.0  2008-01-30            37"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.sort_values(by=\"amount\", ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>91</td>\n",
       "      <td>Kazor</td>\n",
       "      <td>Christopher</td>\n",
       "      <td>M</td>\n",
       "      <td>707 Spindletree ave</td>\n",
       "      <td>None</td>\n",
       "      <td>Naperville</td>\n",
       "      <td>IL</td>\n",
       "      <td>60565</td>\n",
       "      <td>-2592.0</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>30</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>52</td>\n",
       "      <td>BYINGTON</td>\n",
       "      <td>MARGARET</td>\n",
       "      <td>E.</td>\n",
       "      <td>2633 MIDDLEBORO LANE N.E.</td>\n",
       "      <td>None</td>\n",
       "      <td>GRAND RAPIDS</td>\n",
       "      <td>MI</td>\n",
       "      <td>495061254</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-03</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>BYERS</td>\n",
       "      <td>BOB</td>\n",
       "      <td>A.</td>\n",
       "      <td>13170 TELFAIR AVENUE</td>\n",
       "      <td>None</td>\n",
       "      <td>SYLMAR</td>\n",
       "      <td>CA</td>\n",
       "      <td>913423573</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-07</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>55</td>\n",
       "      <td>BUSH</td>\n",
       "      <td>KRYSTIE</td>\n",
       "      <td>None</td>\n",
       "      <td>P.O. BOX 61046</td>\n",
       "      <td>None</td>\n",
       "      <td>DENVER</td>\n",
       "      <td>CO</td>\n",
       "      <td>802061046</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>56</td>\n",
       "      <td>BUSH</td>\n",
       "      <td>ERIC</td>\n",
       "      <td>None</td>\n",
       "      <td>P.O. BOX 61046</td>\n",
       "      <td>None</td>\n",
       "      <td>DENVER</td>\n",
       "      <td>CO</td>\n",
       "      <td>802061046</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-06</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>57</td>\n",
       "      <td>BURTON</td>\n",
       "      <td>SUSAN</td>\n",
       "      <td>None</td>\n",
       "      <td>9338 DEER CREEK DRIVE</td>\n",
       "      <td>None</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>336472286</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>58</td>\n",
       "      <td>BURTON</td>\n",
       "      <td>STEVEN</td>\n",
       "      <td>G.</td>\n",
       "      <td>9938 DEER CREEK DRIVE</td>\n",
       "      <td>None</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>33647</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>59</td>\n",
       "      <td>BURTON</td>\n",
       "      <td>GLENN</td>\n",
       "      <td>M.</td>\n",
       "      <td>4404 CHARLESTON COURT</td>\n",
       "      <td>None</td>\n",
       "      <td>TAMPA</td>\n",
       "      <td>FL</td>\n",
       "      <td>336092620</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-05</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>65</td>\n",
       "      <td>BURKE</td>\n",
       "      <td>DONALD</td>\n",
       "      <td>J.</td>\n",
       "      <td>12 LOMPOC</td>\n",
       "      <td>None</td>\n",
       "      <td>RANCHO SANTA MARGA</td>\n",
       "      <td>CA</td>\n",
       "      <td>926881817</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2008-03-11</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name   first_name middle_name                   street_1 street_2                city state        zip  amount        date  candidate_id\n",
       "0  91     Kazor  Christopher           M        707 Spindletree ave     None          Naperville    IL      60565 -2592.0  2008-04-21            32\n",
       "1  30    Buckel        Linda        None              PO Box 683130     None           Park City    UT  840683130 -2300.0  2007-08-14            20\n",
       "2  52  BYINGTON     MARGARET          E.  2633 MIDDLEBORO LANE N.E.     None        GRAND RAPIDS    MI  495061254 -2300.0  2008-03-03            22\n",
       "3  53     BYERS          BOB          A.       13170 TELFAIR AVENUE     None              SYLMAR    CA  913423573 -2300.0  2008-03-07            22\n",
       "4  55      BUSH      KRYSTIE        None             P.O. BOX 61046     None              DENVER    CO  802061046 -2300.0  2008-03-06            22\n",
       "5  56      BUSH         ERIC        None             P.O. BOX 61046     None              DENVER    CO  802061046 -2300.0  2008-03-06            22\n",
       "6  57    BURTON        SUSAN        None      9338 DEER CREEK DRIVE     None               TAMPA    FL  336472286 -2300.0  2008-03-05            22\n",
       "7  58    BURTON       STEVEN          G.      9938 DEER CREEK DRIVE     None               TAMPA    FL      33647 -2300.0  2008-03-05            22\n",
       "8  59    BURTON        GLENN          M.      4404 CHARLESTON COURT     None               TAMPA    FL  336092620 -2300.0  2008-03-05            22\n",
       "9  65     BURKE       DONALD          J.                  12 LOMPOC     None  RANCHO SANTA MARGA    CA  926881817 -2300.0  2008-03-11            22"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors ORDER BY amount;\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>31</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>160</td>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>ELENA</td>\n",
       "      <td>1291 NIGHTINGALE AVENUE</td>\n",
       "      <td>None</td>\n",
       "      <td>MIAMI SPRINGS</td>\n",
       "      <td>FL</td>\n",
       "      <td>331663832</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>2008-01-25</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14</td>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>None</td>\n",
       "      <td>8600 Moody Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Fort Smith</td>\n",
       "      <td>AR</td>\n",
       "      <td>72903</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>16</td>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>None</td>\n",
       "      <td>211 Long Island Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Hot Springs</td>\n",
       "      <td>AR</td>\n",
       "      <td>71913</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-06-12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22</td>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>None</td>\n",
       "      <td>2550 Adamsbrooke Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Conway</td>\n",
       "      <td>AR</td>\n",
       "      <td>72034</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-04-11</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>29</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>34</td>\n",
       "      <td>Buck</td>\n",
       "      <td>Blaine</td>\n",
       "      <td>M</td>\n",
       "      <td>45 Eaton Ave</td>\n",
       "      <td>None</td>\n",
       "      <td>Camden</td>\n",
       "      <td>ME</td>\n",
       "      <td>48431752</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>136</td>\n",
       "      <td>ABRAMOWITZ</td>\n",
       "      <td>NIRA</td>\n",
       "      <td>None</td>\n",
       "      <td>411 HARBOR ROAD</td>\n",
       "      <td>None</td>\n",
       "      <td>SOUTHPORT</td>\n",
       "      <td>CT</td>\n",
       "      <td>68901376</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-09-14</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>6</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>None</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id   last_name first_name middle_name                 street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "0   31      Buckel      Linda        None            PO Box 683130     None      Park City    UT  840683130  4600.0  2007-08-14            20\n",
       "1  160       ABATE      MARIA       ELENA  1291 NIGHTINGALE AVENUE     None  MIAMI SPRINGS    FL  331663832  2600.0  2008-01-25            37\n",
       "2   14       Altes       R.D.        None          8600 Moody Road     None     Fort Smith    AR      72903  2300.0  2007-06-21            16\n",
       "3   16     Anthony       John        None    211 Long Island Drive     None    Hot Springs    AR      71913  2300.0  2007-06-12            16\n",
       "4   22       Baker      David        None   2550 Adamsbrooke Drive     None         Conway    AR      72034  2300.0  2007-04-11            16\n",
       "5   29      Buckel      Linda        None            PO Box 683130     None      Park City    UT  840683130  2300.0  2007-08-14            20\n",
       "6   34        Buck     Blaine           M             45 Eaton Ave     None         Camden    ME   48431752  2300.0  2007-09-30            20\n",
       "7  136  ABRAMOWITZ       NIRA        None          411 HARBOR ROAD     None      SOUTHPORT    CT   68901376  2300.0  2007-09-14            35\n",
       "8    6        Akin       Mike        None         181 Baywood Lane     None     Monticello    AR      71655  1500.0  2007-05-18            16\n",
       "9   10       Allen    John D.        None   1052 Cannon Mill Drive     None  North Augusta    SC      29860  1300.0  2007-06-29            16"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors ORDER BY amount DESC;\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SELECT-COLUMNS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_name</th>\n",
       "      <th>amount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Steven</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Don</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Don</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Don</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Charles</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mike</td>\n",
       "      <td>1500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Rebecca</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Brittni</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>John D.</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>John D.</td>\n",
       "      <td>1300.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  first_name  amount\n",
       "0     Steven   500.0\n",
       "1        Don   250.0\n",
       "2        Don    50.0\n",
       "3        Don   100.0\n",
       "4    Charles   100.0\n",
       "5       Mike  1500.0\n",
       "6    Rebecca   500.0\n",
       "7    Brittni   250.0\n",
       "8    John D.  1000.0\n",
       "9    John D.  1300.0"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[['first_name', 'amount']].head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Steven', 500), ('Don', 250), ('Don', 50), ('Don', 100), ('Charles', 100), ('Mike', 1500), ('Rebecca', 500), ('Brittni', 250), ('John D.', 1000), ('John D.', 1300), ('John W.', 1000), ('Rebecca', 1000), ('Rebecca', 200), ('R.D.', 2300), ('Dale', 250), ('John', 2300), ('Robert', 500), ('Robert', 100), ('William', 50), ('Omar', 1000), ('Omar', 1000), ('David', 2300), ('David', 250), ('Charles', 1000), ('John', 500), ('Steve', 50), ('Steve', 25), ('Bruce', 100), ('Linda', 2300), ('Linda', -2300), ('Linda', 4600), ('Thomas', 100), ('Jay', 200), ('Blaine', 2300), ('Barbara', 50), ('Barbara', 50), ('Mark M', 460.8), ('Ida', 100), ('Elizabeth', 50), ('John', 500), ('John', -500), ('John', 500), ('John', 700), ('John', -700), ('John', 1000), ('John', 1300), ('John', 200), ('John', 500), ('John', 200.09), ('Ryan', 25), ('HERBERT', -500), ('MARGARET', -2300), ('BOB', -2300), ('AUDREY', -200), ('KRYSTIE', -2300), ('ERIC', -2300), ('SUSAN', -2300), ('STEVEN', -2300), ('GLENN', -2300), ('CRAIG', -500), ('CRAIG', -1000), ('BARBARA', -500), ('SUZANNE', -700), ('GAIL', -2000), ('DONALD', -2300), ('RONALD', -1000), ('BARTLE', -800), ('BARTLE', -1000), ('DANIEL', -100), ('MARGARET', -200), ('WALTER', -100), ('MARJORIE', -100), ('JOHN', -2300), ('IRENE', -2300), ('TIMOTHY', -2300), ('Bryan', -25), ('James', -25), ('Anoop', -100), ('Michael', -503), ('Nina', -50), ('James', -28), ('William', -150), ('David', -330), ('Jeanette', -1000), ('Richard', -2300), ('Robert', -223), ('Alan', -844.8), ('Marcia', -299.5), ('Louise', -34.08), ('Timur', -777.95), ('Christopher', -2592), ('Thomas', -200), ('Joseph', -24.6), ('Philip', -1046), ('Judith', -1100), ('Cathleen', -14.76), ('Bradley', -24.53), ('Bruce', -261), ('Hyeokchan', -39.5), ('Richard', -1000), ('Rebecca', 100), ('Elaine', 25), ('Elaine', 70), ('Elaine', 100), ('Shirley', 50), ('Shirley', 100), ('Rebecca', 100), ('Shirley', 100), ('Shirley', 100), ('Shirley', 100), ('Elizabeth', -350), ('Thomas', -2300), ('Shirley', 100), ('Shirley', 100), ('Sharron', 500), ('Patricia', 200), ('Patricia', 100), ('Jim', 300), ('Jim', 200), ('Carole', 70), ('Carole', 50), ('Carole', 100), ('Barbara', 1000), ('Lin', 250), ('Lin', 100), ('TRAVIS', -500), ('BRIAN', -1000), ('TRAVIS', -1000), ('DEAN', 500), ('HENRY', 100), ('LANNY', 500), ('DIANA', 500), ('KEVIN', 50), ('KEVIN', 150), ('KEVIN', 50), ('NIRA', 2300), ('MICHAEL', 250), ('KEN', 300), ('NASAR', 1000), ('PATRICIA', 75), ('PATRICIA', 25), ('PATRICIA', 75), ('PATRICIA', 75), ('PATRICIA', 25), ('PATRICIA', 25), ('THOMAS', 50), ('WELDON', 100), ('WELDON', 50), ('GERALD', 100), ('GERALD', 100), ('ZAINUL', 500), ('SYBIL', 75), ('SYBIL', 50), ('RONALD', 200), ('RONALD', 100), ('ROBERT', 500), ('MIKE', 1000), ('DAVID', 200), ('PAULINE', 35), ('MARIA', 2600), ('PETER', 25), ('SHIRLEY', 150), ('CHARLES', 300), ('CHARLES', 410), ('CHARLES', 500), ('JOHN', 200), ('MARLING', 100), ('RUDOLPH', 100), ('RODNEY', 500), ('DENIS', 500), ('MERRILL', 120), ('GEORGE', 800), ('PETER', 50), ('SALEM', 1000), ('SALEM', 1300)]\n"
     ]
    }
   ],
   "source": [
    "out=make_query(\"SELECT first_name, amount FROM contributors;\")\n",
    "print(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_name</th>\n",
       "      <th>amount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Steven</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Don</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Don</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Don</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Charles</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Mike</td>\n",
       "      <td>1500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Rebecca</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Brittni</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>John D.</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>John D.</td>\n",
       "      <td>1300.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  first_name  amount\n",
       "0     Steven   500.0\n",
       "1        Don   250.0\n",
       "2        Don    50.0\n",
       "3        Don   100.0\n",
       "4    Charles   100.0\n",
       "5       Mike  1500.0\n",
       "6    Rebecca   500.0\n",
       "7    Brittni   250.0\n",
       "8    John D.  1000.0\n",
       "9    John D.  1300.0"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_frame(out,['first_name', 'amount']).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SELECT-DISTINCT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "last_name     175\n",
       "first_name    175\n",
       "dtype: int64"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[['last_name','first_name']].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "last_name     126\n",
       "first_name    126\n",
       "dtype: int64"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[['last_name','first_name']].drop_duplicates().count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last_name first_name\n",
       "0       Agee     Steven\n",
       "1     Ahrens        Don\n",
       "4       Akin    Charles\n",
       "5       Akin       Mike\n",
       "6       Akin    Rebecca\n",
       "7   Aldridge    Brittni\n",
       "8      Allen    John D.\n",
       "10   Allison    John W.\n",
       "11   Allison    Rebecca\n",
       "13     Altes       R.D."
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[['last_name','first_name']].drop_duplicates().head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "126\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name\n",
       "0      Agee     Steven\n",
       "1    Ahrens        Don\n",
       "2      Akin    Charles\n",
       "3      Akin       Mike\n",
       "4      Akin    Rebecca\n",
       "5  Aldridge    Brittni\n",
       "6     Allen    John D.\n",
       "7   Allison    John W.\n",
       "8   Allison    Rebecca\n",
       "9     Altes       R.D."
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT DISTINCT last_name, first_name FROM contributors;\")\n",
    "print(len(out))\n",
    "make_frame(out,['last_name', 'first_name']).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ASSIGN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>Agee, Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Charles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Mike</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>NaN</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "      <td>Aldridge, Brittni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-06-11</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                      street_1 street_2           city state    zip  amount        date  candidate_id               name\n",
       "0      Agee     Steven         NaN        549 Laurel Branch Road      NaN          Floyd    VA  24091   500.0  2007-06-30            16       Agee, Steven\n",
       "1    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566   250.0  2007-05-16            16        Ahrens, Don\n",
       "2    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566    50.0  2007-06-18            16        Ahrens, Don\n",
       "3    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566   100.0  2007-06-21            16        Ahrens, Don\n",
       "4      Akin    Charles         NaN        10187 Sugar Creek Road      NaN    Bentonville    AR  72712   100.0  2007-06-16            16      Akin, Charles\n",
       "5      Akin       Mike         NaN              181 Baywood Lane      NaN     Monticello    AR  71655  1500.0  2007-05-18            16         Akin, Mike\n",
       "6      Akin    Rebecca         NaN              181 Baywood Lane      NaN     Monticello    AR  71655   500.0  2007-05-18            16      Akin, Rebecca\n",
       "7  Aldridge    Brittni         NaN  808 Capitol Square Place, SW      NaN     Washington    DC  20024   250.0  2007-06-06            16  Aldridge, Brittni\n",
       "8     Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1000.0  2007-06-11            16     Allen, John D.\n",
       "9     Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1300.0  2007-06-29            16     Allen, John D."
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci['name']=dfcwci['last_name']+\", \"+dfcwci['first_name']\n",
    "dfcwci.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "      <th>ucname</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>Agee, Steven</td>\n",
       "      <td>Agee:Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "      <td>Ahrens:Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "      <td>Ahrens:Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "      <td>Ahrens:Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Charles</td>\n",
       "      <td>Akin:Charles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Mike</td>\n",
       "      <td>Akin:Mike</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Rebecca</td>\n",
       "      <td>Akin:Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>NaN</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "      <td>Aldridge, Brittni</td>\n",
       "      <td>Aldridge:Brittni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-06-11</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "      <td>Allen:John D.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "      <td>Allen:John D.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                      street_1 street_2           city state    zip  amount        date  candidate_id               name            ucname\n",
       "0      Agee     Steven         NaN        549 Laurel Branch Road      NaN          Floyd    VA  24091   500.0  2007-06-30            16       Agee, Steven       Agee:Steven\n",
       "1    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566   250.0  2007-05-16            16        Ahrens, Don        Ahrens:Don\n",
       "2    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566    50.0  2007-06-18            16        Ahrens, Don        Ahrens:Don\n",
       "3    Ahrens        Don         NaN          4034 Rennellwood Way      NaN     Pleasanton    CA  94566   100.0  2007-06-21            16        Ahrens, Don        Ahrens:Don\n",
       "4      Akin    Charles         NaN        10187 Sugar Creek Road      NaN    Bentonville    AR  72712   100.0  2007-06-16            16      Akin, Charles      Akin:Charles\n",
       "5      Akin       Mike         NaN              181 Baywood Lane      NaN     Monticello    AR  71655  1500.0  2007-05-18            16         Akin, Mike         Akin:Mike\n",
       "6      Akin    Rebecca         NaN              181 Baywood Lane      NaN     Monticello    AR  71655   500.0  2007-05-18            16      Akin, Rebecca      Akin:Rebecca\n",
       "7  Aldridge    Brittni         NaN  808 Capitol Square Place, SW      NaN     Washington    DC  20024   250.0  2007-06-06            16  Aldridge, Brittni  Aldridge:Brittni\n",
       "8     Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1000.0  2007-06-11            16     Allen, John D.     Allen:John D.\n",
       "9     Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1300.0  2007-06-29            16     Allen, John D.     Allen:John D."
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.assign(ucname=dfcwci.last_name+\":\"+dfcwci.first_name).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Will the above command actually change `dfcwci`?\n",
    "\n",
    "#### What if we wanted to change an existing assignment?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.00</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>Agee, Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "      <td>Buckheit, Bruce</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>Ranganath, Anoop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>Perreault, Louise</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "      <td>ABDELLA, THOMAS</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id               name\n",
       "0         Agee     Steven         NaN       549 Laurel Branch Road      NaN            Floyd    VA      24091  500.00  2007-06-30            16       Agee, Steven\n",
       "27    Buckheit      Bruce         NaN                8904 KAREN DR      NaN          FAIRFAX    VA  220312731  100.00  2007-09-19            20    Buckheit, Bruce\n",
       "77   Ranganath      Anoop         NaN           2507 Willard Drive      NaN  Charlottesville    VA      22903 -100.00  2008-04-21            32   Ranganath, Anoop\n",
       "88   Perreault     Louise         NaN    503 Brockridge Hunt Drive      NaN          Hampton    VA      23666  -34.08  2008-04-21            32  Perreault, Louise\n",
       "145    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340      NaN          FAIRFAX    VA  220308440   50.00  2007-09-30            35    ABDELLA, THOMAS"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.state=='VA']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0           Agee, Steven\n",
       "27       Buckheit, Bruce\n",
       "77      Ranganath, Anoop\n",
       "88     Perreault, Louise\n",
       "145      ABDELLA, THOMAS\n",
       "Name: name, dtype: object"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.loc[dfcwci.state=='VA', 'name']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dfcwci.loc[dfcwci.state=='VA', 'name']=\"junk\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.00</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id  name\n",
       "0         Agee     Steven         NaN       549 Laurel Branch Road      NaN            Floyd    VA      24091  500.00  2007-06-30            16  junk\n",
       "27    Buckheit      Bruce         NaN                8904 KAREN DR      NaN          FAIRFAX    VA  220312731  100.00  2007-09-19            20  junk\n",
       "77   Ranganath      Anoop         NaN           2507 Willard Drive      NaN  Charlottesville    VA      22903 -100.00  2008-04-21            32  junk\n",
       "88   Perreault     Louise         NaN    503 Brockridge Hunt Drive      NaN          Hampton    VA      23666  -34.08  2008-04-21            32  junk\n",
       "145    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340      NaN          FAIRFAX    VA  220308440   50.00  2007-09-30            35  junk"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.query(\"state=='VA'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "Let us see the entire process in SQL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "OperationalError",
     "evalue": "duplicate column name: name",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOperationalError\u001b[0m                          Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-71-82c4fd418964>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0malt\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"ALTER TABLE contributors ADD COLUMN name VARCHAR;\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0malt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mOperationalError\u001b[0m: duplicate column name: name"
     ]
    }
   ],
   "source": [
    "alt=\"ALTER TABLE contributors ADD COLUMN name VARCHAR;\"\n",
    "db.cursor().execute(alt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 'id', 'INTEGER', 1, None, 1),\n",
       " (1, 'last_name', 'VARCHAR', 0, None, 0),\n",
       " (2, 'first_name', 'VARCHAR', 0, None, 0),\n",
       " (3, 'middle_name', 'VARCHAR', 0, None, 0),\n",
       " (4, 'street_1', 'VARCHAR', 0, None, 0),\n",
       " (5, 'street_2', 'VARCHAR', 0, None, 0),\n",
       " (6, 'city', 'VARCHAR', 0, None, 0),\n",
       " (7, 'state', 'VARCHAR', 0, None, 0),\n",
       " (8, 'zip', 'VARCHAR', 0, None, 0),\n",
       " (9, 'amount', 'INTEGER', 0, None, 0),\n",
       " (10, 'date', 'DATETIME', 0, None, 0),\n",
       " (11, 'candidate_id', 'INTEGER', 1, None, 0),\n",
       " (12, 'name', '', 0, None, 0)]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_query(\"PRAGMA table_info(contributors);\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Agee, Steven', 1),\n",
       " ('Ahrens, Don', 2),\n",
       " ('Ahrens, Don', 3),\n",
       " ('Ahrens, Don', 4),\n",
       " ('Akin, Charles', 5),\n",
       " ('Akin, Mike', 6),\n",
       " ('Akin, Rebecca', 7),\n",
       " ('Aldridge, Brittni', 8),\n",
       " ('Allen, John D.', 9),\n",
       " ('Allen, John D.', 10),\n",
       " ('Allison, John W.', 11),\n",
       " ('Allison, Rebecca', 12),\n",
       " ('Allison, Rebecca', 13),\n",
       " ('Altes, R.D.', 14),\n",
       " ('Andres, Dale', 15),\n",
       " ('Anthony, John', 16),\n",
       " ('Arbogast, Robert', 17),\n",
       " ('Arbogast, Robert', 18),\n",
       " ('Ardle, William', 19),\n",
       " ('Atiq, Omar', 20),\n",
       " ('Atiq, Omar', 21),\n",
       " ('Baker, David', 22),\n",
       " ('Bancroft, David', 23),\n",
       " ('Banks, Charles', 24),\n",
       " ('Barbee, John', 25),\n",
       " ('Buckler, Steve', 26),\n",
       " ('Buckler, Steve', 27),\n",
       " ('Buckheit, Bruce', 28),\n",
       " ('Buckel, Linda', 29),\n",
       " ('Buckel, Linda', 30),\n",
       " ('Buckel, Linda', 31),\n",
       " ('Buck, Thomas', 32),\n",
       " ('Buck, Jay', 33),\n",
       " ('Buck, Blaine', 34),\n",
       " ('Buck, Barbara', 35),\n",
       " ('Buck, Barbara', 36),\n",
       " ('Buchman, Mark M', 37),\n",
       " ('Bucher, Ida', 38),\n",
       " ('Buchanek, Elizabeth', 39),\n",
       " ('Buchanan, John', 40),\n",
       " ('Buchanan, John', 41),\n",
       " ('Buchanan, John', 42),\n",
       " ('Buchanan, John', 43),\n",
       " ('Buchanan, John', 44),\n",
       " ('Buchanan, John', 45),\n",
       " ('Buchanan, John', 46),\n",
       " ('Buchanan, John', 47),\n",
       " ('Buchanan, John', 48),\n",
       " ('Buchanan, John', 49),\n",
       " ('Harrison, Ryan', 50),\n",
       " ('BYNUM, HERBERT', 51),\n",
       " ('BYINGTON, MARGARET', 52),\n",
       " ('BYERS, BOB', 53),\n",
       " ('BYERS, AUDREY', 54),\n",
       " ('BUSH, KRYSTIE', 55),\n",
       " ('BUSH, ERIC', 56),\n",
       " ('BURTON, SUSAN', 57),\n",
       " ('BURTON, STEVEN', 58),\n",
       " ('BURTON, GLENN', 59),\n",
       " ('BURKHARDT, CRAIG', 60),\n",
       " ('BURKHARDT, CRAIG', 61),\n",
       " ('BURKHARDT, BARBARA', 62),\n",
       " ('BURKE, SUZANNE', 63),\n",
       " ('BURKE, GAIL', 64),\n",
       " ('BURKE, DONALD', 65),\n",
       " ('BURGERT, RONALD', 66),\n",
       " ('BULL, BARTLE', 67),\n",
       " ('BULL, BARTLE', 68),\n",
       " ('BUKOWSKI, DANIEL', 69),\n",
       " ('BUISSON, MARGARET', 70),\n",
       " ('BUCKLEY, WALTER', 71),\n",
       " ('BUCKLEY, MARJORIE', 72),\n",
       " ('BRUNO, JOHN', 73),\n",
       " ('BRUNO, IRENE', 74),\n",
       " ('BROWN, TIMOTHY', 75),\n",
       " ('Schuff, Bryan', 76),\n",
       " ('Hobbs, James', 77),\n",
       " ('Ranganath, Anoop', 78),\n",
       " ('Nystrom, Michael', 79),\n",
       " ('Muse, Nina', 80),\n",
       " ('Waddell, James', 81),\n",
       " ('Brucks, William', 82),\n",
       " ('Kuehn, David', 83),\n",
       " ('Verster, Jeanette', 84),\n",
       " ('Uihlein, Richard', 85),\n",
       " ('Eskenberry, Robert', 86),\n",
       " ('Froehling, Alan', 87),\n",
       " ('Duryea, Marcia', 88),\n",
       " ('Perreault, Louise', 89),\n",
       " ('Rozenfeld, Timur', 90),\n",
       " ('Kazor, Christopher', 91),\n",
       " ('Lehner, Thomas', 92),\n",
       " ('Plummer, Joseph', 93),\n",
       " ('Raught, Philip', 94),\n",
       " ('Ferrara, Judith', 95),\n",
       " ('Johnson, Cathleen', 96),\n",
       " ('Sanford, Bradley', 97),\n",
       " ('Gaarder, Bruce', 98),\n",
       " ('Choe, Hyeokchan', 99),\n",
       " ('Jacobs, Richard', 100),\n",
       " ('Aaronson, Rebecca', 101),\n",
       " ('Aarons, Elaine', 102),\n",
       " ('Aarons, Elaine', 103),\n",
       " ('Aarons, Elaine', 104),\n",
       " ('Aaron, Shirley', 105),\n",
       " ('Aaron, Shirley', 106),\n",
       " ('Aaronson, Rebecca', 107),\n",
       " ('Aaron, Shirley', 108),\n",
       " ('Aaron, Shirley', 109),\n",
       " ('Aaron, Shirley', 110),\n",
       " ('Reid, Elizabeth', 111),\n",
       " ('Reich, Thomas', 112),\n",
       " ('Aaron, Shirley', 113),\n",
       " ('Aaron, Shirley', 114),\n",
       " ('Aaron, Sharron', 115),\n",
       " ('Aaron, Patricia', 116),\n",
       " ('Aaron, Patricia', 117),\n",
       " ('Aaron, Jim', 118),\n",
       " ('Aaron, Jim', 119),\n",
       " ('Aaron, Carole', 120),\n",
       " ('Aaron, Carole', 121),\n",
       " ('Aaron, Carole', 122),\n",
       " ('Aaron, Barbara', 123),\n",
       " ('Aanonsen, Lin', 124),\n",
       " ('Aanonsen, Lin', 125),\n",
       " ('BOURNE, TRAVIS', 126),\n",
       " ('SECRIST, BRIAN', 127),\n",
       " ('TOLLESTRUP, TRAVIS', 128),\n",
       " ('ACCORD, DEAN', 129),\n",
       " ('ABTS, HENRY', 130),\n",
       " ('ABSHIER, LANNY', 131),\n",
       " ('ABSHIER, DIANA', 132),\n",
       " ('ABREU, KEVIN', 133),\n",
       " ('ABREU, KEVIN', 134),\n",
       " ('ABREU, KEVIN', 135),\n",
       " ('ABRAMOWITZ, NIRA', 136),\n",
       " ('ABRAMS, MICHAEL', 137),\n",
       " ('ABRAMOWITZ, KEN', 138),\n",
       " ('ABOUBAKARE, NASAR', 139),\n",
       " ('ABEGG, PATRICIA', 140),\n",
       " ('ABEGG, PATRICIA', 141),\n",
       " ('ABEGG, PATRICIA', 142),\n",
       " ('ABEGG, PATRICIA', 143),\n",
       " ('ABEGG, PATRICIA', 144),\n",
       " ('ABEGG, PATRICIA', 145),\n",
       " ('ABDELLA, THOMAS', 146),\n",
       " ('ABBOTT, WELDON', 147),\n",
       " ('ABBOTT, WELDON', 148),\n",
       " ('ABBOTT, GERALD', 149),\n",
       " ('ABBOTT, GERALD', 150),\n",
       " ('ABEDIN, ZAINUL', 151),\n",
       " ('ABBOTT, SYBIL', 152),\n",
       " ('ABBOTT, SYBIL', 153),\n",
       " ('ABBOTT, RONALD', 154),\n",
       " ('ABBOTT, RONALD', 155),\n",
       " ('ABBOTT, ROBERT', 156),\n",
       " ('ABBOTT, MIKE', 157),\n",
       " ('ABBOT, DAVID', 158),\n",
       " ('ABBO, PAULINE', 159),\n",
       " ('ABATE, MARIA', 160),\n",
       " ('ABAIR, PETER', 161),\n",
       " ('ABACHERLI, SHIRLEY', 162),\n",
       " ('AARONS, CHARLES', 163),\n",
       " ('AARONS, CHARLES', 164),\n",
       " ('AARONS, CHARLES', 165),\n",
       " ('ABEL, JOHN', 166),\n",
       " ('ABEL, MARLING', 167),\n",
       " ('ABEL, RUDOLPH', 168),\n",
       " ('ABELE, RODNEY', 169),\n",
       " ('ABERCROMBIE, DENIS', 170),\n",
       " ('ABESHAUS, MERRILL', 171),\n",
       " ('ABRAHAM, GEORGE', 172),\n",
       " ('ABRAHAMSON, PETER', 173),\n",
       " ('ABRAHAM, SALEM', 174),\n",
       " ('ABRAHAM, SALEM', 175)]"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out = make_query(\"SELECT id, last_name,first_name from contributors;\")\n",
    "out2 = [(e[1]+\", \"+e[2],e[0]) for e in out]  \n",
    "out2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#could use map or a comprehension here as well\n",
    "alt2=\"UPDATE contributors SET name = ? WHERE id = ?;\"\n",
    "for ele in out2:\n",
    "    db.cursor().execute(alt2, ele)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "db.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>None</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>Agee, Steven</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>None</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>None</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>None</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>None</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>None</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>None</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "      <td>Ahrens, Don</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>None</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Charles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Mike</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "      <td>Akin, Rebecca</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>None</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>None</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "      <td>Aldridge, Brittni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>None</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-06-11</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>None</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "      <td>Allen, John D.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name                      street_1 street_2           city state    zip  amount        date  candidate_id               name\n",
       "0   1      Agee     Steven        None        549 Laurel Branch Road     None          Floyd    VA  24091   500.0  2007-06-30            16       Agee, Steven\n",
       "1   2    Ahrens        Don        None          4034 Rennellwood Way     None     Pleasanton    CA  94566   250.0  2007-05-16            16        Ahrens, Don\n",
       "2   3    Ahrens        Don        None          4034 Rennellwood Way     None     Pleasanton    CA  94566    50.0  2007-06-18            16        Ahrens, Don\n",
       "3   4    Ahrens        Don        None          4034 Rennellwood Way     None     Pleasanton    CA  94566   100.0  2007-06-21            16        Ahrens, Don\n",
       "4   5      Akin    Charles        None        10187 Sugar Creek Road     None    Bentonville    AR  72712   100.0  2007-06-16            16      Akin, Charles\n",
       "5   6      Akin       Mike        None              181 Baywood Lane     None     Monticello    AR  71655  1500.0  2007-05-18            16         Akin, Mike\n",
       "6   7      Akin    Rebecca        None              181 Baywood Lane     None     Monticello    AR  71655   500.0  2007-05-18            16      Akin, Rebecca\n",
       "7   8  Aldridge    Brittni        None  808 Capitol Square Place, SW     None     Washington    DC  20024   250.0  2007-06-06            16  Aldridge, Brittni\n",
       "8   9     Allen    John D.        None        1052 Cannon Mill Drive     None  North Augusta    SC  29860  1000.0  2007-06-11            16     Allen, John D.\n",
       "9  10     Allen    John D.        None        1052 Cannon Mill Drive     None  North Augusta    SC  29860  1300.0  2007-06-29            16     Allen, John D."
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * from contributors;\")\n",
    "make_frame(out,cont_cols+[\"name\"]).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And lets now do an assignment to an existing column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "upd=\"UPDATE contributors SET name = 'junk' WHERE state = 'VA';\"\n",
    "db.cursor().execute(upd)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>None</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.00</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28</td>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>None</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.00</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>78</td>\n",
       "      <td>Ranganath</td>\n",
       "      <td>Anoop</td>\n",
       "      <td>None</td>\n",
       "      <td>2507 Willard Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Charlottesville</td>\n",
       "      <td>VA</td>\n",
       "      <td>22903</td>\n",
       "      <td>-100.00</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>89</td>\n",
       "      <td>Perreault</td>\n",
       "      <td>Louise</td>\n",
       "      <td>None</td>\n",
       "      <td>503 Brockridge Hunt Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>Hampton</td>\n",
       "      <td>VA</td>\n",
       "      <td>23666</td>\n",
       "      <td>-34.08</td>\n",
       "      <td>2008-04-21</td>\n",
       "      <td>32</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>146</td>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>M.</td>\n",
       "      <td>4231 MONUMENT WALL WAY #340</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220308440</td>\n",
       "      <td>50.00</td>\n",
       "      <td>2007-09-30</td>\n",
       "      <td>35</td>\n",
       "      <td>junk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id  last_name first_name middle_name                     street_1 street_2             city state        zip  amount        date  candidate_id  name\n",
       "0    1       Agee     Steven        None       549 Laurel Branch Road     None            Floyd    VA      24091  500.00  2007-06-30            16  junk\n",
       "1   28   Buckheit      Bruce        None                8904 KAREN DR     None          FAIRFAX    VA  220312731  100.00  2007-09-19            20  junk\n",
       "2   78  Ranganath      Anoop        None           2507 Willard Drive     None  Charlottesville    VA      22903 -100.00  2008-04-21            32  junk\n",
       "3   89  Perreault     Louise        None    503 Brockridge Hunt Drive     None          Hampton    VA      23666  -34.08  2008-04-21            32  junk\n",
       "4  146    ABDELLA     THOMAS          M.  4231 MONUMENT WALL WAY #340     None          FAIRFAX    VA  220308440   50.00  2007-09-30            35  junk"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * from contributors where state='VA';\")\n",
    "make_frame(out,cont_cols+[\"name\"]).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### No DROP COLUMN in SQLITE\n",
    "\n",
    "Its available in other databases. Here you must just re-create your database, or know about this gotcha from the start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "OperationalError",
     "evalue": "near \"DROP\": syntax error",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOperationalError\u001b[0m                          Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-79-f04a1e6f97c0>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0malt\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"ALTER TABLE contributors DROP COLUMN name;\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcursor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0malt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mdb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcommit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mOperationalError\u001b[0m: near \"DROP\": syntax error"
     ]
    }
   ],
   "source": [
    "alt=\"ALTER TABLE contributors DROP COLUMN name;\"\n",
    "db.cursor().execute(alt)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its much simpler in Pandas, of-course"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "del dfcwci['name']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### AGGREGATE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.750000e+02</td>\n",
       "      <td>175.000000</td>\n",
       "      <td>175.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.780014e+08</td>\n",
       "      <td>3.418114</td>\n",
       "      <td>28.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.628278e+08</td>\n",
       "      <td>1028.418999</td>\n",
       "      <td>7.823484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.474000e+03</td>\n",
       "      <td>-2592.000000</td>\n",
       "      <td>16.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>9.336700e+04</td>\n",
       "      <td>-175.000000</td>\n",
       "      <td>20.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.233313e+08</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>32.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.816946e+08</td>\n",
       "      <td>300.000000</td>\n",
       "      <td>35.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.951532e+08</td>\n",
       "      <td>4600.000000</td>\n",
       "      <td>37.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                zip       amount  candidate_id\n",
       "count  1.750000e+02   175.000000    175.000000\n",
       "mean   3.780014e+08     3.418114     28.000000\n",
       "std    3.628278e+08  1028.418999      7.823484\n",
       "min    2.474000e+03 -2592.000000     16.000000\n",
       "25%    9.336700e+04  -175.000000     20.000000\n",
       "50%    3.233313e+08   100.000000     32.000000\n",
       "75%    7.816946e+08   300.000000     35.000000\n",
       "max    9.951532e+08  4600.000000     37.000000"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4600.0"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.amount.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last_name first_name middle_name       street_1 street_2       city state        zip  amount        date  candidate_id\n",
       "30    Buckel      Linda         NaN  PO Box 683130      NaN  Park City    UT  840683130  4600.0  2007-08-14            20"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.amount==dfcwci.amount.max()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(4600,)]"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT MAX(amount) AS maxamt FROM contributors;\")\n",
    "out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(31, 'Buckel', 'Linda', None, 'PO Box 683130', None, 'Park City', 'UT', '840683130', 4600, '2007-08-14', 20, 'Buckel, Linda', 4600)]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>maxamt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>31</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "      <td>Buckel, Linda</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name       street_1 street_2       city state        zip  amount        date  candidate_id         maxamt\n",
       "0  31    Buckel      Linda        None  PO Box 683130     None  Park City    UT  840683130    4600  2007-08-14            20  Buckel, Linda"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT *, MAX(amount) AS maxamt FROM contributors;\")\n",
    "print(out)\n",
    "make_frame(out, cont_cols+['maxamt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(175,)]"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT COUNT(amount) AS AMOUNTCOUNT FROM contributors;\")\n",
    "out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(3.418114285714276,)]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT AVG(amount) FROM contributors;\")\n",
    "out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>ELENA</td>\n",
       "      <td>1291 NIGHTINGALE AVENUE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>MIAMI SPRINGS</td>\n",
       "      <td>FL</td>\n",
       "      <td>331663832</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>2008-01-25</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    last_name first_name middle_name                 street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "30     Buckel      Linda         NaN            PO Box 683130      NaN      Park City    UT  840683130  4600.0  2007-08-14            20\n",
       "159     ABATE      MARIA       ELENA  1291 NIGHTINGALE AVENUE      NaN  MIAMI SPRINGS    FL  331663832  2600.0  2008-01-25            37"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[dfcwci.amount > dfcwci.amount.max() - 2300]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This also shows us a subselect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>31</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>4600</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>160</td>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>ELENA</td>\n",
       "      <td>1291 NIGHTINGALE AVENUE</td>\n",
       "      <td>None</td>\n",
       "      <td>MIAMI SPRINGS</td>\n",
       "      <td>FL</td>\n",
       "      <td>331663832</td>\n",
       "      <td>2600</td>\n",
       "      <td>2008-01-25</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id last_name first_name middle_name                 street_1 street_2           city state        zip  amount        date  candidate_id\n",
       "0   31    Buckel      Linda        None            PO Box 683130     None      Park City    UT  840683130    4600  2007-08-14            20\n",
       "1  160     ABATE      MARIA       ELENA  1291 NIGHTINGALE AVENUE     None  MIAMI SPRINGS    FL  331663832    2600  2008-01-25            37"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors WHERE amount > (select (MAX(amount) - 2300) FROM contributors);\")\n",
    "make_frame(out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aso `MIN`, `SUM`, `AVG`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GROUP-AGG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>state</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AK</th>\n",
       "      <td>2985459621</td>\n",
       "      <td>1210.00</td>\n",
       "      <td>111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AR</th>\n",
       "      <td>864790</td>\n",
       "      <td>14200.00</td>\n",
       "      <td>192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AZ</th>\n",
       "      <td>860011121</td>\n",
       "      <td>120.00</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CA</th>\n",
       "      <td>14736360720</td>\n",
       "      <td>-5013.73</td>\n",
       "      <td>600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CO</th>\n",
       "      <td>2405477834</td>\n",
       "      <td>-5823.00</td>\n",
       "      <td>111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CT</th>\n",
       "      <td>68901376</td>\n",
       "      <td>2300.00</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DC</th>\n",
       "      <td>800341853</td>\n",
       "      <td>-1549.91</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FL</th>\n",
       "      <td>8970626520</td>\n",
       "      <td>-4050.00</td>\n",
       "      <td>803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IA</th>\n",
       "      <td>50266</td>\n",
       "      <td>250.00</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ID</th>\n",
       "      <td>83648</td>\n",
       "      <td>-261.00</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IL</th>\n",
       "      <td>3042068689</td>\n",
       "      <td>-5586.80</td>\n",
       "      <td>175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KS</th>\n",
       "      <td>66215</td>\n",
       "      <td>-330.00</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>KY</th>\n",
       "      <td>402597029</td>\n",
       "      <td>-200.00</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LA</th>\n",
       "      <td>1406043327</td>\n",
       "      <td>1300.00</td>\n",
       "      <td>74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MA</th>\n",
       "      <td>123026638</td>\n",
       "      <td>-83.00</td>\n",
       "      <td>208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MD</th>\n",
       "      <td>416287617</td>\n",
       "      <td>300.00</td>\n",
       "      <td>55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ME</th>\n",
       "      <td>165647170</td>\n",
       "      <td>2520.00</td>\n",
       "      <td>122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MI</th>\n",
       "      <td>2426973485</td>\n",
       "      <td>-1265.00</td>\n",
       "      <td>164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MN</th>\n",
       "      <td>1102338918</td>\n",
       "      <td>322.00</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MO</th>\n",
       "      <td>64111</td>\n",
       "      <td>100.00</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NC</th>\n",
       "      <td>27502</td>\n",
       "      <td>500.00</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NH</th>\n",
       "      <td>32564424</td>\n",
       "      <td>-24.60</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NJ</th>\n",
       "      <td>70254993</td>\n",
       "      <td>-817.45</td>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NV</th>\n",
       "      <td>3575889763</td>\n",
       "      <td>725.00</td>\n",
       "      <td>144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NY</th>\n",
       "      <td>606129991</td>\n",
       "      <td>-6474.50</td>\n",
       "      <td>233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OH</th>\n",
       "      <td>176071</td>\n",
       "      <td>450.00</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>OK</th>\n",
       "      <td>2202499044</td>\n",
       "      <td>800.00</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PA</th>\n",
       "      <td>540499020</td>\n",
       "      <td>-2146.00</td>\n",
       "      <td>145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RI</th>\n",
       "      <td>58065892</td>\n",
       "      <td>200.00</td>\n",
       "      <td>70</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SC</th>\n",
       "      <td>296214789</td>\n",
       "      <td>2400.00</td>\n",
       "      <td>69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TN</th>\n",
       "      <td>37188</td>\n",
       "      <td>-25.00</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TX</th>\n",
       "      <td>6221452245</td>\n",
       "      <td>1985.24</td>\n",
       "      <td>302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UT</th>\n",
       "      <td>9251153394</td>\n",
       "      <td>5050.00</td>\n",
       "      <td>340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VA</th>\n",
       "      <td>440691831</td>\n",
       "      <td>515.92</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WA</th>\n",
       "      <td>2941290251</td>\n",
       "      <td>-500.00</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               zip    amount  candidate_id\n",
       "state                                     \n",
       "AK      2985459621   1210.00           111\n",
       "AR          864790  14200.00           192\n",
       "AZ       860011121    120.00            37\n",
       "CA     14736360720  -5013.73           600\n",
       "CO      2405477834  -5823.00           111\n",
       "CT        68901376   2300.00            35\n",
       "DC       800341853  -1549.91           102\n",
       "FL      8970626520  -4050.00           803\n",
       "IA           50266    250.00            16\n",
       "ID           83648   -261.00            32\n",
       "IL      3042068689  -5586.80           175\n",
       "KS           66215   -330.00            32\n",
       "KY       402597029   -200.00            22\n",
       "LA      1406043327   1300.00            74\n",
       "MA       123026638    -83.00           208\n",
       "MD       416287617    300.00            55\n",
       "ME       165647170   2520.00           122\n",
       "MI      2426973485  -1265.00           164\n",
       "MN      1102338918    322.00           100\n",
       "MO           64111    100.00            20\n",
       "NC           27502    500.00            16\n",
       "NH        32564424    -24.60            32\n",
       "NJ        70254993   -817.45            64\n",
       "NV      3575889763    725.00           144\n",
       "NY       606129991  -6474.50           233\n",
       "OH          176071    450.00            80\n",
       "OK      2202499044    800.00           102\n",
       "PA       540499020  -2146.00           145\n",
       "RI        58065892    200.00            70\n",
       "SC       296214789   2400.00            69\n",
       "TN           37188    -25.00            32\n",
       "TX      6221452245   1985.24           302\n",
       "UT      9251153394   5050.00           340\n",
       "VA       440691831    515.92           135\n",
       "WA      2941290251   -500.00            90"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.groupby(\"state\").sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "state\n",
       "AK     403.333333\n",
       "AR    1183.333333\n",
       "AZ     120.000000\n",
       "CA    -217.988261\n",
       "CO   -1455.750000\n",
       "CT    2300.000000\n",
       "DC    -309.982000\n",
       "FL    -135.000000\n",
       "IA     250.000000\n",
       "ID    -261.000000\n",
       "IL    -931.133333\n",
       "KS    -330.000000\n",
       "KY    -200.000000\n",
       "LA     650.000000\n",
       "MA     -13.833333\n",
       "MD     150.000000\n",
       "ME     630.000000\n",
       "MI    -253.000000\n",
       "MN     107.333333\n",
       "MO     100.000000\n",
       "NC     500.000000\n",
       "NH     -24.600000\n",
       "NJ    -408.725000\n",
       "NV     181.250000\n",
       "NY    -809.312500\n",
       "OH     112.500000\n",
       "OK     266.666667\n",
       "PA    -429.200000\n",
       "RI     100.000000\n",
       "SC     800.000000\n",
       "TN     -25.000000\n",
       "TX     220.582222\n",
       "UT     459.090909\n",
       "VA     103.184000\n",
       "WA    -166.666667\n",
       "Name: amount, dtype: float64"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.groupby(\"state\")['amount'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['VA', 'CA', 'AR', 'DC', 'SC', 'IA', 'OH', 'NC', 'UT', 'MO', 'IL',\n",
       "       'ME', 'FL', 'MD', 'MI', 'CO', 'WA', 'NY', 'TX', 'KY', 'PA', 'TN',\n",
       "       'MA', 'MN', 'KS', 'NJ', 'NH', 'ID', 'OK', nan, 'NV', 'CT', 'RI',\n",
       "       'AK', 'LA', 'AZ'], dtype=object)"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.state.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>sum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>None</td>\n",
       "      <td>-500.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AK</td>\n",
       "      <td>1210.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AR</td>\n",
       "      <td>14200.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>AZ</td>\n",
       "      <td>120.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>CA</td>\n",
       "      <td>-5013.73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>CO</td>\n",
       "      <td>-5823.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>CT</td>\n",
       "      <td>2300.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DC</td>\n",
       "      <td>-1549.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>FL</td>\n",
       "      <td>-4050.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>IA</td>\n",
       "      <td>250.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>ID</td>\n",
       "      <td>-261.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>IL</td>\n",
       "      <td>-5586.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>KS</td>\n",
       "      <td>-330.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>KY</td>\n",
       "      <td>-200.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>LA</td>\n",
       "      <td>1300.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>MA</td>\n",
       "      <td>-83.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>MD</td>\n",
       "      <td>300.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>ME</td>\n",
       "      <td>2520.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>MI</td>\n",
       "      <td>-1265.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>MN</td>\n",
       "      <td>322.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>MO</td>\n",
       "      <td>100.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NC</td>\n",
       "      <td>500.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NH</td>\n",
       "      <td>-24.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NJ</td>\n",
       "      <td>-817.45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NV</td>\n",
       "      <td>725.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>NY</td>\n",
       "      <td>-6474.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>OH</td>\n",
       "      <td>450.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>OK</td>\n",
       "      <td>800.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>PA</td>\n",
       "      <td>-2146.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>RI</td>\n",
       "      <td>200.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>SC</td>\n",
       "      <td>2400.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>TN</td>\n",
       "      <td>-25.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>TX</td>\n",
       "      <td>1985.24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>UT</td>\n",
       "      <td>5050.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>VA</td>\n",
       "      <td>515.92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>WA</td>\n",
       "      <td>-500.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   state       sum\n",
       "0   None   -500.00\n",
       "1     AK   1210.00\n",
       "2     AR  14200.00\n",
       "3     AZ    120.00\n",
       "4     CA  -5013.73\n",
       "5     CO  -5823.00\n",
       "6     CT   2300.00\n",
       "7     DC  -1549.91\n",
       "8     FL  -4050.00\n",
       "9     IA    250.00\n",
       "10    ID   -261.00\n",
       "11    IL  -5586.80\n",
       "12    KS   -330.00\n",
       "13    KY   -200.00\n",
       "14    LA   1300.00\n",
       "15    MA    -83.00\n",
       "16    MD    300.00\n",
       "17    ME   2520.00\n",
       "18    MI  -1265.00\n",
       "19    MN    322.00\n",
       "20    MO    100.00\n",
       "21    NC    500.00\n",
       "22    NH    -24.60\n",
       "23    NJ   -817.45\n",
       "24    NV    725.00\n",
       "25    NY  -6474.50\n",
       "26    OH    450.00\n",
       "27    OK    800.00\n",
       "28    PA  -2146.00\n",
       "29    RI    200.00\n",
       "30    SC   2400.00\n",
       "31    TN    -25.00\n",
       "32    TX   1985.24\n",
       "33    UT   5050.00\n",
       "34    VA    515.92\n",
       "35    WA   -500.00"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT state,SUM(amount) FROM contributors GROUP BY state;\")\n",
    "make_frame(out, legend=['state','sum'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DELETE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ahrens</td>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0      Agee     Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   250.0  2007-05-16            16\n",
       "2    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566    50.0  2007-06-18            16\n",
       "3    Ahrens        Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   100.0  2007-06-21            16\n",
       "4      Akin    Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In-place drops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>last_name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Agee</th>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ahrens</th>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-05-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ahrens</th>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-06-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ahrens</th>\n",
       "      <td>Don</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4034 Rennellwood Way</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pleasanton</td>\n",
       "      <td>CA</td>\n",
       "      <td>94566</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-21</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Akin</th>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "last_name                                                                                                                    \n",
       "Agee          Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "Ahrens           Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   250.0  2007-05-16            16\n",
       "Ahrens           Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566    50.0  2007-06-18            16\n",
       "Ahrens           Don         NaN    4034 Rennellwood Way      NaN   Pleasanton    CA  94566   100.0  2007-06-21            16\n",
       "Akin         Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2=dfcwci.copy()\n",
    "df2.set_index('last_name', inplace=True)\n",
    "df2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>last_name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Agee</th>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Akin</th>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Akin</th>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Akin</th>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Aldridge</th>\n",
       "      <td>Brittni</td>\n",
       "      <td>NaN</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          first_name middle_name                      street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "last_name                                                                                                                          \n",
       "Agee          Steven         NaN        549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "Akin         Charles         NaN        10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16\n",
       "Akin            Mike         NaN              181 Baywood Lane      NaN   Monticello    AR  71655  1500.0  2007-05-18            16\n",
       "Akin         Rebecca         NaN              181 Baywood Lane      NaN   Monticello    AR  71655   500.0  2007-05-18            16\n",
       "Aldridge     Brittni         NaN  808 Capitol Square Place, SW      NaN   Washington    DC  20024   250.0  2007-06-06            16"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.drop(['Ahrens'], inplace=True)\n",
    "df2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>NaN</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                      street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0      Agee     Steven         NaN        549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1      Akin    Charles         NaN        10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16\n",
       "2      Akin       Mike         NaN              181 Baywood Lane      NaN   Monticello    AR  71655  1500.0  2007-05-18            16\n",
       "3      Akin    Rebecca         NaN              181 Baywood Lane      NaN   Monticello    AR  71655   500.0  2007-05-18            16\n",
       "4  Aldridge    Brittni         NaN  808 Capitol Square Place, SW      NaN   Washington    DC  20024   250.0  2007-06-06            16"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.reset_index(inplace=True)\n",
    "df2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The recommended way to do it is to create a new dataframe. This might be impractical is things are very large."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>NaN</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-06-11</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>P.O. Box 1089</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Conway</td>\n",
       "      <td>AR</td>\n",
       "      <td>72033</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3206 Summit Court</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>AR</td>\n",
       "      <td>72227</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-04-25</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3206 Summit Court</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>AR</td>\n",
       "      <td>72227</td>\n",
       "      <td>200.0</td>\n",
       "      <td>2007-06-12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last_name first_name middle_name                      street_1 street_2           city state    zip  amount        date  candidate_id\n",
       "0       Agee     Steven         NaN        549 Laurel Branch Road      NaN          Floyd    VA  24091   500.0  2007-06-30            16\n",
       "4       Akin    Charles         NaN        10187 Sugar Creek Road      NaN    Bentonville    AR  72712   100.0  2007-06-16            16\n",
       "5       Akin       Mike         NaN              181 Baywood Lane      NaN     Monticello    AR  71655  1500.0  2007-05-18            16\n",
       "6       Akin    Rebecca         NaN              181 Baywood Lane      NaN     Monticello    AR  71655   500.0  2007-05-18            16\n",
       "7   Aldridge    Brittni         NaN  808 Capitol Square Place, SW      NaN     Washington    DC  20024   250.0  2007-06-06            16\n",
       "8      Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1000.0  2007-06-11            16\n",
       "9      Allen    John D.         NaN        1052 Cannon Mill Drive      NaN  North Augusta    SC  29860  1300.0  2007-06-29            16\n",
       "10   Allison    John W.         NaN                 P.O. Box 1089      NaN         Conway    AR  72033  1000.0  2007-05-18            16\n",
       "11   Allison    Rebecca         NaN             3206 Summit Court      NaN    Little Rock    AR  72227  1000.0  2007-04-25            16\n",
       "12   Allison    Rebecca         NaN             3206 Summit Court      NaN    Little Rock    AR  72227   200.0  2007-06-12            16"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci=dfcwci[dfcwci.last_name!='Ahrens']\n",
    "dfcwci.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sqlite3.Cursor at 0x1133f8ab0>"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "drow=\"DELETE FROM contributors WHERE last_name=\\\"Ahrens\\\";\"\n",
    "db.cursor().execute(drow)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>None</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>None</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>None</td>\n",
       "      <td>808 Capitol Square Place, SW</td>\n",
       "      <td>None</td>\n",
       "      <td>Washington</td>\n",
       "      <td>DC</td>\n",
       "      <td>20024</td>\n",
       "      <td>250.0</td>\n",
       "      <td>2007-06-06</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>9</td>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>None</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-06-11</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10</td>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>None</td>\n",
       "      <td>1052 Cannon Mill Drive</td>\n",
       "      <td>None</td>\n",
       "      <td>North Augusta</td>\n",
       "      <td>SC</td>\n",
       "      <td>29860</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>2007-06-29</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>11</td>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>None</td>\n",
       "      <td>P.O. Box 1089</td>\n",
       "      <td>None</td>\n",
       "      <td>Conway</td>\n",
       "      <td>AR</td>\n",
       "      <td>72033</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>12</td>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>3206 Summit Court</td>\n",
       "      <td>None</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>AR</td>\n",
       "      <td>72227</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>2007-04-25</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>13</td>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>None</td>\n",
       "      <td>3206 Summit Court</td>\n",
       "      <td>None</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>AR</td>\n",
       "      <td>72227</td>\n",
       "      <td>200.0</td>\n",
       "      <td>2007-06-12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name                      street_1 street_2           city state    zip  amount        date  candidate_id\n",
       "0   1      Agee     Steven        None        549 Laurel Branch Road     None          Floyd    VA  24091   500.0  2007-06-30            16\n",
       "1   5      Akin    Charles        None        10187 Sugar Creek Road     None    Bentonville    AR  72712   100.0  2007-06-16            16\n",
       "2   6      Akin       Mike        None              181 Baywood Lane     None     Monticello    AR  71655  1500.0  2007-05-18            16\n",
       "3   7      Akin    Rebecca        None              181 Baywood Lane     None     Monticello    AR  71655   500.0  2007-05-18            16\n",
       "4   8  Aldridge    Brittni        None  808 Capitol Square Place, SW     None     Washington    DC  20024   250.0  2007-06-06            16\n",
       "5   9     Allen    John D.        None        1052 Cannon Mill Drive     None  North Augusta    SC  29860  1000.0  2007-06-11            16\n",
       "6  10     Allen    John D.        None        1052 Cannon Mill Drive     None  North Augusta    SC  29860  1300.0  2007-06-29            16\n",
       "7  11   Allison    John W.        None                 P.O. Box 1089     None         Conway    AR  72033  1000.0  2007-05-18            16\n",
       "8  12   Allison    Rebecca        None             3206 Summit Court     None    Little Rock    AR  72227  1000.0  2007-04-25            16\n",
       "9  13   Allison    Rebecca        None             3206 Summit Court     None    Little Rock    AR  72227   200.0  2007-06-12            16"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.commit()\n",
    "out=make_query(\"SELECT * FROM contributors;\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LIMIT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>None</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>None</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>None</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>None</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>None</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0   1      Agee     Steven        None  549 Laurel Branch Road     None        Floyd    VA  24091     500  2007-06-30            16\n",
       "1   5      Akin    Charles        None  10187 Sugar Creek Road     None  Bentonville    AR  72712     100  2007-06-16            16\n",
       "2   6      Akin       Mike        None        181 Baywood Lane     None   Monticello    AR  71655    1500  2007-05-18            16"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out=make_query(\"SELECT * FROM contributors LIMIT 3;\")\n",
    "make_frame(out).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>NaN</td>\n",
       "      <td>549 Laurel Branch Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Floyd</td>\n",
       "      <td>VA</td>\n",
       "      <td>24091</td>\n",
       "      <td>500.0</td>\n",
       "      <td>2007-06-30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10187 Sugar Creek Road</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bentonville</td>\n",
       "      <td>AR</td>\n",
       "      <td>72712</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-06-16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181 Baywood Lane</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Monticello</td>\n",
       "      <td>AR</td>\n",
       "      <td>71655</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>2007-05-18</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  last_name first_name middle_name                street_1 street_2         city state    zip  amount        date  candidate_id\n",
       "0      Agee     Steven         NaN  549 Laurel Branch Road      NaN        Floyd    VA  24091   500.0  2007-06-30            16\n",
       "4      Akin    Charles         NaN  10187 Sugar Creek Road      NaN  Bentonville    AR  72712   100.0  2007-06-16            16\n",
       "5      Akin       Mike         NaN        181 Baywood Lane      NaN   Monticello    AR  71655  1500.0  2007-05-18            16"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci[0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indexes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "crind=\"CREATE INDEX amount_ix ON contributors(amount);\"\n",
    "db.cursor().execute(crind)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CREATE TABLE \"candidates\" (\r\n",
      "    \"id\" INTEGER PRIMARY KEY  NOT NULL ,\r\n",
      "    \"first_name\" VARCHAR,\r\n",
      "    \"last_name\" VARCHAR,\r\n",
      "    \"middle_name\" VARCHAR,\r\n",
      "    \"party\" VARCHAR NOT NULL\r\n",
      ");\r\n",
      "CREATE TABLE \"contributors\" (\r\n",
      "    \"id\" INTEGER PRIMARY KEY  AUTOINCREMENT  NOT NULL,\r\n",
      "    \"last_name\" VARCHAR,\r\n",
      "    \"first_name\" VARCHAR,\r\n",
      "    \"middle_name\" VARCHAR,\r\n",
      "    \"street_1\" VARCHAR,\r\n",
      "    \"street_2\" VARCHAR,\r\n",
      "    \"city\" VARCHAR,\r\n",
      "    \"state\" VARCHAR,\r\n",
      "    \"zip\" VARCHAR,\r\n",
      "    \"amount\" INTEGER,\r\n",
      "    \"date\" DATETIME,\r\n",
      "    \"candidate_id\" INTEGER NOT NULL, name,\r\n",
      "    FOREIGN KEY(candidate_id) REFERENCES candidates(id)\r\n",
      ");\r\n",
      "CREATE INDEX amount_ix ON contributors(amount);\r\n"
     ]
    }
   ],
   "source": [
    "! echo \".schema\" | sqlite3 /tmp/cancont.db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "crind=\"DROP INDEX amount_ix;\"\n",
    "db.cursor().execute(crind)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CREATE TABLE \"candidates\" (\n",
      "    \"id\" INTEGER PRIMARY KEY  NOT NULL ,\n",
      "    \"first_name\" VARCHAR,\n",
      "    \"last_name\" VARCHAR,\n",
      "    \"middle_name\" VARCHAR,\n",
      "    \"party\" VARCHAR NOT NULL\n",
      ");\n",
      "CREATE TABLE \"contributors\" (\n",
      "    \"id\" INTEGER PRIMARY KEY  AUTOINCREMENT  NOT NULL,\n",
      "    \"last_name\" VARCHAR,\n",
      "    \"first_name\" VARCHAR,\n",
      "    \"middle_name\" VARCHAR,\n",
      "    \"street_1\" VARCHAR,\n",
      "    \"street_2\" VARCHAR,\n",
      "    \"city\" VARCHAR,\n",
      "    \"state\" VARCHAR,\n",
      "    \"zip\" VARCHAR,\n",
      "    \"amount\" INTEGER,\n",
      "    \"date\" DATETIME,\n",
      "    \"candidate_id\" INTEGER NOT NULL, name,\n",
      "    FOREIGN KEY(candidate_id) REFERENCES candidates(id)\n",
      ");\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "echo \".schema\" | sqlite3 /tmp/cancont.db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "crind=\"CREATE INDEX amount_ix ON contributors(amount);\"\n",
    "db.cursor().execute(crind)\n",
    "db.commit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Relationships: JOINs are Cartesian Products."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simple subselect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>party</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>33</td>\n",
       "      <td>Joseph</td>\n",
       "      <td>Biden</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>36</td>\n",
       "      <td>Samuel</td>\n",
       "      <td>Brownback</td>\n",
       "      <td>NaN</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>34</td>\n",
       "      <td>Hillary</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>R.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>39</td>\n",
       "      <td>Christopher</td>\n",
       "      <td>Dodd</td>\n",
       "      <td>J.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>26</td>\n",
       "      <td>John</td>\n",
       "      <td>Edwards</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id   first_name  last_name middle_name party\n",
       "0  33       Joseph      Biden         NaN     D\n",
       "1  36       Samuel  Brownback         NaN     R\n",
       "2  34      Hillary    Clinton          R.     D\n",
       "3  39  Christopher       Dodd          J.     D\n",
       "4  26         John    Edwards         NaN     D"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcand.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "obamaid=dfcand.query(\"last_name=='Obama'\")['id'].values[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-07-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>25.0</td>\n",
       "      <td>2007-08-16</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last_name first_name middle_name         street_1 street_2        city state        zip  amount        date  candidate_id\n",
       "25   Buckler      Steve         NaN  24351 Armada Dr      NaN  Dana Point    CA  926291306    50.0  2007-07-30            20\n",
       "26   Buckler      Steve         NaN  24351 Armada Dr      NaN  Dana Point    CA  926291306    25.0  2007-08-16            20\n",
       "27  Buckheit      Bruce         NaN    8904 KAREN DR      NaN     FAIRFAX    VA  220312731   100.0  2007-09-19            20\n",
       "28    Buckel      Linda         NaN    PO Box 683130      NaN   Park City    UT  840683130  2300.0  2007-08-14            20\n",
       "29    Buckel      Linda         NaN    PO Box 683130      NaN   Park City    UT  840683130 -2300.0  2007-08-14            20"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obamacontrib=dfcwci.query(\"candidate_id==%i\" % obamaid)\n",
    "obamacontrib.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>street_1</th>\n",
       "      <th>street_2</th>\n",
       "      <th>city</th>\n",
       "      <th>state</th>\n",
       "      <th>zip</th>\n",
       "      <th>amount</th>\n",
       "      <th>date</th>\n",
       "      <th>candidate_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>26</td>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>None</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>None</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>50.0</td>\n",
       "      <td>2007-07-30</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>27</td>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>None</td>\n",
       "      <td>24351 Armada Dr</td>\n",
       "      <td>None</td>\n",
       "      <td>Dana Point</td>\n",
       "      <td>CA</td>\n",
       "      <td>926291306</td>\n",
       "      <td>25.0</td>\n",
       "      <td>2007-08-16</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>None</td>\n",
       "      <td>8904 KAREN DR</td>\n",
       "      <td>None</td>\n",
       "      <td>FAIRFAX</td>\n",
       "      <td>VA</td>\n",
       "      <td>220312731</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2007-09-19</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>29</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>30</td>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>None</td>\n",
       "      <td>PO Box 683130</td>\n",
       "      <td>None</td>\n",
       "      <td>Park City</td>\n",
       "      <td>UT</td>\n",
       "      <td>840683130</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>2007-08-14</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id last_name first_name middle_name         street_1 street_2        city state        zip  amount        date  candidate_id\n",
       "0  26   Buckler      Steve        None  24351 Armada Dr     None  Dana Point    CA  926291306    50.0  2007-07-30            20\n",
       "1  27   Buckler      Steve        None  24351 Armada Dr     None  Dana Point    CA  926291306    25.0  2007-08-16            20\n",
       "2  28  Buckheit      Bruce        None    8904 KAREN DR     None     FAIRFAX    VA  220312731   100.0  2007-09-19            20\n",
       "3  29    Buckel      Linda        None    PO Box 683130     None   Park City    UT  840683130  2300.0  2007-08-14            20\n",
       "4  30    Buckel      Linda        None    PO Box 683130     None   Park City    UT  840683130 -2300.0  2007-08-14            20"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "russiandollsel=\"\"\"\n",
    "SELECT * FROM contributors WHERE \n",
    "    candidate_id = (SELECT id from candidates WHERE last_name = 'Obama');\n",
    "\"\"\"\n",
    "out=make_query(russiandollsel)\n",
    "make_frame(out).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### implicit join"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('Buckler', 'Steve', 50, 'Obama'), ('Buckler', 'Steve', 25, 'Obama'), ('Buckheit', 'Bruce', 100, 'Obama'), ('Buckel', 'Linda', 2300, 'Obama'), ('Buckel', 'Linda', -2300, 'Obama'), ('Buckel', 'Linda', 4600, 'Obama'), ('Buck', 'Thomas', 100, 'Obama'), ('Buck', 'Jay', 200, 'Obama'), ('Buck', 'Blaine', 2300, 'Obama'), ('Buck', 'Barbara', 50, 'Obama'), ('Buck', 'Barbara', 50, 'Obama'), ('Buchman', 'Mark M', 460.8, 'Obama'), ('Bucher', 'Ida', 100, 'Obama'), ('Buchanek', 'Elizabeth', 50, 'Obama'), ('Buchanan', 'John', 500, 'Obama'), ('Buchanan', 'John', -500, 'Obama'), ('Buchanan', 'John', 500, 'Obama'), ('Buchanan', 'John', 700, 'Obama'), ('Buchanan', 'John', -700, 'Obama'), ('Buchanan', 'John', 1000, 'Obama'), ('Buchanan', 'John', 1300, 'Obama'), ('Buchanan', 'John', 200, 'Obama'), ('Buchanan', 'John', 500, 'Obama'), ('Buchanan', 'John', 200.09, 'Obama'), ('Harrison', 'Ryan', 25, 'Obama')]\n"
     ]
    }
   ],
   "source": [
    "implicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    contributors.last_name, contributors.first_name, contributors.amount, candidates.last_name \n",
    "FROM \n",
    "    contributors, candidates \n",
    "WHERE contributors.candidate_id = candidates.id\n",
    "AND candidates.last_name = 'Obama';\n",
    "\"\"\"\n",
    "out=make_query(implicitjoinsel)\n",
    "print(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>contributors.last_name</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>contributors.amount</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>50.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>25.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  contributors.last_name contributors.first_name  contributors.amount candidates.last_name\n",
       "0                Buckler                   Steve                 50.0                Obama\n",
       "1                Buckler                   Steve                 25.0                Obama\n",
       "2               Buckheit                   Bruce                100.0                Obama\n",
       "3                 Buckel                   Linda               2300.0                Obama\n",
       "4                 Buckel                   Linda              -2300.0                Obama"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_frame(out, legend=[\"contributors.last_name\", \n",
    "            \"contributors.first_name\", \"contributors.amount\", \"candidates.last_name\"]).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's expand to not just include Obama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "172\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>contributors.last_name</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>contributors.amount</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>250.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  contributors.last_name contributors.first_name  contributors.amount candidates.last_name\n",
       "0                   Agee                  Steven                500.0             Huckabee\n",
       "1                   Akin                 Charles                100.0             Huckabee\n",
       "2                   Akin                    Mike               1500.0             Huckabee\n",
       "3                   Akin                 Rebecca                500.0             Huckabee\n",
       "4               Aldridge                 Brittni                250.0             Huckabee"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "implicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    contributors.last_name, contributors.first_name, contributors.amount, candidates.last_name \n",
    "FROM \n",
    "    contributors, candidates \n",
    "WHERE contributors.candidate_id = candidates.id;\n",
    "\"\"\"\n",
    "out=make_query(implicitjoinsel)\n",
    "print(len(out))\n",
    "make_frame(out, legend=[\"contributors.last_name\", \n",
    "            \"contributors.first_name\", \"contributors.amount\", \"candidates.last_name\"]).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explicit INNER JOIN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![inner join](http://pandas.pydata.org/pandas-docs/stable/_images/merging_merge_on_key_inner.png)\n",
    "\n",
    "(from http://pandas.pydata.org/pandas-docs/stable/merging.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name_x</th>\n",
       "      <th>first_name_x</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>id</th>\n",
       "      <th>last_name_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Andres</td>\n",
       "      <td>Dale</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Bancroft</td>\n",
       "      <td>David</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Banks</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Barbee</td>\n",
       "      <td>John</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Jay</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>ABEDIN</td>\n",
       "      <td>ZAINUL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>ROBERT</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>MIKE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>ABBOT</td>\n",
       "      <td>DAVID</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>ABBO</td>\n",
       "      <td>PAULINE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>ABAIR</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>ABACHERLI</td>\n",
       "      <td>SHIRLEY</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>MARLING</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>RUDOLPH</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>ABELE</td>\n",
       "      <td>RODNEY</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>ABERCROMBIE</td>\n",
       "      <td>DENIS</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>ABESHAUS</td>\n",
       "      <td>MERRILL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>GEORGE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>ABRAHAMSON</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>172 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name_x first_name_x  candidate_id  id last_name_y\n",
       "0           Agee       Steven            16  16    Huckabee\n",
       "1           Akin      Charles            16  16    Huckabee\n",
       "2           Akin         Mike            16  16    Huckabee\n",
       "3           Akin      Rebecca            16  16    Huckabee\n",
       "4       Aldridge      Brittni            16  16    Huckabee\n",
       "5          Allen      John D.            16  16    Huckabee\n",
       "6          Allen      John D.            16  16    Huckabee\n",
       "7        Allison      John W.            16  16    Huckabee\n",
       "8        Allison      Rebecca            16  16    Huckabee\n",
       "9        Allison      Rebecca            16  16    Huckabee\n",
       "10         Altes         R.D.            16  16    Huckabee\n",
       "11        Andres         Dale            16  16    Huckabee\n",
       "12       Anthony         John            16  16    Huckabee\n",
       "13      Arbogast       Robert            16  16    Huckabee\n",
       "14      Arbogast       Robert            16  16    Huckabee\n",
       "15         Ardle      William            16  16    Huckabee\n",
       "16          Atiq         Omar            16  16    Huckabee\n",
       "17          Atiq         Omar            16  16    Huckabee\n",
       "18         Baker        David            16  16    Huckabee\n",
       "19      Bancroft        David            16  16    Huckabee\n",
       "20         Banks      Charles            16  16    Huckabee\n",
       "21        Barbee         John            16  16    Huckabee\n",
       "22       Buckler        Steve            20  20       Obama\n",
       "23       Buckler        Steve            20  20       Obama\n",
       "24      Buckheit        Bruce            20  20       Obama\n",
       "25        Buckel        Linda            20  20       Obama\n",
       "26        Buckel        Linda            20  20       Obama\n",
       "27        Buckel        Linda            20  20       Obama\n",
       "28          Buck       Thomas            20  20       Obama\n",
       "29          Buck          Jay            20  20       Obama\n",
       "..           ...          ...           ...  ..         ...\n",
       "142      ABDELLA       THOMAS            35  35      Romney\n",
       "143       ABBOTT       WELDON            35  35      Romney\n",
       "144       ABBOTT       WELDON            35  35      Romney\n",
       "145       ABBOTT       GERALD            35  35      Romney\n",
       "146       ABBOTT       GERALD            35  35      Romney\n",
       "147       ABEDIN       ZAINUL            37  37      McCain\n",
       "148       ABBOTT        SYBIL            37  37      McCain\n",
       "149       ABBOTT        SYBIL            37  37      McCain\n",
       "150       ABBOTT       RONALD            37  37      McCain\n",
       "151       ABBOTT       RONALD            37  37      McCain\n",
       "152       ABBOTT       ROBERT            37  37      McCain\n",
       "153       ABBOTT         MIKE            37  37      McCain\n",
       "154        ABBOT        DAVID            37  37      McCain\n",
       "155         ABBO      PAULINE            37  37      McCain\n",
       "156        ABATE        MARIA            37  37      McCain\n",
       "157        ABAIR        PETER            37  37      McCain\n",
       "158    ABACHERLI      SHIRLEY            37  37      McCain\n",
       "159       AARONS      CHARLES            37  37      McCain\n",
       "160       AARONS      CHARLES            37  37      McCain\n",
       "161       AARONS      CHARLES            37  37      McCain\n",
       "162         ABEL         JOHN            37  37      McCain\n",
       "163         ABEL      MARLING            37  37      McCain\n",
       "164         ABEL      RUDOLPH            37  37      McCain\n",
       "165        ABELE       RODNEY            37  37      McCain\n",
       "166  ABERCROMBIE        DENIS            37  37      McCain\n",
       "167     ABESHAUS      MERRILL            37  37      McCain\n",
       "168      ABRAHAM       GEORGE            37  37      McCain\n",
       "169   ABRAHAMSON        PETER            37  37      McCain\n",
       "170      ABRAHAM        SALEM            37  37      McCain\n",
       "171      ABRAHAM        SALEM            37  37      McCain\n",
       "\n",
       "[172 rows x 5 columns]"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols_wanted=['last_name_x', 'first_name_x', 'candidate_id', 'id', 'last_name_y']\n",
    "dfcwci.merge(dfcand, left_on=\"candidate_id\", right_on=\"id\")[cols_wanted]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>contributors.last_name</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  contributors.last_name contributors.first_name candidates.last_name\n",
       "0                   Agee                  Steven             Huckabee\n",
       "1                   Akin                 Charles             Huckabee\n",
       "2                   Akin                    Mike             Huckabee\n",
       "3                   Akin                 Rebecca             Huckabee\n",
       "4               Aldridge                 Brittni             Huckabee"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "explicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    contributors.last_name, contributors.first_name, candidates.last_name \n",
    "FROM \n",
    "    contributors JOIN candidates \n",
    "ON contributors.candidate_id = candidates.id;\n",
    "\"\"\"\n",
    "out=make_query(explicitjoinsel)\n",
    "make_frame(out, legend=[\"contributors.last_name\", \n",
    "            \"contributors.first_name\",  \"candidates.last_name\"]).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count(contributors.id)</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25</td>\n",
       "      <td>Lin</td>\n",
       "      <td>Clinton</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>25</td>\n",
       "      <td>TIMOTHY</td>\n",
       "      <td>Giuliani</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>22</td>\n",
       "      <td>John</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>25</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>25</td>\n",
       "      <td>Ryan</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>25</td>\n",
       "      <td>Richard</td>\n",
       "      <td>Paul</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>25</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   count(contributors.id) contributors.first_name candidates.last_name\n",
       "0                      25                     Lin              Clinton\n",
       "1                      25                 TIMOTHY             Giuliani\n",
       "2                      22                    John             Huckabee\n",
       "3                      25                   SALEM               McCain\n",
       "4                      25                    Ryan                Obama\n",
       "5                      25                 Richard                 Paul\n",
       "6                      25                  GERALD               Romney"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "explicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    COUNT(contributors.id), contributors.first_name, candidates.last_name \n",
    "FROM \n",
    "    contributors JOIN candidates \n",
    "ON contributors.candidate_id = candidates.id\n",
    "\n",
    "GROUP BY candidates.last_name;\n",
    "\"\"\"\n",
    "out=make_query(explicitjoinsel)\n",
    "print(len(out))\n",
    "make_frame(out, legend=[\"count(contributors.id)\", \n",
    "            \"contributors.first_name\",  \"candidates.last_name\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(make_query(\"SELECT DISTINCT id, last_name FROM candidates;\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>last_name_y</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Clinton</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>34.0</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Giuliani</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Huckabee</th>\n",
       "      <th>count</th>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>16.0</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">McCain</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Obama</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>20.0</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Paul</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>32.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">Romney</th>\n",
       "      <th>count</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>35.0</td>\n",
       "      <td>35.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   candidate_id    id\n",
       "last_name_y                          \n",
       "Clinton     count          25.0  25.0\n",
       "            mean           34.0  34.0\n",
       "            std             0.0   0.0\n",
       "            min            34.0  34.0\n",
       "            25%            34.0  34.0\n",
       "            50%            34.0  34.0\n",
       "            75%            34.0  34.0\n",
       "            max            34.0  34.0\n",
       "Giuliani    count          25.0  25.0\n",
       "            mean           22.0  22.0\n",
       "            std             0.0   0.0\n",
       "            min            22.0  22.0\n",
       "            25%            22.0  22.0\n",
       "            50%            22.0  22.0\n",
       "            75%            22.0  22.0\n",
       "            max            22.0  22.0\n",
       "Huckabee    count          22.0  22.0\n",
       "            mean           16.0  16.0\n",
       "            std             0.0   0.0\n",
       "            min            16.0  16.0\n",
       "            25%            16.0  16.0\n",
       "            50%            16.0  16.0\n",
       "            75%            16.0  16.0\n",
       "            max            16.0  16.0\n",
       "McCain      count          25.0  25.0\n",
       "            mean           37.0  37.0\n",
       "            std             0.0   0.0\n",
       "            min            37.0  37.0\n",
       "            25%            37.0  37.0\n",
       "            50%            37.0  37.0\n",
       "            75%            37.0  37.0\n",
       "            max            37.0  37.0\n",
       "Obama       count          25.0  25.0\n",
       "            mean           20.0  20.0\n",
       "            std             0.0   0.0\n",
       "            min            20.0  20.0\n",
       "            25%            20.0  20.0\n",
       "            50%            20.0  20.0\n",
       "            75%            20.0  20.0\n",
       "            max            20.0  20.0\n",
       "Paul        count          25.0  25.0\n",
       "            mean           32.0  32.0\n",
       "            std             0.0   0.0\n",
       "            min            32.0  32.0\n",
       "            25%            32.0  32.0\n",
       "            50%            32.0  32.0\n",
       "            75%            32.0  32.0\n",
       "            max            32.0  32.0\n",
       "Romney      count          25.0  25.0\n",
       "            mean           35.0  35.0\n",
       "            std             0.0   0.0\n",
       "            min            35.0  35.0\n",
       "            25%            35.0  35.0\n",
       "            50%            35.0  35.0\n",
       "            75%            35.0  35.0\n",
       "            max            35.0  35.0"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.merge(dfcand, left_on=\"candidate_id\", right_on=\"id\")[cols_wanted].groupby('last_name_y').describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outer JOIN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### left outer (contributors on candidates)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![left outer](http://pandas.pydata.org/pandas-docs/stable/_images/merging_merge_on_key_left.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name_x</th>\n",
       "      <th>first_name_x</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>id</th>\n",
       "      <th>last_name_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Andres</td>\n",
       "      <td>Dale</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Bancroft</td>\n",
       "      <td>David</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Banks</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Barbee</td>\n",
       "      <td>John</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Jay</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>ABEDIN</td>\n",
       "      <td>ZAINUL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>ROBERT</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>MIKE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>ABBOT</td>\n",
       "      <td>DAVID</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>ABBO</td>\n",
       "      <td>PAULINE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>ABAIR</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>ABACHERLI</td>\n",
       "      <td>SHIRLEY</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>MARLING</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>RUDOLPH</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>ABELE</td>\n",
       "      <td>RODNEY</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>ABERCROMBIE</td>\n",
       "      <td>DENIS</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>ABESHAUS</td>\n",
       "      <td>MERRILL</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>GEORGE</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>ABRAHAMSON</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>172 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name_x first_name_x  candidate_id  id last_name_y\n",
       "0           Agee       Steven            16  16    Huckabee\n",
       "1           Akin      Charles            16  16    Huckabee\n",
       "2           Akin         Mike            16  16    Huckabee\n",
       "3           Akin      Rebecca            16  16    Huckabee\n",
       "4       Aldridge      Brittni            16  16    Huckabee\n",
       "5          Allen      John D.            16  16    Huckabee\n",
       "6          Allen      John D.            16  16    Huckabee\n",
       "7        Allison      John W.            16  16    Huckabee\n",
       "8        Allison      Rebecca            16  16    Huckabee\n",
       "9        Allison      Rebecca            16  16    Huckabee\n",
       "10         Altes         R.D.            16  16    Huckabee\n",
       "11        Andres         Dale            16  16    Huckabee\n",
       "12       Anthony         John            16  16    Huckabee\n",
       "13      Arbogast       Robert            16  16    Huckabee\n",
       "14      Arbogast       Robert            16  16    Huckabee\n",
       "15         Ardle      William            16  16    Huckabee\n",
       "16          Atiq         Omar            16  16    Huckabee\n",
       "17          Atiq         Omar            16  16    Huckabee\n",
       "18         Baker        David            16  16    Huckabee\n",
       "19      Bancroft        David            16  16    Huckabee\n",
       "20         Banks      Charles            16  16    Huckabee\n",
       "21        Barbee         John            16  16    Huckabee\n",
       "22       Buckler        Steve            20  20       Obama\n",
       "23       Buckler        Steve            20  20       Obama\n",
       "24      Buckheit        Bruce            20  20       Obama\n",
       "25        Buckel        Linda            20  20       Obama\n",
       "26        Buckel        Linda            20  20       Obama\n",
       "27        Buckel        Linda            20  20       Obama\n",
       "28          Buck       Thomas            20  20       Obama\n",
       "29          Buck          Jay            20  20       Obama\n",
       "..           ...          ...           ...  ..         ...\n",
       "142      ABDELLA       THOMAS            35  35      Romney\n",
       "143       ABBOTT       WELDON            35  35      Romney\n",
       "144       ABBOTT       WELDON            35  35      Romney\n",
       "145       ABBOTT       GERALD            35  35      Romney\n",
       "146       ABBOTT       GERALD            35  35      Romney\n",
       "147       ABEDIN       ZAINUL            37  37      McCain\n",
       "148       ABBOTT        SYBIL            37  37      McCain\n",
       "149       ABBOTT        SYBIL            37  37      McCain\n",
       "150       ABBOTT       RONALD            37  37      McCain\n",
       "151       ABBOTT       RONALD            37  37      McCain\n",
       "152       ABBOTT       ROBERT            37  37      McCain\n",
       "153       ABBOTT         MIKE            37  37      McCain\n",
       "154        ABBOT        DAVID            37  37      McCain\n",
       "155         ABBO      PAULINE            37  37      McCain\n",
       "156        ABATE        MARIA            37  37      McCain\n",
       "157        ABAIR        PETER            37  37      McCain\n",
       "158    ABACHERLI      SHIRLEY            37  37      McCain\n",
       "159       AARONS      CHARLES            37  37      McCain\n",
       "160       AARONS      CHARLES            37  37      McCain\n",
       "161       AARONS      CHARLES            37  37      McCain\n",
       "162         ABEL         JOHN            37  37      McCain\n",
       "163         ABEL      MARLING            37  37      McCain\n",
       "164         ABEL      RUDOLPH            37  37      McCain\n",
       "165        ABELE       RODNEY            37  37      McCain\n",
       "166  ABERCROMBIE        DENIS            37  37      McCain\n",
       "167     ABESHAUS      MERRILL            37  37      McCain\n",
       "168      ABRAHAM       GEORGE            37  37      McCain\n",
       "169   ABRAHAMSON        PETER            37  37      McCain\n",
       "170      ABRAHAM        SALEM            37  37      McCain\n",
       "171      ABRAHAM        SALEM            37  37      McCain\n",
       "\n",
       "[172 rows x 5 columns]"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.merge(dfcand, left_on=\"candidate_id\", right_on=\"id\", how=\"left\")[cols_wanted]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count(contributors.id)</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>contributors.candidate_id</th>\n",
       "      <th>candidates.id</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25</td>\n",
       "      <td>Lin</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>34</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>25</td>\n",
       "      <td>TIMOTHY</td>\n",
       "      <td>Giuliani</td>\n",
       "      <td>22</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>22</td>\n",
       "      <td>John</td>\n",
       "      <td>Huckabee</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>25</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>McCain</td>\n",
       "      <td>37</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>25</td>\n",
       "      <td>Ryan</td>\n",
       "      <td>Obama</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>25</td>\n",
       "      <td>Richard</td>\n",
       "      <td>Paul</td>\n",
       "      <td>32</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>25</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>Romney</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   count(contributors.id) contributors.first_name contributors.candidate_id  candidates.id  candidates.last_name\n",
       "0                      25                     Lin                   Clinton             34                    34\n",
       "1                      25                 TIMOTHY                  Giuliani             22                    22\n",
       "2                      22                    John                  Huckabee             16                    16\n",
       "3                      25                   SALEM                    McCain             37                    37\n",
       "4                      25                    Ryan                     Obama             20                    20\n",
       "5                      25                 Richard                      Paul             32                    32\n",
       "6                      25                  GERALD                    Romney             35                    35"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "explicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    COUNT(contributors.id), contributors.first_name, candidates.last_name,\n",
    "        contributors.candidate_id, candidates.id\n",
    "FROM \n",
    "    contributors LEFT OUTER JOIN candidates \n",
    "ON contributors.candidate_id = candidates.id\n",
    "\n",
    "GROUP BY candidates.last_name;\n",
    "\"\"\"\n",
    "out=make_query(explicitjoinsel)\n",
    "make_frame(out, legend=[\"count(contributors.id)\", \"contributors.first_name\",  \n",
    "            \"contributors.candidate_id\", \"candidates.id\", \"candidates.last_name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### right outer (contributors on candidates) = left outer (candidates on contributors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We do it thus as sqlite has no support for right outer or plain outer. If it did we could write:\n",
    "\n",
    "```sql\n",
    "SELECT \n",
    "    COUNT(contributors.id), contributors.first_name, candidates.last_name \n",
    "FROM \n",
    "    contributors RIGHT OUTER JOIN candidates \n",
    "ON contributors.candidate_id = candidates.id\n",
    "\n",
    "GROUP BY candidates.last_name;\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![right outer](http://pandas.pydata.org/pandas-docs/stable/_images/merging_merge_on_key_right.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name_x</th>\n",
       "      <th>first_name_x</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>id</th>\n",
       "      <th>last_name_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Andres</td>\n",
       "      <td>Dale</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Bancroft</td>\n",
       "      <td>David</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Banks</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Barbee</td>\n",
       "      <td>John</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Jay</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>ROBERT</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>MIKE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>ABBOT</td>\n",
       "      <td>DAVID</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>ABBO</td>\n",
       "      <td>PAULINE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>ABAIR</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>ABACHERLI</td>\n",
       "      <td>SHIRLEY</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>MARLING</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>RUDOLPH</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>ABELE</td>\n",
       "      <td>RODNEY</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>ABERCROMBIE</td>\n",
       "      <td>DENIS</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>ABESHAUS</td>\n",
       "      <td>MERRILL</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>GEORGE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>ABRAHAMSON</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>33</td>\n",
       "      <td>Biden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36</td>\n",
       "      <td>Brownback</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39</td>\n",
       "      <td>Dodd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26</td>\n",
       "      <td>Edwards</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24</td>\n",
       "      <td>Gravel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30</td>\n",
       "      <td>Hunter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31</td>\n",
       "      <td>Kucinich</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>179</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29</td>\n",
       "      <td>Richardson</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38</td>\n",
       "      <td>Tancredo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>41</td>\n",
       "      <td>Thompson</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>182 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name_x first_name_x  candidate_id  id last_name_y\n",
       "0           Agee       Steven          16.0  16    Huckabee\n",
       "1           Akin      Charles          16.0  16    Huckabee\n",
       "2           Akin         Mike          16.0  16    Huckabee\n",
       "3           Akin      Rebecca          16.0  16    Huckabee\n",
       "4       Aldridge      Brittni          16.0  16    Huckabee\n",
       "5          Allen      John D.          16.0  16    Huckabee\n",
       "6          Allen      John D.          16.0  16    Huckabee\n",
       "7        Allison      John W.          16.0  16    Huckabee\n",
       "8        Allison      Rebecca          16.0  16    Huckabee\n",
       "9        Allison      Rebecca          16.0  16    Huckabee\n",
       "10         Altes         R.D.          16.0  16    Huckabee\n",
       "11        Andres         Dale          16.0  16    Huckabee\n",
       "12       Anthony         John          16.0  16    Huckabee\n",
       "13      Arbogast       Robert          16.0  16    Huckabee\n",
       "14      Arbogast       Robert          16.0  16    Huckabee\n",
       "15         Ardle      William          16.0  16    Huckabee\n",
       "16          Atiq         Omar          16.0  16    Huckabee\n",
       "17          Atiq         Omar          16.0  16    Huckabee\n",
       "18         Baker        David          16.0  16    Huckabee\n",
       "19      Bancroft        David          16.0  16    Huckabee\n",
       "20         Banks      Charles          16.0  16    Huckabee\n",
       "21        Barbee         John          16.0  16    Huckabee\n",
       "22       Buckler        Steve          20.0  20       Obama\n",
       "23       Buckler        Steve          20.0  20       Obama\n",
       "24      Buckheit        Bruce          20.0  20       Obama\n",
       "25        Buckel        Linda          20.0  20       Obama\n",
       "26        Buckel        Linda          20.0  20       Obama\n",
       "27        Buckel        Linda          20.0  20       Obama\n",
       "28          Buck       Thomas          20.0  20       Obama\n",
       "29          Buck          Jay          20.0  20       Obama\n",
       "..           ...          ...           ...  ..         ...\n",
       "152       ABBOTT       ROBERT          37.0  37      McCain\n",
       "153       ABBOTT         MIKE          37.0  37      McCain\n",
       "154        ABBOT        DAVID          37.0  37      McCain\n",
       "155         ABBO      PAULINE          37.0  37      McCain\n",
       "156        ABATE        MARIA          37.0  37      McCain\n",
       "157        ABAIR        PETER          37.0  37      McCain\n",
       "158    ABACHERLI      SHIRLEY          37.0  37      McCain\n",
       "159       AARONS      CHARLES          37.0  37      McCain\n",
       "160       AARONS      CHARLES          37.0  37      McCain\n",
       "161       AARONS      CHARLES          37.0  37      McCain\n",
       "162         ABEL         JOHN          37.0  37      McCain\n",
       "163         ABEL      MARLING          37.0  37      McCain\n",
       "164         ABEL      RUDOLPH          37.0  37      McCain\n",
       "165        ABELE       RODNEY          37.0  37      McCain\n",
       "166  ABERCROMBIE        DENIS          37.0  37      McCain\n",
       "167     ABESHAUS      MERRILL          37.0  37      McCain\n",
       "168      ABRAHAM       GEORGE          37.0  37      McCain\n",
       "169   ABRAHAMSON        PETER          37.0  37      McCain\n",
       "170      ABRAHAM        SALEM          37.0  37      McCain\n",
       "171      ABRAHAM        SALEM          37.0  37      McCain\n",
       "172          NaN          NaN           NaN  33       Biden\n",
       "173          NaN          NaN           NaN  36   Brownback\n",
       "174          NaN          NaN           NaN  39        Dodd\n",
       "175          NaN          NaN           NaN  26     Edwards\n",
       "176          NaN          NaN           NaN  24      Gravel\n",
       "177          NaN          NaN           NaN  30      Hunter\n",
       "178          NaN          NaN           NaN  31    Kucinich\n",
       "179          NaN          NaN           NaN  29  Richardson\n",
       "180          NaN          NaN           NaN  38    Tancredo\n",
       "181          NaN          NaN           NaN  41    Thompson\n",
       "\n",
       "[182 rows x 5 columns]"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.merge(dfcand, left_on=\"candidate_id\", right_on=\"id\", how=\"right\")[cols_wanted]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count(contributors.id)</th>\n",
       "      <th>contributors.first_name</th>\n",
       "      <th>contributors.candidate_id</th>\n",
       "      <th>candidates.id</th>\n",
       "      <th>candidates.last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Biden</td>\n",
       "      <td>NaN</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Brownback</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>25</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>34.0</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Dodd</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Edwards</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>25</td>\n",
       "      <td>WALTER</td>\n",
       "      <td>Giuliani</td>\n",
       "      <td>22.0</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Gravel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>22</td>\n",
       "      <td>William</td>\n",
       "      <td>Huckabee</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Hunter</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Kucinich</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>25</td>\n",
       "      <td>ZAINUL</td>\n",
       "      <td>McCain</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>25</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>Obama</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>25</td>\n",
       "      <td>William</td>\n",
       "      <td>Paul</td>\n",
       "      <td>32.0</td>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Richardson</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>25</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>Romney</td>\n",
       "      <td>35.0</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Tancredo</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>Thompson</td>\n",
       "      <td>NaN</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    count(contributors.id) contributors.first_name contributors.candidate_id  candidates.id  candidates.last_name\n",
       "0                        0                    None                     Biden            NaN                    33\n",
       "1                        0                    None                 Brownback            NaN                    36\n",
       "2                       25                  Thomas                   Clinton           34.0                    34\n",
       "3                        0                    None                      Dodd            NaN                    39\n",
       "4                        0                    None                   Edwards            NaN                    26\n",
       "5                       25                  WALTER                  Giuliani           22.0                    22\n",
       "6                        0                    None                    Gravel            NaN                    24\n",
       "7                       22                 William                  Huckabee           16.0                    16\n",
       "8                        0                    None                    Hunter            NaN                    30\n",
       "9                        0                    None                  Kucinich            NaN                    31\n",
       "10                      25                  ZAINUL                    McCain           37.0                    37\n",
       "11                      25                  Thomas                     Obama           20.0                    20\n",
       "12                      25                 William                      Paul           32.0                    32\n",
       "13                       0                    None                Richardson            NaN                    29\n",
       "14                      25                  WELDON                    Romney           35.0                    35\n",
       "15                       0                    None                  Tancredo            NaN                    38\n",
       "16                       0                    None                  Thompson            NaN                    41"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "explicitjoinsel=\"\"\"\n",
    "SELECT \n",
    "    COUNT(contributors.id), contributors.first_name, candidates.last_name, \n",
    "        contributors.candidate_id, candidates.id\n",
    "FROM \n",
    "    candidates LEFT OUTER JOIN contributors \n",
    "ON contributors.candidate_id = candidates.id\n",
    "\n",
    "GROUP BY candidates.last_name;\n",
    "\"\"\"\n",
    "out=make_query(explicitjoinsel)\n",
    "make_frame(out, legend=[\"count(contributors.id)\", \"contributors.first_name\",  \n",
    "                    \"contributors.candidate_id\", \"candidates.id\", \"candidates.last_name\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### full outer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "also not supported by sqlite\n",
    "\n",
    "```sql\n",
    "SELECT \n",
    "    COUNT(contributors.id), contributors.first_name, candidates.last_name \n",
    "FROM \n",
    "    contributors FULL OUTER JOIN candidates \n",
    "ON contributors.candidate_id = candidates.id\n",
    "\n",
    "GROUP BY candidates.last_name;\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![outer](http://pandas.pydata.org/pandas-docs/stable/_images/merging_merge_on_key_outer.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name_x</th>\n",
       "      <th>first_name_x</th>\n",
       "      <th>candidate_id</th>\n",
       "      <th>id</th>\n",
       "      <th>last_name_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Andres</td>\n",
       "      <td>Dale</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Bancroft</td>\n",
       "      <td>David</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Banks</td>\n",
       "      <td>Charles</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Barbee</td>\n",
       "      <td>John</td>\n",
       "      <td>16.0</td>\n",
       "      <td>16</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Jay</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>ROBERT</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>MIKE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>ABBOT</td>\n",
       "      <td>DAVID</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>ABBO</td>\n",
       "      <td>PAULINE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>ABAIR</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>ABACHERLI</td>\n",
       "      <td>SHIRLEY</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>MARLING</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>RUDOLPH</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>ABELE</td>\n",
       "      <td>RODNEY</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>ABERCROMBIE</td>\n",
       "      <td>DENIS</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>ABESHAUS</td>\n",
       "      <td>MERRILL</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>GEORGE</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>ABRAHAMSON</td>\n",
       "      <td>PETER</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>37.0</td>\n",
       "      <td>37</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>33</td>\n",
       "      <td>Biden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36</td>\n",
       "      <td>Brownback</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39</td>\n",
       "      <td>Dodd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>175</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26</td>\n",
       "      <td>Edwards</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24</td>\n",
       "      <td>Gravel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30</td>\n",
       "      <td>Hunter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31</td>\n",
       "      <td>Kucinich</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>179</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29</td>\n",
       "      <td>Richardson</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38</td>\n",
       "      <td>Tancredo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>41</td>\n",
       "      <td>Thompson</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>182 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     last_name_x first_name_x  candidate_id  id last_name_y\n",
       "0           Agee       Steven          16.0  16    Huckabee\n",
       "1           Akin      Charles          16.0  16    Huckabee\n",
       "2           Akin         Mike          16.0  16    Huckabee\n",
       "3           Akin      Rebecca          16.0  16    Huckabee\n",
       "4       Aldridge      Brittni          16.0  16    Huckabee\n",
       "5          Allen      John D.          16.0  16    Huckabee\n",
       "6          Allen      John D.          16.0  16    Huckabee\n",
       "7        Allison      John W.          16.0  16    Huckabee\n",
       "8        Allison      Rebecca          16.0  16    Huckabee\n",
       "9        Allison      Rebecca          16.0  16    Huckabee\n",
       "10         Altes         R.D.          16.0  16    Huckabee\n",
       "11        Andres         Dale          16.0  16    Huckabee\n",
       "12       Anthony         John          16.0  16    Huckabee\n",
       "13      Arbogast       Robert          16.0  16    Huckabee\n",
       "14      Arbogast       Robert          16.0  16    Huckabee\n",
       "15         Ardle      William          16.0  16    Huckabee\n",
       "16          Atiq         Omar          16.0  16    Huckabee\n",
       "17          Atiq         Omar          16.0  16    Huckabee\n",
       "18         Baker        David          16.0  16    Huckabee\n",
       "19      Bancroft        David          16.0  16    Huckabee\n",
       "20         Banks      Charles          16.0  16    Huckabee\n",
       "21        Barbee         John          16.0  16    Huckabee\n",
       "22       Buckler        Steve          20.0  20       Obama\n",
       "23       Buckler        Steve          20.0  20       Obama\n",
       "24      Buckheit        Bruce          20.0  20       Obama\n",
       "25        Buckel        Linda          20.0  20       Obama\n",
       "26        Buckel        Linda          20.0  20       Obama\n",
       "27        Buckel        Linda          20.0  20       Obama\n",
       "28          Buck       Thomas          20.0  20       Obama\n",
       "29          Buck          Jay          20.0  20       Obama\n",
       "..           ...          ...           ...  ..         ...\n",
       "152       ABBOTT       ROBERT          37.0  37      McCain\n",
       "153       ABBOTT         MIKE          37.0  37      McCain\n",
       "154        ABBOT        DAVID          37.0  37      McCain\n",
       "155         ABBO      PAULINE          37.0  37      McCain\n",
       "156        ABATE        MARIA          37.0  37      McCain\n",
       "157        ABAIR        PETER          37.0  37      McCain\n",
       "158    ABACHERLI      SHIRLEY          37.0  37      McCain\n",
       "159       AARONS      CHARLES          37.0  37      McCain\n",
       "160       AARONS      CHARLES          37.0  37      McCain\n",
       "161       AARONS      CHARLES          37.0  37      McCain\n",
       "162         ABEL         JOHN          37.0  37      McCain\n",
       "163         ABEL      MARLING          37.0  37      McCain\n",
       "164         ABEL      RUDOLPH          37.0  37      McCain\n",
       "165        ABELE       RODNEY          37.0  37      McCain\n",
       "166  ABERCROMBIE        DENIS          37.0  37      McCain\n",
       "167     ABESHAUS      MERRILL          37.0  37      McCain\n",
       "168      ABRAHAM       GEORGE          37.0  37      McCain\n",
       "169   ABRAHAMSON        PETER          37.0  37      McCain\n",
       "170      ABRAHAM        SALEM          37.0  37      McCain\n",
       "171      ABRAHAM        SALEM          37.0  37      McCain\n",
       "172          NaN          NaN           NaN  33       Biden\n",
       "173          NaN          NaN           NaN  36   Brownback\n",
       "174          NaN          NaN           NaN  39        Dodd\n",
       "175          NaN          NaN           NaN  26     Edwards\n",
       "176          NaN          NaN           NaN  24      Gravel\n",
       "177          NaN          NaN           NaN  30      Hunter\n",
       "178          NaN          NaN           NaN  31    Kucinich\n",
       "179          NaN          NaN           NaN  29  Richardson\n",
       "180          NaN          NaN           NaN  38    Tancredo\n",
       "181          NaN          NaN           NaN  41    Thompson\n",
       "\n",
       "[182 rows x 5 columns]"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfcwci.merge(dfcand, left_on=\"candidate_id\", right_on=\"id\", how=\"outer\")[cols_wanted]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When and how to use which?\n",
    "\n",
    "See this:\n",
    "\n",
    "http://blog.codinghorror.com/a-visual-explanation-of-sql-joins/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas /SQL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>first_name</th>\n",
       "      <th>last_name</th>\n",
       "      <th>middle_name</th>\n",
       "      <th>party</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20</td>\n",
       "      <td>Barack</td>\n",
       "      <td>Obama</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>24</td>\n",
       "      <td>Mike</td>\n",
       "      <td>Gravel</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26</td>\n",
       "      <td>John</td>\n",
       "      <td>Edwards</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>29</td>\n",
       "      <td>Bill</td>\n",
       "      <td>Richardson</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>31</td>\n",
       "      <td>Dennis</td>\n",
       "      <td>Kucinich</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>33</td>\n",
       "      <td>Joseph</td>\n",
       "      <td>Biden</td>\n",
       "      <td></td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>34</td>\n",
       "      <td>Hillary</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>R.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>39</td>\n",
       "      <td>Christopher</td>\n",
       "      <td>Dodd</td>\n",
       "      <td>J.</td>\n",
       "      <td>D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id   first_name   last_name middle_name party\n",
       "0  20       Barack       Obama                 D\n",
       "1  24         Mike      Gravel                 D\n",
       "2  26         John     Edwards                 D\n",
       "3  29         Bill  Richardson                 D\n",
       "4  31       Dennis    Kucinich                 D\n",
       "5  33       Joseph       Biden                 D\n",
       "6  34      Hillary     Clinton          R.     D\n",
       "7  39  Christopher        Dodd          J.     D"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_sql(\"SELECT * FROM candidates WHERE party= 'D';\", db)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last_name</th>\n",
       "      <th>first_name</th>\n",
       "      <th>amount</th>\n",
       "      <th>last_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Agee</td>\n",
       "      <td>Steven</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Charles</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Mike</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Akin</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Aldridge</td>\n",
       "      <td>Brittni</td>\n",
       "      <td>250.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Allen</td>\n",
       "      <td>John D.</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Allison</td>\n",
       "      <td>John W.</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Allison</td>\n",
       "      <td>Rebecca</td>\n",
       "      <td>200.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Altes</td>\n",
       "      <td>R.D.</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Andres</td>\n",
       "      <td>Dale</td>\n",
       "      <td>250.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Anthony</td>\n",
       "      <td>John</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Arbogast</td>\n",
       "      <td>Robert</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Ardle</td>\n",
       "      <td>William</td>\n",
       "      <td>50.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Atiq</td>\n",
       "      <td>Omar</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Baker</td>\n",
       "      <td>David</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Bancroft</td>\n",
       "      <td>David</td>\n",
       "      <td>250.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Banks</td>\n",
       "      <td>Charles</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Barbee</td>\n",
       "      <td>John</td>\n",
       "      <td>500.0</td>\n",
       "      <td>Huckabee</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>50.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Buckler</td>\n",
       "      <td>Steve</td>\n",
       "      <td>25.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Buckheit</td>\n",
       "      <td>Bruce</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>-2300.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Buckel</td>\n",
       "      <td>Linda</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Thomas</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Buck</td>\n",
       "      <td>Jay</td>\n",
       "      <td>200.0</td>\n",
       "      <td>Obama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>142</th>\n",
       "      <td>ABDELLA</td>\n",
       "      <td>THOMAS</td>\n",
       "      <td>50.0</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>144</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>WELDON</td>\n",
       "      <td>50.0</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>GERALD</td>\n",
       "      <td>100.0</td>\n",
       "      <td>Romney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>ABEDIN</td>\n",
       "      <td>ZAINUL</td>\n",
       "      <td>500.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>75.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>SYBIL</td>\n",
       "      <td>50.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>200.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>RONALD</td>\n",
       "      <td>100.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>ROBERT</td>\n",
       "      <td>500.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>ABBOTT</td>\n",
       "      <td>MIKE</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>ABBOT</td>\n",
       "      <td>DAVID</td>\n",
       "      <td>200.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>ABBO</td>\n",
       "      <td>PAULINE</td>\n",
       "      <td>35.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>ABATE</td>\n",
       "      <td>MARIA</td>\n",
       "      <td>2600.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>ABAIR</td>\n",
       "      <td>PETER</td>\n",
       "      <td>25.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>ABACHERLI</td>\n",
       "      <td>SHIRLEY</td>\n",
       "      <td>150.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>300.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>410.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>AARONS</td>\n",
       "      <td>CHARLES</td>\n",
       "      <td>500.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>JOHN</td>\n",
       "      <td>200.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>MARLING</td>\n",
       "      <td>100.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>ABEL</td>\n",
       "      <td>RUDOLPH</td>\n",
       "      <td>100.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>ABELE</td>\n",
       "      <td>RODNEY</td>\n",
       "      <td>500.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>ABERCROMBIE</td>\n",
       "      <td>DENIS</td>\n",
       "      <td>500.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>ABESHAUS</td>\n",
       "      <td>MERRILL</td>\n",
       "      <td>120.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>GEORGE</td>\n",
       "      <td>800.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>ABRAHAMSON</td>\n",
       "      <td>PETER</td>\n",
       "      <td>50.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>ABRAHAM</td>\n",
       "      <td>SALEM</td>\n",
       "      <td>1300.0</td>\n",
       "      <td>McCain</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>172 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       last_name first_name  amount last_name\n",
       "0           Agee     Steven   500.0  Huckabee\n",
       "1           Akin    Charles   100.0  Huckabee\n",
       "2           Akin       Mike  1500.0  Huckabee\n",
       "3           Akin    Rebecca   500.0  Huckabee\n",
       "4       Aldridge    Brittni   250.0  Huckabee\n",
       "5          Allen    John D.  1000.0  Huckabee\n",
       "6          Allen    John D.  1300.0  Huckabee\n",
       "7        Allison    John W.  1000.0  Huckabee\n",
       "8        Allison    Rebecca  1000.0  Huckabee\n",
       "9        Allison    Rebecca   200.0  Huckabee\n",
       "10         Altes       R.D.  2300.0  Huckabee\n",
       "11        Andres       Dale   250.0  Huckabee\n",
       "12       Anthony       John  2300.0  Huckabee\n",
       "13      Arbogast     Robert   500.0  Huckabee\n",
       "14      Arbogast     Robert   100.0  Huckabee\n",
       "15         Ardle    William    50.0  Huckabee\n",
       "16          Atiq       Omar  1000.0  Huckabee\n",
       "17          Atiq       Omar  1000.0  Huckabee\n",
       "18         Baker      David  2300.0  Huckabee\n",
       "19      Bancroft      David   250.0  Huckabee\n",
       "20         Banks    Charles  1000.0  Huckabee\n",
       "21        Barbee       John   500.0  Huckabee\n",
       "22       Buckler      Steve    50.0     Obama\n",
       "23       Buckler      Steve    25.0     Obama\n",
       "24      Buckheit      Bruce   100.0     Obama\n",
       "25        Buckel      Linda  2300.0     Obama\n",
       "26        Buckel      Linda -2300.0     Obama\n",
       "27        Buckel      Linda  4600.0     Obama\n",
       "28          Buck     Thomas   100.0     Obama\n",
       "29          Buck        Jay   200.0     Obama\n",
       "..           ...        ...     ...       ...\n",
       "142      ABDELLA     THOMAS    50.0    Romney\n",
       "143       ABBOTT     WELDON   100.0    Romney\n",
       "144       ABBOTT     WELDON    50.0    Romney\n",
       "145       ABBOTT     GERALD   100.0    Romney\n",
       "146       ABBOTT     GERALD   100.0    Romney\n",
       "147       ABEDIN     ZAINUL   500.0    McCain\n",
       "148       ABBOTT      SYBIL    75.0    McCain\n",
       "149       ABBOTT      SYBIL    50.0    McCain\n",
       "150       ABBOTT     RONALD   200.0    McCain\n",
       "151       ABBOTT     RONALD   100.0    McCain\n",
       "152       ABBOTT     ROBERT   500.0    McCain\n",
       "153       ABBOTT       MIKE  1000.0    McCain\n",
       "154        ABBOT      DAVID   200.0    McCain\n",
       "155         ABBO    PAULINE    35.0    McCain\n",
       "156        ABATE      MARIA  2600.0    McCain\n",
       "157        ABAIR      PETER    25.0    McCain\n",
       "158    ABACHERLI    SHIRLEY   150.0    McCain\n",
       "159       AARONS    CHARLES   300.0    McCain\n",
       "160       AARONS    CHARLES   410.0    McCain\n",
       "161       AARONS    CHARLES   500.0    McCain\n",
       "162         ABEL       JOHN   200.0    McCain\n",
       "163         ABEL    MARLING   100.0    McCain\n",
       "164         ABEL    RUDOLPH   100.0    McCain\n",
       "165        ABELE     RODNEY   500.0    McCain\n",
       "166  ABERCROMBIE      DENIS   500.0    McCain\n",
       "167     ABESHAUS    MERRILL   120.0    McCain\n",
       "168      ABRAHAM     GEORGE   800.0    McCain\n",
       "169   ABRAHAMSON      PETER    50.0    McCain\n",
       "170      ABRAHAM      SALEM  1000.0    McCain\n",
       "171      ABRAHAM      SALEM  1300.0    McCain\n",
       "\n",
       "[172 rows x 4 columns]"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_sql(implicitjoinsel, db)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is very useful if the database is big and out of memory. Sqlite3 is the only db2api database supported. For any other database you should use `SQLAlchemy`. See, for eg: https://plot.ly/ipython-notebooks/big-data-analytics-with-pandas-and-sqlite/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "db.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Useful Links"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- http://sebastianraschka.com/Articles/sqlite3_database.html and  http://sebastianraschka.com/Articles/2014_sqlite_in_python_tutorial.html#unique_indexes\n",
    "- https://github.com/tthibo/SQL-Tutorial\n",
    "- chrisalbon.com\n",
    "\n",
    "And especially for R users:\n",
    "\n",
    "- https://cran.r-project.org/web/packages/dplyr/vignettes/introduction.html\n",
    "- https://gist.github.com/TomAugspurger/6e052140eaa5fdb6e8c0/"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}