databases/03 Strings and regular expressions.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:427e7806790659def09104ed67e700b6c22a56c7c4194d50a000188b7858f968"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#String operations and regular expressions\n",
      "\n",
      "Today we talk about strings. When we have a string, we might want to ask whether it has particular characteristics---does it start with a particular character? Does it contain within it another string?---or try to extract smaller parts of the string, like the first fifteen characters, or say, the part of the string inside parentheses. Or we may want to transform the string into another string altogether, by (for example) converting its characters to upper case, or replacing substrings within it with other substrings. Today we discuss how to do these things in Python.\n",
      "\n",
      "##Simple string checks\n",
      "\n",
      "There are a number of functions, methods and operators that can tell us whether or not a Python string matches certain characteristics. Let's talk about the `in` operator first:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"foo\" in \"buffoon\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 1,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"foo\" in \"reginald\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `in` operator takes one expression evaluating to a string on the left and another on the right, and returns `True` if the string on the left occurs somewhere inside of the string on the right.\n",
      "\n",
      "We can check to see if a string begins with or ends with another string using that string's `.startswith()` and `.endswith()` methods, respectively:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"foodie\".startswith(\"foo\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 3,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"foodie\".endswith(\"foo\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `.isdigit()` method returns `True` if Python thinks the string could represent an integer, and `False` otherwise:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"foodie\".isdigit()\n",
      "print \"4567\".isdigit()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "False\n",
        "True\n"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And the `.islower()` and `.isupper()` methods return `True` if the string is in all lower case or all upper case, respectively (and `False` otherwise)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"foodie\".islower()\n",
      "print \"foodie\".isupper()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "True\n",
        "False\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"YELLING ON THE INTERNET\".islower()\n",
      "print \"YELLING ON THE INTERNET\".isupper()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "False\n",
        "True\n"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Finding substrings\n",
      "\n",
      "The `in` operator discussed above will tell us if a substring occurs in some other string. If we want to know *where* that substring occurs, we can use the `.find()` method. The `.find()` method takes a single parameter between its parentheses: an expression evaluating to a string, which will be searched for within the string whose `.find()` method was called. If the substring is found, the entire expression will evaluate to the index at which the substring is found. If the substring is not found, the expression evaluates to `-1`. To demonstrate:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"Now is the winter of our discontent\".find(\"win\")\n",
      "print \"Now is the winter of our discontent\".find(\"lose\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "11\n",
        "-1\n"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `.count()` method will return the number of times a particular substring is found within the larger string:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"I got rhythm, I got music, I got my man, who could ask for anything more\".count(\"I got\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "3\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##String slices\n",
      "\n",
      "As has been alluded to previously, string slices work exactly like list slices---except you're getting characters from the string, instead of elements from a list. Observe:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "message = \"bungalow\"\n",
      "message[3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "'g'"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "message[1:6]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "'ungal'"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "message[:3]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "'bun'"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "message[2:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "'ngalow'"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "message[-2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "'o'"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Combine this with the `find()` method and you can do things like write expressions that evaluate to everything from where a substring matches, up to the end of the string:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "shakespeare = \"Now is the winter of our discontent\"\n",
      "substr_index = shakespeare.find(\"win\")\n",
      "print shakespeare[substr_index:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "winter of our discontent\n"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Simple string transformations\n",
      "\n",
      "Python strings have a number of different methods which, when called on a string, return a copy of that string with a simple transformation applied to it. These are helpful for normalizing and cleaning up data, or preparing it to be displayed.\n",
      "\n",
      "Let's start with `.lower()`, which evaluates to a copy of the string in all lower case:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"ARGUMENTATION! DISAGREEMENT! STRIFE!\".lower()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "'argumentation! disagreement! strife!'"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The converse of `.lower()` is `.upper()`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"e.e. cummings is. not. happy about this.\".upper()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "'E.E. CUMMINGS IS. NOT. HAPPY ABOUT THIS.'"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The method `.title()` evaluates to a copy of the string it's called on, replacing every letter at the beginning of a word in the string with a capital letter:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"dr. strangelove, or, how I learned to love the bomb\".title()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "'Dr. Strangelove, Or, How I Learned To Love The Bomb'"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `.strip()` method removes any whitespace from the beginning or end of the string (but not between characters later in the string):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\" got some random whitespace in some places here     \".strip()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        "'got some random whitespace in some places here'"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Finally, the `.replace()` method takes two parameters: a string to find, and a string to replace that string with whenever it's found. You can use this to make sad stories."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"I got rhythm, I got music, I got my man, who could ask for anything more\".replace(\"I got\", \"I used to have\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "'I used to have rhythm, I used to have music, I used to have my man, who could ask for anything more'"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###\"Escape\" sequences in strings\n",
      "\n",
      "Inside of strings that you type into your Python code, there are certain sequences of characters that have a special meaning. These sequences start with a backslash character (`\\`) and allow you to insert into your string characters that would otherwise be difficult to type, or that would go against Python syntax. Here's some code illustrating a few common sequences:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"include \\\"double quotes\\\" (inside of a double-quoted string)\"\n",
      "print 'include \\'single quotes\\' (inside of a single-quoted string)'\n",
      "print \"one\\ttab, two\\ttabs\"\n",
      "print \"new\\nline\"\n",
      "print \"include an actual backslash \\\\ (two backslashes in the string)\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "include \"double quotes\" (inside of a double-quoted string)\n",
        "include 'single quotes' (inside of a single-quoted string)\n",
        "one\ttab, two\ttabs\n",
        "new\n",
        "line\n",
        "include an actual backslash \\ (two backslashes in the string)\n"
       ]
      }
     ],
     "prompt_number": 113
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Regular expressions\n",
      "\n",
      "So far, we've discussed how to write programs and expressions that are able to check whether strings meet very simple criteria, such as \u201cdoes this string begin with a particular character\u201d or \u201cdoes this string contain another string\u201d? But imagine writing a program that performs the following task: find and print all ZIP codes in a string (i.e., a five-character sequence of digits). Give up? Here\u2019s my attempt, using only the tools we\u2019ve discussed so far:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "input_str = \"here's a zip code: 12345. 567 isn't a zip code, but 45678 is. 23456? yet another zip code.\"\n",
      "current = \"\"\n",
      "zips = []\n",
      "for ch in input_str:\n",
      "    if ch in '0123456789':\n",
      "        current += ch\n",
      "    else:\n",
      "        current = \"\"\n",
      "    if len(current) == 5:\n",
      "        zips.append(current)\n",
      "        current = \"\"\n",
      "print zips"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['12345', '45678', '23456']\n"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Basically, we have to iterate over each character in the string, check to see if that character is a digit, append to a string variable if so, continue reading characters until we reach a non-digit character, check to see if we found exactly five digit characters, and add it to a list if so. At the end, we print out the list that has all of our results. Problems with this code: it\u2019s messy; it doesn\u2019t overtly communicate what it\u2019s doing; it\u2019s not easily generalized to other, similar tasks (e.g., if we wanted to write a program that printed out phone numbers from a string, the code would likely look completely different).\n",
      "\n",
      "Our ancient UNIX pioneers had this problem, and in pursuit of a solution, thought to themselves, \"Let\u2019s make a tiny language that allows us to write specifications for textual patterns, and match those patterns against strings. No one will ever have to write fiddly code that checks strings character-by-character ever again.\" And thus regular expressions were born.\n",
      "\n",
      "Here's the code for accomplishing the same task with regular expressions, by the way:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import re\n",
      "zips = re.findall(r\"\\d{5}\", input_str)\n",
      "print zips"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['12345', '45678', '23456']\n"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I\u2019ll allow that the `r\"\\d{5}\"` in there is mighty cryptic (though hopefully it won\u2019t be when you\u2019re done reading this page and/or participating in the associated lecture). But the overall structure of the program is much simpler.\n",
      "\n",
      "###Fetching our corpus\n",
      "\n",
      "For this section of class, we'll be using the subject lines of all e-mails in the [EnronSent corpus](http://verbs.colorado.edu/enronsent/), kindly put into the public domain by the United States Federal Energy Regulatory Commission. Download a copy into your notebook directory like so:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import urllib\n",
      "urllib.urlretrieve(\"https://raw.githubusercontent.com/ledeprogram/courses/master/databases/data/enronsubjects.txt\", \"enronsubjects.txt\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 185,
       "text": [
        "('enronsubjects.txt', <httplib.HTTPMessage instance at 0x1053d8ab8>)"
       ]
      }
     ],
     "prompt_number": 185
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Matching strings with regular expressions\n",
      "\n",
      "The most basic operation that regular expressions perform is matching strings: you\u2019re asking the computer whether a particular string matches some description. We're going to be using regular expressions to print only those lines from our `enronsubjects.txt` corpus that match particular sequences. Let's load our corpus into a list of lines first:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "subjects = [x.strip() for x in open(\"enronsubjects.txt\").readlines()]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 186
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can check whether or not a pattern matches a given string in Python with the `re.search()` function. The first parameter to search is the regular expression you're trying to match; the second parameter is the string you're matching against.\n",
      "\n",
      "Here's an example, using a very simple regular expression. The following code prints out only those lines in our Enron corpus that match the (very simple) regular expression `shipping`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import re\n",
      "[line for line in subjects if re.search(\"shipping\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 187,
       "text": [
        "['FW: How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'How to use UPS for shipping on the internet',\n",
        " 'lng shipping/mosk meeting in tokyo 2nd of feb',\n",
        " 'lng shipping/mosk meeting in tokyo 2nd of feb',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping',\n",
        " 'lng shipping',\n",
        " 'lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping']"
       ]
      }
     ],
     "prompt_number": 187
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "At its simplest, a regular expression matches a string if that string contains exactly the characters you've specified in the regular expression. So the expression `shipping` matches strings that contain exactly the sequences of `s`, `h`, `i`, `p`, `p`, `i`, `n`, and `g` in a row. If the regular expression matches, `re.search()` evaluates to `True` and the matching line is included in the evaluation of the list comprehension.\n",
      "\n",
      "> BONUS TECH TIP: `re.search()` doesn't actually evaluate to `True` or `False`---it evaluates to either a `Match` object if a match is found, or `None` if no match was found. Those two count as `True` and `False` for the purposes of an `if` statement, though."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Metacharacters: character classes\n",
      "\n",
      "The \"shipping\" example is pretty boring. (There was hardly any fan fiction in there at all.) Let's go a bit deeper into detail with what you can do with regular expressions. There are certain characters or strings of characters that we can insert into a regular expressions that have special meaning. For example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(\"sh.pping\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 101,
       "text": [
        "['FW: How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'FW: How to use UPS for shipping on the internet',\n",
        " 'How to use UPS for shipping on the internet',\n",
        " \"FW: We've been shopping!\",\n",
        " 'Re: Start shopping...',\n",
        " 'Start shopping...',\n",
        " 'lng shipping/mosk meeting in tokyo 2nd of feb',\n",
        " 'lng shipping/mosk meeting in tokyo 2nd of feb',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping',\n",
        " 'lng shipping',\n",
        " 'lng shipping',\n",
        " 'Re: lng shipping',\n",
        " 'lng shipping',\n",
        " 'FW: Online shopping',\n",
        " 'Online shopping']"
       ]
      }
     ],
     "prompt_number": 101
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In a regular expression, the character `.` means \"match any character here.\" So, using the regular expression `sh.pping`, we get lines that match `shipping` but also `shopping`. The `.` is an example of a regular expression *metacharacter*---a character (or string of characters) that has a special meaning.\n",
      "\n",
      "Here are a few more metacharacters. These metacharacters allow you to say that a character belonging to a particular *class* of characters should be matched in a particular position:\n",
      "\n",
      "| metacharacter | meaning |\n",
      "|---------------|---------|\n",
      "| `.` | match any character |\n",
      "| `\\w` | match any alphanumeric (\"*w*ord\") character (lowercase and capital letters, 0 through 9, underscore) |\n",
      "| `\\s` | match any whitespace character (i.e., space and tab) |\n",
      "| `\\S` | match any non-whitespace character (the inverse of \\s) |\n",
      "| `\\d` | match any digit (0 through 9) |\n",
      "| `\\.` | match a literal `.` |\n",
      "\n",
      "Here, for example, is a (clearly imperfect) regular expression to search for all subject lines containing a time of day:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\d:\\d\\d\\wm\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 111,
       "text": [
        "['RE: 3:17pm',\n",
        " '3:17pm',\n",
        " \"RE: It's On!!! - 2:00pm Today\",\n",
        " \"FW: It's On!!! - 2:00pm Today\",\n",
        " \"It's On!!! - 2:00pm Today\",\n",
        " 'Re: Registration Confirmation: Larry Summers on 12/6 at 1:45pm (was',\n",
        " 'Re: Conference Call today 2/9/01 at 11:15am PST',\n",
        " 'Conference Call today 2/9/01 at 11:15am PST',\n",
        " '5/24 1:00pm conference call.',\n",
        " '5/24 1:00pm conference call.',\n",
        " 'FW: 07:33am EDT 15-Aug-01 Prudential Securities (C',\n",
        " 'FW: 07:33am EDT 15-Aug-01 Prudential Securities (C',\n",
        " '07:33am EDT 15-Aug-01 Prudential Securities (C',\n",
        " \"Re: Updated Mar'00 Requirements Received at 11:25am from CES\",\n",
        " \"Re: Updated Mar'00 Requirements Received at 11:25am from CES\",\n",
        " \"Re: Updated Mar'00 Requirements Received at 11:25am from CES\",\n",
        " \"Updated Mar'00 Requirements Received at 11:25am from CES\",\n",
        " 'Reminder: Legal Team Meeting -- Friday, 9:00am Houston time',\n",
        " 'Thursday, March 7th 1:30-3:00pm: REORIENTATION',\n",
        " 'Meeting at 2:00pm Friday',\n",
        " 'Meeting at 2:00pm Friday',\n",
        " 'Fw: 12:30pm Deadline for changes to letters or contracts today',\n",
        " '12:30pm Deadline for changes to letters or contracts today',\n",
        " 'Johnathan actually resigned at 9:00am this morning',\n",
        " 'FW: Enron Conference Call Today, 11:00am CST',\n",
        " 'Enron Conference Call Today, 11:00am CST',\n",
        " 'Meeting, Wednesday, January 23 at 10:00am at the Houstonian',\n",
        " 'RE: TVA Meeting, Wednesday June13, 1:15pm, EB3125b',\n",
        " 'TVA Meeting, Wednesday June13, 1:15pm, EB3125b',\n",
        " 'Re: Dabhol Update: Conference Call Thursday, Dec. 28, 8:00am',\n",
        " 'Dabhol Update: Conference Call Thursday, Dec. 28, 8:00am Houston time',\n",
        " 'FW: Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'Fw: Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'RE: Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'Fw: Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'Victoria Ashley Jones Born 5/25/01 7:31am.',\n",
        " 'RE: UCSF Cogen Calculation Conf Call, 10/12/01 at 8:00am PST',\n",
        " 'UCSF Cogen Calculation Conf Call, 10/12/01 at 8:00am PST',\n",
        " 'FW: Confirmation:  UCSF Cogen Conf Call. 10/22/02 at 8:00am',\n",
        " '=09RE: Confirmation:  UCSF Cogen Conf Call. 10/22/02 at 8:00am PST/=',\n",
        " '=09Confirmation:  UCSF Cogen Conf Call. 10/22/02 at 8:00am PST/10:0=',\n",
        " 'RE: Confirmation:  UCSF Cogen Conf Call. 10/22/02 at 8:00am',\n",
        " '=09Confirmation:  UCSF Cogen Conf Call. 10/22/02 at 8:00am PST/10:0=',\n",
        " 'Re: March expenses - deadline 04-04-01 2:00pm',\n",
        " 'Cirque - Jan 24 5:00pm show']"
       ]
      }
     ],
     "prompt_number": 111
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here's that regular expression again: `r\"\\d:\\d\\d\\wm\"`. I'm going to show you how to read this, one unit at a time.\n",
      "\n",
      "\"Hey, regular expression engine. Tell me if you can find this pattern in the current string. First of all, look for any number (`\\d`). If you find that, look for a colon right after it (`:`). If you find that, look for another number right after it (`\\d`). If you find *that*, look for any alphanumeric character---you know, a letter, a number, an underscore. If you find  that, then look for a `m`. Good? If you found all of those things in a row, then the pattern matched.\""
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "####But what about that weirdo `r\"\"`?\n",
      "\n",
      "Python provides another way to include string literals in your program, in addition to the single- and double-quoted strings we've already discussed. The r\"\" string literal, or \"raw\" string, includes all characters inside the quotes literally, without interpolating special escape characters. Here's an example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"this is\\na test\"\n",
      "print r\"this is\\na test\"\n",
      "print \"I love \\\\ backslashes!\"\n",
      "print r\"I love \\ backslashes!\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "this is\n",
        "a test\n",
        "this is\\na test\n",
        "I love \\ backslashes!\n",
        "I love \\ backslashes!\n"
       ]
      }
     ],
     "prompt_number": 114
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As you can see, whereas a double- or single-quoted string literal interprets `\\n` as a new line character, the raw quoted string includes those characters as they were literally written. More importantly, for our purposes at least, is the fact that, in the raw quoted string, we only need to write one backslash in order to get a literal backslash in our string.\n",
      "\n",
      "Why is this important? Because regular expressions use backslashes all the time, and we don't want Python to try to interpret those backslashes as special characters. (Inside a regular string, we'd have to write a simple regular expression like `\\b\\w+\\b` as `\\\\b\\\\w+\\\\b`---yecch.)\n",
      "\n",
      "So the basic rule of thumb is this: use r\"\" to quote any regular expressions in your program. All of the examples you'll see below will use this convention."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Character classes in-depth\n",
      "\n",
      "You can define your own character classes by enclosing a list of characters, or range of characters, inside square brackets:\n",
      "\n",
      "| regex | explanation |\n",
      "|-------|-------------|\n",
      "| `[aeiou]` | matches any vowel |\n",
      "| `[02468]` | matches any even digit |\n",
      "| `[a-z]` | matches any lower-case letter |\n",
      "| `[A-Z]` | matches any upper-case character |\n",
      "| `[^0-9]` | matches any non-digit (the ^ inverts the class, matches anything not in the list) |\n",
      "| `[Ee]` | matches either `E` or `e` |\n",
      "\n",
      "Let's find every subject line where we have four or more vowels in a row:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"[aeiou][aeiou][aeiou][aeiou]\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 121,
       "text": [
        "['Re: Natural gas quote for Louiisiana-Pacific (L-P)',\n",
        " 'WooooooHoooooo more Vacation',\n",
        " 'Re: Clickpaper Counterparties waiting to clear the work queue',\n",
        " 'Gooooooooooood Bye!',\n",
        " 'Gooooooooooood Bye!',\n",
        " 'RE: Hello Sweeeeetie',\n",
        " 'Hello Sweeeeetie',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'Re: FW: Wasss Uuuuuup STG?',\n",
        " 'RE: Rrrrrrrooooolllllllllllll TIDE!!!!!!!!',\n",
        " 'Rrrrrrrooooolllllllllllll TIDE!!!!!!!!',\n",
        " 'FW: The Osama Bin Laden Song ( Soooo Funny !! )',\n",
        " 'Fw: The Osama Bin Laden Song ( Soooo Funny !! )',\n",
        " 'The Osama Bin Laden Song ( Soooo Funny !! )',\n",
        " 'RE: duuuuhhhhh',\n",
        " 'RE: duuuuhhhhh',\n",
        " 'RE: duuuuhhhhh',\n",
        " 'duuuuhhhhh',\n",
        " 'RE: duuuuhhhhh',\n",
        " 'duuuuhhhhh',\n",
        " 'RE: FPL Queue positions 1-15',\n",
        " 'Re: FPL Queue positions 1-15',\n",
        " 'Re: Helloooooo!!!',\n",
        " 'Re: Helloooooo!!!',\n",
        " 'Fw: FW: OOOooooops',\n",
        " 'FW: FW: OOOooooops',\n",
        " 'Re: yeeeeha',\n",
        " 'yeeeeha',\n",
        " 'yahoooooooooooooooooooo',\n",
        " 'RE: yahoooooooooooooooooooo',\n",
        " 'RE: yahoooooooooooooooooooo',\n",
        " 'yahoooooooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " \"FW: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " \"RE: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " \"RE: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " 'Re: skiiiiiiiiing',\n",
        " 'skiiiiiiiiing',\n",
        " 'scuba dooooooooooooo',\n",
        " 'RE: scuba dooooooooooooo',\n",
        " 'RE: scuba dooooooooooooo',\n",
        " 'scuba dooooooooooooo',\n",
        " 'Re: skiiiiiiiing',\n",
        " 'skiiiiiiiing',\n",
        " 'Re: skiiiiiiiing',\n",
        " 'Re: skiiiiiiiiing',\n",
        " \"RE: Clickpaper CP's awaiting migration in work queue's 06/27/01\",\n",
        " \"FW: Clickpaper CP's awaiting migration in work queue's 06/27/01\",\n",
        " \"Clickpaper CP's awaiting migration in work queue's 06/27/01\",\n",
        " 'RE:  Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'FW: Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'Re: FW: Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'FW: Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'FW: Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'Fw: Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'Sequoia Adv. Pro.: Draft Stipulation and Order',\n",
        " 'i would have done this but i was toooo busy.....']"
       ]
      }
     ],
     "prompt_number": 121
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Metacharacters: anchors\n",
      "\n",
      "The next important kind of metacharacter is the *anchor*. An anchor doesn't match a character, but matches a particular place in a string.\n",
      "\n",
      "| anchor | meaning |\n",
      "|--------|---------|\n",
      "| `^` | match at beginning of string |\n",
      "| `$` | match at end of string |\n",
      "| `\\b` | match at word boundary |\n",
      "\n",
      "> Note: `^` in a character class has a different meaning from `^` outside a character class!\n",
      "\n",
      "> Note #2: If you want to search for a literal dollar sign (`$`), you need to put a backslash in front of it, like so: `\\$`\n",
      "\n",
      "Now we have enough regular expression knowledge to do some fairly sophisticated matching. As an example, all the subject lines that begin with the string `New York`, regardless of whether or not the initial letters were capitalized:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"^[Nn]ew [Yy]ork\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 127,
       "text": [
        "['New York Details',\n",
        " 'New York Power Authority',\n",
        " 'New York Power Authority',\n",
        " 'New York Power Authority',\n",
        " 'New York Power Authority',\n",
        " 'New York',\n",
        " 'New York',\n",
        " 'New York',\n",
        " 'New York, etc.',\n",
        " 'New York, etc.',\n",
        " 'New York sites',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York',\n",
        " 'New York',\n",
        " 'New York City Marathon Guaranteed Entry',\n",
        " 'new york rest reviews',\n",
        " 'New York State Electric & Gas Corporation (\"NYSEG\")',\n",
        " 'New York State Electric & Gas Corporation (\"NYSEG\")',\n",
        " 'New York State Electric & Gas Corporation (\"NYSEG\")',\n",
        " 'New York State Electric & Gas (\"NYSEG\")',\n",
        " 'New York regulatory restriccions',\n",
        " 'New York regulatory restriccions',\n",
        " 'New York Bar Numbers']"
       ]
      }
     ],
     "prompt_number": 127
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Every subject line that ends with an ellipsis:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\.\\.\\.$\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 130,
       "text": [
        "['Re: Inquiry....',\n",
        " 'Re: Inquiry....',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'RE: the candidate we spoke about this morning...',\n",
        " 'the candidate we spoke about this morning...',\n",
        " 'Re: Hmmmmm........',\n",
        " 'Hmmmmm........',\n",
        " 'FW: Bumping into the husband....',\n",
        " 'FW: Bumping into the husband....',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'Henry Hub instead of NYMEX...',\n",
        " 'Henry Hub instead of NYMEX...',\n",
        " 'Re: Henry Hub instead of NYMEX...',\n",
        " 'Henry Hub instead of NYMEX...',\n",
        " 'Transcanada Trade...',\n",
        " 'Transcanada Trade...',\n",
        " 'Here is the Article---no picture though...',\n",
        " 'Here is the Article---no picture though...',\n",
        " 'Re: ooops....',\n",
        " 'Re: ooops....',\n",
        " 'Re: ooops....',\n",
        " 'ooops....',\n",
        " 'FW: A crossroads we have all been at ...',\n",
        " 'FW: A crossroads we have all been at ...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'FW: follow up > FW: Caltech-developed arbitrage trading technolog\\ty being assessed by Reliant Energy right now...',\n",
        " 'follow up > FW: Caltech-developed arbitrage  trading technology being assessed by Reliant Energy right  now...',\n",
        " 'Caltech-developed arbitrage trading technology  being assessed by Reliant Energy right now...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " \"RE: okay here's what i got on the euro...\",\n",
        " \"okay here's what i got on the euro...\",\n",
        " \"RE: okay here's what i got on the euro...\",\n",
        " 'RE: first of all...',\n",
        " 'RE: first of all...',\n",
        " 'RE: try this one...',\n",
        " 'RE: try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'Re: try this one...',\n",
        " 'try this one...',\n",
        " 'RE: Follow up for Hardware Request....',\n",
        " 'Follow up for Hardware Request....',\n",
        " 'RE: Yahoo - GE Lighting Launches National Energy Program ...',\n",
        " 'RE: cheer up...',\n",
        " 'cheer up...',\n",
        " 'RE: Leaving Enron.....',\n",
        " 'Leaving Enron.....',\n",
        " 'Fwd: Revenge is a sweet thing...',\n",
        " 'Fwd: Revenge is a sweet thing...',\n",
        " 'Fwd: Revenge is a sweet thing...',\n",
        " 'Re: Got bored and...',\n",
        " 'Got bored and...',\n",
        " 'Re: Fw: [txhmed] interesting ...',\n",
        " 'Re: all Hector wants for christmas...',\n",
        " 'all Hector wants for christmas...',\n",
        " \"Re: Check out Leni's website...\",\n",
        " \"Check out Leni's website...\",\n",
        " 'my ....',\n",
        " 'my ....',\n",
        " 'Re: Just a little something to make you smile.......',\n",
        " 'Just a little something to make you smile.......',\n",
        " 'Just a little something to make you smile.......',\n",
        " 'Steamboat Vacation information...',\n",
        " 'Steamboat Vacation information...',\n",
        " 'FW: FW: (fwd) FW:  Warning from HFD...',\n",
        " 'Fw: (fwd) FW: Warning from HFD...',\n",
        " 'FW: (fwd) FW: Warning from HFD...',\n",
        " 'RE: Yeah Orange....',\n",
        " 'Yeah Orange....',\n",
        " 'RE: Yeah Orange....',\n",
        " 'RE: Yeah Orange....',\n",
        " 'Re: one last thing...',\n",
        " 'one last thing...',\n",
        " 'If you are stuck...',\n",
        " \"Re: FW: You've Been in Corporate America Too Long When...\",\n",
        " \"Re: It's true what they say...\",\n",
        " \"It's true what they say...\",\n",
        " 'Re: Todd & Things....',\n",
        " 'Todd & Things....',\n",
        " 'Re: testing....',\n",
        " \"Re: Don't send a dad...\",\n",
        " \"Don't send a dad...\",\n",
        " 'James is coming...',\n",
        " 'Re: Congratulations, etc...................',\n",
        " 'Congratulations, etc...................',\n",
        " 'RE: Fancy meeting you....',\n",
        " 'Fancy meeting you....',\n",
        " 'FW: In the spirit of cooperation...',\n",
        " 'In the spirit of cooperation...',\n",
        " 'In the spirit of cooperation...',\n",
        " 'RE: Back on the Block....',\n",
        " 'Back on the Block....',\n",
        " 'RE: A little humor for the new year....',\n",
        " 'A little humor for the new year....',\n",
        " 'RE: Infrastructure Prevents...',\n",
        " 'Infrastructure Prevents...',\n",
        " 'Re: FW: Could you please....',\n",
        " 'RE: FW: Could you please....',\n",
        " 'RE: FW: Could you please....',\n",
        " 'Re: FW: Could you please....',\n",
        " 'Re: Just Checking...',\n",
        " 'Just Checking...',\n",
        " 'RE: Vacation...',\n",
        " 'RE: Vacation...',\n",
        " 'Vacation...',\n",
        " 'FW: Vacation....',\n",
        " 'RE: Vacation....',\n",
        " 'Vacation....',\n",
        " 'FW: AA has left the building...',\n",
        " 'AA has left the building...',\n",
        " 'Re: It has been a while...',\n",
        " \"Re: Fw: it ain't easy.....\",\n",
        " 'FW: Message from Boeing.......',\n",
        " 'FW: Message from Boeing.......',\n",
        " 'FW: Message from Boeing.......',\n",
        " 'Message from Boeing.......',\n",
        " 'RE: I AM THANKFUL FOR ......',\n",
        " 'Re: I AM THANKFUL FOR ......',\n",
        " 'RE: I AM THANKFUL FOR ......',\n",
        " 'Re: I AM THANKFUL FOR ......',\n",
        " 'RE: I AM THANKFUL FOR ......',\n",
        " 'Re: I AM THANKFUL FOR ......',\n",
        " 'FW: I AM THANKFUL FOR ......',\n",
        " 'FW: I AM THANKFUL FOR ......',\n",
        " 'Re: Back in the saddle again...',\n",
        " 'Re: Back in the saddle again...',\n",
        " 'Re: Back in the saddle again...',\n",
        " 'RE: I know this sounds crazy but...',\n",
        " 'I know this sounds crazy but...',\n",
        " 'RE: By the way...',\n",
        " 'By the way...',\n",
        " \"Re: Fwd: Why you don't drink till you pass out.....\",\n",
        " \"Fwd: Why you don't drink till you pass out.....\",\n",
        " \"Fwd: Why you don't drink till you pass out.....\",\n",
        " \"Fwd: Why you don't drink till you pass out.....\",\n",
        " \"Why you don't drink till you pass out.....\",\n",
        " \"Fw: Ads you won't see...\",\n",
        " \"Fw: Ads you won't see...\",\n",
        " \"RE: FW: Nostradamus' prediction on WW3..................\",\n",
        " \"Re: FW: Nostradamus' prediction on WW3..................\",\n",
        " 'Lets get this ball rolling....',\n",
        " 'RE: Lets get the ball rolling......',\n",
        " 'RE: Lets get the ball rolling......',\n",
        " 'Lets get the ball rolling......',\n",
        " 'Tell me that....................',\n",
        " 'Re: Tell me that....................',\n",
        " 'FW: YOU WANT TO KNOW ABOUT THIS....',\n",
        " 'FW: YOU WANT TO KNOW ABOUT THIS....',\n",
        " 'Re: FW: Cash Balance Plan...',\n",
        " \"Don't forget...\",\n",
        " 'RE: Seeking info...',\n",
        " 'Seeking info...',\n",
        " 'FW: I thought you might be interested...',\n",
        " 'I thought you might be interested...',\n",
        " 'RE: Party Date...',\n",
        " 'Party Date...',\n",
        " 'Re: FW: RE: Coming Home Soon...',\n",
        " 'RE: FW: RE: Coming Home Soon...',\n",
        " 'FW: Weekend Events.........',\n",
        " 'FW: Weekend Events.........',\n",
        " 'Weekend Events.........',\n",
        " 'Re: your famous...',\n",
        " 'Re: your famous...',\n",
        " 'Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Parting is such sweet sorrow...',\n",
        " 'Parting is such sweet sorrow...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: Even the best laid plans...',\n",
        " 'Re: FW: Never let a guy take a message.....',\n",
        " 'Re: And the winners are...',\n",
        " 'RE: And the winners are...',\n",
        " 'Re: And the winners are...',\n",
        " \"Re: WSJ: PG&E's Huge losses...\",\n",
        " 'RE: Here it is...',\n",
        " 'RE: Here it is...',\n",
        " 'RE: Here it is...',\n",
        " 'Here it is...',\n",
        " 'Re:RE: Here it is...',\n",
        " 'Re:RE: Here it is...',\n",
        " 'Re:Here it is...',\n",
        " 'Eeegads...',\n",
        " \"Re: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"Re: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"Re: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"RE: FW: ok, it's a little excessive, but...\",\n",
        " \"Re: FW: ok, it's a little excessive, but...\",\n",
        " 'RE: Eeegads...',\n",
        " 'Eeegads...',\n",
        " 'help...',\n",
        " 'Re: Well...',\n",
        " 'RE: Well...',\n",
        " 'Re: Well...',\n",
        " 'Re: You forgot your wine....',\n",
        " \"RE: tell me it isn't true...\",\n",
        " \"tell me it isn't true...\",\n",
        " 'How You Should Act...........',\n",
        " 'RE: If you go a run this afternoon....',\n",
        " 'If you go a run this afternoon....',\n",
        " 'FW: You still suck at baseball....',\n",
        " 'You still suck at baseball....',\n",
        " 'FW: Primary Authority Plus...',\n",
        " 'Primary Authority Plus...',\n",
        " 'Fw: Primary Authority Plus...',\n",
        " 'Primary Authority Plus...',\n",
        " 'RE: Would like to help...',\n",
        " 'Would like to help...',\n",
        " 'FW: Would like to help...',\n",
        " 'Would like to help...',\n",
        " 'from my red neck uncle...',\n",
        " 'Your Chapters.ca Coupons ...',\n",
        " 'Your Chapters.ca Coupons ...',\n",
        " 'funny stuff about your mother...',\n",
        " 'RE: Reply to this....',\n",
        " 'RE: Reply to this....',\n",
        " 'RE: Reply to this....',\n",
        " 'RE: Reply to this....',\n",
        " 'RE: Reply to this....',\n",
        " 'RE: Reply to this....',\n",
        " 'Re: Reply to this....',\n",
        " 'Re: Reply to this....',\n",
        " 'Re: Plans...',\n",
        " 'Re: Plans...',\n",
        " 'Re: Plans...',\n",
        " 'Re: Howdy Stranger...',\n",
        " 'Help! Canadians need weather...',\n",
        " 'FW: Hilarious....',\n",
        " 'FW: Hilarious....',\n",
        " 'FW: Hilarious....',\n",
        " 'Fw: Hilarious....',\n",
        " 'FW: Hilarious....',\n",
        " \"RE: Dan's coming to town...\",\n",
        " \"Re: Dan's coming to town...\",\n",
        " \"Dan's coming to town...\",\n",
        " 'FW: Interesting...',\n",
        " 'FW: Interesting...',\n",
        " \"RE: Haven't heard from you yet......\",\n",
        " \"=09Haven't heard from you yet......\",\n",
        " 'RE: A pleasant thought for long term investors...',\n",
        " 'RE: A pleasant thought for long term investors...',\n",
        " 'A pleasant thought for long term investors...',\n",
        " 'A pleasant thought for long term investors...',\n",
        " 'RE: New Digits....',\n",
        " 'New Digits....',\n",
        " 'RE: Things we wish we could say at work...',\n",
        " 'FW: Things we wish we could say at work...',\n",
        " 'RE: PBM merger...',\n",
        " 'FW: PBM merger...',\n",
        " 'PBM merger...',\n",
        " 'funny........',\n",
        " 'Questions.........',\n",
        " 'RE: You have 44 hours remaining...',\n",
        " 'You have 44 hours remaining...',\n",
        " 'FW: guidelines....',\n",
        " 'guidelines....',\n",
        " 'Re: Daily California Update.....',\n",
        " 'Daily California Update.....',\n",
        " 'Daily California Update.....',\n",
        " 'ETS on the Move...',\n",
        " 'ETS on the Move...',\n",
        " \"Re: HELLO I'M HERE AGAIN...\",\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Re: Next time you see me....',\n",
        " 'RE: I need your help...',\n",
        " 'I need your help...',\n",
        " 'RE: 2 nd version of Plan...',\n",
        " '2 nd version of Plan...',\n",
        " 'FW: 2 nd version of Plan...',\n",
        " 'RE: 2 nd version of Plan...',\n",
        " '2 nd version of Plan...',\n",
        " 'Last night...',\n",
        " 'Last night...',\n",
        " 'Re: Last night...',\n",
        " 'Last night...',\n",
        " 'Hey Chris..........',\n",
        " 'Hey Chris..........',\n",
        " 'FW: Thanks!...',\n",
        " 'Re: Drinks...',\n",
        " 'Drinks...',\n",
        " 'RE: Advantages of being a man...',\n",
        " 'RE: Advantages of being a man...',\n",
        " 'Re: Advantages of being a man...',\n",
        " 'Re: Advantages of being a man...',\n",
        " 'FW: Now we know....',\n",
        " 'FW: Fun for when your bored....',\n",
        " 'FW: Fun for when your bored....',\n",
        " 'Fun for when your bored....',\n",
        " \"FW: Robin & Peter Vint's going away party -  Friday March 15th  -\\t boo hoo.....\",\n",
        " 'RE: This weekend...',\n",
        " 'This weekend...',\n",
        " 'RE: Moving on...',\n",
        " 'Moving on...',\n",
        " 'FW: A Very Cold Winter...',\n",
        " 'Fwd: Just ask a child...',\n",
        " 'Fwd: Just ask a child...',\n",
        " 'Just ask a child...',\n",
        " 'Fwd: FW: Fwd[3]:FW: For the Sportsman in all of us...',\n",
        " 'Fwd: FW: Fwd[3]:FW: For the Sportsman in all of us...',\n",
        " 'Fwd: FW: Fwd[3]:FW: For the Sportsman in all of us...',\n",
        " 'FW: Fwd[3]:FW: For the Sportsman in all of us...',\n",
        " 'About the release tomorrow...',\n",
        " 'About the release tomorrow...',\n",
        " 'Re: Two Flatscreens to be moved...',\n",
        " 'Re: Two Flatscreens to be moved...',\n",
        " 'Re: Two Flatscreens to be moved...',\n",
        " 'Two Flatscreens to be moved...',\n",
        " 'Two Flatscreens to be moved...',\n",
        " \"Fw: La medaille d'or...\",\n",
        " \"Fw: La medaille d'or...\",\n",
        " \"FW: La medaille d'or...\",\n",
        " 'Fw: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Fw: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Fwd: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Paul Harvey Story ...Probably Should Circulate This  One...',\n",
        " 'FW: Do you remember.........',\n",
        " 'FW: Do you remember.........',\n",
        " 'FW: Do you remember.........',\n",
        " 'Fw: Priceless Series .........',\n",
        " 'Fw: Priceless Series .........',\n",
        " 'Fwd: Priceless Series .........',\n",
        " 'Re: Surround Sound...',\n",
        " 'Surround Sound...',\n",
        " 'FW: wow...',\n",
        " 'Fwd: wow...',\n",
        " 'wow...',\n",
        " 'FW: Patience...',\n",
        " 'FW: Patience...',\n",
        " 'FW: This is hillarious...',\n",
        " 'This is hillarious...',\n",
        " 'FW: Women...',\n",
        " 'Fw: Women...',\n",
        " 'FW: Women...',\n",
        " 'FW: I have moved, but my Phone has not .....',\n",
        " 'I have moved, but my Phone has not .....',\n",
        " 'FW: two sides to the story....',\n",
        " 'FW: two sides to the story....',\n",
        " 'RE: two sides to the story....',\n",
        " 'RE: two sides to the story....',\n",
        " 'FW: two sides to the story....',\n",
        " 'RE: I have moved, but my Phone has not .....',\n",
        " 'I have moved, but my Phone has not .....',\n",
        " 'FW: Voices from the past...',\n",
        " 'Fw: Voices from the past...',\n",
        " 'Voices from the past...',\n",
        " 'FW: The cost of kids...',\n",
        " 'Fw: The cost of kids...',\n",
        " 'FW: Condom Sense....',\n",
        " 'Fw: Condom Sense....',\n",
        " 'Re: Trying to reach you...',\n",
        " 'RE: Shut in comments and EOG....',\n",
        " 'FW: Shut in comments and EOG....',\n",
        " 'FW: Shut in comments and EOG....',\n",
        " 'FW: Shut in comments and EOG....',\n",
        " 'FW: new \"rules\"...',\n",
        " 'new \"rules\"...',\n",
        " 'RE: O:/ECT_Trading...',\n",
        " 'O:/ECT_Trading...',\n",
        " 'Re: Question...',\n",
        " 'NYMEX email address...',\n",
        " 'Re: NYMEX email address...',\n",
        " 'FW: FYI...',\n",
        " 'FYI...',\n",
        " 'RE: FYI...',\n",
        " 'FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'FW: FYI...',\n",
        " 'FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'RE: FYI...',\n",
        " 'FW: FYI...',\n",
        " 'FYI...',\n",
        " 'FW: For hours of endless revenge..........',\n",
        " 'FW: For hours of endless revenge..........',\n",
        " 'FW: For hours of endless revenge..........',\n",
        " 'For hours of endless revenge..........',\n",
        " 'RE: Three new additions to the world.........',\n",
        " 'Three new additions to the world.........',\n",
        " 'FW: Complete Madness ...',\n",
        " 'FW: Complete Madness ...',\n",
        " 'FW: Complete Madness ...',\n",
        " 'FW: Three new additions to the world.........',\n",
        " 'RE: Three new additions to the world.........',\n",
        " 'Three new additions to the world.........',\n",
        " 'Re: AW: AW: I am sooo sorry...',\n",
        " 'Re: AW: I am sooo sorry...',\n",
        " 'Re: I am sooo sorry...',\n",
        " 'Delta Airlines...',\n",
        " 'Delta Airlines...',\n",
        " 'RE: How to get emails....',\n",
        " 'How to get emails....',\n",
        " 'FW: It could be worse....',\n",
        " 'FW: It could be worse....',\n",
        " 'Fwd: Life...',\n",
        " 'Fwd: Life...',\n",
        " 'Fwd: Life...',\n",
        " 'Life...',\n",
        " 'Fwd: Life...',\n",
        " 'Fwd: Life...',\n",
        " 'Fwd: Life...',\n",
        " 'Life...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: For our Children...',\n",
        " 'FW: FW: I said a prayer for you just now.......',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'FW: Hope you like this poem...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " 'Fwd: So Very True...',\n",
        " \"Fw: YOU KNOW YOU'RE A LATINO IF...\",\n",
        " \"Fw: YOU KNOW YOU'RE A LATINO IF...\",\n",
        " \"RE: YOU KNOW YOU'RE A LATINO IF...\",\n",
        " 'For you ....',\n",
        " 'For you ....',\n",
        " 'For you ....',\n",
        " 'FW: Dying...',\n",
        " 'FW: Dying...',\n",
        " 'FW: Dying...',\n",
        " 'FW: Dying...',\n",
        " 'FW: Dying...',\n",
        " 'RE: FW: You know your at a LATINO birthday party....',\n",
        " 'Fw: FW: You know your at a LATINO birthday party....',\n",
        " 'RE: Follow up for Hardware Request...',\n",
        " 'FW: Follow up for Hardware Request...',\n",
        " 'Follow up for Hardware Request...',\n",
        " 'RE: tonight...',\n",
        " 'tonight...',\n",
        " 'Re: some questions...',\n",
        " 'some questions...',\n",
        " 'Re: some questions...',\n",
        " 'Re: Updating Regulatory Affairs Database.....',\n",
        " 'Updating Regulatory Affairs Database.....',\n",
        " 'RE: Stranger things have happened...',\n",
        " 'Stranger things have happened...',\n",
        " \"RE: Michelle's poem....\",\n",
        " \"Michelle's poem....\",\n",
        " 'RE: FW: Enron Employees Leaving Houston...',\n",
        " 'Fwd: FW: Enron Employees Leaving Houston...',\n",
        " 'FW: Interesting facts about this election...',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'Re: crestar / gulf aos contract....',\n",
        " 'FW: Life......',\n",
        " 'FW: Life......',\n",
        " 'Re: And the Prize Goes To...',\n",
        " 'Re: TIME heals all .....',\n",
        " \"I'm Back...\",\n",
        " 'Re: as you requested...',\n",
        " 'as you requested...',\n",
        " 'Re: as you requested...',\n",
        " 'Re: as you requested...',\n",
        " 'as you requested...',\n",
        " 'Re: as you requested...',\n",
        " 'Re: as you requested...',\n",
        " 'Re: as you requested...',\n",
        " 'as you requested...',\n",
        " 'And the beat goes on...',\n",
        " 'Re: And the beat goes on...',\n",
        " 'Re: Follow up....',\n",
        " 'Greetings...',\n",
        " 'Can you handle...',\n",
        " 'When you get back...',\n",
        " 'Oh knower of all things...',\n",
        " 'Re: Yes, I need your help again ...',\n",
        " 'Another Picture....',\n",
        " 'Another Picture....',\n",
        " 'Per your request...',\n",
        " 'It should be more...',\n",
        " 'Good luck America....',\n",
        " 'Good luck America....',\n",
        " 'Good luck America....',\n",
        " 'Re: Yes but...........',\n",
        " 'Yes but...........',\n",
        " 'Good luck America....',\n",
        " 'Positively the last word .....',\n",
        " 'Positively the last word .....',\n",
        " 'Positively the last word .....',\n",
        " 'Re: Positively the last word .....',\n",
        " 'Positively the last word .....',\n",
        " 'Just Like Chapman...',\n",
        " 'Re: Just Like Chapman...',\n",
        " 'When you come....',\n",
        " 'My Lucky Day...',\n",
        " 'Re: Sorry, one more thing ...',\n",
        " \"Another Gov't Agency Name...\",\n",
        " 'just thinking...',\n",
        " 'Re: just thinking...',\n",
        " 'just thinking...',\n",
        " 'More Koch Masters...',\n",
        " 'No Word...',\n",
        " '400 and counting...',\n",
        " 'Declined: Lunch ...',\n",
        " 'RE: Lunch ...',\n",
        " 'RE: Lunch ...',\n",
        " 'Declined: Lunch ...',\n",
        " 'Re: access to O;...',\n",
        " 'Re: access to O;...',\n",
        " 'access to O;...',\n",
        " 'Our Apologies ...',\n",
        " 'Our Apologies ...',\n",
        " 'Need direction please...',\n",
        " 'Need direction please...',\n",
        " 'Need direction please...',\n",
        " 'Some municipal bonds for you to look at.....',\n",
        " 'Some municipal bonds for you to look at.....',\n",
        " 'Re: Publishable Research......',\n",
        " 'Publishable Research......',\n",
        " 'Publishable Research......',\n",
        " 'Thank-you...',\n",
        " 'Thank-you...',\n",
        " 'Clintons leaving the Whitehouse...',\n",
        " 'Clintons leaving the Whitehouse...',\n",
        " 'Clintons leaving the Whitehouse...',\n",
        " 'I just like hearing it.....',\n",
        " 'I just like hearing it.....',\n",
        " \"I've done it....\",\n",
        " \"I've done it....\",\n",
        " 'FW: Real Options Research...',\n",
        " 'Real Options Research...',\n",
        " 'RE: In light of the events this week....',\n",
        " 'In light of the events this week....',\n",
        " 'RE: In light of the events this week....',\n",
        " 'RE: In light of the events this week....',\n",
        " 'RE: In light of the events this week....',\n",
        " 'In light of the events this week....',\n",
        " 'RE: In light of the events this week....',\n",
        " 'FW: An interesting story Abt. Stanford University ...',\n",
        " 'FW: An interesting story Abt. Stanford University ...',\n",
        " 'RE: Would anyone be interested?....',\n",
        " '=09Would anyone be interested?....',\n",
        " 'FW: Would anyone be interested?....',\n",
        " '=09Would anyone be interested?....',\n",
        " 'RE: Would anyone be interested?....',\n",
        " 'RE: Would anyone be interested?....',\n",
        " 'FW: Would anyone be interested?....',\n",
        " '=09RE: Would anyone be interested?....',\n",
        " '=09FW: Would anyone be interested?....',\n",
        " '=09Would anyone be interested?....',\n",
        " 'RE: Hello...',\n",
        " 'Hello...',\n",
        " 'RE: Czy planujesz pojawic sie w Londynie  ...',\n",
        " 'Czy planujesz pojawic sie w Londynie  ...',\n",
        " 'Re: Conf Call...',\n",
        " 'Conf Call...',\n",
        " 'Midwest ISO information...',\n",
        " 'Midwest ISO information...',\n",
        " 'Need a laugh? Here it is...',\n",
        " 'Time Magazine - Enron Plays the Pipes....',\n",
        " 'Time Magazine - Enron Plays the Pipes....',\n",
        " \"WSJ: PG&E's Huge losses...\",\n",
        " \"FW: Hadn't heard about this Enron Mention...\",\n",
        " \"Hadn't heard about this Enron Mention...\",\n",
        " 'Finally the truth comes out...',\n",
        " 'Finally the truth comes out...',\n",
        " 'RE: however...',\n",
        " 'however...',\n",
        " 'Re: FW: Mike Curry has signed and returned docs.....',\n",
        " \"Rahil Jafry: Carly Fiorina Tops FORTUNE's List of 50 Most Powerful Women in Business   for ...\",\n",
        " \"RE: I'm still here ....\",\n",
        " \"I'm still here ....\",\n",
        " 'RE: Marcello has a favour to ask....',\n",
        " 'Marcello has a favour to ask....',\n",
        " 'RE: Hi...',\n",
        " 'RE: Hi...',\n",
        " 'RE: Hi...',\n",
        " 'Hi...',\n",
        " 'RE: Hi...',\n",
        " 'Hi...',\n",
        " 'RE: Hi...',\n",
        " 'Hi...',\n",
        " 'RE: Shankman...',\n",
        " 'Shankman...',\n",
        " \"Re: I'm Leaving...\",\n",
        " 'RE: floor space...',\n",
        " 'floor space...',\n",
        " 'Re: Sad news...',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fw: Cast your vote..........',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'RE: FW: Careful what you write...',\n",
        " 'RE: FW: Careful what you write...',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'Fwd: FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'FW: Careful what you write...',\n",
        " 'Fwd: something groovy to do...',\n",
        " 'RE: Our tree trimming storey.....',\n",
        " 'RE: Our tree trimming storey.....',\n",
        " 'RE: Our tree trimming storey.....',\n",
        " 'Our tree trimming storey.....',\n",
        " 'RE: Our tree trimming storey.....',\n",
        " 'Our tree trimming storey.....',\n",
        " 'FW: Our tree trimming storey.....',\n",
        " 'Our tree trimming storey.....',\n",
        " 'RE: Just a thought ...',\n",
        " 'FW: Just a thought ...',\n",
        " 'FW: Just a thought ...',\n",
        " 'FW: This is a classic...',\n",
        " 'FW: This is a classic...',\n",
        " 'FW: This is a classic...',\n",
        " 'RE: Advice / Information....',\n",
        " 'Advice / Information....',\n",
        " 'FW: Advice / Information....',\n",
        " 'RE: Advice / Information....',\n",
        " 'RE: Advice /  Information....',\n",
        " 'Advice /  Information....',\n",
        " 'RE: SAC visit...',\n",
        " 'SAC visit...',\n",
        " 'RE: If we were going to pay....',\n",
        " 'If we were going to pay....',\n",
        " 'Re: Dates for the Faculty-Alumni Awards at MU...',\n",
        " 'Dates for the Faculty-Alumni Awards at MU...',\n",
        " 'Re: Scott McNealy wants to hear from you...',\n",
        " 'Scott McNealy wants to hear from you...',\n",
        " 'OK, Jeff, you requested that we be candid about Enron...',\n",
        " 'OK, Jeff, you requested that we be candid about Enron...',\n",
        " 'Kenneth, here are four Christmas articles for you ...',\n",
        " 'Re: oath to you...',\n",
        " 'Re: Re[2]: oath to you...',\n",
        " 'Re: Re[4]: oath to you...',\n",
        " 'Fw: Bad Girl Barbies....',\n",
        " 'Fw: Bad Girl Barbies....',\n",
        " 'Fw: Bad Girl Barbies....',\n",
        " 'Re: Hands on...',\n",
        " 'Re: Hands on...',\n",
        " 'Fw: Something to think about...',\n",
        " 'FW: Something to think about...',\n",
        " 'Re: Fw: Something to think about...',\n",
        " 'Fw: Someone has way too much time on their hands......',\n",
        " 'Fw: Someone has way too much time on their hands......',\n",
        " 'FW: Someone has way too much time on their hands......',\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"Fw: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"FW: What We've Learned From Watching Porn......\",\n",
        " \"Re: Fw: What We've Learned From Watching Porn......\",\n",
        " \"RE: What We've Learned From Watching Porn......\",\n",
        " 'Re: hey...',\n",
        " 'Re: Better get a good backup....',\n",
        " 'Fwd: something groovy to do...',\n",
        " 'Re: Tomorrow...',\n",
        " 'RE: Tomorrow...',\n",
        " 'Re: Hey...',\n",
        " 'Re: Are you still up for...',\n",
        " 'Re: Tomorrow...',\n",
        " 'RE: Tomorrow...',\n",
        " 'Re: Hey...',\n",
        " 'Re: Are you still up for...',\n",
        " 'RE: this one too....',\n",
        " 'this one too....',\n",
        " 'FW: New Darwin Award winners are in...',\n",
        " 'FW: New Darwin Award winners are in...',\n",
        " 'New Darwin Award winners are in...',\n",
        " 'RE: Hi Sweetie...',\n",
        " 'Hi Sweetie...',\n",
        " 'RE: Hi Sweetie...',\n",
        " 'RE: Hi Sweetie...',\n",
        " 'RE: Hi Sweetie...',\n",
        " 'Hi Sweetie...',\n",
        " 'RE: Dear Abby.......',\n",
        " 'FW: Dear Abby.......',\n",
        " 'FW: Dear Abby.......',\n",
        " 'FW: Dear Abby.......',\n",
        " 'Dear Abby.......',\n",
        " 'Dear Abby.......',\n",
        " 'RE: Strong Words...',\n",
        " 'Strong Words...',\n",
        " 'Fwd: something groovy to do...',\n",
        " 'Re: Happy Hanukah and Merry Christmas...',\n",
        " 'FW: As A Promising Energy Professional...',\n",
        " 'As A Promising Energy Professional...',\n",
        " \"RE: Why markers don't make good Christmas gifts...\",\n",
        " \"FW: Why markers don't make good Christmas gifts...\",\n",
        " \"RE: Why markers don't make good Christmas gifts...\",\n",
        " \"FW: Why markers don't make good Christmas gifts...\",\n",
        " \"RE: Why markers don't make good Christmas gifts...\",\n",
        " \"FW: Why markers don't make good Christmas gifts...\",\n",
        " 'FW: Angels are ....',\n",
        " 'FW: Angels are ....',\n",
        " 'RE: Angels are ....',\n",
        " 'RE: Angels are ....',\n",
        " 'FW: Angels are ....',\n",
        " 'FW: Angels are ....',\n",
        " 'Re: Fw: your voice..........',\n",
        " 'Fw: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Fw: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Fw: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Fwd: FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'FW: Paul Harvey Story ...Probably Should Circulate This One...',\n",
        " 'Paul Harvey Story ...Probably Should Circulate This  One...',\n",
        " 'Re: WAR DAMN EAGLE....',\n",
        " 'RE: User list to access different post ids...',\n",
        " 'Re: User list to access different post ids...',\n",
        " \"FW: Too bad stupidity isn't painful...\",\n",
        " \"FW: Too bad stupidity isn't painful...\",\n",
        " \"Re: FW: It couldn't hurt...\",\n",
        " 'RE: This is hillarious...',\n",
        " 'RE: This is hillarious...',\n",
        " 'FW: This is hillarious...',\n",
        " 'This is hillarious...',\n",
        " 'FW: This is hillarious...',\n",
        " 'This is hillarious...',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'FW: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'FW: Mahmassani VaR........',\n",
        " 'FW: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'Mahmassani VaR........',\n",
        " 'FW: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'Mahmassani VaR........',\n",
        " 'RE: Mahmassani VaR........',\n",
        " 'Mahmassani VaR........',\n",
        " 'FW: Weekend Events.........',\n",
        " 'FW: Weekend Events.........',\n",
        " 'Weekend Events.........',\n",
        " 'RE: after the storm...',\n",
        " 'Fw: after the storm...',\n",
        " 'Fwd: after the storm...',\n",
        " 'RE: Memories.......',\n",
        " 'Memories.......',\n",
        " 'FW: Memories.......',\n",
        " 'RE: Memories.......',\n",
        " 'Memories.......',\n",
        " 'RE: Well...',\n",
        " 'Well...',\n",
        " 'RE: Hey...',\n",
        " 'Hey...',\n",
        " 'RE: Paulie-wog...',\n",
        " 'Paulie-wog...',\n",
        " 'FW: They build great outhouses in AR...',\n",
        " 'They build great outhouses in AR...',\n",
        " 'They build great outhouses in AR...',\n",
        " 'RE: Oh happy day...',\n",
        " 'Oh happy day...',\n",
        " 'RE: Oh happy day...',\n",
        " 'RE: Oh happy day...',\n",
        " \"RE: I know you're busy...\",\n",
        " \"I know you're busy...\",\n",
        " 'RE: Good Morning...',\n",
        " 'Good Morning...',\n",
        " 'RE: Arrrrgh....',\n",
        " 'Arrrrgh....',\n",
        " 'I need to call Mark about the docs that Heather sent...',\n",
        " 'a little high...',\n",
        " 'RE: a little high...',\n",
        " 'Some thoughts on anniversary stuff...',\n",
        " 'You probably saw this all ready...',\n",
        " \"HELP I'm drowning....\",\n",
        " 'FW: Drinking quotes...',\n",
        " 'FW: Drinking quotes...',\n",
        " 'FW: Drinking quotes...',\n",
        " 'Ok, it is a slow news day...',\n",
        " 'A blurb from an internal Enron communiciation...',\n",
        " 'FW: To those of you getting married...',\n",
        " 'To those of you getting married...',\n",
        " 'Oh, just a change or two...',\n",
        " \"One email didn't go through...\",\n",
        " \"RE: One email didn't go through...\",\n",
        " \"RE: One email didn't go through...\",\n",
        " \"One email didn't go through...\",\n",
        " 'Dick Westfahl Retirement - Bambi would be proud...',\n",
        " 'Re: Dick Westfahl Retirement - Bambi would be proud...',\n",
        " 'Re: Dick Westfahl Retirement - Bambi would be proud...',\n",
        " 'Just to confuse you...',\n",
        " 'And on this legal front...',\n",
        " \"Re: FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"We'll Miss You Steffy.......\",\n",
        " \"RE: FW: We'll Miss You Steffy.......\",\n",
        " \"RE: FW: We'll Miss You Steffy.......\",\n",
        " \"Re: FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"FW: We'll Miss You Steffy.......\",\n",
        " \"We'll Miss You Steffy.......\",\n",
        " 'Another try...',\n",
        " 'Re: Fw: things to ponder.....',\n",
        " \"FW: so, you've been laid off....\",\n",
        " \"Fwd: so, you've been laid off....\",\n",
        " 'CAF - Tier 1 -  (Req# 600 - Jode Corp) requires your signature...',\n",
        " 'Re: CAF - Tier 1 - OVERDUE - (Req# 599) is overdue...',\n",
        " 'Re: Years ago...',\n",
        " 'Re: Congratulations...',\n",
        " 'Congratulations...',\n",
        " 'Re: quick confirmation....',\n",
        " 'AM I FREE.....',\n",
        " \"FW: If you're bored here's a mensa test...\",\n",
        " \"FW:If you're bored here's a mensa test...\",\n",
        " 'Re: Long Time...',\n",
        " 'Long Time...',\n",
        " 'RE: Long Time...',\n",
        " 'RE: New exchange broker...',\n",
        " 'RE: New exchange broker...',\n",
        " 'Re: New exchange broker...',\n",
        " 'New exchange broker...',\n",
        " 'FW:  Move Related Reminders...',\n",
        " '=09FW:  Move Related Reminders...',\n",
        " '=09 Move Related Reminders...',\n",
        " 'RE: I NEED VERIFICATION...',\n",
        " 'RE: I NEED VERIFICATION...',\n",
        " 'RE: I NEED VERIFICATION...',\n",
        " 'I NEED VERIFICATION...',\n",
        " 'RE: I NEED VERIFICATION...',\n",
        " 'I NEED VERIFICATION...',\n",
        " 'RE: Heartland Steel revised....',\n",
        " 'Heartland Steel revised....',\n",
        " 'FW: Info you requested...',\n",
        " 'Info you requested...',\n",
        " 'Re: Long distance call........',\n",
        " 'Re: FW: Please call this number...',\n",
        " 'Re: Just a short note.........',\n",
        " 'FW: Read Storyline first.........',\n",
        " '? Fwd: Read Storyline first.........',\n",
        " 'Re: Hey, Dad....',\n",
        " 'Re: T-minus 3 days...',\n",
        " 'T-minus 3 days...',\n",
        " \"Re: Don't forget to vote...\",\n",
        " \"Don't forget to vote...\",\n",
        " 'Re: Suds...',\n",
        " 'Suds...',\n",
        " 'Re: TIP OF THE DAY...',\n",
        " 'TIP OF THE DAY...',\n",
        " 'Re: Kristi called me last night....',\n",
        " 'details...',\n",
        " 'details...',\n",
        " 're: ... no subject ...',\n",
        " 'Re: Thinking of you...........',\n",
        " \"And That's It...\",\n",
        " \"And That's It...\",\n",
        " \"And That's It...\",\n",
        " 'FW: Training courses available...',\n",
        " \"Re: Haven't heard from you in a while........\",\n",
        " 'RE: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " 'FW: Dudes...',\n",
        " 'RE: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " 'FW: Dudes...',\n",
        " 'RE: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " 'FW: Dudes...',\n",
        " 'RE: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " 'FW: Dudes...',\n",
        " 'RE: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " 'RE: Some news.....',\n",
        " 'Some news.....',\n",
        " \"FW: We're Still Here...\",\n",
        " \"We're Still Here...\",\n",
        " 'RE: Still here and still ready...',\n",
        " 'Still here and still ready...',\n",
        " 'Still here and still ready...',\n",
        " 'FW: Dudes...',\n",
        " 'Re: Dudes...',\n",
        " 'Dudes...',\n",
        " \"RE: I'm here...\",\n",
        " \"I'm here...\",\n",
        " \"RE: I'm here...\",\n",
        " \"RE: I'm here...\",\n",
        " \"RE: I'm here...\",\n",
        " \"I'm here...\",\n",
        " 'RE: Troutmaster say...',\n",
        " ...]"
       ]
      }
     ],
     "prompt_number": 130
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Every subject line that has the word 'oil' in it"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\b[Oo]il\\b\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 132,
       "text": [
        "['Re: PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'Re: PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " '=09PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Cabot Oil & Gas Marketing Corp. - Amendment and Confirmations to',\n",
        " 'Cabot Oil & Gas Marketing Corp. - Amendment and Confirmations to',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Re: Cabot Oil & Gas Marketing Corp. - 9/99 production - price',\n",
        " 'Cabot Oil & Gas Marketing Corp. - Amendment and Confirmations to',\n",
        " 'Cabot Oil & Gas Marketing Corp. - Amendment and Confirmations to',\n",
        " 'EOTT Crude Oil Tanks',\n",
        " 'Re: Oil Skim + \"Bugs\"',\n",
        " 'Oil Skim + \"Bugs\"',\n",
        " 'Oil Release Incident',\n",
        " 'Oil Release Incident',\n",
        " 'Oil Release Incident',\n",
        " 'RE: Location of the 2002 Institute on Oil & Gas Law & Taxation --',\n",
        " 'Location of the 2002 Institute on Oil & Gas Law & Taxation -- February, 2002',\n",
        " 'RE: Location of the 2002 Institute on Oil & Gas Law & Taxation --',\n",
        " 'RE: Location of the 2002 Institute on Oil & Gas Law & Taxation -- February, 2002',\n",
        " 'RE: Location of the 2002 Institute on Oil & Gas Law & Taxation',\n",
        " 'B & J Gas and Oil',\n",
        " 'Re: B & J Gas and Oil',\n",
        " 'National Oil & Gas Coop',\n",
        " 'Re: National Oil & Gas Coop',\n",
        " 'Re: B & J Gas & Oil',\n",
        " 'Re: B & J Gas & Oil',\n",
        " 'B & J Gas & Oil',\n",
        " 'Re: B & J Gas & Oil',\n",
        " 'Re: B & J Gas & Oil',\n",
        " 'B & J Gas & Oil',\n",
        " 'Re: Maynard Oil - Revised Nom',\n",
        " 'Maynard Oil - Revised Nom',\n",
        " 'Re: Period after commissioning on oil - PPA availability penalties',\n",
        " 'Re: Period after commissioning on oil - PPA availability penalties',\n",
        " 'Re: Period after commissioning on oil - PPA availability penalties',\n",
        " 'Eastern States Oil & Gas',\n",
        " 'Eastern States Oil & Gas',\n",
        " 'Re: Petro-Canada Oil & Gas',\n",
        " 'Petro-Canada Oil & Gas',\n",
        " 'Re: Enron Liquid Fuels, Inc. v. Gulf Oil Limited Partnership',\n",
        " 'Re: Enron Liquid Fuels, Inc. v. Gulf Oil Limited Partnership',\n",
        " 'RE: Andersen Oil & Gas Symposium',\n",
        " 'RE: Andersen Oil & Gas Symposium',\n",
        " 'Andersen Oil & Gas Symposium',\n",
        " 'RE: Andersen Oil & Gas Symposium',\n",
        " 'Andersen Oil & Gas Symposium',\n",
        " 'Re: FW: Andersen Oil & Gas Symposium',\n",
        " 'RE: Colonial Oil Industries, Inc.',\n",
        " 'Colonial Oil Industries, Inc.',\n",
        " 'Colonial Oil Indusries, Inc.',\n",
        " 'RE: Colonial Oil Indusries, Inc.',\n",
        " 'RE: Colonial Oil Indusries, Inc.',\n",
        " 'Colonial Oil Indusries, Inc.',\n",
        " 'Re: Vineyard Oil and Gas deal 662482',\n",
        " 'Vineyard Oil and Gas deal 662482',\n",
        " 'Vineyard Oil and Gas deal 662482',\n",
        " 'Re: Oil & Gas Confirms - Andex',\n",
        " 'Re: National Oil & Gas Coop',\n",
        " 'Mobil Oil Corporation Master Enfolio Agreement',\n",
        " 'Re: Vineyard Oil & Gas: Q56432',\n",
        " 'Re: Vineyard Oil & Gas: Q56432',\n",
        " 'Re: Vineyard Oil & Gas: Q56432',\n",
        " 'Imperial Oil Resources',\n",
        " 'Belco Oil & Gas',\n",
        " 'Belco Oil & Gas',\n",
        " 'United Oil & Minerals, Inc.',\n",
        " 'Belco Oil & Gas Corp.',\n",
        " 'Re: United Oil & Minerals',\n",
        " 'Husky Oil',\n",
        " 'ETA Amendment - Imperial Oil Resources',\n",
        " '(00-323) Margin Change for Crude Oil, Unleaded Gasoline, and',\n",
        " 'Re: (00-323) Margin Change for Crude Oil, Unleaded Gasoline, and',\n",
        " '(00-323) Margin Change for Crude Oil, Unleaded Gasoline, and Heating',\n",
        " 'Re: United Oil & Minerals -Amended Credit W/S',\n",
        " 'Re: US Heating Oil and Unleaded Gas Fin Spreads - Approval',\n",
        " 'US Heating Oil and Unleaded Gas Fin Spreads - Approval',\n",
        " '(00-359) Margin Rate Change for Crude Oil, Unleaded Gasoline, and',\n",
        " '(00-362) Revised Crude Oil Options Expiration Date',\n",
        " 'Re: (00-362) Revised Crude Oil Options Expiration Date',\n",
        " '(00-367) Revised - Crude Oil Futures Expiration Date',\n",
        " '(00-377) Margin Rate Change for Crude Oil, Unleaded Gasoline, and',\n",
        " 'Re: New EOL Product - Crude Oil Financial',\n",
        " 'New EOL Product - Crude Oil Financial',\n",
        " 'NDA-PVM Oil  Associates Limited',\n",
        " 'Re: NDA-PVM Oil Associates Limited',\n",
        " 'BP Oil International Limited',\n",
        " 'NDA - PVM Oil Associates Limited',\n",
        " 'BP Exploration & Oil Inc.',\n",
        " 'BP Oil Supply Company',\n",
        " 'RE: BP Oil Supply Company',\n",
        " 'BP Oil Supply Company',\n",
        " 'ETA Amendment - BP Oil Supply Company',\n",
        " 'Re: bp Oil Supply registration status',\n",
        " 'bp Oil Supply registration status',\n",
        " 'Re: FW: Product Type approval for 3 product types (Heating Oil Fin',\n",
        " 'FW: Product Type approval for 3 product types (Heating Oil Fin=20',\n",
        " 'FW: Product Type approval for 3 product types (Heating Oil Fin=20',\n",
        " 'Product Type approval for 3 product types (Heating Oil Fin Options=',\n",
        " 'Product Type approval for 2 product types (US Residual Fuel Oil 1%=',\n",
        " 'Husky Oil Limited/ECT Canada',\n",
        " 'RE: Belco Oil & Gas/Westport Resources merger',\n",
        " 'RE: Belco Oil & Gas/Westport Resources merger',\n",
        " 'Belco Oil & Gas/Westport Resources merger',\n",
        " 'RE: Hunt Oil Company of Canada',\n",
        " 'Hunt Oil Company of Canada',\n",
        " 'FW: Belco Oil & Gas Corp Merger w/ Westport Resources Corp.',\n",
        " 'FW: Belco Oil & Gas Corp Merger w/ Westport Resources Corp.',\n",
        " 'Belco Oil & Gas Corp Merger w/ Westport Resources Corp.',\n",
        " 'BP Exploration & Oil Inc. Merger Documentation',\n",
        " 'FW: BP Exploration & Oil Inc. Merger Documentation',\n",
        " 'BP Exploration & Oil Inc. Merger Documentation',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Re: Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'Forward oil prices',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'Re: Seismic Data on Oil & Gas field Development via Satellite',\n",
        " 'Re: Seismic Data on Oil & Gas field Development via Satellite',\n",
        " 'Re: exploration data as the root of the energy (oil) supply chain',\n",
        " 'exploration data as the root of the energy (oil) supply chain and',\n",
        " 'Seismic Data on Oil & Gas field Development via Satellite',\n",
        " 'Seismic Data on Oil & Gas field Development via Satellite',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'RE: Crude Oil for Oz',\n",
        " 'FW: Crude Oil for Oz',\n",
        " 'FW: Crude Oil for Oz',\n",
        " 'Crude Oil for Oz',\n",
        " 'FW: GPCM News: 8/20/01:  RBAC Finalizing Schedule in Houston: Oil',\n",
        " '=09GPCM News: 8/20/01:  RBAC Finalizing Schedule in Houston: Oil Co=',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " '=09PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " '=09PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'MOU With India Oil Corp.',\n",
        " 'MOU With India Oil Corp.',\n",
        " 'Iraqui - Oil for food',\n",
        " 'Iraqui - Oil for food',\n",
        " 'Re: Iraqui - Oil for food',\n",
        " 'Iraqui - Oil for food',\n",
        " 'Re: PIRA Oil Briefing',\n",
        " 'PIRA Oil Briefing',\n",
        " 'Telephone call to Steve Hellman, Oil Space',\n",
        " 'Re: PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'PIRA Global Oil and Natural Outlooks- Save these dates.',\n",
        " 'John Arnold Crude Oil Deals',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Cabot Oil & Gas Marketing Corporation',\n",
        " 'Coleman Oil & Gas Term Sheet and LOU',\n",
        " 'Coleman Oil & Gas Term Sheet and LOU',\n",
        " 'Re: Oil & Gas Lease',\n",
        " 'FW: Fw: Oil changes: Men vs. Women',\n",
        " 'Fwd: Fw: Oil changes: Men vs. Women',\n",
        " 'Citation Oil & Gas Spot Enfolio',\n",
        " 'FW: Patina Oil & Gas Docs.',\n",
        " 'Patina Oil & Gas Docs.',\n",
        " 'FW: Pioneer Oil',\n",
        " 'Pioneer Oil',\n",
        " 'FW: Patina Oil and Gas Purchase',\n",
        " 'Patina Oil and Gas Purchase',\n",
        " 'FW: Kennedy Oil and Enron North America Corp.',\n",
        " 'FW: Kennedy Oil and Enron North America Corp.',\n",
        " 'RE: Cutter Oil',\n",
        " 'Cutter Oil',\n",
        " 'RE: Cutter Oil',\n",
        " 'RE: Cutter Oil',\n",
        " 'RE: Cutter Oil',\n",
        " 'Cutter Oil',\n",
        " 'Belco Oil & Gas/Westport Resources merger',\n",
        " 'Merit Gas & Oil, Inc.',\n",
        " 'Re: Cutter Oil Contract',\n",
        " 'Re: Cross Timbers Oil Company - Contact Information',\n",
        " 'Re: Revised Cutter Oil contract',\n",
        " 'Re: Revised Cutter Oil contract',\n",
        " 'Cross Timbers Oil',\n",
        " 'Cutter Oil Company',\n",
        " 'Cutter Oil Company',\n",
        " 'Cutter Oil',\n",
        " 'Murphy Oil',\n",
        " 'Murphy Oil',\n",
        " 'Forest Oil / Energen Resources',\n",
        " 'Forest Oil Corporation',\n",
        " 'Matrix Oil & Gas',\n",
        " 'Irving Oil CA',\n",
        " 'Irving Oil CA',\n",
        " 'Re: Irving Oil CA',\n",
        " 'Re: Cutter Oil Company',\n",
        " 'Re: Cutter Oil Company',\n",
        " 'Re: Cutter Oil Company',\n",
        " 'Jay Bee Oil & Gas',\n",
        " 'Re: Jay Bee Oil & Gas',\n",
        " 'FW: IGI Resources, Inc. and BTA Oil Producers',\n",
        " 'RE: IGI Resources, Inc. and BTA Oil Producers',\n",
        " 'IGI Resources, Inc. and BTA Oil Producers',\n",
        " 'RE: Patina Oil and Gas Purchase',\n",
        " 'FW: Patina Oil and Gas Purchase',\n",
        " 'Patina Oil and Gas Purchase',\n",
        " 'RE: DeBrular/Stevens Oil',\n",
        " 'RE: DeBrular/Stevens Oil',\n",
        " 'DeBrular/Stevens Oil',\n",
        " 'RE: Stevens Oil',\n",
        " 'RE: Stevens Oil',\n",
        " 'Stevens Oil',\n",
        " 'RE: DeBrular/Stevens Oil',\n",
        " 'DeBrular/Stevens Oil',\n",
        " 'FW: Proposed Contract - BTA Oil Producers',\n",
        " 'Proposed Contract - BTA Oil Producers',\n",
        " 'RE: IGI Resources, Inc. and BTA Oil Producers',\n",
        " 'IGI Resources, Inc. and BTA Oil Producers',\n",
        " 'BTA Oil Producers / IGI Resources',\n",
        " 'Re: Cabot Oil & Gas Marketing Corporation',\n",
        " 'Re: Killam Oil Dehydration in the Juanita Lobo Field, Webb County,',\n",
        " 'Re: Colonial Oil',\n",
        " 'Colonial Oil',\n",
        " 'Wall Street Journal Article - Regarding SPR Oil',\n",
        " 'Reminder: Annual Oil Spill Crisis Management Training',\n",
        " 'Re: Gulf Oil Co.',\n",
        " 'BP Oil International Ltd. (\"BP\")',\n",
        " 'Forest Oil Corporation',\n",
        " 'Wiser Oil Confidentiality Agreement',\n",
        " 'Draft term sheet for oil-power spread option pruchase from FPL',\n",
        " 'Re: Draft term sheet for oil-power spread option pruchase from FPL',\n",
        " 'Draft term sheet for oil-power spread option pruchase from FPL',\n",
        " 'Prepaid Oil Swap - Transaction Diagram',\n",
        " 'Prepaid Oil Swap - Transaction Diagram',\n",
        " 'BP Oil Internation Ltd',\n",
        " 'Cross Oil and Refining & Marketing',\n",
        " 'RE: Kern Oil & Refining Company',\n",
        " 'Kern Oil & Refining Company',\n",
        " 'RE: Oil Prepay Rollover',\n",
        " 'Oil Prepay Rollover',\n",
        " 'FW: US Filter Comments on Omnibus and Annex A of a Heating Oil',\n",
        " 'US Filter Comments on Omnibus and Annex A of a Heating Oil Deferred Premium Call',\n",
        " 'FW: ENA Oil Prepay with CSFB/ Morgan Stanley',\n",
        " 'FW: ENA Oil Prepay with CSFB/ Morgan Stanley',\n",
        " 'FW: ENA Oil Prepay with CSFB/ Morgan Stanley',\n",
        " 'FW: ENA Oil Prepay with CSFB/ Morgan Stanley',\n",
        " 'ENA Oil Prepay with CSFB/ Morgan Stanley',\n",
        " 'Re: CPE Credit-Oil Spill Training',\n",
        " 'CPE Credit-Oil Spill Training',\n",
        " 'NE Heating Oil Reserve',\n",
        " 'NE Heating Oil Reserve',\n",
        " 'CERA Monthly Oil Briefing - CERA Alert - December 20, 2000',\n",
        " 'CERA Monthly Oil Briefing - CERA Alert - December 20, 2000',\n",
        " 'CERA Monthly Oil Briefing - CERA Alert - December 20, 2000',\n",
        " 'Re: PIRA World Oil Outlook Presentation',\n",
        " 'PIRA World Oil Outlook Presentation',\n",
        " 'Oil Week Ahead - Dec 4-00',\n",
        " 'Oil Week Ahead - Dec 4-00',\n",
        " 'The Oil Daily Wednesday, Nov. 29, 2000 (pdf)',\n",
        " 'The Oil Daily Wednesday, Nov. 29, 2000 (pdf)',\n",
        " 'The Oil Daily Wednesday, Nov. 29, 2000 (pdf)',\n",
        " 'The Oil Daily, Tuesday, Nov. 28, 2000 (pdf)',\n",
        " 'The Oil Daily, Tuesday, Nov. 28, 2000 (pdf)',\n",
        " 'The Oil Daily, Tuesday, Nov. 28, 2000 (pdf)',\n",
        " 'Reminder: Annual Oil Spill Crisis Management Training',\n",
        " 'Reminder: Annual Oil Spill Crisis Management Training',\n",
        " 'Re: Annual Oil Spill Crisis Management Training',\n",
        " 'Re: Annual Oil Spill Crisis Management Training',\n",
        " 'Fuel Oil',\n",
        " 'Fuel Oil',\n",
        " 'how to go forward in the oil markets',\n",
        " 'how to go forward in the oil markets',\n",
        " 'Re: Oil Prices and Investment',\n",
        " 'Oil Prices and Investment',\n",
        " 'Crude Oil Purchase Agreement',\n",
        " 'Re: US Heating Oil and Unleaded Gas Fin Spreads - Approval',\n",
        " '(00-362) Revised Crude Oil Options Expiration Date',\n",
        " 'Re: (00-362) Revised Crude Oil Options Expiration Date',\n",
        " 'Re: PLEASE RESPOND: US Residual Fuel Oil 1% Fin Spd / US Residual',\n",
        " 'Product Type Approval for 2 product types!! (US Residual Fuel Oil 1%',\n",
        " 'Re: ISDA for Irving Oil',\n",
        " 'Union Oil and ETA for Enron Online',\n",
        " 'Union Oil and ETA for Enron Online',\n",
        " 'RE: Access to the Daily Oil Bulletin online',\n",
        " 'Access to the Daily Oil Bulletin online',\n",
        " 'Fuel Oil Sample',\n",
        " 'Fuel Oil Vanadium Spec',\n",
        " 'RE: Fuel Oil Ash Spec',\n",
        " 'Fuel Oil Ash Spec',\n",
        " 'Fuel Oil!!!',\n",
        " 'Ft. Pierce #2 Fuel Oil',\n",
        " 'Fuel Oil Reimbursement to FPUA',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Fuel Oil Sample',\n",
        " 'Re: Fuel Oil Sample',\n",
        " 'Fuel Oil Sample',\n",
        " 'FYI - Update of Fuel Oil Analysis',\n",
        " 'Update of Fuel Oil Analysis',\n",
        " 'Update of Fuel Oil Analysis',\n",
        " 'Fuel Oil Questions',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'Re: FW: Patina Oil & Gas Docs.',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'RE: FW: Patina Oil & Gas Docs.',\n",
        " 'Re: FW: Patina Oil & Gas Docs.',\n",
        " 'FW: Patina Oil & Gas Docs.',\n",
        " 'Patina Oil & Gas Docs.',\n",
        " 'Patina Oil and Gas Purchase',\n",
        " 'FW: Kennedy Oil',\n",
        " 'Kennedy Oil',\n",
        " 'Kennedy Oil',\n",
        " 'FW: Kennedy Oil',\n",
        " 'Kennedy Oil',\n",
        " 'Kennedy Oil',\n",
        " 'FW: Frontier Oil Corporation Credit Line',\n",
        " 'Frontier Oil Corporation Credit Line',\n",
        " 'Re: No.6 Oil - Dom. Rep.']"
       ]
      }
     ],
     "prompt_number": 132
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Metacharacters: quantifiers\n",
      "\n",
      "Above we had a regular expression that looked like this:\n",
      "\n",
      "    [aeiou][aeiou][aeiou][aeiou]\n",
      "    \n",
      "Typing out all of those things is kind of a pain. Fortunately, there\u2019s a way to specify how many times to match a particular character, using quantifiers. These affect the character that immediately precede them:\n",
      "\n",
      "| quantifier | meaning |\n",
      "|------------|---------|\n",
      "| `{n}` | match exactly n times |\n",
      "| `{n,m}` | match at least n times, but no more than m times |\n",
      "| `{n,}` | match at least n times |\n",
      "| `+` | match at least once (same as {1,}) |\n",
      "| `*` | match zero or more times |\n",
      "| `?` | match one time or zero times |\n",
      "\n",
      "For example, here's an example of a regular expression that finds subjects that contain at least fifteen capital letters in a row:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"[A-Z]{15,}\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 136,
       "text": [
        "['CONGRATULATIONS!',\n",
        " 'CONGRATULATIONS!',\n",
        " 'Re: FW: Fw: Fw: Fw: Fw: Fw: Fw: PLEEEEEEEEEEEEEEEASE READ!',\n",
        " 'ACCOMPLISHMENTS',\n",
        " 'ACCOMPLISHMENTS',\n",
        " 'Re: FW: FORM: BILATERAL CONFIDENTIALITY AGREEMENT',\n",
        " 'FORM: BILATERAL CONFIDENTIALITY AGREEMENT',\n",
        " 'Re: CONGRATULATIONS!',\n",
        " 'CONGRATULATIONS!',\n",
        " 'Re: ORDER ACKNOWLEDGEMENT',\n",
        " 'ORDER ACKNOWLEDGEMENT',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'Re: CONGRATULATIONS',\n",
        " 'CONGRATULATIONS',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'RE: CONGRATULATIONS',\n",
        " 'Re: CONGRATULATIONS',\n",
        " 'CONGRATULATIONS',\n",
        " 'Re: VEPCO INTERCONNECTION AGREEMENT',\n",
        " 'VEPCO INTERCONNECTION AGREEMENT',\n",
        " 'Re: VEPCO INTERCONNECTION AGREEMENT',\n",
        " 'Re: VEPCO INTERCONNECTION AGREEMENT',\n",
        " 'VEPCO INTERCONNECTION AGREEMENT',\n",
        " 'Re: CONGRATULATIONS !',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'Re: FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'FW: WASSSAAAAAAAAAAAAAABI!',\n",
        " 'RE: NOOOOOOOOOOOOOOOO',\n",
        " 'NOOOOOOOOOOOOOOOO',\n",
        " 'RE: NOOOOOOOOOOOOOOOO',\n",
        " 'CONGRATULATIONS!!!!!!!!!!!!!',\n",
        " 'RE: CONGRATULATIONS!!!!!!!!!!!!!',\n",
        " 'Re: CONGRATULATIONS!!!!!!!!!!!!!',\n",
        " 'CONGRATULATIONS',\n",
        " 'Re: CONFIDENTIALITY/CONFLICTS ISSUES MEETING',\n",
        " 'CONFIDENTIALITY/CONFLICTS ISSUES MEETING',\n",
        " 'GOALS AND ACCOMPLISHMENTS',\n",
        " 'ACCOMPLISHMENTS',\n",
        " 'Re: CONGRATULATIONS!',\n",
        " 'RE: STANDARDIZATION OF TANKER FREIGHT WORDING',\n",
        " 'RE: STANDARDIZATION OF TANKER FREIGHT WORDING',\n",
        " 'Re: STANDARDIZATION OF TANKER FREIGHT WORDING',\n",
        " 'STANDARDIZATION OF TANKER FREIGHT WORDING',\n",
        " 'BRRRRRRRRRRRRRRRRRRRRR',\n",
        " 'Re: CONGRATULATIONS !!!',\n",
        " 'CONGRATULATIONS !!!',\n",
        " 'RE: Mtg. to discuss assignment of customers. Transmission list:  P/LEGAL/PROJECTNETCO/NETCOTRANSMISSION.XLS',\n",
        " 'RE: Mtg. to discuss assignment of customers. Transmission list:  P/LEGAL/PROJECTNETCO/NETCOTRANSMISSION.XLS',\n",
        " 'Mtg. to discuss assignment of customers. Transmission list:  P/LEGAL/PROJECTNETCO/NETCOTRANSMISSION.XLS',\n",
        " 'FW: NEW WEATHER SWAPS ON THE INTERCONTINENTAL EXCHANGE',\n",
        " 'NEW WEATHER SWAPS ON THE INTERCONTINENTAL EXCHANGE']"
       ]
      }
     ],
     "prompt_number": 136
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Lines that contain five consecutive vowels:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"[aeiou]{5}\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 137,
       "text": [
        "['WooooooHoooooo more Vacation',\n",
        " 'Gooooooooooood Bye!',\n",
        " 'Gooooooooooood Bye!',\n",
        " 'RE: Hello Sweeeeetie',\n",
        " 'Hello Sweeeeetie',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'FW: Waaasssaaaaabi !',\n",
        " 'Re: FW: Wasss Uuuuuup STG?',\n",
        " 'RE: Rrrrrrrooooolllllllllllll TIDE!!!!!!!!',\n",
        " 'Rrrrrrrooooolllllllllllll TIDE!!!!!!!!',\n",
        " 'Re: Helloooooo!!!',\n",
        " 'Re: Helloooooo!!!',\n",
        " 'Fw: FW: OOOooooops',\n",
        " 'FW: FW: OOOooooops',\n",
        " 'yahoooooooooooooooooooo',\n",
        " 'RE: yahoooooooooooooooooooo',\n",
        " 'RE: yahoooooooooooooooooooo',\n",
        " 'yahoooooooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " 'RE: I hate yahooooooooooooooo',\n",
        " 'I hate yahooooooooooooooo',\n",
        " \"FW: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " \"RE: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " \"RE: duuuuuuuuuuuuuuuuude...........what's up?\",\n",
        " 'Re: skiiiiiiiiing',\n",
        " 'skiiiiiiiiing',\n",
        " 'scuba dooooooooooooo',\n",
        " 'RE: scuba dooooooooooooo',\n",
        " 'RE: scuba dooooooooooooo',\n",
        " 'scuba dooooooooooooo',\n",
        " 'Re: skiiiiiiiing',\n",
        " 'skiiiiiiiing',\n",
        " 'Re: skiiiiiiiing',\n",
        " 'Re: skiiiiiiiiing']"
       ]
      }
     ],
     "prompt_number": 137
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Count the number of lines that are e-mail forwards, regardless of whether the subject line begins with `Fw:`, `FW:`, `Fwd:` or `FWD:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len([line for line in subjects if re.search(r\"^F[Ww]d?:\", line)])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 140,
       "text": [
        "20159"
       ]
      }
     ],
     "prompt_number": 140
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Lines that have the word `news` in them and end in an exclamation point:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\b[Nn]ews\\b.*!$\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 144,
       "text": [
        "['RE: Christmas Party News!',\n",
        " 'FW: Christmas Party News!',\n",
        " 'Christmas Party News!',\n",
        " 'Good News!',\n",
        " 'Good News--Twice!',\n",
        " 'Re: VERY Interesting News!',\n",
        " 'Great News!',\n",
        " 'Re: Great News!',\n",
        " 'News Flash!',\n",
        " 'RE: News Flash!',\n",
        " 'RE: News Flash!',\n",
        " 'News Flash!',\n",
        " 'RE: Good News!',\n",
        " 'RE: Good News!',\n",
        " 'RE: Good News!',\n",
        " 'RE: Good News!',\n",
        " 'Good News!',\n",
        " 'RE: Good News!!!',\n",
        " 'Good News!!!',\n",
        " 'RE: Big News!',\n",
        " 'Big News!',\n",
        " 'Individual.com - News From a Friend!',\n",
        " 'Individual.com - News From a Friend!',\n",
        " 'Re: Individual.com - News From a Friend!',\n",
        " 'RE: We need news!',\n",
        " '=09We need news!',\n",
        " 'RE: Big News!',\n",
        " 'FW: Big News!',\n",
        " 'RE: Big News!',\n",
        " 'FW: Big News!',\n",
        " 'Big News!',\n",
        " 'FW: NW Wine News- Eroica, Sineann, Bergstrom, Hamacher, And more!',\n",
        " '=09NW Wine News- Eroica, Sineann, Bergstrom, Hamacher, And more!',\n",
        " 'RE: Good News!!!',\n",
        " 'Good News!!!',\n",
        " 'Re: Big News!',\n",
        " 'Big News!',\n",
        " 'RE: Good  News!',\n",
        " 'Good  News!']"
       ]
      }
     ],
     "prompt_number": 144
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Metacharacters: alternation\n",
      "\n",
      "One final bit of regular expression syntax: alternation.\n",
      "\n",
      "* `(?:x|y)`: match either x or y\n",
      "* `(?:x|y|z)`: match x, y or z\n",
      "* etc.\n",
      "\n",
      "So for example, if you wanted to count every subject line that begins with either `Re:` or `Fwd:`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len([line for line in subjects if re.search(r\"^(?:Re|Fwd):\", line)])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 174,
       "text": [
        "39901"
       ]
      }
     ],
     "prompt_number": 174
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Every subject line that mentions a primary color:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\b(?:[Rr]ed|[Yy]ellow|[Bb]lue)\\b\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 175,
       "text": [
        "['Re: Blue Dolphin Pipe',\n",
        " 'Blue Dolphin Pipe',\n",
        " 'FW: Red Rock expansion',\n",
        " 'FW: Red Rock expansion',\n",
        " 'Red Rock expansion',\n",
        " 'Re: Red Rock GE/NP Emissions',\n",
        " 'Re: Red Rock GE/NP Emissions',\n",
        " 'Re: Red Rock GE/NP Emissions',\n",
        " 'Red Rock GE/NP Emissions',\n",
        " 'Air Permit Delay, Red Rock Expansion',\n",
        " 'RE: Red Rock Air Permits Heads up!',\n",
        " 'RE: Red Rock Air Permits Heads up!',\n",
        " 'Red Rock Air Permits Heads up!',\n",
        " 'RE: FW: Red Rock Expansion Station 4',\n",
        " 'RE: FW: Red Rock Expansion Station 4',\n",
        " 'RE: FW: Red Rock Expansion Station 4',\n",
        " 'Re: FW: Red Rock Expansion Station 4',\n",
        " 'FW: Red Rock Expansion Station 4',\n",
        " 'Red Rock Expansion Station 4',\n",
        " 'summary of red rock contracts',\n",
        " 'Re: Yellow Book',\n",
        " 'Re: Now we have red ones',\n",
        " 'Re: Red Herring Res',\n",
        " 'Blue Jean Shirts',\n",
        " 'Blue Jean Shirts',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Red Rock Delay $$ Impact',\n",
        " 'from my red neck uncle...',\n",
        " 'Blue Cross ?',\n",
        " 'Re: REPLY TO: Power Rate Offer: Weserner Exposition, Red Deer',\n",
        " 'Red Rock Expansion Filing',\n",
        " 'Red Rock Expansion Filing',\n",
        " 'Red Rock open season',\n",
        " 'Red Rock open season',\n",
        " 'Re: Final Red Rock',\n",
        " 'Final Red Rock',\n",
        " 'Re: Red Rock shipper letter',\n",
        " 'Red Rock shipper letter',\n",
        " 'Re: Red Rock posting',\n",
        " 'Red Rock posting',\n",
        " 'Re: Red Rock posting',\n",
        " 'Re: Red Rock posting',\n",
        " 'Re: Red Rock posting',\n",
        " 'Red Rock posting',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar update',\n",
        " 'Red Cedar update',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar',\n",
        " 'Re: Red Cedar - Contract Approval Request',\n",
        " 'Red Cedar - Contract Approval Request',\n",
        " 'Re: Red Cedar',\n",
        " 'Re: Red Cedar',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar',\n",
        " 'FW: Red Rock (Transwestern) Cashflow',\n",
        " 'RE: Red Rock (Transwestern) Cashflow',\n",
        " 'RE: Red Rock (Transwestern) Cashflow',\n",
        " 'Red Rock (Transwestern) Cashflow',\n",
        " 'FW: Red Rock additions',\n",
        " 'FW: Red Rock additions',\n",
        " 'FW: Red Rock additions',\n",
        " 'RE: Red Rock additions',\n",
        " 'Red Rock additions',\n",
        " 'FW: Honest Answers from the Guy in the Red..',\n",
        " 'Fw: Honest Answers from the Guy in the Red..',\n",
        " 'Honest Answers from the Guy in the Red..',\n",
        " 'Re: Blue Range',\n",
        " 'Blue Range',\n",
        " 'Re: Blue Range',\n",
        " 'Re: Blue Range Resource Corporation',\n",
        " 'Re: Blue Range Resource Corporation',\n",
        " 'FW: Red Rock additions',\n",
        " 'FW: Red Rock additions',\n",
        " 'RE: Red Rock additions',\n",
        " 'Red Rock additions',\n",
        " 'Red Rock expansion',\n",
        " 'SD 750 case w/ Red Hawk lateral',\n",
        " 'Aquila Red Lake storage project',\n",
        " 'Red Lake Storage project',\n",
        " 'FW: New Uniforms for the other Big Red',\n",
        " 'New Uniforms for the other Big Red',\n",
        " 'FW: Red Lake Storage w/Kevin Hyatt',\n",
        " 'Red Lake Storage w/Kevin Hyatt',\n",
        " 'Follow up from Red Lake meeting',\n",
        " 'Firm Rec/Del questions from potential Aquila Red Lake shippers',\n",
        " 'FW: Red Lake Info',\n",
        " 'FW: Red Lake Info',\n",
        " 'RE: Red Lake Info',\n",
        " 'Red Lake Info',\n",
        " 'Blue Files',\n",
        " 'Blue Flame Propane Inc.',\n",
        " 'Re: Blue Flame Propane Inc.',\n",
        " 'Blue Flame Propane Inc.',\n",
        " 'Re: Ideas on Blue/Green Files',\n",
        " 'FW: Red Hat no longer sings \"When I\\'m 64\"',\n",
        " 'Red Hat no longer sings \"When I\\'m 64\"',\n",
        " 'SECURITY AND BUG NEWS ALERT: Users offer tips on foiling Code Red',\n",
        " 'Red Herring Article',\n",
        " 'Red Herring Article',\n",
        " 'RE: Red Herring',\n",
        " 'Red Herring',\n",
        " 'FW: Red Herring e-mail',\n",
        " 'Red Herring e-mail',\n",
        " 'FW: Transwestern Red Rock Permitting Status',\n",
        " 'Transwestern Red Rock Permitting Status',\n",
        " 'RE: Red Banner Screen Shots',\n",
        " 'FW: Red Banner Screen Shots',\n",
        " 'Red Banner Screen Shots',\n",
        " 'Re: Blue Range Resource Corporation - Eligible Financial Contracts',\n",
        " 'Blue Range Recovery',\n",
        " 'Blue Range Recovery',\n",
        " 'Thank you from Red Herring',\n",
        " 'RE: Red Lake Storage project',\n",
        " 'Red Lake Storage project',\n",
        " 'Aquila Red Lake/TW strategy meeting',\n",
        " 'FW: Red Rock Receipts',\n",
        " 'RE: Red Rock Receipts',\n",
        " 'Red Rock Receipts',\n",
        " 'FW: Project Status Review Meeting for Red Rock and GTB',\n",
        " 'Updated: Project Status Review Meeting for Red Rock and GTB',\n",
        " 'FW: Red Rock Briefing',\n",
        " 'Red Rock Briefing',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'Amending Red Rock Contracts',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'RE: Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'FW: Amending Red Rock Contracts',\n",
        " 'Amending Red Rock Contracts',\n",
        " 'RE: Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'RE: Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'RE: Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'RE: Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'Commissioner.COM Trade Offered by Blue Balls',\n",
        " 'Blue dog override letter',\n",
        " 'Just kidding, this is the real Blue dog override letter',\n",
        " 'Just kidding, this is the real Blue dog override letter',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Override letter related to the Blue Dog turbines',\n",
        " 'Re: Blue Dog Change Order',\n",
        " 'Blue Dog Change Order',\n",
        " 'Blue Dog Change Order',\n",
        " 'LJM/Blue Girl turbines',\n",
        " 'Blue Dog Change Order',\n",
        " 'Blue Dog Change Order',\n",
        " 'Blue Dog Change Order',\n",
        " 'Re: Blue Dog Change Order',\n",
        " 'Re: Blue Dog Change Order',\n",
        " 'Re: Blue Dog Change Order',\n",
        " 'Re: Blue Dog - Comments to Change Order #1',\n",
        " 'Blue Dog - Comments to Change Order #1',\n",
        " 'Letter agreement re: Blue Dog',\n",
        " 'Form for Blue Dog',\n",
        " 'Letter agreement re: Blue Dog',\n",
        " 'Letter agreement re: Blue Dog',\n",
        " 'Re: FW: Form for Blue Dog',\n",
        " 'FW: Form for Blue Dog',\n",
        " 'Form for Blue Dog',\n",
        " 'Re: Blue Dog Amended LLC',\n",
        " 'Blue Dog Amended LLC',\n",
        " 'Re: Dual Fuel Configuration - Blue Dog #2',\n",
        " 'Re: Blue Dog Amended LLC',\n",
        " 'Re: Blue Dog Amended LLC',\n",
        " 'Blue Dog Amended LLC',\n",
        " 'Re: FW: Blue Dog #2',\n",
        " 'FW: Blue Dog #2',\n",
        " 'FW: Blue Dog #2',\n",
        " 'Re: FW: Blue Dog #2',\n",
        " 'Re: FW: Blue Dog #2',\n",
        " 'Re: FW: Blue Dog #2',\n",
        " 'FW: Blue Dog #2',\n",
        " 'FW: Blue Dog #2',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'RE: Blue Dog #2',\n",
        " 'RE: Blue Dog #2',\n",
        " 'Re: FW: Blue Dog #2',\n",
        " 'Re: FW: Blue Dog #2   << OLE Object: StdOleLink >>',\n",
        " 'FW: Blue Dog #2',\n",
        " 'FW: Blue Dog #2',\n",
        " 'ENA/Blue Dog: First Amended and Restated LLC Agreement',\n",
        " 'ENA/Blue Dog: First Amended and Restated LLC Agreement',\n",
        " 'Blue Dog LLC agreement',\n",
        " 'Blue Dog LLC agreement',\n",
        " 'Blue Dog LLC agreement',\n",
        " 'Re: ENA/Blue Dog:',\n",
        " 'ENA/Blue Dog:',\n",
        " 'Re: ENA/Project Blue Dog: Preliminary Documents',\n",
        " 'ENA/Project Blue Dog: Preliminary Documents',\n",
        " 'ENA/Project Blue Dog & Salmon: Preliminary Documents',\n",
        " 'ENA/Project Blue Dog & Salmon: Preliminary Documents',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog change orders',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Blue Dog assignment language',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Re: Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Re: ENRON Blue Dog Max 2 X 7 EA GTG',\n",
        " 'Re: ENRON Blue Dog Max 2 X 7 EA GTG',\n",
        " 'ENRON Blue Dog Max 2 X 7 EA GTG',\n",
        " 'NW my comments in blue',\n",
        " 'NW my comments in blue',\n",
        " 'NW my comments in blue',\n",
        " 'RE: ENA/Blue Dog: Revised Letter Agreement- my comments',\n",
        " 'RE: ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Re: Blue Dog change orders',\n",
        " 'Re: Blue dog',\n",
        " 'Blue dog',\n",
        " 'Standard acknowledgement from GE - applicable to Blue Dog?',\n",
        " 'Blue Dog change order',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: Revised Letter Agreement',\n",
        " 'ENA/Blue Dog: LLC Agreement and Letter Agreement',\n",
        " 'ENA/Blue Dog: LLC Agreement and Letter Agreement',\n",
        " 'Re: FW: ENA/Blue Dog: Marked Documents',\n",
        " 'RE: Blue Dog Turbines',\n",
        " 'RE: Blue Dog Turbines',\n",
        " 'RE: Blue Dog Turbines',\n",
        " 'Blue Dog Turbines',\n",
        " 'ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'Re: ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'Re: ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'Re: ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Closing Checklist and ENA Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Marked',\n",
        " 'ENA/Blue Dog: Marked',\n",
        " 'ENA/Blue Dog: conference call with Paul Hastings',\n",
        " 'ENA/Blue Dog: conference call with Paul Hastings',\n",
        " 'Blue Dog (Northwestern) assignment',\n",
        " 'ENA/Blue Dog: Conference call details',\n",
        " 'ENA/Blue Dog: Conference call details',\n",
        " 'Change Order 2 Blue Dog',\n",
        " 'RE: Change Order 2 Blue Dog',\n",
        " 'RE: Change Order 2 Blue Dog',\n",
        " 'Change Order 2 Blue Dog',\n",
        " 'RE: Change Order 2 Blue Dog',\n",
        " 'RE: Change Order 2 Blue Dog',\n",
        " 'Change Order 2 Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'RE: FW: Blue Dog Max, comments on draft CO No. 2, rev of 4/23/01',\n",
        " 'Re: ENA/Blue Dog: Escrow',\n",
        " 'ENA/Blue Dog: Escrow',\n",
        " 'Re: payment of invoices, Blue Dog Max',\n",
        " 'Re: payment of invoices, Blue Dog Max',\n",
        " 'payment of invoices, Blue Dog Max',\n",
        " 'payment of invoices, Blue Dog Max',\n",
        " 'Re: Blue Dog Meeting Today',\n",
        " 'Blue Dog Meeting Today',\n",
        " 'Re: payment of invoices, Blue Dog Max',\n",
        " 'Re: payment of invoices, Blue Dog Max',\n",
        " 'Re: payment of invoices, Blue Dog Max',\n",
        " 'payment of invoices, Blue Dog Max',\n",
        " 'payment of invoices, Blue Dog Max',\n",
        " 'Re: Blue Dog - Letter to GE',\n",
        " 'Blue Dog - Letter to GE',\n",
        " 'Friday Meeting re: Blue Dog',\n",
        " 'Friday Meeting re: Blue Dog',\n",
        " 'Re: Friday Meeting re: Blue Dog',\n",
        " 'Re: Friday Meeting re: Blue Dog',\n",
        " 'Friday Meeting re: Blue Dog',\n",
        " 'Re: Blue Dog Max monthly report',\n",
        " 'Re: Blue Dog Max monthly report',\n",
        " 'Blue Dog Max monthly report',\n",
        " 'Blue Dog Max monthly report',\n",
        " 'Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'Re: FW: Blue Dog',\n",
        " 'FW: Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'RE: Blue Dog',\n",
        " 'Re: Enron Blue Dog Assignment',\n",
        " 'Enron Blue Dog Assignment',\n",
        " 'Re: Blue Dog',\n",
        " 'Blue Dog',\n",
        " 'Re: ENA/Blue Dog: Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Incumbency Certificate',\n",
        " 'RE: ENA/Blue Dog: Incumbency Certificate',\n",
        " 'RE: ENA/Blue Dog: Incumbency Certificate',\n",
        " 'Re: ENA/Blue Dog: Incumbency Certificate',\n",
        " 'ENA/Blue Dog: Incumbency Certificate',\n",
        " 'Re: Enron Blue Dog Assignment',\n",
        " 'Re: Enron Blue Dog Assignment',\n",
        " 'Enron Blue Dog Assignment',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'FW: CA for Blue Ridge',\n",
        " 'CA for Blue Ridge',\n",
        " 'FW: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'FW: CA for Blue Ridge',\n",
        " 'CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'RE: CA for Blue Ridge',\n",
        " 'FW: CA for Blue Ridge',\n",
        " 'CA for Blue Ridge',\n",
        " 'FW: ENA/Blue Dog: Execution Sets',\n",
        " 'ENA/Blue Dog: Execution Sets',\n",
        " 'Blue Dog',\n",
        " 'Blue Dog closing list',\n",
        " 'Re: Golf at Doral Blue Course - January 3rd',\n",
        " 'Golf at Doral Blue Course - January 3rd',\n",
        " 'Re: Golf at Doral Blue Course - January 3rd',\n",
        " 'Re: Your Blue Note',\n",
        " 'Re: Your Blue Note',\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " \"Re: Lichtenstein's Blue Note\",\n",
        " 'Re: Agreement - Updated in Bold (Red).',\n",
        " 'Re: Agreement - Updated in Bold (Red).',\n",
        " 'FW: Red Rock filing',\n",
        " 'Red Rock filing',\n",
        " 'Red Rock Meeting',\n",
        " 'Transwestern Red Rock Expansion',\n",
        " 'FW: Transwestern Red Rock Expansion',\n",
        " 'Transwestern Red Rock Expansion',\n",
        " 'RE: Transwestern Red Rock Expansion',\n",
        " '=09FW: Transwestern Red Rock Expansion',\n",
        " '=09Transwestern Red Rock Expansion',\n",
        " 'Transwestern Red Rock Expansion- FERC Extension Letter',\n",
        " 'RE: Transwestern Red Rock Expansion- FERC Extension Letter',\n",
        " 'RE: Transwestern Red Rock Expansion- FERC Extension Letter',\n",
        " 'Transwestern Red Rock Expansion- FERC Extension Letter',\n",
        " 'Transwestern Red Rock Expansion; Extension Request',\n",
        " 'Re: Blue Range Resource Corporation',\n",
        " 'blue book value',\n",
        " 'Black Marlin / Blue Dolphin',\n",
        " 'Black Marlin / Blue Dolphin',\n",
        " 'Re: Blue Grass Synfuel LLC, et al, v. ENA',\n",
        " 'RE: Blue Ribbon Panel',\n",
        " 'Blue Ribbon Panel',\n",
        " 'FW: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'RE: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'FW: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'FW: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'FW: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'FW: One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'One Fish, Two Fish, Yellow Fish, Enron',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins (CORRECTION)',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins (CORRECTION)',\n",
        " 'RE: TW Capacity Affected for Red Rock Tie-ins',\n",
        " 'TW Capacity Affected for Red Rock Tie-ins=20',\n",
        " 'Re: Red Cedar Receipts',\n",
        " 'Red Cedar Receipts',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins',\n",
        " '=09RE: TW Capacity Affected for Red Rock Tie-ins',\n",
        " '=09RE: TW Capacity Affected for Red Rock Tie-ins',\n",
        " 'TW Capacity Affected for Red Rock Tie-ins=20',\n",
        " 'Re: Red Cedar Contract',\n",
        " 'Red Cedar Contract',\n",
        " 'TW/Red Cedar deal',\n",
        " 'more Red Cedar',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar update',\n",
        " 'New red cedar contract',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar deal',\n",
        " 'Red Cedar letter',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar Letter',\n",
        " 'Red Cedar Letter',\n",
        " 'Red Cedar',\n",
        " 'Red Cedar agency agreement',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Re: Red Meat',\n",
        " 'Red Meat',\n",
        " 'Southern Ute Indian Tribe d/b/a Red Willow Production Company',\n",
        " 'Bank of America/ERMS executed master blue file',\n",
        " 'Bank of America/ERMS executed master blue file',\n",
        " 'SmartPortfolio.Com Update: Markets Soar on Blue Chip and Tech Rally',\n",
        " 'SmartPortfolio.Com Update: Markets Soar on Blue Chip and Tech Rall=',\n",
        " 'need a red file',\n",
        " 'Re: Jan/Red Jan nat gas spreads',\n",
        " 'Re: Jan/Red Jan nat gas spreads',\n",
        " 'Blue Sox',\n",
        " 'RE: Blue Sox',\n",
        " 'Blue Sox',\n",
        " 'Here are my changes to GTC bullet letter revision.doc (in blue I',\n",
        " 'Gas at Blue Diamond',\n",
        " 'FW: The Red, White and Blue',\n",
        " 'FW: The Red, White and Blue',\n",
        " 'The Red, White and  Blue',\n",
        " 'FW: Red Rock filing',\n",
        " 'FW: Red Rock filing',\n",
        " 'Red Rock filing',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins (CORRECTION)',\n",
        " 'RE: TW Capacity Affected for Red Rock Tie-ins (CORRECTION)',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins (CORRECTION)',\n",
        " 'RE: TW Capacity Affected for Red Rock Tie-ins',\n",
        " 'TW Capacity Affected for Red Rock Tie-ins=20',\n",
        " 'FW: Air Permit Delay, Red Rock Expansion',\n",
        " 'FW: Air Permit Delay, Red Rock Expansion',\n",
        " 'Air Permit Delay, Red Rock Expansion',\n",
        " 'Letter of Credit $ 5,500,000 in support of Transwestern Pipeline Red Rock Expansion',\n",
        " 'Letter of Credit $ 5,500,000 in support of Transwestern Pipeline Red Rock Expansion',\n",
        " 'RE: EDG and Red Cedar',\n",
        " 'FW: EDG and Red Cedar',\n",
        " 'RE: EDG and Red Cedar',\n",
        " 'FW: EDG and Red Cedar',\n",
        " 'RE: EDG and Red Cedar',\n",
        " 'FW: EDG and Red Cedar',\n",
        " 'EDG and Red Cedar',\n",
        " 'Red Rock impact if delayed',\n",
        " 'FW: Red Rock Weekly Reports',\n",
        " 'Red Rock Weekly Reports',\n",
        " 'Declined: Red Rock PSR',\n",
        " 'Red Lake Storage',\n",
        " 'RE: ReL Red Pepper Soup Recipe',\n",
        " 'ReL Red Pepper Soup Recipe',\n",
        " 'RE: Red Soup Recipe',\n",
        " 'Red Soup Recipe',\n",
        " 'FW: TW Capacity Affected for Red Rock Tie-ins',\n",
        " '=09TW Capacity Affected for Red Rock Tie-ins=20',\n",
        " 'FW: Red-Neck Horseshoes',\n",
        " 'FW: Red-Neck Horseshoes']"
       ]
      }
     ],
     "prompt_number": 175
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Capturing what matches\n",
      "\n",
      "The `re.search()` function allows us to check to see *whether or not* a string matches a regular expression. Sometimes we want to find out not just if the string matches, but also to what, exactly, in the string matched. In other words, we want to *capture* whatever it was that matched.\n",
      "\n",
      "The easiest way to do this is with the `re.findall()` function, which takes a regular expression and a string to match it against, and returns a list of all parts of the string that the regular expression matched. Here's an example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import re\n",
      "print re.findall(r\"\\b\\w{5}\\b\", \"alpha beta gamma delta epsilon zeta eta theta\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 154,
       "text": [
        "['alpha', 'gamma', 'delta', 'theta']"
       ]
      }
     ],
     "prompt_number": 154
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The regular expression above, `\\b\\w{5}\\b`, is a regular expression that means \"find me strings of five non-white space characters between word boundaries\"---in other words, find me five-letter words. The `re.findall()` method returns a list of strings---not just telling us whether or not the string matched, but which parts of the string matched.\n",
      "\n",
      "For the following `re.findall()` examples, we'll be operating on the entire file of subject lines as a single string, instead of using a list comprehension for individual subject lines. Here's how to read in the entire file as one string, instead of as a list of strings:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "all_subjects = open(\"enronsubjects.txt\").read()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 159
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Having done that, let's write a regular expression that finds all domain names in the subject lines:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.findall(r\"\\b\\w+\\.(?:com|net|org)\", all_subjects)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 188,
       "text": [
        "['enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Forbes.com',\n",
        " 'Cortlandtwines.com',\n",
        " 'Cortlandtwines.com',\n",
        " 'Match.com',\n",
        " 'Amazon.com',\n",
        " 'Amazon.com',\n",
        " 'Ticketmaster.com',\n",
        " 'Ticketmaster.com',\n",
        " 'Concierge.com',\n",
        " 'Concierge.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'HoustonChronicle.com',\n",
        " 'HoustonChronicle.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'har.com',\n",
        " 'Concierge.com',\n",
        " 'Concierge.com',\n",
        " 'washingtonpost.com',\n",
        " 'washingtonpost.com',\n",
        " 'washingtonpost.com',\n",
        " 'washingtonpost.com',\n",
        " 'ESPN.com',\n",
        " 'ESPN.com',\n",
        " 'ESPN.com',\n",
        " 'enron.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'CommodityLogic.com',\n",
        " 'CommodityLogic.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'MarketWatch.com',\n",
        " 'MarketWatch.com',\n",
        " 'MarketWatch.com',\n",
        " 'INSIDER.com',\n",
        " 'INSIDER.com',\n",
        " 'ArdorNY.com',\n",
        " 'ArdorNY.com',\n",
        " 'ArdorNY.com',\n",
        " 'ArdorNY.com',\n",
        " 'ArdorNY.com',\n",
        " 'yahoo.com',\n",
        " 'governmentguide.com',\n",
        " 'SmartPrice.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Headhunter.net',\n",
        " 'Headhunter.net',\n",
        " 'Headhunter.net',\n",
        " 'Headhunter.net',\n",
        " 'Headhunter.net',\n",
        " 'Headhunter.net',\n",
        " 'enron.com',\n",
        " 'merckmedco.com',\n",
        " 'merckmedco.com',\n",
        " 'turnonthetruth.com',\n",
        " 'DefensiveDriver.com',\n",
        " 'DefensiveDriver.com',\n",
        " 'PrimeShot.com',\n",
        " 'PrimeShot.com',\n",
        " 'PrimeShot.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Clickpaper.com',\n",
        " 'Clickpaper.com',\n",
        " 'Clickpaper.com',\n",
        " 'Clickpaper.com',\n",
        " 'enron.com',\n",
        " 'MichaelMcDermott.com',\n",
        " 'MichaelMcDermott.com',\n",
        " 'MichaelMcDermott.com',\n",
        " 'FUZZY.com',\n",
        " 'lptrixie.com',\n",
        " 'lptrixie.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'CapacityCenter.com',\n",
        " 'CapacityCenter.com',\n",
        " 'dow.com',\n",
        " 'dow.com',\n",
        " 'Center.com',\n",
        " 'Center.com',\n",
        " 'INO.com',\n",
        " 'INO.com',\n",
        " 'INO.com',\n",
        " 'shockwave.com',\n",
        " 'shockwave.com',\n",
        " 'enron.com',\n",
        " 'StudentMagazine.com',\n",
        " 'myuhc.com',\n",
        " 'myuhc.com',\n",
        " 'southwest.com',\n",
        " 'southwest.com',\n",
        " 'southwest.com',\n",
        " 'Alamo.com',\n",
        " 'Alamo.com',\n",
        " 'Alamo.com',\n",
        " 'Alamo.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'clanpages.com',\n",
        " 'clanpages.com',\n",
        " 'clanpages.com',\n",
        " 'Amazon.com',\n",
        " 'Amazon.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Paper.com',\n",
        " 'enron.com',\n",
        " 'fitnessheaven.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'MarketWatch.com',\n",
        " 'MarketWatch.com',\n",
        " 'MarketWatch.com',\n",
        " 'MarketWatch.com',\n",
        " 'EnronX.org',\n",
        " 'EnronX.org',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'Colonize.com',\n",
        " 'CareerPath.com',\n",
        " 'HoustonStreet.com',\n",
        " 'Braodcast.com',\n",
        " 'RedMeteor.com',\n",
        " 'Broker.com',\n",
        " 'Broker.com',\n",
        " 'Broker.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'ClickPaper.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnergyGateway.com',\n",
        " 'Credit2B.com',\n",
        " 'EnronCredit.com',\n",
        " 'Credit2B.com',\n",
        " 'Credit2B.com',\n",
        " 'Anywhere.com',\n",
        " 'Credit2B.com',\n",
        " 'EnronCredit.com',\n",
        " 'PaperExchange.com',\n",
        " 'PaperExchange.com',\n",
        " 'enron.com',\n",
        " 'marcus.net',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'eCredit.com',\n",
        " 'eCredit.com',\n",
        " 'eCredit.com',\n",
        " 'Chematch.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'boxmind.com',\n",
        " 'CERA.com',\n",
        " 'CERA.com',\n",
        " 'CERA.com',\n",
        " 'sharperimage.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'educationplanet.com',\n",
        " 'educationplanet.com',\n",
        " 'mathmistakes.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'ft.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'PowerMarketers.com',\n",
        " 'PowerMarketers.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'FT.com',\n",
        " 'FT.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'homestead.com',\n",
        " 'FT.com',\n",
        " 'FT.com',\n",
        " 'FT.com',\n",
        " 'FT.com',\n",
        " 'insiderSCORES.com',\n",
        " 'insiderSCORES.com',\n",
        " 'insiderSCORES.com',\n",
        " 'insiderSCORES.com',\n",
        " 'Insiderscores.com',\n",
        " 'Insiderscores.com',\n",
        " 'marshweb.com',\n",
        " 'marshweb.com',\n",
        " 'Credit.com',\n",
        " 'Credit.com',\n",
        " 'Credit.com',\n",
        " 'Credit.com',\n",
        " 'Credit.com',\n",
        " 'FT.com',\n",
        " 'FT.com',\n",
        " 'Powermarketers.com',\n",
        " 'Powermarketers.com',\n",
        " 'Enroncredit.com',\n",
        " 'Enroncredit.com',\n",
        " 'Enroncredit.com',\n",
        " 'Enroncredit.com',\n",
        " 'Enroncredit.com',\n",
        " 'enerfax.com',\n",
        " 'libertyforelian.org',\n",
        " 'enerfax.com',\n",
        " 'enerfax.com',\n",
        " 'powermarketers.com',\n",
        " 'Ynot.com',\n",
        " 'Ynot.com',\n",
        " 'bluemountain.com',\n",
        " 'Amazon.com',\n",
        " 'FinMath.com',\n",
        " 'Amazon.com',\n",
        " 'FinMath.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'thewoodlands.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Street.com',\n",
        " 'Street.com',\n",
        " 'nexant.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'econlib.org',\n",
        " 'latimes.com',\n",
        " 'aimnet.com',\n",
        " 'blades.com',\n",
        " 'siliconvalley.com',\n",
        " 'reactionsnet.com',\n",
        " 'reactionsnet.com',\n",
        " 'go.com',\n",
        " 'reactionsnet.com',\n",
        " 'reactionsnet.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'reactionsnet.com',\n",
        " 'reactionsnet.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Expedia.com',\n",
        " 'Expedia.com',\n",
        " 'Travelocity.com',\n",
        " 'Travelocity.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'netcoonline.com',\n",
        " 'netcoonline.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'ubsenergy.com',\n",
        " 'ubswenergy.com',\n",
        " 'Fool.com',\n",
        " 'yahoo.com',\n",
        " 'citienergy.com',\n",
        " 'CVS.com',\n",
        " 'CVS.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'Compaq.com',\n",
        " 'Compaq.com',\n",
        " 'Compaq.com',\n",
        " 'enron.com',\n",
        " 'ESPN.com',\n",
        " 'sekurity.com',\n",
        " 'sekurity.com',\n",
        " 'enron.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'taxclaity.com',\n",
        " 'enron.com',\n",
        " 'BIGWORDS.com',\n",
        " 'al.com',\n",
        " 'al.com',\n",
        " 'clanpages.com',\n",
        " 'clanpages.com',\n",
        " 'clanpages.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'Match.com',\n",
        " 'iWon.com',\n",
        " 'iWon.com',\n",
        " 'Individual.com',\n",
        " 'Individual.com',\n",
        " 'Individual.com',\n",
        " 'Edmunds.com',\n",
        " 'Quicken.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'SurveySavvy.com',\n",
        " 'Clickpaper.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'Nodocero.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'ClickPaper.com',\n",
        " 'EnergyPrism.com',\n",
        " 'EnergyPrism.com',\n",
        " 'EnergyPrism.com',\n",
        " 'EnergyPrism.com',\n",
        " 'InfrastructureWorld.com',\n",
        " 'InfrastructureWorld.com',\n",
        " 'InfrastructureWorld.com',\n",
        " 'edftrading.com',\n",
        " 'edftrading.com',\n",
        " 'edftrading.com',\n",
        " 'edftrading.com',\n",
        " 'enron.com',\n",
        " 'WeatherMarkets.com',\n",
        " 'WeatherMarkets.com',\n",
        " 'Amazon.com',\n",
        " 'StoneAge.com',\n",
        " 'StoneAge.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'hubserve.com',\n",
        " 'hubserve.com',\n",
        " 'AGA.org',\n",
        " 'AGA.org',\n",
        " 'u2.com',\n",
        " 'U2.com',\n",
        " 'U2.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'ABCNEWS.com',\n",
        " 'FitRx.com',\n",
        " 'FitRx.com',\n",
        " 'FitRx.com',\n",
        " 'FitRx.com',\n",
        " 'Dictionary.com',\n",
        " 'Dictionary.com',\n",
        " '1400smith.com',\n",
        " '1400smith.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'Omaha.com',\n",
        " 'Omaha.com',\n",
        " 'Omaha.com',\n",
        " 'Omaha.com',\n",
        " 'Omaha.com',\n",
        " 'Omaha.com',\n",
        " 'MyFamily.com',\n",
        " 'MyFamily.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Quote.com',\n",
        " 'Quote.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Grassy.com',\n",
        " 'Grassy.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'SmartMoney.com',\n",
        " 'SmartMoney.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'TheStreet.com',\n",
        " 'TheStreet.com',\n",
        " 'hannaandersson.com',\n",
        " 'hannaandersson.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'akingump.com',\n",
        " 'akingump.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'brcepat.com',\n",
        " 'brcepat.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'southwest.com',\n",
        " 'southwest.com',\n",
        " 'southwest.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'Rediff.com',\n",
        " 'Rediff.com',\n",
        " 'Rediff.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'ScottPaul.com',\n",
        " 'ScottPaul.com',\n",
        " 'EnronCredit.com',\n",
        " 'ScottPaul.com',\n",
        " 'ScottPaul.com',\n",
        " 'SpeakOut.com',\n",
        " 'ScottPaul.com',\n",
        " 'EnronCredit.com',\n",
        " 'Nice.com',\n",
        " 'Nice.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'WSJ.com',\n",
        " 'RealMoney.com',\n",
        " 'RealMoney.com',\n",
        " 'Abestkitchen.com',\n",
        " 'Abestkitchen.com',\n",
        " 'ZanyBrainy.com',\n",
        " 'ZanyBrainy.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'alan.com',\n",
        " 'enron.com',\n",
        " 'alan.com',\n",
        " 'enron.com',\n",
        " 'Markets.com',\n",
        " 'Markets.com',\n",
        " 'BadMojo09092hotmail.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Dictionary.com',\n",
        " 'enron.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'CareerPath.com',\n",
        " 'Paper.com',\n",
        " 'merckmedco.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnergyGateway.com',\n",
        " 'Amazon.com',\n",
        " 'Amazon.com',\n",
        " 'Amazon.com',\n",
        " 'HoustonChronicle.com',\n",
        " 'HoustonChronicle.com',\n",
        " 'HoustonChronicle.com',\n",
        " 'HoustonStreet.com',\n",
        " 'HoustonStreet.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'Agency.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'risk.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronCredit.com',\n",
        " 'EnronOnline.com',\n",
        " 'EnronOnline.com',\n",
        " 'EnronOnline.com',\n",
        " 'EnronOnline.com',\n",
        " 'EnronOnline.com',\n",
        " 'enron.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'NYTimes.com',\n",
        " 'Travelocity.com',\n",
        " 'Travelocity.com',\n",
        " 'Industrialinfo.com',\n",
        " 'Amazon.com',\n",
        " 'Amazon.com',\n",
        " 'quote.com',\n",
        " 'quote.com',\n",
        " 'Expedia.com',\n",
        " 'Expedia.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'ubid.com',\n",
        " 'Fingerhut.com',\n",
        " 'Fingerhut.com',\n",
        " 'Blair.com',\n",
        " 'Blair.com',\n",
        " 'continental.com',\n",
        " 'continental.com',\n",
        " 'GOPUSA.com',\n",
        " 'GOPUSA.com',\n",
        " 'GOPUSA.com',\n",
        " 'autobytel.com',\n",
        " 'autobytel.com',\n",
        " 'RedMeteor.com',\n",
        " 'RedMeteor.com',\n",
        " 'RedMeteor.com',\n",
        " 'EnronCredit.com',\n",
        " 'Bid4me.com',\n",
        " 'Bid4me.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'hotmail.com',\n",
        " 'hotmail.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'UBSWenergy.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'flashyourrack.com',\n",
        " 'Quicken.com',\n",
        " 'Quicken.com',\n",
        " 'Quicken.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'enron.com',\n",
        " 'WeatherMarkets.com',\n",
        " 'WeatherMarkets.com',\n",
        " 'freeyellow.com',\n",
        " 'freeyellow.com']"
       ]
      }
     ],
     "prompt_number": 188
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Every time the string `New York` is found, along with the word that comes directly afterward:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.findall(r\"New York \\b\\w+\\b\", all_subjects)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 161,
       "text": [
        "['New York Details',\n",
        " 'New York Details',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York Times',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York on',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York Times',\n",
        " 'New York City',\n",
        " 'New York City',\n",
        " 'New York City',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Power',\n",
        " 'New York Mercantile',\n",
        " 'New York Mercantile',\n",
        " 'New York Branch',\n",
        " 'New York City',\n",
        " 'New York Energy',\n",
        " 'New York Energy',\n",
        " 'New York Energy',\n",
        " 'New York Energy',\n",
        " 'New York Energy',\n",
        " 'New York sites',\n",
        " 'New York sites',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York Hotel',\n",
        " 'New York City',\n",
        " 'New York City',\n",
        " 'New York City',\n",
        " 'New York City',\n",
        " 'New York voice',\n",
        " 'New York State',\n",
        " 'New York State',\n",
        " 'New York State',\n",
        " 'New York State',\n",
        " 'New York State',\n",
        " 'New York State',\n",
        " 'New York Inc',\n",
        " 'New York Office',\n",
        " 'New York Office',\n",
        " 'New York regulatory',\n",
        " 'New York regulatory',\n",
        " 'New York regulatory',\n",
        " 'New York regulatory',\n",
        " 'New York Bar',\n",
        " 'New York Bar']"
       ]
      }
     ],
     "prompt_number": 161
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And just to bring things full-circle, everything that looks like a zip code, sorted:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "sorted(re.findall(r\"\\b\\d{5}\\b\", all_subjects))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 163,
       "text": [
        "['00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00003',\n",
        " '00010',\n",
        " '00010',\n",
        " '00458',\n",
        " '01003',\n",
        " '02177',\n",
        " '06716',\n",
        " '06736',\n",
        " '06736',\n",
        " '06752',\n",
        " '06752',\n",
        " '06752',\n",
        " '06752',\n",
        " '06752',\n",
        " '06980',\n",
        " '06980',\n",
        " '10000',\n",
        " '10000',\n",
        " '11111',\n",
        " '11111',\n",
        " '11111',\n",
        " '11111',\n",
        " '11111',\n",
        " '11385',\n",
        " '11385',\n",
        " '11385',\n",
        " '11385',\n",
        " '11385',\n",
        " '11781',\n",
        " '11781',\n",
        " '12357',\n",
        " '12357',\n",
        " '12578',\n",
        " '12590',\n",
        " '12619',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '14790',\n",
        " '15444',\n",
        " '15444',\n",
        " '15444',\n",
        " '15444',\n",
        " '15444',\n",
        " '15444',\n",
        " '15692',\n",
        " '15692',\n",
        " '15692',\n",
        " '15692',\n",
        " '15692',\n",
        " '15692',\n",
        " '15956',\n",
        " '16995',\n",
        " '19785',\n",
        " '19818',\n",
        " '20001',\n",
        " '20001',\n",
        " '20267',\n",
        " '20267',\n",
        " '20267',\n",
        " '20267',\n",
        " '20721',\n",
        " '20721',\n",
        " '20721',\n",
        " '20721',\n",
        " '20721',\n",
        " '20721',\n",
        " '20721',\n",
        " '20740',\n",
        " '20740',\n",
        " '20740',\n",
        " '20740',\n",
        " '20740',\n",
        " '20740',\n",
        " '20740',\n",
        " '20747',\n",
        " '20747',\n",
        " '20747',\n",
        " '20748',\n",
        " '20748',\n",
        " '20748',\n",
        " '21349',\n",
        " '21349',\n",
        " '21865',\n",
        " '22027',\n",
        " '22027',\n",
        " '22069',\n",
        " '22069',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '22585',\n",
        " '23231',\n",
        " '24194',\n",
        " '24194',\n",
        " '24468',\n",
        " '24468',\n",
        " '24468',\n",
        " '24468',\n",
        " '24468',\n",
        " '24468',\n",
        " '24468',\n",
        " '24690',\n",
        " '24690',\n",
        " '24690',\n",
        " '24690',\n",
        " '24924',\n",
        " '24924',\n",
        " '24924',\n",
        " '24924',\n",
        " '25374',\n",
        " '25374',\n",
        " '25374',\n",
        " '25374',\n",
        " '25407',\n",
        " '25672',\n",
        " '25672',\n",
        " '25672',\n",
        " '25672',\n",
        " '25672',\n",
        " '25672',\n",
        " '25841',\n",
        " '25841',\n",
        " '25841',\n",
        " '25841',\n",
        " '26486',\n",
        " '26490',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26511',\n",
        " '26532',\n",
        " '26606',\n",
        " '26635',\n",
        " '26819',\n",
        " '26819',\n",
        " '26819',\n",
        " '26819',\n",
        " '26862',\n",
        " '26862',\n",
        " '27190',\n",
        " '27190',\n",
        " '27190',\n",
        " '27190',\n",
        " '27190',\n",
        " '27190',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27239',\n",
        " '27252',\n",
        " '27252',\n",
        " '27252',\n",
        " '27253',\n",
        " '27253',\n",
        " '27253',\n",
        " '27253',\n",
        " '27253',\n",
        " '27253',\n",
        " '27253',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27291',\n",
        " '27496',\n",
        " '27496',\n",
        " '27579',\n",
        " '27579',\n",
        " '27579',\n",
        " '27579',\n",
        " '27600',\n",
        " '27600',\n",
        " '27606',\n",
        " '27606',\n",
        " '27606',\n",
        " '27606',\n",
        " '27606',\n",
        " '27641',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '29667',\n",
        " '30643',\n",
        " '30643',\n",
        " '30643',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '34342',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35830',\n",
        " '35842',\n",
        " '35842',\n",
        " '35842',\n",
        " '35842',\n",
        " '35874',\n",
        " '35874',\n",
        " '35874',\n",
        " '37877',\n",
        " '38616',\n",
        " '38616',\n",
        " '38927',\n",
        " '38927',\n",
        " '38927',\n",
        " '38927',\n",
        " '39474',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39764',\n",
        " '39833',\n",
        " '39833',\n",
        " '39833',\n",
        " '39833',\n",
        " '39833',\n",
        " '39833',\n",
        " '39833',\n",
        " '40387',\n",
        " '40387',\n",
        " '41038',\n",
        " '41598',\n",
        " '42029',\n",
        " '42066',\n",
        " '42150',\n",
        " '42306',\n",
        " '42342',\n",
        " '42343',\n",
        " '42354',\n",
        " '42361',\n",
        " '42363',\n",
        " '42371',\n",
        " '42375',\n",
        " '42661',\n",
        " '42750',\n",
        " '42764',\n",
        " '42789',\n",
        " '42789',\n",
        " '42896',\n",
        " '42913',\n",
        " '42933',\n",
        " '42934',\n",
        " '42975',\n",
        " '42976',\n",
        " '47477',\n",
        " '47477',\n",
        " '50250',\n",
        " '50569',\n",
        " '51407',\n",
        " '51407',\n",
        " '51407',\n",
        " '51407',\n",
        " '51407',\n",
        " '52330',\n",
        " '52330',\n",
        " '53091',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '55358',\n",
        " '56565',\n",
        " '58900',\n",
        " '58900',\n",
        " '58911',\n",
        " '62039',\n",
        " '62039',\n",
        " '62164',\n",
        " '62164',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64231',\n",
        " '64937',\n",
        " '64937',\n",
        " '64937',\n",
        " '64937',\n",
        " '65066',\n",
        " '65066',\n",
        " '65066',\n",
        " '65066',\n",
        " '65066',\n",
        " '65066',\n",
        " '65066',\n",
        " '65187',\n",
        " '65403',\n",
        " '66394',\n",
        " '66547',\n",
        " '67133',\n",
        " '67207',\n",
        " '70197',\n",
        " '70197',\n",
        " '70996',\n",
        " '70996',\n",
        " '77257',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '77349',\n",
        " '78032',\n",
        " '78032',\n",
        " '78033',\n",
        " '78033',\n",
        " '78158',\n",
        " '78158',\n",
        " '78728',\n",
        " '78728',\n",
        " '78728',\n",
        " '78728',\n",
        " '78728',\n",
        " '78728',\n",
        " '80110',\n",
        " '83017',\n",
        " '83017',\n",
        " '83017',\n",
        " '83829',\n",
        " '87541',\n",
        " '90593',\n",
        " '90593',\n",
        " '90593',\n",
        " '92886',\n",
        " '92886',\n",
        " '93394',\n",
        " '93481',\n",
        " '93481',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93836',\n",
        " '93871',\n",
        " '94074',\n",
        " '96724',\n",
        " '96724',\n",
        " '96731',\n",
        " '96731']"
       ]
      }
     ],
     "prompt_number": 163
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Full example: finding the dollar value of the Enron e-mail subject corpus\n",
      "\n",
      "Here's an example that combines our regular expression prowess with our ability to do smaller manipulations on strings. We want to find all dollar amounts in the subject lines, and then figure out what their sum is.\n",
      "\n",
      "To understand what we're working with, let's start by writing a list comprehension that finds strings that just have the dollar sign (`$`) in them:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[line for line in subjects if re.search(r\"\\$\", line)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 164,
       "text": [
        "['Re: APEA - $228,204 hit',\n",
        " 'Re: APEA - $228,204 hit',\n",
        " 'DJ Cal-ISO Pays $10M To Avoid Rolling Blackouts Wed -Sources, DJ',\n",
        " 'DJ Cal-ISO Pays $10M To Avoid Rolling Blackouts Wed -Sources, DJ',\n",
        " 'DJ Cal-ISO Pays $10M To Avoid Rolling Blackouts Wed -Sources, DJ',\n",
        " 'DJ Cal-ISO Pays $10M To Avoid Rolling Blackouts Wed -Sources, DJ',\n",
        " 'Goldman Comment re: Enron issued this morning - Revised Price Target of $68/share',\n",
        " 'RE: Goldman Sachs $2.19 Natural GAs',\n",
        " 'Goldman Sachs $2.19 Natural GAs',\n",
        " 'RE: $25 million',\n",
        " '$25 million',\n",
        " 'RE: $25 million loan from EDf',\n",
        " '$25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " '$25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " 'RE: $25 million loan from EDf',\n",
        " '$25 million loan from EDf',\n",
        " 'A$M and its \"second tier\" status',\n",
        " 'A$M and its \"second tier\" status',\n",
        " 'A$M and its \"second tier\" status',\n",
        " 'UT/a$m business school and engineering school comparisons',\n",
        " 'Re: $',\n",
        " '$',\n",
        " 'Re: $',\n",
        " '$',\n",
        " '$$$$',\n",
        " 'FFL $$',\n",
        " 'RE: shipper imbal $$ collected',\n",
        " 'shipper imbal $$ collected',\n",
        " \"Oneok's Strangers Gas Payment $820,000\",\n",
        " \"Oneok's Strangers Gas Payment $820,000\",\n",
        " 'Another $40 Million?',\n",
        " 'FW: Entergy and FPL Group Agree to a $27 Billion Merger Of Equals',\n",
        " 'FW: Entergy and FPL Group Agree to a $27 Billion Merger Of Equals',\n",
        " 'Over $50 -- You made it happen!',\n",
        " 'Over $50 -- You made it happen!',\n",
        " 'FW: Co 0530 CINY 40781075  $5,356.46  FX Funding',\n",
        " 'Co 0530 CINY 40781075  $5,356.46  FX Funding',\n",
        " 'FW: Outstanding Young Alumni Travel Value to Amsterdam from $895',\n",
        " 'Outstanding Young Alumni Travel Value to Amsterdam from $895',\n",
        " 'RE: Modesto 7 MW COB deal @$19.3.',\n",
        " 'RE: Modesto 7 MW COB deal @$19.3.',\n",
        " 'Modesto 7 MW COB deal @$19.3.',\n",
        " 'Modesto 7 MW COB deal @$19.3.',\n",
        " 'RE: -$870K prior month adjustments',\n",
        " '-$870K prior month adjustments',\n",
        " 'RE: -$141,000 P&L hit on 8/13/01',\n",
        " '-$141,000 P&L hit on 8/13/01',\n",
        " '$$$',\n",
        " 'Re: DWR Stranded costs: $21 billion',\n",
        " 'CAISO cuts refund estimate to $6.1B from $8.9B',\n",
        " \"State's Power Purchases Costlier Than Projected Tab is $6 million a\",\n",
        " 'Fwd: Edison gets more time; Calif. may sell $14 bln bonds',\n",
        " 'Edison gets more time; Calif. may sell $14 bln bonds',\n",
        " 'Re: IDEA RE ISSUE OF UTILS IN CALIF WANTING $100 PRICE CAP',\n",
        " 'Back to $250 Cap in California',\n",
        " 'Energy Secretary Announces $350MM to Upgrade Path 15',\n",
        " 'RE: $.01 surcharge as \"tax\"',\n",
        " 'FW: $.01 surcharge as \"tax\"',\n",
        " 'FW: $.01 surcharge as \"tax\"',\n",
        " '$.01 surcharge as \"tax\"',\n",
        " \"California's $12.5 Bln Bond Sale May Be Salvaged, Official Says;\",\n",
        " \"RE: California's $12.5 Bln Bond Sale May Be Salvaged, Official\",\n",
        " \"RE: California's $12.5 Bln Bond Sale May Be Salvaged, Official Says; DWR Contract Renegotiation Is Key\",\n",
        " \"California's $12.5 Bln Bond Sale May Be Salvaged, Official Says; DWR Contract Renegotiation Is Key\",\n",
        " 'Re: Royal Bank of Canada - Wire ($2,529,352.58)',\n",
        " 'Free $10 Three Team Parlay',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'Blue Girl - $1.2MM option expires today - need to know whether to',\n",
        " 'FW: Economic Times article: FIs may take over Enron for $700-800m',\n",
        " 'FW: Economic Times article: FIs may take over Enron for $700-800m',\n",
        " 'FW: Economic Times article: FIs may take over Enron for $700-800m',\n",
        " 'Red Rock Delay $$ Impact',\n",
        " 'HandsFree Kits - $2',\n",
        " 'HandsFree Kits - $2',\n",
        " 'Re: The $10 you owe me',\n",
        " 'The $10 you owe me',\n",
        " 'RE: Enron files for Chapter 11 owing US$13B',\n",
        " 'Enron files for Chapter 11 owing US$13B',\n",
        " 'RE: $ allocation',\n",
        " '$ allocation',\n",
        " 'Re: Last chance: Save $100 on a future airline ticket',\n",
        " 'Re: ECS and the $500k reduction',\n",
        " 'Re: ECS and the $500k reduction',\n",
        " 'Re: ECS and the $500k reduction',\n",
        " 'Re: ECS and the $500k reduction',\n",
        " 'ECS and the $500k reduction',\n",
        " 'ECS and the $500k reduction',\n",
        " 'ECS and the $500k reduction',\n",
        " 'ECS and the $500k reduction',\n",
        " 'ECS and the $500k reduction',\n",
        " 'FW: Free Shipping & $1,300 in Savings',\n",
        " 'Free Shipping & $1,300 in Savings',\n",
        " 'RE: Free Shipping & $1,300 in Savings',\n",
        " 'RE: Free Shipping & $1,300 in Savings',\n",
        " 'FW: Free Shipping & $1,300 in Savings',\n",
        " 'Free Shipping & $1,300 in Savings',\n",
        " 'RE: Dynegy Is Mulling $2 Billion Investment In Enron in Possible',\n",
        " 'FW: Dynegy Is Mulling $2 Billion Investment In Enron in Possible \\tStep Toward Merger',\n",
        " 'FW: Dynegy Is Mulling $2 Billion Investment In Enron in Possible Step Toward Merger',\n",
        " 'Dynegy Is Mulling $2 Billion Investment In Enron in Possible Step Toward Merger',\n",
        " 'Peoples Gas --> $5,000 Invoice for Summer-Winter Exchange 6-1-00 to',\n",
        " 'Peoples Gas --> $5,000 Invoice for Summer-Winter Exchange 6-1-00 to',\n",
        " 'Peoples Gas --> $5,000 Invoice for Summer-Winter Exchange 6-1-00 to',\n",
        " 'Peoples Gas --> $5,000 Invoice for Summer-Winter Exchange 6-1-00 to',\n",
        " 'Re: short fall $971,443.11 for Wis Elect Power',\n",
        " 'Re: short fall $971,443.11 for Wis Elect Power',\n",
        " 'Re: short fall $971,443.11 for Wis Elect Power',\n",
        " 'Re: short fall $971,443.11 for Wis Elect Power',\n",
        " 'Re: short fall $971,443.11 for Wis Elect Power',\n",
        " 'short fall $971,443.11 for Wis Elect Power',\n",
        " 'RE: Q&A for NNG/TW Supported $1Billion Line of Credit',\n",
        " 'Q&A for NNG/TW Supported $1Billion Line of Credit',\n",
        " 'FW: Deals from $39 in our Las Vegas store!',\n",
        " '=09Deals from $39 in our Las Vegas store!',\n",
        " 'A trip worth $10,000 could be yours',\n",
        " 'A trip worth $10,000 could be yours',\n",
        " '142,000,000 Email Addresses for ONLY $149!!!!',\n",
        " \"Lou's $50,000\",\n",
        " \"Lou's $50,000\",\n",
        " \"Lou's $50,000\",\n",
        " 'Summary of $ at Risk for Customs',\n",
        " 'Summary of $ at Risk for Customs',\n",
        " 'Summary of $ at Risk for Customs',\n",
        " \"Calling All Investors: The New Power Company's IPO Priced at $21\",\n",
        " \"Calling All Investors: The New Power Company's IPO Priced at $21 P=\",\n",
        " 'Fenosa and Enron to Invest $550 Million in Dominican Republic',\n",
        " \"Enron Brazil To Invest $455 Million In Gas Distribution '01-'04\",\n",
        " 'RE: $5 million for 90 days?- how quaint!',\n",
        " 'FW: $5 million for 90 days?- how quaint!',\n",
        " '$5 million for 90 days?- how quaint!',\n",
        " 'RE: Wind $7MM',\n",
        " 'RE: Wind $7MM',\n",
        " 'RE: Wind $7MM',\n",
        " 'Wind $7MM',\n",
        " 'RE: Wind $7MM',\n",
        " 'Wind $7MM',\n",
        " 'Re: Counting the Cal ISO Votes for a $100 Price Cap',\n",
        " 'RE: C$ swap between EIM/ENA',\n",
        " 'C$ swap between EIM/ENA',\n",
        " \"Re: Where's My $20\",\n",
        " \"Re: Where's My $20\",\n",
        " \"Re: Where's My $20\",\n",
        " \"Re: Where's My $20\",\n",
        " 'Re: $100',\n",
        " 'Re: $100',\n",
        " 'Re: $100',\n",
        " \"Re: Where's My $20\",\n",
        " \"Re: Where's My $20\",\n",
        " 'RE: Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'Fw: Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'RE: Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'Fw: Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'Eric Schroeder has just sent you $29.75 with PayPal',\n",
        " 'RE: What are you talking about $1600?',\n",
        " 'Re: What are you talking about $1600?',\n",
        " 'RE: What are you talking about $1600?',\n",
        " 'RE: What are you talking about $1600?',\n",
        " '=09Re: What are you talking about $1600?',\n",
        " 'What are you talking about $1600?',\n",
        " 'What are you talking about $1600?',\n",
        " 'FW: Enron Seeks $2 Billion Cash Infusion As It Faces an Escalating',\n",
        " 'FW: Enron Seeks $2 Billion Cash Infusion As It Faces an Escalating Fiscal Crisis',\n",
        " 'Enron Seeks $2 Billion Cash Infusion As It Faces an Escalating Fiscal Crisis',\n",
        " 'The new, correct price is $67,776,700',\n",
        " 'Re: Demar request for $2.7 mm to pay out the Skandinavian now',\n",
        " 'Re: Demar request for $2.7 mm to pay out the Skandinavian now',\n",
        " 'RE: Transactions exceeding $100mil',\n",
        " 'Our benefits are about $50 per month higher with UBS',\n",
        " 'RE: $9.6MM EOL Gas Daily Issue',\n",
        " '$9.6MM EOL Gas Daily Issue',\n",
        " 'FW: NEAL - ITIN ONLY/$212.50',\n",
        " 'FW: NEAL - ITIN ONLY/$212.50',\n",
        " 'NEAL - ITIN ONLY/$212.50',\n",
        " 'FW: NEAL - ITIN ONLY/$212.50',\n",
        " 'FW: NEAL - ITIN ONLY/$212.50',\n",
        " 'NEAL - ITIN ONLY/$212.50',\n",
        " 'FW: Duke $',\n",
        " 'Duke $',\n",
        " 'RE: Duke $',\n",
        " 'FW: Duke $',\n",
        " 'FW: Duke $',\n",
        " 'Duke $',\n",
        " '$$$$',\n",
        " '$$$$',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'FW: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'FW: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'FW: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'FW: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'RE: Wire Detail for 10/25/01 wire for  $195,209.95',\n",
        " 'FW: DYN($42/sh)/ENE($7/sh) Merger At Risk. - Simmons and Company',\n",
        " 'FW: DYN($42/sh)/ENE($7/sh) Merger At Risk. - Simmons and Company latest thoughts',\n",
        " 'FW: DYN($42/sh)/ENE($7/sh) Merger At Risk. - Simmons and Company latest thoughts',\n",
        " \"FW: Re-Allocaton of $'s\",\n",
        " \"RE: Re-Allocaton of $'s\",\n",
        " \"Re-Allocaton of $'s\",\n",
        " \"Re-Allocaton of $'s\",\n",
        " 'RE: Wind $7MM',\n",
        " 'FW: Wind $7MM',\n",
        " 'Wind $7MM',\n",
        " 'RE: $9.92????????????',\n",
        " '$9.92????????????',\n",
        " 'RE: Below $10',\n",
        " 'Below $10',\n",
        " 'FW: Comments on the Status of ENE ($16/sh).',\n",
        " 'FW: Comments on the Status of ENE ($16/sh).',\n",
        " 'FW: Comments on the Status of ENE ($16/sh).',\n",
        " 'Breaking News : Williams Ordered to Pay $8 Million Refund to',\n",
        " 'Breaking News : Williams Ordered to Pay $8 Million Refund to Cal-ISO',\n",
        " 'Coho $500mm lawsuit against Hicks Muse',\n",
        " 'Coho $500mm lawsuit against Hicks Muse',\n",
        " 'Coho $500mm lawsuit against Hicks Muse',\n",
        " 'Re: $$$$',\n",
        " '$$$$',\n",
        " 'Perd $',\n",
        " 'Re: $80 million',\n",
        " 'Re: $80 million',\n",
        " '$80 million',\n",
        " '$80 million',\n",
        " 'Re: $80 million',\n",
        " '$80 million',\n",
        " '$80 million',\n",
        " 'Re: Calif Atty Gen Offers $50M Reward In Pwr Supplier',\n",
        " 'Financial Disclosure of $1.2 Billion Equity Adjustment',\n",
        " 'ENE: Despite Bounce It Appears Cheap; Yet $102 Target Likely a Late',\n",
        " 'ENE: Despite Bounce It Appears Cheap; Yet $102 Target Likely a Late 2002 Event:',\n",
        " 'Is it worth $200?',\n",
        " 'RE: #@$ !!!!!!!!',\n",
        " '$#%:#@$ !!!!!!!!',\n",
        " 'RE: @%$$@!!!',\n",
        " '=09@%$$@!!!',\n",
        " 'Special Offer: Switch to ShareBuilder and Get $50!',\n",
        " 'Amendment to Enron Corp. $25 Million guaranty of Enron Credit Inc.',\n",
        " 'RE: Amendment to Enron Corp. $25 Million guaranty of Enron Credit',\n",
        " 'Goldman Sach $ repo docs',\n",
        " 'Re: Goldman Sach $ repo docs',\n",
        " 'RE: Amendment to Enron Corp. $25 Million guaranty of Enron Credit',\n",
        " 'FW: Goldmans $1.5m',\n",
        " 'Goldmans $1.5m',\n",
        " 'FW: $1.5 Check',\n",
        " '$1.5 Check',\n",
        " 'RE: Goldman Sachs $',\n",
        " 'Goldman Sachs $',\n",
        " 'RE: TODAY ONLY - SAVE UP TO $120 EXTRA ON AIRLINE TICKETS!',\n",
        " 'RE: TODAY ONLY - SAVE UP TO $120 EXTRA ON AIRLINE TICKETS!',\n",
        " 'RE: $.01 surcharge as \"tax\"',\n",
        " 'RE: $.01 surcharge as \"tax\"',\n",
        " 'RE: $.01 surcharge as \"tax\"',\n",
        " 'FW: $.01 surcharge as \"tax\"',\n",
        " 'FW: $.01 surcharge as \"tax\"',\n",
        " '$.01 surcharge as \"tax\"',\n",
        " \"FW: PennFuture's E-Cubed - The $45 Million Rip Off\",\n",
        " \"=09PennFuture's E-Cubed - The $45 Million Rip Off\",\n",
        " 'RE: PaPUC assessment of $147,000 to Enron',\n",
        " 'Re: PaPUC assessment of $147,000 to Enron',\n",
        " 'PaPUC assessment of $147,000 to Enron',\n",
        " \"RE: ASAP!! EES' objections to PaPUC assessment of $147,000\",\n",
        " \"ASAP!! EES' objections to PaPUC assessment of $147,000\",\n",
        " 'RE: Pennsylvania $147,000 EES Assessment',\n",
        " '=09Pennsylvania $147,000 EES Assessment',\n",
        " 'FW: CAEM Study: Gas Dereg Has Saved Consumers $600B',\n",
        " 'CAEM Study: Gas Dereg Has Saved Consumers $600B',\n",
        " 'PaPUC assessment of $147,000 to Enron',\n",
        " \"RE: ASAP!! EES' objections to PaPUC assessment of $147,000\",\n",
        " \"ASAP!! EES' objections to PaPUC assessment of $147,000\",\n",
        " 'FW: Energy Novice to Be Paid $240,000',\n",
        " 'Energy Novice to Be Paid $240,000',\n",
        " 'RE:  $22.8 schedule C for BPA deal',\n",
        " '$22.8 schedule C for BPA deal',\n",
        " '$22.8 schedule C for BPA deal',\n",
        " 'origination $100k to Laird Dyer',\n",
        " 'Cd$ CME letter',\n",
        " 'Cd$ CME letter',\n",
        " '$',\n",
        " 'RE: $',\n",
        " 'RE: $',\n",
        " 'Re: $',\n",
        " 'RE: $',\n",
        " 'GET RICH ON $6.00 !!!',\n",
        " 'RE: Thoughts on the world of energy (OSX $77, XNG $183, XOI 496)',\n",
        " 'FW: Letter of Credit $ 5,500,000 in support of Transwestern',\n",
        " 'Letter of Credit $ 5,500,000 in support of Transwestern Pipeline Red Rock Expansion',\n",
        " 'Letter of Credit $ 5,500,000 in support of Transwestern Pipeline Red Rock Expansion',\n",
        " 'FW: shipper imbal $$ collected',\n",
        " 'shipper imbal $$ collected',\n",
        " 'FW: shipper imbal $$ collected',\n",
        " 'RE: shipper imbal $$ collected',\n",
        " 'shipper imbal $$ collected',\n",
        " 'FW: shipper imbal $$ collected',\n",
        " 'RE: shipper imbal $$ collected',\n",
        " 'shipper imbal $$ collected',\n",
        " \"FW: $$'s allocated to TW\",\n",
        " \"$$'s allocated to TW\",\n",
        " 'RE: email to USG confirming our decision not to require more LOC $',\n",
        " 'email to USG confirming our decision not to require more LOC $',\n",
        " '$',\n",
        " 'Re: Calpine Confirms $4.6B, 10-Yr Calif. Power Sales',\n",
        " 'RE: $2.15 bn Enron Metals Inventory Financings Closed',\n",
        " 'RE: $2.15 bn Enron Metals Inventory Financings Closed',\n",
        " 'FW: Thayer Aerospace Awarded $130 Million Vought Aircraft Contract',\n",
        " 'FW: Thayer Aerospace Awarded $130 Million Vought Aircraft Contract',\n",
        " 'Thayer Aerospace Awarded $130 Million Vought Aircraft Contract to',\n",
        " 're: mid-columbia $1 mm Schedule E difference',\n",
        " '$0.25 scheduling fee.',\n",
        " 'MPC $',\n",
        " 'RE: $10',\n",
        " 'RE: $10',\n",
        " 'RE: $10',\n",
        " '$10',\n",
        " 'RE: $10',\n",
        " '$10',\n",
        " 'FW: *** Gold/TSE GL/$US/CPI/TSE MM/CRB Bloomberg charts ***',\n",
        " 'FW: *** Gold/TSE GL/$US/CPI/TSE MM/CRB Bloomberg charts ***',\n",
        " 'FW: *** Gold/TSE GL/$US/CPI/TSE MM/CRB Bloomberg charts ***',\n",
        " 'FW: Summer Fare Sale From $128 Return!',\n",
        " 'Summer Fare Sale From $128 Return!']"
       ]
      }
     ],
     "prompt_number": 164
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Based on this data, we can guess at the steps we'd need to do in order to figure out these values. We're going to ignore anything that doesn't have \"k\", \"million\" or \"billion\" after it as chump change. So what we need to find is: a dollar sign, followed by any series of numbers (or a period), followed potentially by a space (but sometimes not), followed by a \"k\", \"m\" or \"b\" (which will sometimes start the word \"million\" or \"billion\" but sometimes not... so we won't bother looking).\n",
      "\n",
      "Here's how I would translate that into a regular expression:\n",
      "\n",
      "    \\$[0-9.]+ ?(?:[Kk]|[Mm]|[Bb])\n",
      "    \n",
      "We can use `re.findall()` to capture all instances where we found this regular expression in the text. Here's what that would look like:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.findall(r\"\\$[0-9.]+ ?(?:[Kk]|[Mm]|[Bb])\", all_subjects)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 182,
       "text": [
        "['$10M',\n",
        " '$10M',\n",
        " '$10M',\n",
        " '$10M',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$25 m',\n",
        " '$40 M',\n",
        " '$27 B',\n",
        " '$27 B',\n",
        " '$870K',\n",
        " '$870K',\n",
        " '$21 b',\n",
        " '$6.1B',\n",
        " '$8.9B',\n",
        " '$6 m',\n",
        " '$14 b',\n",
        " '$14 b',\n",
        " '$350M',\n",
        " '$12.5 B',\n",
        " '$12.5 B',\n",
        " '$12.5 B',\n",
        " '$12.5 B',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$1.2M',\n",
        " '$13B',\n",
        " '$13B',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$500k',\n",
        " '$2 B',\n",
        " '$2 B',\n",
        " '$2 B',\n",
        " '$2 B',\n",
        " '$1B',\n",
        " '$1B',\n",
        " '$550 M',\n",
        " '$455 M',\n",
        " '$5 m',\n",
        " '$5 m',\n",
        " '$5 m',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$2 B',\n",
        " '$2 B',\n",
        " '$2 B',\n",
        " '$2.7 m',\n",
        " '$2.7 m',\n",
        " '$100m',\n",
        " '$9.6M',\n",
        " '$9.6M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$7M',\n",
        " '$8 M',\n",
        " '$8 M',\n",
        " '$500m',\n",
        " '$500m',\n",
        " '$500m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$80 m',\n",
        " '$50M',\n",
        " '$1.2 B',\n",
        " '$25 M',\n",
        " '$25 M',\n",
        " '$25 M',\n",
        " '$1.5m',\n",
        " '$1.5m',\n",
        " '$45 M',\n",
        " '$45 M',\n",
        " '$600B',\n",
        " '$600B',\n",
        " '$100k',\n",
        " '$4.6B',\n",
        " '$2.15 b',\n",
        " '$2.15 b',\n",
        " '$130 M',\n",
        " '$130 M',\n",
        " '$130 M',\n",
        " '$1 m']"
       ]
      }
     ],
     "prompt_number": 182
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If we want to actually make a sum, though, we're going to need to do a little massaging."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "total_value = 0\n",
      "dollar_amounts = re.findall(r\"\\$\\d+ ?(?:[Kk]|[Mm]|[Bb])\", all_subjects)\n",
      "for amount in dollar_amounts:\n",
      "    # the last character will be 'k', 'm', or 'b'; \"normalize\" by making lowercase.\n",
      "    multiplier = amount[-1].lower()\n",
      "    # trim off the beginning $ and ending multiplier value\n",
      "    amount = amount[1:-1]\n",
      "    # remove any remaining whitespace\n",
      "    amount = amount.strip()\n",
      "    # convert to a floating-point number\n",
      "    float_amount = float(amount)\n",
      "    # multiply by an amount, based on what the last character was\n",
      "    if multiplier == 'k':\n",
      "        float_amount = float_amount * 1000\n",
      "    elif multiplier == 'm':\n",
      "        float_amount = float_amount * 1000000\n",
      "    elif multiplier == 'b':\n",
      "        float_amount = float_amount * 1000000000\n",
      "    # add to total value\n",
      "    total_value = total_value + float_amount\n",
      "\n",
      "print total_value"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1.34965734e+12\n"
       ]
      }
     ],
     "prompt_number": 183
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The number is so big that Python decided to use scientific notation! If we convert to an integer, we get around that problem:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print int(total_value)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1349657340000\n"
       ]
      }
     ],
     "prompt_number": 184
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "That's over one trillion dollars! Nice work, guys."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Conclusion\n",
      "\n",
      "Regular expressions are a great way to take some raw text and find the parts that are of interest to you. Python's string methods and string slicing syntax are a great way to massage and clean up data. You know them both now, which makes you powerful. But as powerful as you are, you have only scratched the surface of your potential! We only scratched the surface of what's possible with regular expressions. Here's some further reading:\n",
      "\n",
      "* [egrep for Linguists](http://stts.se/egrep_for_linguists/egrep_for_linguists.html) explains how to use regular expressions using the command-line tool `egrep` (which I recommend becoming familiar with!)\n",
      "* Once you've mastered the basics, check the official [Python regular expressions HOWTO](https://docs.python.org/2.7/howto/regex.html). The official [Python documentation on regular expressions](https://docs.python.org/2/library/re.html) is a deep dive on the subject.\n"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}