Lecture 03.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:65df4b03fd494ef51e8a5215511c1f69fc5fc52794e6fbca54e58e90d1f0fb31"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Today we'll cover:\n",
      "\n",
      "1. [Running OS (operating system) commands](#OS-interface)\n",
      "2. [Handling command line arguments](#Command-line-arguments)\n",
      "3. [Math](#Math)\n",
      "4. [Pattern matching](#Pattern-matching)\n",
      "5. [Dates and times](#Dates-and-times)\n",
      "6. [Performance measurement](#Performance-measurement)"
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "OS interface"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.getcwd()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "'/Users/tewaria/Development/stats607a-fall2014'"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.chdir('homeworks')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.getcwd()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "'/Users/tewaria/Development/stats607a-fall2014/homeworks'"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.chdir('..') # change directory to parent directory"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.getcwd()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "'/Users/tewaria/Development/stats607a-fall2014'"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[os.getenv(var) for var in ['HOME', 'PATH']] # return the environment variables HOME and PATH"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "['/Users/tewaria',\n",
        " '/Users/tewaria/anaconda/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/local/munki:/usr/texbin']"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can list files in a directory using `listdir`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.listdir('homeworks') # list files in directory homeworks"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "['assignment_one_kmeans.py',\n",
        " 'assignment_one_nbayes.py',\n",
        " 'assignment_one_optimization.py',\n",
        " 'datasets',\n",
        " 'hw1.aux',\n",
        " 'hw1.log',\n",
        " 'hw1.out',\n",
        " 'hw1.pdf',\n",
        " 'hw1.synctex.gz',\n",
        " 'hw1.tex',\n",
        " 'hw1_ver1.0.pdf',\n",
        " 'seeds_dataset.txt',\n",
        " 'stats607-header.tex']"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Command line arguments"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When you run a command on the OS shell, such as `ls -l -a`, the arguments supplied to it (`-l` and `-a` in this case) are called command line arguments.\n",
      "\n",
      "Now let us put the following python commands in a script called `print_cmdline_args.py`:\n",
      "\n",
      "    import sys\n",
      "    \n",
      "    print 'This python\\'s script name is ' + sys.argv[0]\n",
      "    \n",
      "    if len(sys.argv) > 1:\n",
      "        print 'It was called with the following arguments:',\n",
      "        for arg in sys.argv[1:]:\n",
      "            print arg,\n",
      "        print\n",
      "    else:\n",
      "        print 'It was called with no arguments.'"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "os.system('python print_cmdline_args.py')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "0"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Well... That just told us that the command executed successfully. What if we want to see the output?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print os.popen('python print_cmdline_args.py').read()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "This python's script name is print_cmdline_args.py\n",
        "It was called with no arguments.\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In Python 2.7 or later, one can also use the `subprocess.check_output()` function from the `subprocess` module.\n",
      "Let us now call the script with some arguments."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print os.popen('python print_cmdline_args.py having arguments that enlighten are fun in scripts as well as in life!').read()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "This python's script name is print_cmdline_args.py\n",
        "It was called with the following arguments: having arguments that enlighten are fun in scripts as well as in life!\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Math"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import math"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print dir(math)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['__doc__', '__file__', '__name__', '__package__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import random"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 14
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print 'A randomly picked name from the math module directory: ' \\\n",
      "        + random.choice(dir(math)[4:]) # choose a random element from a sequence"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "A randomly picked name from the math module directory: fabs\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.choice(\"STATS 607A\") # also works for strings"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "'7'"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "my_list = range(-5,5); print my_list"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[-5, -4, -3, -2, -1, 0, 1, 2, 3, 4]\n"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.shuffle(my_list) # shuffles list in place"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print my_list"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[-4, 1, -3, 4, 0, -5, -1, -2, 3, 2]\n"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.sample(my_list, 5) # random sample of 5 (drawn without replacement)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "[2, -4, 4, 1, -5]"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.randrange(10,20) # equivalent to, but faster than, random.choice(range(10,20))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "11"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.random() # floating point number in [0,1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 22,
       "text": [
        "0.7184199130428679"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.uniform(0,10) # returns a + (b-a)*random()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "8.910448593893147"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "random.gauss(0, 1) # the ubiquitious standard normal (args are mean, std dev)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "-1.5938778111370666"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Pattern matching"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Regular expressions are a powerful way to find and extract complicated patterns in string. The Python module `re` offers powerful regular expression matching operations."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import re"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's create a list with valid and invalid email addresses."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "addr_list = ['someone@somewhere.com', 'prof@university.edu', \\\n",
      "             'employee@company,com', 'student@school.edu', \\\n",
      "             'silly@com', 'missing_at#server.org']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "What is a valid email address anyways? The actual answer can get a bit complicated (see [this question](http://stackoverflow.com/questions/201323/using-a-regular-expression-to-validate-an-email-address)), but for today's lecture, we'll define a valid email address as anything of the form:\n",
      "\n",
      "`username@server.domain`\n",
      "\n",
      "where `username` can be any non-empty string of letters and numbers, `server` can be any non-empty string of letters and numbers, and `domain` is any non-empty string of letters."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_pattern = r'[a-zA-Z0-9]+@[a-zA-Z0-9]+\\.[a-zA-Z]+' # a raw string with a regular expression"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There are several things to note about this regular expression:\n",
      "\n",
      "1. `[]` is used to match a set of characters\n",
      "2. `+` means \"one or more repetitions of\"\n",
      "3. `@` matches the character `@` (and nothing else)\n",
      "4. `.` matches any single character. So, if we're actually looking for a period we have to escape it.\n",
      "5. But Python itself interprets escape sequences in strings. So we created a raw string (that starts with `r`)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.match(email_pattern, addr_list[0])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "<_sre.SRE_Match at 0x105c50c60>"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.match(email_pattern, addr_list[1])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "<_sre.SRE_Match at 0x105c50d30>"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Hmmm... What's going on? `re.match()` just seems to return objects but we want to know whether the pattern matched or not. Let's try on an invalid email address."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "re.match(email_pattern, addr_list[2])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Okay, so re.match returns `None` if no match occurs."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, now let us validate all addresses. When you know a pattern will be matched a lot, it is useful (for efficiency) to compile the regular expression."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog = re.compile(email_pattern)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`re.compile()` returns a `RegexObject` whose `match()` method can be called later."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog.match('a@b.c')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "<_sre.SRE_Match at 0x105c50ed0>"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Remember our list of addresses..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print addr_list"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['someone@somewhere.com', 'prof@university.edu', 'employee@company,com', 'student@school.edu', 'silly@com', 'missing_at#server.org']\n"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[addr for addr in addr_list if email_prog.match(addr)] # pick valid ones"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "['someone@somewhere.com', 'prof@university.edu', 'student@school.edu']"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[bool(email_prog.match(addr)) for addr in addr_list] # print True for valid one, False for invalid one"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 35,
       "text": [
        "[True, True, False, True, False, False]"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@university.edu'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We would like to allow for underscores in the username. In Python regular expressions `\\w` is equivalent to `[a-zA-Z0-9_]`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog = re.compile(r'\\w+@[a-zA-Z0-9]+\\.[a-zA-Z]+')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 37
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@university.edu'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 38,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "However, what about the string `'a_user@university.edu and a lot of other stuff`?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@university.edu and a lot of other stuff'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Actually `match` just tells you whether the pattern occurs at the beginning of the string. To make sure, there's nothing after the pattern we meed the regular expression to match the entire string not just a prefic of the string. The character `$` has a special meaning in regular expression and matches the end of the string."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog = re.compile(r'\\w+@[a-zA-Z0-9]+\\.[a-zA-Z]+$')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 40
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@university.edu and a lot of other stuff'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 41,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now what about `a_user@dept.university.edu`?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@dept.university.edu'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can allow the `server` pattern `[a-zA-Z0-9]+\\.` itself to repeat 1 or more times!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog = re.compile(r'\\w+@([a-zA-Z0-9]+\\.)+[a-zA-Z]+$')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 43
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "bool(email_prog.match('a_user@dept.university.edu'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's extract the users from all email addresses."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "addr_list.append('a_user@dept.university.edu'); print addr_list"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['someone@somewhere.com', 'prof@university.edu', 'employee@company,com', 'student@school.edu', 'silly@com', 'missing_at#server.org', 'a_user@dept.university.edu']\n"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog = re.compile(r'(\\w+)@([a-zA-Z0-9]+\\.)+[a-zA-Z]+$') # note how we surrounded the user pattern in parentheses"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `group` method returns the subgroups of the match. `group(0)` returns the entire string. `group(1)` returns the first subgroup. `group(2)` returns the second subgroup, and so on."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog.match(addr_list[0]).group(1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "'someone'"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "email_prog.match(addr_list[0]).group(2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "'somewhere.'"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[email_prog.match(addr).group(1) for addr in addr_list if email_prog.match(addr)] # extract users from valid email addresses"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "['someone', 'prof', 'student', 'a_user']"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Dates and times"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from datetime import date"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 50
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "now = date.today()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 51
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "next_new_year_day = date(now.year+1, 1, 1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 52
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print next_new_year_day"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2015-01-01\n"
       ]
      }
     ],
     "prompt_number": 53
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print \"There are %d days until the next New Year's Day.\" % (next_new_year_day - now).days"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "There are 108 days until the next New Year's Day.\n"
       ]
      }
     ],
     "prompt_number": 54
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "type(next_new_year_day - now) # difference of two dates gives a timedelta object"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 55,
       "text": [
        "datetime.timedelta"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Performance measurement"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from timeit import Timer"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 56
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "map_and_lambda_timer = Timer('map(lambda x: x**2, mylist)','mylist = range(10)') # args are code to be timed, set up code"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 57
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "list_comprehension_timer = Timer('[x**2 for x in mylist]','mylist = range(10)')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 58
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "map_and_lambda_timer.timeit()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 59,
       "text": [
        "1.8721342086791992"
       ]
      }
     ],
     "prompt_number": 59
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "list_comprehension_timer.timeit()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 60,
       "text": [
        "1.3079421520233154"
       ]
      }
     ],
     "prompt_number": 60
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By default, `timeit()` repeats the code a million times. But we can supply an argument."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "map_and_lambda_timer.timeit(2*10**6) # time 2 million executions"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 61,
       "text": [
        "3.8341610431671143"
       ]
      }
     ],
     "prompt_number": 61
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "list_comprehension_timer.timeit(2*10**6) # list comprehension is a little bit faster"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 62,
       "text": [
        "2.367954969406128"
       ]
      }
     ],
     "prompt_number": 62
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "map_and_builtin_timer = Timer('map(abs, mylist)','mylist = range(-5,5)') # no lambda function, using built-in abs"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 63
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "list_comp_builtin_timer = Timer('[abs(x) for x in mylist]','mylist = range(10)')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 64
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "map_and_builtin_timer.timeit()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 65,
       "text": [
        "0.931617021560669"
       ]
      }
     ],
     "prompt_number": 65
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "list_comp_builtin_timer.timeit()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 66,
       "text": [
        "1.380180835723877"
       ]
      }
     ],
     "prompt_number": 66
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now list comprehension is slower!"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}