Python on the Command Line.ipynb

{
 "metadata": {
  "name": "Python on the Command Line"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for x in range(5):\n",
      "    "
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python --version\n",
      "Python 2.7.2\n",
      "$ python\n",
      "Python 2.7.2 (default, Oct 11 2012, 20:14:37) \n",
      "[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin\n",
      "Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n",
      ">>> exit()\n",
      "$ echo \"import sys; print sys.path\" > pypath.py\n",
      "$ python pypath.py\n",
      "[lots of text, this is where python looks for library code]"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Copying from the Notebook to the command-line\n",
      "\n",
      "There are times when you will want to test things out in the Notebook, but create a re-usable script on the command line once you have things working. Or you may start in a Notebook but realize you will need to read in data, or write out results. While entire notebooks can be saved as Python using `File -> Download As -> Python (.py)`, saving a script from a single cell requires you to copy and paste. For Windows and Linux users, once you have copied the script, open the file you want to save it as in a text editor, paste, and save. Mac users can do this, or use the command-line shortcut: `pbpaste > filename.py` where you replace \"filename\" with the actual name of the script you would like to save."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## A simple program that accepts arguments\n",
      "\n",
      "We've seen how arguments work in bash scripts using `$1`, `$2`, etc. We can get arguments in our Python programs too, using the `sys.argv` list. \n",
      "\n",
      "### args.py\n",
      "\n",
      "    import sys\n",
      "    for arg in sys.argv:\n",
      "        print arg\n",
      "        "
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python args.py\n",
      "args.py\n",
      "$ python args.py one two three\n",
      "args.py\n",
      "one\n",
      "two\n",
      "three"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## A library *and* a program\n",
      "\n",
      "This script can be run as a program, using the special line `if __name__ == '__main__'` to detect that it is being run as a script and not imported as a library. This way, one script can do double-duty. The first argument (`sys.argv[0]`) is always the name of the script being run, so we skip that with the slice notation `[1:]` which says \"give me a copy of this list from the second element (since lists start counting at 0) to the end.\"\n",
      "\n",
      "## allcaps.py\n",
      "\n",
      "    # define a function to use later\n",
      "    def capitalize(str):\n",
      "        return str.upper()\n",
      "\n",
      "    # we'll start here when run as a program, not because the name \"main\" is magic, but because of\n",
      "    # the two lines at the bottom\n",
      "    def main():\n",
      "        import sys\n",
      "        for arg in sys.argv[1:]:\n",
      "            print capitalize(arg)\n",
      "\n",
      "    if __name__ == '__main__': # this line is magical and only succeeds when the script is called as a program\n",
      "        main() # this line is not magical, we can put any python code here"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "python allcaps.py abraham lincoln\n",
      "ABRAHAM\n",
      "LINCOLN\n",
      "python allcaps.py \"abraham lincoln\"\n",
      "ABRAHAM LINCOLN\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## A program that uses a library\n",
      "\n",
      "Here we have a simple program that uses a function from the previous script we wrote. Since this script is so simple, and doesn't double as a library, we don't bother with the `if __name__ == '__main__'` trick and just put all our code at the top level. Any time your code starts to grow beyond a handful of lines you will want to break it up into functions and have a `main()` function to kick things off. Even if you do not intend a script to be run directly, because it only contains libarary functions for other scripts, you can use `if __name__ == '__main__'` to run a test harness to test and demonstrate the use of the functions you've included.\n",
      "\n",
      "### capit.py\n",
      "\n",
      "    from allcaps import capitalize\n",
      "\n",
      "    for word in '''several species of small furry animals gathered together in a cave and grooving with a pict'''.split():\n",
      "        print capitalize(word)"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python capit.py\n",
      "SEVERAL\n",
      "SPECIES\n",
      "OF\n",
      "SMALL\n",
      "FURRY\n",
      "ANIMALS\n",
      "GATHERED\n",
      "TOGETHER\n",
      "IN\n",
      "A\n",
      "CAVE\n",
      "AND\n",
      "GROOVING\n",
      "WITH\n",
      "A\n",
      "PICT"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## A One-Line Web Server\n",
      "\n",
      "Sometimes you need a quick web server, to share some files, show a quick example web page, test a form, etc. Python makes this easy. You will generally run the server on a specific port, like 8080 or 8888 because of the permissions required to run on the standard port 80. To access your server on that port you will use the url `http://localhost:8080/` replacing \"8080\" with the port you choose, if it is different. To run a server on port 8080 (for instance):"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python -m SimpleHTTPServer 8080"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Reading from a file\n",
      "\n",
      "There are several ways to read from a file. We can accept data from standard input, which allows us to pipe data to a program on the command-line like we did with `sort` and `uniq`. Or we can pass one or more filenames as arguments, then open and read the files. We can read all of the file at once, or we can go through it line by line. I will demonstrate these three different techniques.\n",
      "\n",
      "### filein.py\n",
      "\n",
      "    import sys\n",
      "\n",
      "    def readLinesFromStdin():\n",
      "        return sys.stdin.readlines()\n",
      "\n",
      "    def readDataFromStdin():\n",
      "        return sys.stdin.read()\n",
      "\n",
      "    def readLinesFromFile(filename):\n",
      "        return open(filename).readlines()\n",
      "\n",
      "    def readDataFromFile(filename):\n",
      "        return open(filename).read()\n",
      "\n",
      "    def main():\n",
      "        if len(sys.argv) > 1:\n",
      "            if sys.argv[1] == '-l':\n",
      "                # Use read lines mode\n",
      "                if len(sys.argv) > 2:\n",
      "                    # We have filename(s)\n",
      "                    for filename in sys.argv[2:]:\n",
      "                        lines = readLinesFromFile(filename)\n",
      "                        print 'read', len(lines), 'lines from', filename\n",
      "                else:\n",
      "                    lines = readLinesFromStdin()\n",
      "                    print 'read', len(lines), 'lines from stdin'\n",
      "            else:\n",
      "                for filename in sys.argv[1:]:\n",
      "                    data = readDataFromFile(filename)\n",
      "                    print 'read', len(data), 'characters from', filename\n",
      "        else:\n",
      "            data = readDataFromStdin()\n",
      "            print 'read', len(data), 'characters from stdin'\n",
      "\n",
      "    if __name__ == '__main__':\n",
      "        main()\n",
      "\n"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python filein.py args.py\n",
      "read 42 characters from args.py\n",
      "$ cat args.py | python filein.py\n",
      "read 42 characters from stdin\n",
      "$ python filein.py -l args.py\n",
      "read 3 lines from args.py\n",
      "$ cat args.py | python filein.py -l\n",
      "read 3 lines from stdin\n",
      "$ python filein.py -l args.py filein.py\n",
      "read 3 lines from args.py\n",
      "read 37 lines from filein.py"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Writing out results\n",
      "\n",
      "When we are done, we also want to show the user the result of our work. So far we have been outputting everything to `sys.stdout`, which is simply the default if we call `print` without giving it a different file to write to. We also have glossed over closing files (since they close automatically at the end of the function we opened them in) and opening them in write modes, since they open in read mode by default.\n",
      "\n",
      "### fileout.py\n",
      "\n",
      "    def appendToFiles(filenames):\n",
      "        for filename in filenames:\n",
      "            out = open(filename, 'a') # 'a' opens in \"append\" mode\n",
      "            print >> out, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'\n",
      "            out.close()\n",
      "\n",
      "    def overwriteFiles(filenames):\n",
      "        for filename in filenames:\n",
      "            out = open(filename, 'w') # 'w' opens in \"write\" mode, which erases existing content of the file\n",
      "            print >> out, '1234567890'\n",
      "            out.close()\n",
      "\n",
      "    def main():\n",
      "        appendToFiles(['data1.txt', 'data2.txt'])\n",
      "        overwriteFiles(['data3.txt', 'data4.txt'])\n",
      "\n",
      "    if __name__ == '__main__':\n",
      "        main()"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python fileout.py\n",
      "$ cat data*.txt\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "1234567890\n",
      "1234567890\n",
      "$ python fileout.py\n",
      "$ cat data*.txt\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "1234567890\n",
      "1234567890\n",
      "$ python fileout.py\n",
      "$ cat data*.txt\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n",
      "1234567890\n",
      "1234567890\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n",
      "## Calling shell programs from Python\n",
      "\n",
      "We can call Python from the shell, but we can also call shell scripts from Python. It's like Inception, but for code!\n",
      "\n",
      "### shell.py\n",
      "\n",
      "    import subprocess\n",
      "\n",
      "    def ls():\n",
      "        output = subprocess.check_output(['ls']) # output is a string of everything returned\n",
      "        directory = output.split('\\n') # split the string on newlines\n",
      "        directory.pop() # throw away the last, empty line\n",
      "        for filename in directory:\n",
      "            print filename\n",
      "\n",
      "    def lsl():\n",
      "        output = subprocess.check_output(['ls', '-l', '-h'])\n",
      "        directory = output.split('\\n') # split the string on newlines\n",
      "        directory.pop() # throw away the last, empty line\n",
      "        for filename in directory:\n",
      "            print filename\n",
      "\n",
      "    def main():\n",
      "        print 'ls:'\n",
      "        ls()\n",
      "        print\n",
      "        print 'ls -l -h:'\n",
      "        lsl()\n",
      "        print\n",
      "\n",
      "    if __name__ == '__main__':\n",
      "        main()\n",
      "\n",
      "    \n"
     ]
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "$ python shell.py\n",
      "Python on the Command Line.ipynb\n",
      "abc.txt\n",
      "allcaps.py\n",
      "allcaps.pyc\n",
      "args.py\n",
      "capit.py\n",
      "data1.txt\n",
      "data2.txt\n",
      "data3.txt\n",
      "data4.txt\n",
      "fileout.py\n",
      "pypath.py\n",
      "readin.py\n",
      "shell.py\n",
      "\n",
      "ls -l -h:\n",
      "total 128\n",
      "-rw-r--r--  1 delza  staff    11K  7 Jun 10:20 Python on the Command Line.ipynb\n",
      "-rw-r--r--  1 delza  staff     8B  7 Jun 10:11 abc.txt\n",
      "-rw-r--r--  1 delza  staff   467B  7 Jun 00:35 allcaps.py\n",
      "-rw-r--r--  1 delza  staff   599B  7 Jun 00:35 allcaps.pyc\n",
      "-rw-r--r--  1 delza  staff    42B  7 Jun 00:22 args.py\n",
      "-rw-r--r--  1 delza  staff   177B  7 Jun 00:35 capit.py\n",
      "-rw-r--r--  1 delza  staff    81B  7 Jun 10:18 data1.txt\n",
      "-rw-r--r--  1 delza  staff    81B  7 Jun 10:18 data2.txt\n",
      "-rw-r--r--  1 delza  staff    11B  7 Jun 10:18 data3.txt\n",
      "-rw-r--r--  1 delza  staff    11B  7 Jun 10:18 data4.txt\n",
      "-rw-r--r--  1 delza  staff   563B  7 Jun 10:17 fileout.py\n",
      "-rw-r--r--  1 delza  staff    27B  7 Jun 00:13 pypath.py\n",
      "-rw-r--r--  1 delza  staff   1.0K  7 Jun 09:50 readin.py\n",
      "-rw-r--r--  1 delza  staff   664B  7 Jun 11:20 shell.py"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}