python-ipython/py03v03_python_getting_started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python Getting Started"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div id='toc-container'><script type='text/javascript'>\n",
       "$(function() {\n",
       "    function regenTOC(){\n",
       "        element = $(\"#toc-container\");\n",
       "\n",
       "\tvar toc = document.createElement(\"div\");\n",
       "\t$(toc).attr(\"class\", \"table-of-contents\");\n",
       "\n",
       "\tvar curLevel = 0;\n",
       "\tvar containerStack = [toc];\n",
       "\tvar levelOfTag = {\"h2\": 1, \"h3\": 2, \"h4\": 3, \"h5\": 4};\n",
       "\n",
       "\tfunction pushLevel() {\n",
       "            var list = document.createElement(\"ul\");\n",
       "            containerStack.push(list);\n",
       "            curLevel++;\n",
       "\t}\n",
       "\t\n",
       "\tfunction popLevel() {\n",
       "            var lastContainer = containerStack.pop();\n",
       "            $(lastContainer).appendTo(containerStack[containerStack.length - 1]);\n",
       "            curLevel--;\n",
       "\t}\n",
       "\t\n",
       "\t$(\".text_cell_render :header\").each(function (i, elem) {\n",
       "            var level = levelOfTag[ elem.tagName.toLowerCase() ];\n",
       "\n",
       "            if (level === undefined)\n",
       "\t\treturn;\n",
       "\n",
       "            while (curLevel < level)\n",
       "\t\tpushLevel();\n",
       "            while (curLevel > level)\n",
       "\t\tpopLevel();\n",
       "            \n",
       "            var listItem = document.createElement(\"li\");\n",
       "            var link = document.createElement(\"a\");\n",
       "            $(link)\n",
       "\t\t.text($(elem).contents().first().text()) // Remove the pilcrow sign\n",
       "\t\t.attr(\"href\", \"#\" + $(elem).attr(\"id\"))\n",
       "\t\t.appendTo(listItem);\n",
       "            $(listItem).appendTo(containerStack[containerStack.length - 1]);\n",
       "\t});\n",
       "\t\n",
       "\twhile (curLevel > 0)\n",
       "            popLevel();\n",
       "\n",
       "        $(\"<a class='btn-update' href='#'>Update</a>\")\n",
       "          .click(regenTOC).prependTo(toc);\n",
       "\n",
       "\t$(toc).prepend(\"<div class='title'>Contents</div>\")\n",
       "          .wrap(\"<div class='toc-headings'/>\");\n",
       "\n",
       "        $(element).empty();\n",
       "        $(element).append(toc);\n",
       "    }\n",
       "\n",
       "    if (typeof(IPython) !== 'undefined')\n",
       "        $([IPython.events]).on('notebook_loaded.Notebook', regenTOC);\n",
       "    regenTOC();\n",
       "});\n",
       "\n",
       "</script></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import addutils.toc ; addutils.toc.js(ipy_notebook=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After learning the basic Python concepts, there are still some skills to learn to start working effectively.\n",
    "\n",
    "In this Notebook we will see how to manage functions and how to work with inport, namespaces and packages.\n",
    "Then we will see how to read and write external data and how to manage the external environment.\n",
    "Since most of our customers are working on Windows-based systems, this notebook is mainly oriented to this specific OS. Nevertheless many concepts you will find here can be applied to Mac or Linux systems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".text_cell_render @font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "div.cell {\n",
       "    width: 900px;\n",
       "    margin-left: 0% !important;\n",
       "    margin-right: 0%;\n",
       "}\n",
       "\n",
       "code {\n",
       "    font-size:10pt;\n",
       "}\n",
       "\n",
       ".text_cell_render  h1 {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 10, 88, 126 );\n",
       "    font-size:28pt;\n",
       "}\n",
       ".text_cell_render h2 {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 10, 88, 126 );\n",
       "    font-size:24pt;\n",
       "}\n",
       ".text_cell_render h3 {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 10, 88, 126 );\n",
       "    font-size:20pt;\n",
       "}\n",
       ".text_cell_render h4 {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 10, 88, 126 );\n",
       "    font-size:18pt;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-weight: 300;\n",
       "    font-size: 11pt;\n",
       "    color: rgb( 48, 48, 48 );\n",
       "    font-style: italic;\n",
       "    margin-bottom: .5em;\n",
       "    margin-top: 0.5em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render ul {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 90, 90, 90 );\n",
       "    font-size:11pt;\n",
       "    line-height: 185%;\n",
       "}\n",
       "\n",
       ".text_cell_render yp {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 90, 90, 90 );\n",
       "    font-size:11pt;\n",
       "}\n",
       "\n",
       ".text_cell_render strong {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 30, 30, 30 );\n",
       "    font-size:11pt;\n",
       "}\n",
       "\n",
       ".text_cell_render a:link {\n",
       "    font-family: Tahoma, sans-serif;\n",
       "    color: rgb( 10, 88, 126 );\n",
       "    font-size:11pt;\n",
       "}\n",
       "\n",
       ".text_cell_render a:visited {\n",
       "    color:rgb( 10, 88, 126 );\n",
       "}\n",
       "\n",
       ".text_cell_render {\n",
       "    font-family: Helvetica, Courier, Computer Modern, \"Helvetica Neue\", Arial, Geneva, sans-serif;\n",
       "    color: rgb( 84, 84, 84 );\n",
       "    font-size:11pt;\n",
       "    line-height: 125%;\n",
       "    font-size: 100%;\n",
       "    width:800px;\n",
       "}\n",
       "\n",
       ".CodeMirror {\n",
       "    font-family: Courier, \"Source Code Pro\", source-code-pro,Consolas, monospace;\n",
       "}\n",
       "\n",
       ".warning {\n",
       "    color: rgb( 240, 20, 20 );\n",
       "}\n",
       "\n",
       "/* Pandas tables */\n",
       "/*\n",
       ".rendered_html td {\n",
       "    text-align: right;\n",
       "}\n",
       "*/\n",
       "\n",
       "table.dataframe td {\n",
       "    text-align: right;\n",
       "}\n",
       "\n",
       ".output .table-of-contents {\n",
       "    border: 1px #cecece solid;\n",
       "    background-color: #fafafa;\n",
       "    padding-top: 10px;\n",
       "    padding-bottom: 5px;\n",
       "    padding-right: 15px;\n",
       "    padding-left: 0px;\n",
       "    margin-bottom: 20px;\n",
       "    display: inline-block;\n",
       "    position: relative;\n",
       "}\n",
       "\n",
       ".output .table-of-contents ul {\n",
       "    list-style-type: none;\n",
       "    padding-left: 20px;\n",
       "}\n",
       "\n",
       ".output .table-of-contents .title {\n",
       "    font-weight: bold;\n",
       "    font-height: 11pt;\n",
       "    padding-left: 20px; /* looks better if it's the same to the <ul> */\n",
       "}\n",
       "\n",
       ".output .table-of-contents .btn-update {\n",
       "    position: absolute;\n",
       "    float: right;\n",
       "    right: 11px;\n",
       "    top: 4px;\n",
       "    font-size: 9pt;\n",
       "}\n",
       "\n",
       "</style>\n",
       "<script>\n",
       "    MathJax.Hub.Config({\n",
       "                        TeX: {\n",
       "                           extensions: [\"AMSmath.js\"]\n",
       "                           },\n",
       "                displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "                \"HTML-CSS\": {\n",
       "                    styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "                }\n",
       "        });\n",
       "</script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from addutils import css_notebook\n",
    "css_notebook()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'/home/sebastiano/Code/addfor/tutorial_release/python-ipython'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 Comment your code !"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If there was a single thing you have to keep in mind, well, this is it: **comment your code!**\n",
    "This is particulary important when you start to have structured code involving classes and functions and when you start to collaborate with someone else.\n",
    "As you see in the following code there are two types of comments in Python:\n",
    "```python\n",
    "    # Single line comments are defined by the # sign\n",
    "    '''\n",
    "    Milti-line comments are defined using\n",
    "    three consecutive single quote signs.\n",
    "    '''\n",
    "```\n",
    "**But remember:**  \n",
    "\n",
    "\n",
    "* Comments must be used just to explain what can't be understood by reading the code\n",
    "* Bad, obvious and out-of-date comments are much worst than no-comment\n",
    "* First write clear code, then add comments to explain what isn't obvious"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Local Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Local functions are used to avoid code repetition and to give a tidy face to your code. Have a look to the code in the next cell and notice the following things:\n",
    "\n",
    "1. Always use intelligible names for variables. In this case we used `spacing_string`\n",
    "2. Function arguments can have default values. This mean that you don't have to reassign all the values every time you call a function: you will define just the values you need. *Notice: mandatory (non-default) values come always first in the function call definition*\n",
    "3. Function arguments are named: you can specify the arguments in any order when calling the function\n",
    "4. Functions can be defined anywhere in the code (better at the beginning of the code), and always before being called\n",
    "5. Arguments are passed by reference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----\n",
      "*********\n"
     ]
    }
   ],
   "source": [
    "def local(spacing_string, n=5):\n",
    "    '''Print n carriage returns\n",
    "       \"spacing_string\" must be provided\n",
    "       \"n\" can be omitted and gets the default value'''\n",
    "    # Variables defined inside functions are local\n",
    "    print spacing_string*n\n",
    "\n",
    "local('-')     # n = 5 (default value)\n",
    "local('*',n=9) # n = 9 (named argument)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Since you wrote a nice description for you function\n",
    "# you can invoke help with help(local) or alternatively with local?\n",
    "local?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 External Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "External functions are saved in external files. As an example you will  find in this folder a file named `my_module.py`. This is the code:\n",
    "\n",
    "    # my_module.py =========================================================\n",
    "    ''' This is the documentation\n",
    "          for my example module'''\n",
    "    \n",
    "    '''Naming convenctions:\n",
    "        Function names must be lowercase separated by underscores '_'\n",
    "        An underscore '_' at the beginning of a functuin name means the function is\n",
    "        intended to be private BUT Python do NOT enforce this rule:\n",
    "        you can break the code if you want! '''\n",
    "    \n",
    "    # Variables defined here are available for ALL the functions in the module\n",
    "    # When module is imported, these variable can be accessed with:\n",
    "    #    module_name.variable_name\n",
    "    module_variable = 1\n",
    "\n",
    "    # Constants are named in uppercase\n",
    "    MODULE_CONSTANT = 1\n",
    "    \n",
    "    def my_function(name):\n",
    "        ''' This is the documentation for \"my_function\"'''\n",
    "        # Variables defined inside functions are local\n",
    "        (first, family) = name                       # Arguments bundled in a Tuple\n",
    "        return _my_private_function(first, family)\n",
    "    \n",
    "    def _my_private_function(first_name, second_name):\n",
    "        ''' This is the documentation for \"_my_private_function\"'''\n",
    "        full_name = first_name + ' [' + second_name.upper() + ']'\n",
    "        return full_name\n",
    "        \n",
    "    if __name__ == '__main__':\n",
    "        ''' This is a Unit Test: use \"run my_module\" from Python interpreter'''\n",
    "        print 'This is the testing code:'\n",
    "        print my_function(('Johnn', 'Doe'))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**my_function(name)** accepts a tuple made of two strings and calls **_my_private_function.** Functions whos name begins with '_' are meant to be private and cannot be called from outside. Lets try a call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rick [BAYES]\n"
     ]
    }
   ],
   "source": [
    "import addutils.my_module as my\n",
    "print my.my_function(('rick', 'bayes'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Try by yourself*** &nbsp;the following commands:\n",
    "\n",
    "    my.MODULE_CONSTANT\n",
    "    my.module_variable\n",
    "    my.my_function?\n",
    "    my?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "my?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Testing your code with **if __name__ == '__main__':**\n",
    "To write reliable code, one of the most important things is to do continuous testing. In Python there is an easy way to test your code every time you modify your functions. When check **__name__ == '__main__'** is True, it means that the module has been called from the command line. You can use this check to write your Unit Testing code:\n",
    "\n",
    "    if __name__ == '__main__':\n",
    "        ''' This is a Unit Test: use \"run my_module\" from Python interpreter'''\n",
    "        print 'This is the testing code:'\n",
    "        print my_function(('John', 'Doe'))\n",
    "\n",
    "Try to call your module from the command line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is the testing code:\n",
      "Johnn [DOE]\n"
     ]
    }
   ],
   "source": [
    "%run -m utilities.my_module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Private Methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Methods whose names start with '_' are meant to be private: this means you aren't supposed to access it. This is an example:\n",
    "\n",
    "    def _my_private_function(first_name, second_name):\n",
    "        ...\n",
    "        return full_name\n",
    "\n",
    "If you **\"import my_module as my\"** and try to type **my.[TAB]** you'll see just the public methods and variables.\n",
    "\n",
    "Actually Python allows you to call private methods anyway but we advice you to do it just when you'll be much more proficient in using this language. Try if you want:\n",
    "\n",
    "    my._my_private_function('John', 'McEnroe')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Try by yourself*** &nbsp;some more examples:\n",
    "    \n",
    "    # Explore other private methods with: my._ + TAB\n",
    "    import numpy as np\n",
    "    print my.__doc__\n",
    "    print my.my_function.__doc__\n",
    "    name = ('Graham', 'Chapman')\n",
    "    print my.my_function(name)\n",
    "    my?               # Module documentation: OBJECT INTROSPECTION\n",
    "    my??              # will also show the function's source code if possible\n",
    "    np.*load*?        # ? can be also used to search the namespace\n",
    "    my.my_function?   # Module function documentation\n",
    "    help(my)          # Module Help: notice private functions not listed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Anonymous Functions (lambda functions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python supports the creation of anonymous functions (i.e. functions that are not bound to a name) at runtime, using a construct called \"lambda\".\n",
    "\n",
    "This piece of code shows the difference between a normal function definition **f** and a lambda function **g**:\n",
    "\n",
    "    def f (x):\n",
    "        return x**2\n",
    "\n",
    "    g = lambda x: x**2\n",
    "\n",
    "    print f(4), g(4)\n",
    "\n",
    "Note that the lambda definition does not include a **'return'** statement -- it always contains an expression which is returned. Also note that you can put a lambda definition anywhere a function is expected, and you don't have to assign it to a variable at all. \n",
    "\n",
    "Check the following code to have an idea of the typical usage for lambda functions: here we sanitize a list of strings by **'mapping'** a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Alabama', 'Georgia', ' Georgia', ' Georgia', 'Florida']\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "states = [' Alabama ', 'Georgia!', '  ## Georgia', ' ? georgia', 'FlOrIda']\n",
    "cleaner = lambda str: re.sub('[!#?]', '', str.strip()).title()\n",
    "print map(cleaner, states)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 File I/O"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Simple I/O"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python is very easy to work with files. \n",
    "***Try by yourself*** &nbsp;this self-explaining code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First Second\n",
      "10     0.32432\n",
      "20  1.324\n",
      "21 7.237923\n",
      "36 .83298932\n",
      "56        237.327823\n"
     ]
    }
   ],
   "source": [
    "import os.path\n",
    "path = os.path.join(os.path.curdir, \"example_data\", \"my_input.txt\")\n",
    "ifile = open(path, 'r')\n",
    "for l in ifile: # ifile is an iterator\n",
    "    print l,    # ',' is for suppressing the newline '\\n'\n",
    "ifile.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['First Second\\n', '10     0.32432\\n', '20  1.324\\n', '21 7.237923\\n', '36 .83298932\\n', '56        237.327823\\n']\n"
     ]
    }
   ],
   "source": [
    "# Read all the lines in a list\n",
    "ifile = open(path, 'r')\n",
    "lines = ifile.readlines()\n",
    "print lines\n",
    "ifile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read a file, format and write back"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First Second\n",
      "\n",
      "10      3.243e-01\n",
      "\n",
      "20      1.324e+00\n",
      "\n",
      "21      7.238e+00\n",
      "\n",
      "36      8.330e-01\n",
      "\n",
      "56      2.373e+02\n",
      "\n"
     ]
    }
   ],
   "source": [
    "ifile = open(path)         # 'read mode' is default\n",
    "path_2 = os.path.join(os.path.curdir, \"tmp\", \"my_input2.txt\")\n",
    "ofile = open(path_2, 'w')   # Open Output file in 'write mode'\n",
    "for line in ifile:                   # Read ONE line at a time\n",
    "    s = line.split()\n",
    "    try:\n",
    "        ofile.write('{:g} {:14.3e}\\n'.format(float(s[0]), float(s[1])))\n",
    "        print '{:g} {:14.3e}\\n'.format(float(s[0]), float(s[1]))\n",
    "    except:\n",
    "        ofile.write('{0} {1}\\n'.format(s[0], s[1]))\n",
    "        print '{0} {1}\\n'.format(s[0], s[1])\n",
    "# Notice: 'print' automatically adds a newline at the of the string\n",
    "\n",
    "ifile.close()\n",
    "ofile.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When it's possible **use the \"with\" syntax**, this will close the file automatically in case of an exception preventing the program flow to reach the **'close'** statements. This is also considered a ***\"more pythonic\"*** style."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First\t Second\n",
      "10\t   0.324\n",
      "20\t   1.324\n",
      "21\t   7.238\n",
      "36\t   0.833\n",
      "56\t 237.328\n"
     ]
    }
   ],
   "source": [
    "with open(path) as fid:\n",
    "    for line in fid:\n",
    "        s = line.split()\n",
    "        try:\n",
    "            print '{:.5g}\\t{:8.3f}'.format(float(s[0]), float(s[1]))\n",
    "        except:\n",
    "            print '{0}\\t {1}'.format(s[0], s[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Pickle / cPickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the most common way to serialize and save to disk any type of Python object. Mind that if you need to save complex and structured data and share it, cPickle is not the preferred method: consider instead of using a specific file format like [hdf5](http://en.wikipedia.org/wiki/Hierarchical_Data_Format)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['one', 'two', 'three']\n"
     ]
    }
   ],
   "source": [
    "import cPickle                          # cPickle is like Pickle but faster\n",
    "ls = ['one', 'two', 'three']\n",
    "with file('tmp/out_ascii.pkl', 'w') as f:   # Can choose an arbitrary extension\n",
    "    cPickle.dump(ls, f, 0)                  # dump with protocol '0': readable ASCII\n",
    "with file('tmp/out_compb.pkl', 'w') as f:   # Can choose an arbitrary extension\n",
    "    cPickle.dump(ls, f, 2)                  # dump with protocol '2': compressed bin\n",
    "\n",
    "with open('tmp/out_compb.pkl') as f:\n",
    "    d2 = cPickle.load(f)                # Protocol is automatically detected\n",
    "print d2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 Operating System"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 General Info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python gives you extensive possibilities to access you PC operating system. There are three modules in the Python Standard Library that you must be aware of:\n",
    "\n",
    "* [sys — System-specific parameters and functions](http://docs.python.org/library/sys.html)\n",
    "* [os — Miscellaneous operating system interfaces](http://docs.python.org/library/os.html)\n",
    "* [glob — Unix style pathname pattern expansion](http://docs.python.org/library/glob.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 sys — System-specific parameters and functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try some example commands by running the following cells"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'linux2'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sys \n",
    "# Platform identifier\n",
    "sys.platform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.7.9 |Anaconda 2.1.0 (64-bit)| (default, Mar  9 2015, 16:20:48) \\n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Version number of the Python interpreter\n",
    "sys.version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "/opt/anaconda/lib/python27.zip\n",
      "/opt/anaconda/lib/python2.7\n",
      "/opt/anaconda/lib/python2.7/plat-linux2\n",
      "/opt/anaconda/lib/python2.7/lib-tk\n",
      "/opt/anaconda/lib/python2.7/lib-old\n",
      "/opt/anaconda/lib/python2.7/lib-dynload\n",
      "/opt/anaconda/lib/python2.7/site-packages/setuptools-14.0-py2.7.egg\n",
      "/opt/anaconda/lib/python2.7/site-packages\n",
      "/opt/anaconda/lib/python2.7/site-packages/PIL\n",
      "/opt/anaconda/lib/python2.7/site-packages/Sphinx-1.2.3-py2.7.egg\n",
      "/home/sebastiano/src/bokeh\n",
      "/opt/anaconda/lib/python2.7/site-packages/IPython/extensions\n",
      "/home/sebastiano/.ipython\n"
     ]
    }
   ],
   "source": [
    "# PYTHONPATH: Folders in which looking for modules\n",
    "for p in sys.path:\n",
    "    print p.strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/anaconda\n"
     ]
    }
   ],
   "source": [
    "# Shows where the Python files are installed\n",
    "print sys.exec_prefix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Information about the float DataType\n",
    "sys.float_info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9223372036854775807"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Largest (simple) positive integer supported\n",
    "sys.maxint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9223372036854775807"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Maximum size lists, strings, dicts can have\n",
    "sys.maxsize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 os — Miscellaneous operating system interfaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try some example commands by running the following cells"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              SSH_ASKPASS: /usr/libexec/openssh/gnome-ssh-askpass\n",
      "                   GOPATH: /home/sebastiano/Code/go/\n",
      "            LOADEDMODULES: \n",
      "                     LESS: -R\n",
      "                   GOROOT: /usr/lib/golang/\n",
      "AND MORE ...\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "for counter, osvariable in enumerate(os.environ):\n",
    "    if counter >= 5:\n",
    "        print 'AND MORE ...'\n",
    "        break\n",
    "    print '{0:>25}: {1}'.format(osvariable,os.environ[osvariable])\n",
    "else:\n",
    "    print '============ No more OS Variables ============'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# How to check a system variable:\n",
    "if 'NUMBER_OF_PROCESSORS' in os.environ:\n",
    "    print \"Number of processors in this machine:\", os.environ['NUMBER_OF_PROCESSORS']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/sebastiano/Code/addfor/tutorial_release/python-ipython\n"
     ]
    }
   ],
   "source": [
    "# Working directory\n",
    "print os.getcwd()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['.ipynb_checkpoints', 'example_data', 'index.ipynb', 'index.nbconvert.ipynb', 'py01v03_ipython_notebook_introduction.ipynb', 'py01v03_ipython_notebook_introduction.nbconvert.ipynb', 'py02v03_python_basics.ipynb', 'py02v03_python_basics.nbconvert.ipynb', 'py03v03_python_getting_started.ipynb', 'py04v03_python_style_guide.ipynb', 'py05v03_python_more_examples.ipynb', 'py06v03_python_object_oriented.ipynb', 'py07v03_python_speed-up_with_C.ipynb', 'py08v03_Unicode.ipynb', 'py09v03_python_regular_expressions.ipynb', 'rerun-python-ipython.sh', 'tmp', 'utilities']\n"
     ]
    }
   ],
   "source": [
    "# List the files in the current directory\n",
    "print sorted(os.listdir(os.getcwd())) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Difference between **os.name** and **sys.platform**: sys.platform will distinguish between linux, other unixes, and OS X  while os.name is \"posix\" for all of them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'posix'"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The history saving thread hit an unexpected error (OperationalError('attempt to write a readonly database',)).History will not be written to the database.\n"
     ]
    }
   ],
   "source": [
    "os.name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('', '/Users/dani/myfile.py')\n",
      "('/Users/dani', 'myfile.py')\n"
     ]
    }
   ],
   "source": [
    "# Correctly handle paths, and filenames\n",
    "# os.name can be 'posix', 'nt', 'mac', 'os2', 'ce', 'java', 'riscos'\n",
    "if os.name == 'posix':\n",
    "    full_path = \"/Users/dani/myfile.py\"\n",
    "elif os.name == 'nt':\n",
    "    full_path = 'C:\\\\myfile.py'\n",
    "\n",
    "print os.path.splitdrive(full_path)\n",
    "print os.path.split(full_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "if os.name == 'posix':\n",
    "    os.system('ls')\n",
    "else:\n",
    "    os.system('dir')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 glob — Unix style pathname pattern expansion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try some example commands by running the following cells"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n"
     ]
    }
   ],
   "source": [
    "import glob\n",
    "print glob.glob(\"*.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "Visit [www.add-for.com](<http://www.add-for.com/IT>) for more tutorials and updates.\n",
    "\n",
    "This work is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-sa/4.0/\">Creative Commons Attribution-ShareAlike 4.0 International License</a>."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}