00-python-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scientific Python Basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepared by: Cindee Madison, Thomas Kluyver (Any errors are our own)  \n",
    "Thanks to: Justin Kitzes, Matt Davis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Individual things\n",
    "\n",
    "The most basic component of any programming language are \"things\", also called variables or\n",
    "(in special cases) objects.\n",
    "\n",
    "The most common basic \"things\" in Python are integers, floats, strings, booleans, and\n",
    "some special objects of various types. We'll meet many of these as we go through the lesson."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ To run the code in a cell quickly, press Ctrl-Enter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ To quickly create a new cell below an existing one, type Ctrl-m then b.\n",
    "Other shortcuts for making, deleting, and moving cells are in the menubar at the top of the\n",
    "screen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A thing\n",
    "2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Use print to show multiple things in the same cell\n",
    "# Note that you can use single or double quotes for strings\n",
    "print(2)\n",
    "print('hello')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Things can be stored as variables\n",
    "a = 2\n",
    "b = 'hello'\n",
    "c = True  # This is case sensitive\n",
    "print(a, b, c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# The type function tells us the type of thing we have\n",
    "print(type(a))\n",
    "print(type(b))\n",
    "print(type(c))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# What happens when a new variable point to a previous variable?\n",
    "a = 1\n",
    "b = a\n",
    "a = 2\n",
    "## What is b?\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Commands that operate on things\n",
    "\n",
    "Just storing data in variables isn't much use to us. Right away, we'd like to start performing\n",
    "operations and manipulations on data and variables.\n",
    "\n",
    "There are three very common means of performing an operation on a thing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Use an operator\n",
    "\n",
    "All of the basic math operators work like you think they should for numbers. They can also\n",
    "do some useful operations on other things, like strings. There are also boolean operators that\n",
    "compare quantities and give back a `bool` variable as a result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Standard math operators work as expected on numbers\n",
    "a = 2\n",
    "b = 3\n",
    "print(a + b)\n",
    "print(a * b)\n",
    "print(a ** b)  # a to the power of b (a^b does something completely different!)\n",
    "print(a / b)   # Careful with dividing integers if you use Python 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# There are also operators for strings\n",
    "print('hello' + 'world')\n",
    "print('hello' * 3)\n",
    "#print('hello' / 3)  # You can't do this!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Boolean operators compare two things\n",
    "a = (1 > 3)\n",
    "b = (3 == 3)\n",
    "print(a)\n",
    "print(b)\n",
    "print(a or b)\n",
    "print(a and b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Use a function\n",
    "\n",
    "These will be very familiar to anyone who has programmed in any language, and work like you\n",
    "would expect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# There are thousands of functions that operate on things\n",
    "print(type(3))\n",
    "print(len('hello'))\n",
    "print(round(3.3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ To find out what a function does, you can type it's name and then a question mark to\n",
    "get a pop up help window. Or, to see what arguments it takes, you can type its name, an open\n",
    "parenthesis, and hit tab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "round?\n",
    "#round(\n",
    "round(3.14159, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ Many useful functions are not in the Python built in library, but are in external\n",
    "scientific packages. These need to be imported into your Python notebook (or program) before\n",
    "they can be used. Probably the most important of these are numpy and matplotlib."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Many useful functions are in external packages\n",
    "# Let's meet numpy\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# To see what's in a package, type the name, a period, then hit tab\n",
    "#np?\n",
    "np."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Some examples of numpy functions and \"things\"\n",
    "print(np.sqrt(4))\n",
    "print(np.pi)  # Not a function, just a variable\n",
    "print(np.sin(np.pi))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Use a method\n",
    "\n",
    "Before we get any farther into the Python language, we have to say a word about \"objects\". We\n",
    "will not be teaching object oriented programming in this workshop, but you will encounter objects\n",
    "throughout Python (in fact, even seemingly simple things like ints and strings are actually\n",
    "objects in Python).\n",
    "\n",
    "In the simplest terms, you can think of an object as a small bundled \"thing\" that contains within\n",
    "itself both data and functions that operate on that data. For example, strings in Python are\n",
    "objects that contain a set of characters and also various functions that operate on the set of\n",
    "characters. When bundled in an object, these functions are called \"methods\".\n",
    "\n",
    "Instead of the \"normal\" `function(arguments)` syntax, methods are called using the\n",
    "syntax `variable.method(arguments)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A string is actually an object\n",
    "a = 'hello, world'\n",
    "print(type(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Objects have bundled methods\n",
    "#a.\n",
    "print(a.capitalize())\n",
    "print(a.replace('l', 'X'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1 - Conversion\n",
    "\n",
    "Throughout this lesson, we will successively build towards a program that will calculate the\n",
    "variance of some measurements,  in this case `Height in Metres`.  The first thing we want to do is convert from an antiquated measurement system.\n",
    "\n",
    "To change inches into metres we use the following equation (conversion factor is rounded)\n",
    "\n",
    "## $metre = \\frac{inches}{39}$\n",
    "\n",
    "1. Create a variable for the conversion factor, called `inches_in_metre`.\n",
    "1. Create a variable (`inches`) for your height in inches, as inaccurately as you want.\n",
    "2. Divide `inches` by `inches_in_metre`, and store the result in a new variable, `metres`.\n",
    "1. Print the result\n",
    "\n",
    "__Bonus__\n",
    "\n",
    "Convert from feet and inches to metres."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ A 'gotcha' for all python 2 users (it was changed in python 3) is the result of integer division. To make it work the obvious way, either:\n",
    "\n",
    "1. `inches_in_metre = 39.`  (add the decimal to cast to a float, or use 39.4 to be more accurate)\n",
    "2. `from __future__ import division` -  Put this at the **top** of the code and it will work "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Collections of things\n",
    "\n",
    "While it is interesting to explore your own height, in science we work with larger  slightly more complex datasets. In this example, we are interested in the characteristics and distribution of heights. Python provides us with a number of objects to handle collections of things.\n",
    "\n",
    "Probably 99% of your work in scientific Python will use one of four types of collections:\n",
    "`lists`, `tuples`, `dictionaries`, and `numpy arrays`. We'll look quickly at each of these and what\n",
    "they can do for you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Lists\n",
    "\n",
    "Lists are probably the handiest and most flexible type of container. \n",
    "\n",
    "Lists are declared with square brackets []. \n",
    "\n",
    "Individual elements of a list can be selected using the syntax `a[ind]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lists are created with square bracket syntax\n",
    "a = ['blueberry', 'strawberry', 'pineapple']\n",
    "print(a, type(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lists (and all collections) are also indexed with square brackets\n",
    "# NOTE: The first index is zero, not one\n",
    "print(a[0])\n",
    "print(a[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "## You can also count from the end of the list\n",
    "print('last item is:', a[-1])\n",
    "print('second to last item is:', a[-2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# you can access multiple items from a list by slicing, using a colon between indexes\n",
    "# NOTE: The end value is not inclusive\n",
    "print('a =', a)\n",
    "print('get first two:', a[0:2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# You can leave off the start or end if desired\n",
    "print(a[:2])\n",
    "print(a[2:])\n",
    "print(a[:])\n",
    "print(a[:-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lists are objects, like everything else, and have methods such as append\n",
    "a.append('banana')\n",
    "print(a)\n",
    "\n",
    "a.append([1,2])\n",
    "print(a)\n",
    "\n",
    "a.pop()\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ A 'gotcha' for some new Python users is that many collections, including lists,\n",
    "actually store pointers to data, not the data itself. \n",
    "\n",
    "Remember when we set `b=a` and then changed `a`?\n",
    "\n",
    "What happens when we do this in a list?\n",
    "\n",
    "__HELP:__ look into the `copy` module\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "a = 1\n",
    "b = a\n",
    "a = 2\n",
    "## What is b?\n",
    "print('What is b?', b)\n",
    "\n",
    "a = [1, 2, 3]\n",
    "b = a\n",
    "print('original b', b)\n",
    "a[0] = 42\n",
    "print('What is b after we change a ?', b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### EXERCISE 2 - Store a bunch of heights (in metres) in a list\n",
    "\n",
    "1. Ask five people around you for their heights (in metres).\n",
    "2. Store these in a list called `heights`.\n",
    "3. Append your own height, calculated above in the variable *metres*, to the list.\n",
    "4. Get the first height from the list and print it.\n",
    "\n",
    "__Bonus__\n",
    "\n",
    "1. Extract the last value in two different ways: first, by using the index for\n",
    "the last item in the list, and second, presuming that you do not know how long the list is.\n",
    "\n",
    "__HINT:__ **len()** can be used to find the length of a collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Tuples\n",
    "\n",
    "We won't say a whole lot about tuples except to mention that they basically work just like lists, with\n",
    "two major exceptions:\n",
    "\n",
    "1. You declare tuples using () instead of []\n",
    "1. Once you make a tuple, you can't change what's in it (referred to as immutable)\n",
    "\n",
    "You'll see tuples come up throughout the Python language, and over time you'll develop a feel for when\n",
    "to use them. \n",
    "\n",
    "In general, they're often used instead of lists:\n",
    "\n",
    "1. to group items when the position in the collection is critical, such as coord = (x,y)\n",
    "1. when you want to make prevent accidental modification of the items, e.g. shape = (12,23)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "xy = (23, 45)\n",
    "print(xy[0])\n",
    "xy[0] = \"this won't work with a tuple\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Anatomy of a traceback error\n",
    "\n",
    "Traceback errors are `raised` when you try to do something with code it isn't meant to do.  It is also meant to be informative, but like many things, it is not always as informative as we would like.\n",
    "\n",
    "Looking at our error:\n",
    "\n",
    "    TypeError                                 Traceback (most recent call last)\n",
    "    <ipython-input-25-4d15943dd557> in <module>()\n",
    "          1 xy = (23, 45)\n",
    "          2 xy[0]\n",
    "    ----> 3 xy[0] = 'this wont work with a tuple'\n",
    "\n",
    "    TypeError: 'tuple' object does not support item assignment\n",
    "    \n",
    "1. The command you tried to run raise a **TypeError**  This suggests you are using a variable in a way that its **Type** doesnt support\n",
    "2. the arrow ----> points to the line where the error occurred, In this case on line 3 of your code form the above line.\n",
    "3. Learning how to **read** a traceback error is an important skill to develop, and helps you know how to ask questions about what has gone wrong in your code.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Dictionaries\n",
    "\n",
    "Dictionaries are the collection to use when you want to store and retrieve things by their names\n",
    "(or some other kind of key) instead of by their position in the collection. A good example is a set\n",
    "of model parameters, each of which has a name and a value. Dictionaries are declared using {}."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Make a dictionary of model parameters\n",
    "convertors = {'inches_in_feet' : 12,\n",
    "              'inches_in_metre' : 39}\n",
    "\n",
    "print(convertors)\n",
    "print(convertors['inches_in_feet'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "## Add a new key:value pair\n",
    "convertors['metres_in_mile'] = 1609.34\n",
    "print(convertors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Raise a KEY error\n",
    "print(convertors['blueberry'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Numpy arrays (ndarrays)\n",
    "\n",
    "Even though numpy arrays (often written as ndarrays, for n-dimensional arrays) are not part of the\n",
    "core Python libraries, they are so useful in scientific Python that we'll include them here in the \n",
    "core lesson. Numpy arrays are collections of things, all of which must be the same type, that work\n",
    "similarly to lists (as we've described them so far). The most important are:\n",
    "\n",
    "1. You can easily perform elementwise operations (and matrix algebra) on arrays\n",
    "1. Arrays can be n-dimensional\n",
    "1. There is no equivalent to append, although arrays can be concatenated\n",
    "\n",
    "Arrays can be created from existing collections such as lists, or instantiated \"from scratch\" in a \n",
    "few useful ways.\n",
    "\n",
    "When getting started with scientific Python, you will probably want to try to use ndarrays whenever\n",
    "possible, saving the other types of collections for those cases when you have a specific reason to use\n",
    "them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# We need to import the numpy library to have access to it \n",
    "# We can also create an alias for a library, this is something you will commonly see with numpy\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Make an array from a list\n",
    "alist = [2, 3, 4]\n",
    "blist = [5, 6, 7]\n",
    "a = np.array(alist)\n",
    "b = np.array(blist)\n",
    "print(a, type(a))\n",
    "print(b, type(b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Do arithmetic on arrays\n",
    "print(a**2)\n",
    "print(np.sin(a))\n",
    "print(a * b)\n",
    "print(a.dot(b), np.dot(a, b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Boolean operators work on arrays too, and they return boolean arrays\n",
    "print(a > 2)\n",
    "print(b == 6)\n",
    "\n",
    "c = a > 2\n",
    "print(c)\n",
    "print(type(c))\n",
    "print(c.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Indexing arrays\n",
    "print(a[0:2])\n",
    "\n",
    "c = np.random.rand(3,3)\n",
    "print(c)\n",
    "print('\\n')\n",
    "print(c[1:3,0:2])\n",
    "\n",
    "c[0,:] = a\n",
    "print('\\n')\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Arrays can also be indexed with other boolean arrays\n",
    "print(a)\n",
    "print(b)\n",
    "print(a > 2)\n",
    "print(a[a > 2])\n",
    "print(b[a > 2])\n",
    "\n",
    "b[a == 3] = 77\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# ndarrays have attributes in addition to methods\n",
    "#c.\n",
    "print(c.shape)\n",
    "print(c.prod())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# There are handy ways to make arrays full of ones and zeros\n",
    "print(np.zeros(5), '\\n')\n",
    "print(np.ones(5), '\\n')\n",
    "print(np.identity(5), '\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# You can also easily make arrays of number sequences\n",
    "print(np.arange(0, 10, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### EXERCISE 3 - Using Arrays for simple analysis\n",
    "\n",
    "Revisit your list of heights\n",
    "\n",
    "1. turn it into an array\n",
    "2. calculate the mean\n",
    "3. create a mask of all heights greater than a certain value (your choice)\n",
    "4. find the mean of the masked heights\n",
    "\n",
    "__BONUS__\n",
    "\n",
    "1. find the number of heights greater than your threshold\n",
    "2. mean() can take an optional argument called axis, which allows you to calculate the mean across different axes, eg across rows or across columns. Create an array with two dimensions (not equal sized) and calculate the mean across rows and mean across columns. Use 'shape' to understand how the means are calculated.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Repeating yourself\n",
    "\n",
    "So far, everything that we've done could, in principle, be done by hand calculation. In this section\n",
    "and the next, we really start to take advantage of the power of programming languages to do things\n",
    "for us automatically.\n",
    "\n",
    "We start here with ways to repeat yourself. The two most common ways of doing this are known as for\n",
    "loops and while loops. For loops in Python are useful when you want to cycle over all of the items\n",
    "in a collection (such as all of the elements of an array), and while loops are useful when you want to\n",
    "cycle for an indefinite amount of time until some condition is met.\n",
    "\n",
    "The basic examples below will work for looping over lists, tuples, and arrays. Looping over dictionaries\n",
    "is a bit different, since there is a key and a value for each item in a dictionary. Have a look at the\n",
    "Python docs for more information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A basic for loop - don't forget the white space!\n",
    "wordlist = ['hi', 'hello', 'bye']\n",
    "for word in wordlist:\n",
    "    print(word + '!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note on indentation**: Notice the indentation once we enter the for loop.  Every idented statement after the for loop declaration is part of the for loop.  This rule holds true for while loops, if statements, functions, etc. Required identation is one of the reasons Python is such a beautiful language to read.\n",
    "\n",
    "If you do not have consistent indentation you will get an `IndentationError`.  Fortunately, most code editors will ensure your indentation is correction.\n",
    "\n",
    "__NOTE__ In Python the default is to use four (4) spaces for each indentation, most editros can be configured to follow this guide."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Indentation error: Fix it!\n",
    "for word in wordlist:\n",
    "    new_word = word.capitalize()\n",
    "   print(new_word + '!') # Bad indent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Sum all of the values in a collection using a for loop\n",
    "numlist = [1, 4, 77, 3]\n",
    "\n",
    "total = 0\n",
    "for num in numlist:\n",
    "    total = total + num\n",
    "    \n",
    "print(\"Sum is\", total)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Often we want to loop over the indexes of a collection, not just the items\n",
    "print(wordlist)\n",
    "\n",
    "for i, word in enumerate(wordlist):\n",
    "    print(i, word, wordlist[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# While loops are useful when you don't know how many steps you will need,\n",
    "# and want to stop once a certain condition is met.\n",
    "step = 0\n",
    "prod = 1\n",
    "while prod < 100:\n",
    "    step = step + 1\n",
    "    prod = prod * 2\n",
    "    print(step, prod)\n",
    "    \n",
    "print('Reached a product of', prod, 'at step number', step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TIP:__ Once we start really generating useful and large collections of data, it becomes unwieldy to\n",
    "inspect our results manually. The code below shows how to make a very simple plot of an array.\n",
    "We'll do much more plotting later on, this is just to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load up pylab, a useful plotting library\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Make some x and y data and plot it\n",
    "y = np.arange(100)**2\n",
    "plt.plot(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### EXERCISE 4 - Variance\n",
    "\n",
    "We can now calculate the variance of the heights we collected before.\n",
    "\n",
    "As a reminder, **sample variance** is the calculated from the sum of squared differences of each observation from the mean:\n",
    "\n",
    "### $variance = \\frac{\\Sigma{(x-mean)^2}}{n-1}$\n",
    "\n",
    "where **mean** is the mean of our observations, **x** is each individual observation, and **n** is the number of observations.\n",
    "\n",
    "First, we need to calculate the mean:\n",
    "\n",
    "1. Create a variable `total` for the sum of the heights.\n",
    "2. Using a `for` loop, add each height to `total`.\n",
    "3. Find the mean by dividing this by the number of measurements, and store it as `mean`.\n",
    "\n",
    "__Note__: To get the number of things in a list, use `len(the_list)`.\n",
    "\n",
    "Now we'll use another loop to calculate the variance:\n",
    "\n",
    "1. Create a variable `sum_diffsq` for the sum of squared differences.\n",
    "2. Make a second `for` loop over `heights`.\n",
    "  - At each step, subtract the height from the mean and call it `diff`. \n",
    "  - Square this and call it `diffsq`.\n",
    "  - Add `diffsq` on to `sum_diffsq`.\n",
    "3. Divide `diffsq` by `n-1` to get the variance.\n",
    "4. Display the variance.\n",
    "\n",
    "__Note__: To square a number in Python, use `**`, eg. `5**2`.\n",
    "\n",
    "__Bonus__\n",
    "\n",
    "1. Test whether `variance` is larger than 0.01, and print out a line that says \"variance more than 0.01: \"\n",
    "followed by the answer (either True or False)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Making choices\n",
    "\n",
    "Often we want to check if a condition is True and take one action if it is, and another action if the\n",
    "condition is False. We can achieve this in Python with an if statement.\n",
    "\n",
    "__TIP:__ You can use any expression that returns a boolean value (True or False) in an if statement.\n",
    "Common boolean operators are ==, !=, <, <=, >, >=. You can also use `is` and `is not` if you want to\n",
    "check if two variables are identical in the sense that they are stored in the same location in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A simple if statement\n",
    "x = 3\n",
    "if x > 0:\n",
    "    print('x is positive')\n",
    "elif x < 0:\n",
    "    print('x is negative')\n",
    "else:\n",
    "    print('x is zero')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# If statements can rely on boolean variables\n",
    "x = -1\n",
    "test = (x > 0)\n",
    "print(type(test)); print(test)\n",
    "\n",
    "if test:\n",
    "    print('Test was true')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Creating chunks with functions and modules\n",
    "\n",
    "One way to write a program is to simply string together commands, like the ones described above, in a long\n",
    "file, and then to run that file to generate your results. This may work, but it can be cognitively difficult\n",
    "to follow the logic of programs written in this style. Also, it does not allow you to reuse your code\n",
    "easily - for example, what if we wanted to run our logistic growth model for several different choices of\n",
    "initial parameters?\n",
    "\n",
    "The most important ways to \"chunk\" code into more manageable pieces is to create functions and then\n",
    "to gather these functions into modules, and eventually packages. Below we will discuss how to create\n",
    "functions and modules. A third common type of \"chunk\" in Python is classes, but we will not be covering\n",
    "object-oriented programming in this workshop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# We've been using functions all day\n",
    "x = 3.333333\n",
    "print(round(x, 2))\n",
    "print(np.sin(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# It's very easy to write your own functions\n",
    "def multiply(x, y):\n",
    "    return x*y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Once a function is \"run\" and saved in memory, it's available just like any other function\n",
    "print(type(multiply))\n",
    "print(multiply(4, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# It's useful to include docstrings to describe what your function does\n",
    "def say_hello(time, people):\n",
    "    '''\n",
    "    Function says a greeting. Useful for engendering goodwill\n",
    "    '''\n",
    "    return 'Good ' + time + ', ' + people"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Docstrings**: A docstring is a special type of comment that tells you what a function does.  You can see them when you ask for help about a function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "say_hello('afternoon', 'friends')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# All arguments must be present, or the function will return an error\n",
    "say_hello('afternoon')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Keyword arguments can be used to make some arguments optional by giving them a default value\n",
    "# All mandatory arguments must come first, in order\n",
    "def say_hello(time, people='friends'):\n",
    "    return 'Good ' + time + ', ' + people"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "say_hello('afternoon')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "say_hello('afternoon', 'students')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### EXERCISE 5 - Creating a variance function\n",
    "\n",
    "Finally, let's turn our variance calculation into a function that we can use over and over again. \n",
    "Copy your code from Exercise 4 into the box below, and do the following:\n",
    "\n",
    "1. Turn your code into a function called `calculate_variance` that takes a list of values and returns their variance.\n",
    "1. Write a nice docstring describing what your function does.\n",
    "1. In a subsequent cell, call your function with different sets of numbers to make sure it works.\n",
    "\n",
    "__Bonus__\n",
    "\n",
    "1. Refactor your function by pulling out the section that calculates the mean into another function, and calling that inside your `calculate_variance` function.\n",
    "2. Make sure it can works properly when all the data are integers as well.\n",
    "3. Give a better error message when it's passed an empty list. Use the web to find out how to raise exceptions in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### EXERCISE 6 - Putting the `calculate_mean` and `calculate_variance` function(s) in a module\n",
    "\n",
    "We can make our functions more easily reusable by placing them into modules that we can import, just\n",
    "like we have been doing with `numpy`. It's pretty simple to do this.\n",
    "\n",
    "1. Copy your function(s) into a new text file, in the same directory as this notebook,\n",
    "called `stats.py`.\n",
    "1. In the cell below, type `import stats` to import the module. Type `stats.` and hit tab to see the available\n",
    "functions in the module. Try calculating the variance of a number of samples of heights (or other random numbers) using your imported module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}