DSFromScratch/Chap2/054-Chap2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Crash Course in Python (skipping the first part)\n",
    "\n",
    "#### Sets\n",
    "A set represents a collection of *distinct* elements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = set()\n",
    "s.add(1)\n",
    "s.add(2)\n",
    "s.add(2) # s is still {1,2}\n",
    "x = len(s) # equals 2\n",
    "y = 2 in s # equals True\n",
    "x = 3 in s # equals False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we have a large collection of items that we want to use for a membership test, a set is more appropriate than a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stopwords_list = [\"a\",\"an\",\"at\"] + [\"yet\",\"you\"]\n",
    "\n",
    "\"zip\" in stopwords_list # False, but have to check every element\n",
    "\n",
    "stopwords_set = set(stopwords_list) # If we just want a yes/no if a word is in a collection\n",
    "\n",
    "\"Zip\" in stopwords_set # very fast to check"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second reason is to find distinct items in a collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "item_list = [1,2,3,1,2,3]\n",
    "num_items = len(item_list) # 6\n",
    "item_set = set(item_list)  # {1,2,3} - these are the distinct values\n",
    "num_distinct_items = len(item_set) # 3\n",
    "distinct_item_list = list(item_set) # [1,2,3] - turning the distinct items into a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Control Flow\n",
    "You can perform an action conditionally using if:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if 1 > 2:\n",
    "    message = \"if only 1 were greater than two...\"\n",
    "elif 1 > 3:\n",
    "    message = \"elif stands for 'else if'\"\n",
    "else:\n",
    "    message = \"when all else fails use else (if you want to)\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also write *ternary*(of three parts) if-then-else on one line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "parity = \"even\" if x % 2 == 0 else \"odd\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more complex logic, use continue and break:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "for x in range(10):\n",
    "    if x == 3:\n",
    "        continue # go immediately to the next iteration\n",
    "    if x == 5:\n",
    "        break # quit the loop entirely\n",
    "    print (x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Truthiness\n",
    "Booleans work the same in Python as in most other languages, except they're capitalized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "one_is_less_than_two = 1 < 2 # equals True\n",
    "true_equals_false = True == False # equals False\n",
    "print (one_is_less_than_two)\n",
    "print(true_equals_false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python uses None to indicate a nonexistent value. It is similar to other languages' null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "x = None\n",
    "print (x == None)\n",
    "print (x is None)  # also True, more Pythonic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following are all False\n",
    "\n",
    "- False\n",
    "- None\n",
    "- [] (an empty list)\n",
    "- {} (an empty dict)\n",
    "- \"\"\n",
    "- set()\n",
    "- 0\n",
    "- 0.0\n",
    "\n",
    "Pretty much anything else gets treated as `True`. This allows you use `if` to test for empty lists of empty strings etc.\n",
    "\n",
    "To ensure that a value is definitely a number (when it could possibly be None):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "x = None\n",
    "safe_x = x or 0 # converts any potential Nones to 0. This is saying safe_x is the number x if x is not-None. If x is not\n",
    "                # a number the first term will be False and safe_x is converted to 0.\n",
    "\n",
    "print(safe_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More Advanced Features\n",
    "\n",
    "#### Sorting\n",
    "Sort sorts a list \"in place\". Sorted returns a new list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = [4,1,2,3]\n",
    "y = sorted(x) # y is [1,2,3,4], x is unchanged\n",
    "x.sort() # now x is [1,2,3,4]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Specify reverse = True to sort elements from largest to smallest. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-4, 3, -2, 1]\n"
     ]
    }
   ],
   "source": [
    "# sort the list by absolute value from largest to smallest\n",
    "x = sorted([-4,1,-2,3], key=abs, reverse=True)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List Comprehensions\n",
    "\n",
    "When you want to transform a list into another list, by choosing only certain elements, or transforming elements, or both."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "even_numbers = [x for x in range(5) if x % 2 == 0] # [0,2,4]\n",
    "\n",
    "squares = [x * x for x in range(5)] #[0, 1, 4, 9, 16]\n",
    "\n",
    "even_squares = [x * x for x in even_numbers] # [0, 4, 16]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly you can turn lists into dictionaries or sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "square_dict = {x : x * x for x in range(5)} # {0:0, 1:1, 2:4, 3:9, 4:16}\n",
    "\n",
    "square_set = {x * x for x in [1, -1]} # {1}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you don't need the value from a list, it's conventional to use an underscore as the variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "zeros = [0 for _ in even_numbers] # has the same length as even_numbers\n",
    "print(zeros)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A list comprehension can include multiple fors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pairs = [(x,y) for x in range(10) for y in range(10)] # 100 pairs (0,0) (0,1)....(9,8),(9,9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Later `for`s can use the results of earlier ones: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (1, 7), (1, 8), (1, 9), (2, 3), (2, 4), (2, 5), (2, 6), (2, 7), (2, 8), (2, 9), (3, 4), (3, 5), (3, 6), (3, 7), (3, 8), (3, 9), (4, 5), (4, 6), (4, 7), (4, 8), (4, 9), (5, 6), (5, 7), (5, 8), (5, 9), (6, 7), (6, 8), (6, 9), (7, 8), (7, 9), (8, 9)]\n"
     ]
    }
   ],
   "source": [
    "increasing_pairs = [(x,y) for x in range(10) for y in range(x+1, 10)]\n",
    "print (increasing_pairs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generators and Iterators\n",
    "\n",
    "When you only want to iterate over parts of a large list. It is not efficient to loop over the whole list.\n",
    "A *generator* is something you can iterate over, but whose values are produced only as needed. \n",
    "\n",
    "One way to create generators is with functions and the yield operator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def lazy_range(n):\n",
    "    \"\"\"a lazy version of range\"\"\"\n",
    "    i = 0\n",
    "    while i < n: # only looking at values less than n.\n",
    "        yield i\n",
    "        i += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following loop will consume the `yielded` values one at a time until none are left:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in lazy_range(10): # operations will only be performed on results less than 10.\n",
    "    (i)/3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python 3 the `range()` function IS lazy. \n",
    "This means you could create an infinite sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def natural_numbers():\n",
    "    \"\"\"returns 1,2,3, ...\"\"\"\n",
    "    n = 1\n",
    "    while True:\n",
    "        yield n\n",
    "        n += 1\n",
    "        \n",
    "# Though you probably shouldn't iterate over it without using some kind of break logic. WHY?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The flip side of a generator is that you can only iterate through a generator once, if you need to do it multiple times, you need to either recreate the generator each time of use a list. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generator alternative\n",
    "\n",
    "Use comprehensions wrapped in parentheses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object <genexpr> at 0x7f95e40a8830>\n"
     ]
    }
   ],
   "source": [
    "lazy_evens_below_20 = (i for i in lazy_range(20) if i % 2 == 0)\n",
    "print(lazy_evens_below_20)\n",
    "\n",
    "# What is this used for? What have we stored in lazy_evens_below_20 and how can it be used?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Randomness\n",
    "\n",
    "Generating random numbers is important in Data Science. We will use the `random` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.5262103693956621, 0.12765972847670115, 0.0008494817011335254, 0.14652841728643295]\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "# random.random() produces numbers uniformly between 0 and 1.\n",
    "four_uniform_randoms = [random.random() for _ in range(4)]\n",
    "print(four_uniform_randoms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can set the internal state with random.seed if you want to get reproducible results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2267058593810488\n",
      "0.46300735781502145\n",
      "0.46300735781502145\n"
     ]
    }
   ],
   "source": [
    "random.seed(8) # set the seed to 8\n",
    "print (random.random()) \n",
    "\n",
    "random.seed(9) # set the seed to 10\n",
    "print (random.random()) \n",
    "\n",
    "random.seed(9) # set the seed to 10\n",
    "print (random.random()) \n",
    "\n",
    "# random.random() will always give the same number if you use the same seed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "random.randrange takes 1 or 2 arguments and returns an element chosen randomly from the corresponding range():"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random.randrange(10) # choose randomly from range (10) = [0,1,...., 9]\n",
    "random.randrange(3,6) # choose randomly from range (3,6) = [3,4,5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "random.shuffle randomly reorders the elements of a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9, 8, 0, 4, 1, 3, 5, 7, 2, 6]\n"
     ]
    }
   ],
   "source": [
    "up_to_ten = list(range(10))  # had to convert to a list as range is a generator object in Python 3\n",
    "random.shuffle(up_to_ten)\n",
    "print (up_to_ten)\n",
    "\n",
    "# results change each time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to randomly pick on element from a list, use `random.choice`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alice\n"
     ]
    }
   ],
   "source": [
    "my_best_friend = random.choice([\"Alice\", \"Bob\", \"Charlie\"])\n",
    "print(my_best_friend)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choose a sample of elements, without replacement use `random.sample`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24, 50, 47, 56, 6, 18]\n"
     ]
    }
   ],
   "source": [
    "lottery_numbers = range(60)\n",
    "winning_numbers = random.sample(lottery_numbers, 6)\n",
    "print(winning_numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Choose a sample of elements *with* replacement, you can make multiple calls to `random.choice`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4, 3, 6, 4]\n"
     ]
    }
   ],
   "source": [
    "# using list comprehension\n",
    "four_with_replacement = [random.choice(range(10)) for _ in range(4)]\n",
    "print(four_with_replacement)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Regular Expressions\n",
    "\n",
    "Just a few examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "print(all([\n",
    "        not re.match(\"a\", \"cat\"), # cat doesn't start with an \"a\"\n",
    "        re.search(\"a\",\"cat\"),     # cat doesn't have an 'a' in it\n",
    "        not re.search(\"c\",\"dog\"), # 'dog' doesn't have a 'c' in it\n",
    "        3 == len(re.split(\"[ab]\", \"carbs\")),  # split on a or b to give ['c','r','s']\n",
    "        \"R-D-\" == re.sub(\"[0-9]\", \"-\", \"R2D2\")])) # replace digits with dashes\n",
    "# All these statements are true"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}