docs/Module2_EssentialsOfPython/Generators_and_Comprehensions.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. meta::\n",
    "   :description: Topic: generators comprehensions and efficiency, Difficulty: Easy, Category: Section\n",
    "   :keywords: generator, range, list comprehension, generator comprehension, nested comprehensions, inline for-loop, filtered, iterator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generators & Comprehension Expressions\n",
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "**Note**:\n",
    "\n",
    "There are reading-comprehension exercises included throughout the text. These are meant to help you put your reading to practice. Solutions for the exercises are included at the bottom of this page.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introducing Generators\n",
    "Now we introduce an important type of object called a **generator**, which allows us to generate arbitrarily-many items in a series, without having to store them all in memory at once.  \n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Definition**:\n",
    "\n",
    "A **generator** is a special kind of iterator, which stores the instructions for how to *generate* each of its members, in order, along with its current state of iterations. It generates each member, one at a time, only as it is requested via iteration.\n",
    "</div>\n",
    "\n",
    "Recall that a list readily stores all of its members; you can access any of its contents via indexing. A generator, on the other hand, *does not store any items*. Instead, it stores the instructions for generating each of its members, and stores its iteration state; this means that the generator will know if it has generated its second member, and will thus generate its third member the next time it is iterated on. \n",
    "\n",
    "The whole point of this is that you can use a generator to produce a long sequence of items, without having to store them all in memory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `range` generator\n",
    "An extremely popular built-in generator is `range`, which, given the values: \n",
    "\n",
    "- 'start' (inclusive, default=0)\n",
    "- 'stop' (exclusive)\n",
    "- 'step' (default=1) \n",
    "\n",
    "will generate the corresponding sequence of integers (from start to stop, using the step size) upon iteration. Consider the following example usages of `range`:\n",
    "```python\n",
    "# start: 2  (included)\n",
    "#  stop: 7  (excluded)\n",
    "#  step: 1  (default)\n",
    "for i in range(2, 7):\n",
    "    print(i)\n",
    "# prints: 2.. 3.. 4.. 5.. 6\n",
    "```\n",
    "***\n",
    "```python\n",
    "# start:  1  (included)\n",
    "#  stop: 10  (excluded)\n",
    "#  step:  2\n",
    "for i in range(1, 10, 2):\n",
    "    print(i)\n",
    "# prints: 1.. 3.. 5.. 7.. 9\n",
    "```\n",
    "***\n",
    "```python\n",
    "# A very common use case!\n",
    "# start:  0  (default, included)\n",
    "#  stop:  5  (excluded)\n",
    "#  step:  1  (default)\n",
    "for i in range(5):\n",
    "    print(i)\n",
    "# prints: 0.. 1.. 2.. 3.. 4\n",
    "```\n",
    "\n",
    "Because `range` is a generator, the command `range(5)` will simply store the instructions needed to produce the sequence of numbers 0-4, whereas the list `[0, 1, 2, 3, 4]` stores all of these items in memory at once. For short sequences, this seems to be a rather paltry savings; this is not the case for long sequences. The following graph compares the memory consumption used when defining a generator for the sequence of numbers $0-N$ using `range`, compared to storing the sequence in a list:\n",
    "\n",
    "![Memory consumption figure](attachments/Mem_Consumption_Generator.png)\n",
    "\n",
    "Given our discussion of generators, it should make sense that the memory consumed simply by defining `range(N)` is independent of $N$, whereas the memory consumed by the list grows linearly with $N$ (for large $N$).\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Takeaway**: \n",
    "\n",
    "`range` is a built-in generator, which generates sequences of integers.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Using `range`**:\n",
    "\n",
    "Using `range` in a for-loop, print the numbers 10-1, in sequence.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating your own generator: generator comprehensions\n",
    "Python provides a sleek syntax for defining a simple generator in a single line of code; this expression is known as a **generator comprehension**. The following syntax is extremely useful and will appear very frequently in Python code:\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "**Definition**: \n",
    "\n",
    "The syntax \n",
    "<br>\n",
    "`(<expression> for <var> in <iterable> [if <condition>])`\n",
    "<br>\n",
    "specifies the general form for a **generator comprehension**. This produces a generator, whose instructions for generating its members are provided within the parenthetical statement. \n",
    "</div>\n",
    "\n",
    "Written in a long form, the pseudo-code for \n",
    "```\n",
    "(<expression> for <var> in <iterable> if <condition>)\n",
    "``` \n",
    "is:\n",
    "\n",
    "```\n",
    "for <var> in <iterable>:\n",
    "    if bool(<condition>):\n",
    "        yield <expression>\n",
    "```\n",
    "\n",
    "The following expression defines a generator for all the even numbers in 0-99:\n",
    "```python\n",
    "# when iterated over, `even_gen` will generate 0.. 2.. 4.. ... 98\n",
    "even_gen = (i for i in range(100) if i%2 == 0)\n",
    "```\n",
    "\n",
    "The `if <condition>` clause in the generator expression is optional. The generator comprehension \n",
    "```\n",
    "(<expression> for <var> in <iterable>)\n",
    "``` \n",
    "corresponds to:\n",
    "\n",
    "```\n",
    "for <var> in <iterable>:\n",
    "    yield <expression>\n",
    "```\n",
    "\n",
    "For example:\n",
    "```python\n",
    "# when iterated over, `example_gen` will generate 0/2.. 9/2.. 21/2.. 32/2\n",
    "example_gen = (i/2 for i in [0, 9, 21, 32])\n",
    "\n",
    "for item in example_gen:\n",
    "    print(item)\n",
    "# prints: 0.0.. 4.5.. 10.5.. 16.0\n",
    "```\n",
    "\n",
    "`<expression>` can be any valid single-line of Python code that returns an object:\n",
    "```python\n",
    "((i, i**2, i**3) for i in range(10))\n",
    "# will generate:\n",
    "# (0, 0, 0)\n",
    "# (1, 1, 1)\n",
    "# (2, 4, 8)\n",
    "# (3, 9, 27)\n",
    "# (4, 16, 64)\n",
    "# (5, 25, 125)\n",
    "# (6, 36, 216)\n",
    "# (7, 49, 343)\n",
    "# (8, 64, 512)\n",
    "# (9, 81, 729)\n",
    "```\n",
    "\n",
    "This means that `<expression>` can even involve inline if-else statements!\n",
    "```python\n",
    "((\"apple\" if i < 3 else \"pie\") for i in range(6))\n",
    "# will generate:\n",
    "# 'apple'..\n",
    "# 'apple'..\n",
    "# 'apple'..\n",
    "# 'pie'..\n",
    "# 'pie'..\n",
    "# 'pie'\n",
    "```\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Takeaway**:  \n",
    "\n",
    "A generator comprehension is a single-line specification for defining a generator in Python. It is absolutely essential to learn this syntax in order to write simple and readable code.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "**Note**:  \n",
    "\n",
    "Generator comprehensions are **not** the only method for defining generators in Python. One can define a generator similar to the way one can define a function (which we will encounter soon). [See this section of the official Python tutorial](https://docs.python.org/3/tutorial/classes.html#generators) if you are interested in diving deeper into generators.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Writing a Generator Comprehension**:\n",
    "\n",
    "Using a generator comprehension, define a generator for the series:\n",
    "```\n",
    "(0, 2).. (1, 3).. (2, 4).. (4, 6).. (5, 7)\n",
    "```\n",
    "\n",
    "Note that (3, 5) is *not* in the series.\n",
    "\n",
    "Iterate over the generator and print its contents to verify your solution.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Storing generators\n",
    "Just like we saw with the `range` generator, defining a generator using a comprehension does *not* perform any computations or consume any memory beyond defining the rules for producing the sequence of data. See what happens when we try to print this generator:\n",
    "```python\n",
    "# will generate 0, 1, 4, 9, 25, ..., 9801\n",
    ">>> gen = (i**2 for i in range(100))\n",
    ">>> print(gen)\n",
    "<generator object <genexpr> at 0x000001E768FE8A40>\n",
    "```\n",
    "This output simply indicates that `gen` stores a generator-expression at the memory address `0x000001E768FE8A40`; this is simply where the instructions for generating our sequence of squared numbers is stored. `gen` will not produce any results until we iterate over it. For this reason, generators cannot be inspected in the same way that lists and other sequences can be. You **cannot** do the following:\n",
    "```python\n",
    "# you **cannot** do the following...\n",
    ">>> gen = (i**2 for i in range(100))\n",
    "\n",
    "# query the length of a generator\n",
    ">>> len(gen)\n",
    "TypeError: object of type 'generator' has no len()\n",
    "    \n",
    "# index into a generator\n",
    ">>> gen[2]\n",
    "TypeError: 'generator' object is not subscriptable\n",
    "```\n",
    "\n",
    "The sole exception to this is the `range` generator, for which all of these inspections are valid."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Consuming generators\n",
    "We can feed this to any function that accepts iterables. For instance, we can feed `gen` to the built-in `sum` function, which sums the contents of an iterable:\n",
    "```python\n",
    ">>> gen = (i**2 for i in range(100))\n",
    ">>> sum(gen)  # computes the sum 0 + 1 + 4 + 9 + 25 + ... + 9801\n",
    "328350\n",
    "```\n",
    "This computes the sum of the sequence of numbers *without ever storing the full sequence of numbers in memory*. In fact, only two numbers need be stored during any given iteration of the sum: the current value of the sum, and the number being added to it.\n",
    "\n",
    "What happens if we run this command a second time:\n",
    "```python\n",
    "# computes the sum of ... nothing!\n",
    "# `gen` has already been consumed!\n",
    ">>> sum(gen)\n",
    "0\n",
    "```\n",
    "It may be surprising to see that the sum now returns 0. This is because **a generator is exhausted after it is iterated over in full**. You must redefine the generator if you want to iterate over it again; fortunately, defining a generator requires very few resources, so this is not a point of concern.\n",
    "\n",
    "You can also check for membership in a generator, but this also consumes the generator:\n",
    "\n",
    "```python\n",
    "# checking for membership consumes a generator until\n",
    "# it finds that item (consuming the entire generator\n",
    "# if the item is not contained within it)\n",
    ">>> gen = (i for i in range(1, 11))\n",
    ">>> 5 in gen  # first 5 elements are consumed \n",
    "True\n",
    "\n",
    "# 1-5 are no longer contained in gen\n",
    "# this check consumes the entire generator!\n",
    ">>> 5 in gen  \n",
    "False\n",
    "\n",
    ">>> sum(gen)\n",
    "0\n",
    "```\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Takeaway**:\n",
    "\n",
    "A generator can only be iterated over once, after which it is exhausted and must be re-defined in order to be iterated over again.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chaining comprehensions\n",
    "Because generators are iterables, they can be fed into subsequent generator comprehensions. That is, they can be \"chained\" together.\n",
    "```python\n",
    "# chaining two generator comprehensions\n",
    "\n",
    "# generates 400.. 100.. 0.. 100.. 400 \n",
    ">>> gen_1 = (i**2 for i in [-20, -10, 0, 10, 20])\n",
    "\n",
    "# iterates through gen_1, excluding any numbers whose absolute value is greater than 150\n",
    ">>> gen_2 = (j for j in gen_1 if abs(j) <= 150)\n",
    "\n",
    "# computing 100 + 0 + 100\n",
    ">>> sum(gen_2)\n",
    "200\n",
    "```\n",
    "\n",
    "This is equivalent to:\n",
    "\n",
    "```python\n",
    "total = 0\n",
    "for i in [-20, -10, 0, 10, 20]:\n",
    "    j = i ** 2\n",
    "    if j <= 150:\n",
    "        total += j\n",
    "\n",
    "# total is now 200\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using generator comprehensions on the fly\n",
    "A feature of Python, that can make your code supremely readable and intuitive, is that generator comprehensions can be fed *directly* into functions that operate on iterables. That is,\n",
    "\n",
    "```python\n",
    ">>> gen = (i**2 for i in range(100))\n",
    ">>> sum(gen)\n",
    "328350\n",
    "```\n",
    "\n",
    "can be simplified as:\n",
    "\n",
    "```python\n",
    ">>> sum(i**2 for i in range(100))\n",
    "328350\n",
    "```\n",
    "\n",
    "If you want your code to compute the finite harmonic series: $\\sum_{k=1}^{100} \\frac{1}{n} = 1 + \\frac{1}{2} + ... + \\frac{1}{100}$, you can simply write:\n",
    "```python\n",
    ">>> sum(1/n for n in range(1, 101))\n",
    "5.187377517639621\n",
    "```\n",
    "\n",
    "This convenient syntax works for any function that expects an iterable as an argument, such as the `list` function and `all` function:\n",
    "```python\n",
    "# providing generator expressions as arguments to functions\n",
    "# that operate on iterables\n",
    ">>> list(i**2 for i in range(10))\n",
    "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n",
    "\n",
    ">>> all(i < 10 for i in [1, 3, 5, 7])\n",
    "True\n",
    "\n",
    ">>> \", \".join(str(i) for i in [10, 200, 4000, 80000])\n",
    "'10, 200, 4000, 80000'\n",
    "```\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Takeaway**:\n",
    "\n",
    "A generator comprehension can be specified directly as an argument to a function, wherever a single iterable is expected as an input to that function.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Using Generator Comprehensions on the Fly**:\n",
    "\n",
    "In a single line, compute the sum of all of the odd-numbers in 0-100.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterating over generators using `next`\n",
    "The built-in function `next` allows you manually \"request\" the next member of a generator, or more generally, any kind of *iterator*. Calling `next` on an exhausted iterator will raise a `StopIteration` signal.\n",
    "\n",
    "```python\n",
    "# consuming an iterator using `next`\n",
    ">>> short_gen = (i/2 for i in [1, 2, 3])\n",
    "\n",
    ">>> next(short_gen)\n",
    "0.5\n",
    "\n",
    ">>> next(short_gen)\n",
    "1.0\n",
    "\n",
    ">>> next(short_gen)\n",
    "1.5\n",
    "\n",
    ">>> next(short_gen)\n",
    "StopIteration                             \n",
    "Traceback (most recent call last)\n",
    "<ipython-input-5-ed60a54ccf0b> in <module>()\n",
    "----> 1 next(short_gen)\n",
    "\n",
    "StopIteration:\n",
    "```\n",
    "\n",
    "This is a great tool for retrieving content from a generator, or any iterator, without having to perform a for-loop over it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterables vs. Iterators\n",
    "This subsection is *not* essential to your basic understanding of the material. I am including it to prevent this text from being misleading to those who already know quite a bit about Python. **This is a bit advanced, feel free to skip it...**\n",
    "\n",
    "There is a bit of confusing terminology to be cleared up: an iterable is not the same thing as an iterator.\n",
    "\n",
    "An *iterator* object stores its current state of iteration and \"yields\" each of its members in order, on demand via `next`, until it is exhausted. As we've seen, a generator is an example of an iterator. We now must understand that every iterator is an iterable, but not every iterable is an iterator.\n",
    "\n",
    "An *iterable* is an object that *can* be iterated over but does not necessarily have all the machinery of an iterator. For example, sequences (e.g lists, tuples, and strings) and other containers (e.g. dictionaries and sets) do not keep track of their own state of iteration. Thus you cannot call `next` on one of these outright: \n",
    "\n",
    "```python\n",
    "# a list is an example of an iterable that is *not*\n",
    "# an iterator - you cannot call `next` on it.\n",
    ">>> x = [1, 2, 3]\n",
    ">>> next(x)\n",
    "---------------------------------------------------------------------------\n",
    "TypeError                                 Traceback (most recent call last)\n",
    "<ipython-input-19-b9d20096048c> in <module>()\n",
    "----> 1 next([1,2])\n",
    "\n",
    "TypeError: 'list' object is not an iterator\n",
    "```\n",
    "In order to iterate over, say, a list you must first pass it to the built-in `iter` function. This function will return an *iterator* for that list, which stores its state of iteration and the instructions to yield each one of the list's members:\n",
    "```python\n",
    "# any iterable can be fed to `iter`\n",
    "# to produce an iterator for that object\n",
    ">>> x = [1, 2, 3]\n",
    ">>> x_it = iter(x)  # `x_it` is an iterator\n",
    ">>> next(x_it)\n",
    "1\n",
    ">>> next(x_it)\n",
    "2\n",
    ">>> next(x_it)\n",
    "3\n",
    "```\n",
    "In this way, a list is an *iterable* but not an *iterator*, which is also the case for tuples, strings, sets, and dictionaries.\n",
    "\n",
    "Python actually creates an iterator \"behind the scenes\", whenever you perform a for-loop over an iterable like a list. It feeds that iterable to `iter`, and then proceeds to call `next` on the resulting iterator for each of the for-loop's iterations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List & Tuple Comprehensions\n",
    "Using generator comprehensions to initialize lists is so useful that Python actually reserves a specialized syntax for it, known as the list comprehension. A **list comprehension** is a syntax for constructing a list, which exactly mirrors the generator comprehension syntax:\n",
    "\n",
    "```\n",
    "[<expression> for <var> in <iterable> {if <condition}]\n",
    "```\n",
    "\n",
    "For example, if we want to create a list of square-numbers, we can simply write:\n",
    "```python\n",
    "# a simple list comprehension\n",
    ">>> [i**2 for i in range(10)]\n",
    "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n",
    "```\n",
    "\n",
    "This produces the exact same result as feeding the `list` function a generator comprehension. However, using a list comprehension is slightly more efficient than is feeding the `list` function a generator comprehension.\n",
    "\n",
    "Let's appreciate how economical list comprehensions are. The following code stores words that contain the letter \"o\", in a list:\n",
    "```python\n",
    "words_with_o = []\n",
    "word_collection = ['Python', 'Like', 'You', 'Mean', 'It']\n",
    "\n",
    "for word in word_collection:\n",
    "    if \"o\" in word.lower():\n",
    "        words_with_o.append(word)\n",
    "```\n",
    "\n",
    "This can be written in a single line, using a list comprehension:\n",
    "```python\n",
    ">>> word_collection = ['Python', 'Like', 'You', 'Mean', 'It']\n",
    ">>> words_with_o = [word for word in word_collection if \"o\" in word.lower()]\n",
    ">>> words_with_o\n",
    "['Python', 'You']\n",
    "```\n",
    "\n",
    "Tuples can be created using comprehension expressions too, but we must explicitly invoke the `tuple` constructor since parentheses are already reserved for defining a generator-comprehension.\n",
    "```python\n",
    "# creating a tuple using a comprehension expression\n",
    ">>> tuple(i**2 for i in range(5))\n",
    "(0, 1, 4, 9, 16)\n",
    "```\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Takeaway**:\n",
    "\n",
    " The comprehensions-statement is an extremely useful syntax for creating simple and complicated lists and tuples alike.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nesting comprehensions\n",
    "It can be useful to nest comprehension expressions within one another, although this should be used sparingly.\n",
    "```python\n",
    "# Nested list comprehensions.\n",
    "# This creates a 3x4 \"matrix\" (list of lists) of zeros.\n",
    ">>> [[0 for col in range(4)] for row in range(3)]\n",
    "[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: List Comprehensions**:\n",
    "\n",
    "Use a list comprehension to create a list that contains the string \"hello\" 100 times.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Fancier List Comprehensions**:\n",
    "\n",
    "Use the inline `if-else` statement (discussed earlier in this module), along with a list comprehension, to create the list:\n",
    "\n",
    "```python\n",
    "['hello',\n",
    " 'goodbye',\n",
    " 'hello',\n",
    " 'goodbye',\n",
    " 'hello',\n",
    " 'goodbye',\n",
    " 'hello',\n",
    " 'goodbye',\n",
    " 'hello',\n",
    " 'goodbye']\n",
    "```\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Tuple Comprehensions**:\n",
    "\n",
    "Use a tuple-comprehension to extract comma-separated numbers from a string, converting them into a tuple of floats. I.e. `\"3.2,2.4,99.8\"` should become `(3.2, 2.4, 99.8)`. You will want to use the built-in string function [str.split](https://docs.python.org/3/library/stdtypes.html#str.split).\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Translating a For-Loop**:\n",
    "\n",
    "Replicate the functionality of the the following code by writing a list comprehension.\n",
    "```python\n",
    "# skip all non-lowercased letters (including punctuation)\n",
    "# append 1 if lowercase letter is equal to \"o\"\n",
    "# append 0 if lowercase letter is not \"o\"\n",
    "out = []\n",
    "for i in \"Hello. How Are You?\":\n",
    "    if i.islower():\n",
    "        out.append(1 if i == \"o\" else 0)\n",
    "```\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Reading Comprehension: Memory Efficiency**:\n",
    "\n",
    "Is there any difference in performance between the following expressions?\n",
    "\n",
    "```python\n",
    "# feeding `sum` a generator comprehension\n",
    "sum(1/n for n in range(1, 101))\n",
    "```\n",
    "\n",
    "```python\n",
    "# feeding `sum` a list comprehension\n",
    "sum([1/n for n in range(1, 101)])\n",
    "```\n",
    "\n",
    "Is one expression preferable over the other? Why?\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Links to Official Documentation\n",
    "\n",
    "- [Generator definition](https://docs.python.org/3/glossary.html#term-generator)\n",
    "- [range](https://docs.python.org/3/library/stdtypes.html#typesseq-range)\n",
    "- [Generator comprehension expressions](https://docs.python.org/3/tutorial/classes.html#generator-expressions)\n",
    "- [Iterator definition](https://docs.python.org/3/glossary.html#term-iterator)\n",
    "- [next](https://docs.python.org/3/library/functions.html#next)\n",
    "- [iter](https://docs.python.org/3/library/functions.html#iter)\n",
    "- [List comprehensions](https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions)\n",
    "- [Nested list comprehensions](https://docs.python.org/3/tutorial/datastructures.html#nested-list-comprehensions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Comprehension Exercise Solutions:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using range: Solution**\n",
    "```python\n",
    "# start=10, stop=0 (excluded), step-size=-1\n",
    ">>> for i in range(10, 0, -1):\n",
    ">>>     print(i, end=\" \") # the \"end\" parameter is to avoid each value taking up a new line\n",
    "10 9 8 7 6 5 4 3 2 1 \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Writing a Generator Comprehension: Solution**\n",
    "```python\n",
    ">>> g = ((n, n+2) for n in range(6) if n != 3)\n",
    ">>> list(g) # convert into a list to print values\n",
    "[(0, 2), (1, 3), (2, 4), (4, 6), (5, 7)]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using Generator Comprehensions on the Fly: Solution**\n",
    "```python\n",
    ">>> sum(range(1, 101, 2))\n",
    "2500\n",
    "```\n",
    "\n",
    "or\n",
    "\n",
    "```python\n",
    ">>> sum(i for i in range(101) if i%2 != 0)\n",
    "2500\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**List Comprehensions: Solution**\n",
    "```python\n",
    ">>> [\"hello\" for i in range(100)]\n",
    "['hello', 'hello', ..., 'hello', 'hello'] # 100 hello's\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fancier List Comprehensions: Solution**\n",
    "```python\n",
    ">>> [(\"hello\" if i%2 == 0 else \"goodbye\") for i in range(10)]\n",
    "['hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye', 'hello', 'goodbye']\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tuple Comprehension: Solution**\n",
    "```python\n",
    ">>> string_of_nums = \"3.2, 2.4, 99.8\"\n",
    ">>> tuple(float(i) for i in string_of_nums.split(\",\"))\n",
    "(3.2, 2.4, 99.8)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Translating a For-Loop: Solution**\n",
    "```python\n",
    ">>> out = [(1 if i == \"o\" else 0) for i in \"Hello. How Are You?\" if i.islower()]\n",
    ">>> out\n",
    "[0, 0, 0, 1, 1, 0, 0, 0, 1, 0]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Memory Efficiency: Solution**\n",
    "\n",
    "It is preferable to use the generator expression `sum(1/n for n in range(1, 101))`, rather than the list comprehension `sum([1/n for n in range(1, 101)])`. Using a list comprehension unnecessarily creates a list of the one hundred numbers, in memory, before feeding the list to `sum`. The generator expression need only produce a single value at a time, as `sum` iterates over it."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "text_representation": {
    "extension": ".md",
    "format_name": "markdown",
    "format_version": "1.3",
    "jupytext_version": "1.13.6"
   }
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}