docs/beta/notebooks/EvoGrammarFuzzer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Evolutionary Grammar Fuzzing\n",
    "\n",
    "In this chapter, we introduce how to implement [search-based test generation](SearchBasedFuzzing.ipynb) on grammars, using _genetic improvement_ operators such as mutation and cross-over on derivation trees."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Prerequisites**\n",
    "\n",
    "* You should have read the [chapter on search-based test generation](SearchBasedFuzzer.ipynb).\n",
    "* You should have read the [chapter on recombining inputs](LangFuzzer.ipynb).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import fuzzingbook_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import SearchBasedFuzzer\n",
    "import LangFuzzer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Grammar-Based Mutation\n",
    "\n",
    "General idea: Take a derivation tree and a matching grammar; apply a random mutation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from Grammars import EXPR_GRAMMAR\n",
    "from GrammarFuzzer import display_tree\n",
    "from Parser import EarleyParser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"230pt\" height=\"431pt\"\n",
       " viewBox=\"0.00 0.00 230.00 431.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 427)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-427 226,-427 226,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-411.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;start&gt;</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-360.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;expr&gt;</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M97.5,-407.9748C97.5,-401.195 97.5,-390.966 97.5,-382.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.0001,-382.0561 97.5,-372.0561 94.0001,-382.0561 101.0001,-382.0561\"/>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<text text-anchor=\"middle\" x=\"47.5\" y=\"-309.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;term&gt;</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M90.1223,-356.9748C82.7369,-349.4417 71.1768,-337.6503 61.958,-328.2472\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"64.4079,-325.7466 54.9079,-321.0561 59.4094,-330.6471 64.4079,-325.7466\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-309.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\"> + </text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>1&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M97.5,-356.9748C97.5,-350.195 97.5,-339.966 97.5,-331.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.0001,-331.0561 97.5,-321.0561 94.0001,-331.0561 101.0001,-331.0561\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<text text-anchor=\"middle\" x=\"145.5\" y=\"-309.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;expr&gt;</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>1&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M104.5826,-356.9748C111.6017,-349.517 122.5487,-337.8857 131.3541,-328.53\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"134.0834,-330.7369 138.3884,-321.0561 128.986,-325.9393 134.0834,-330.7369\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<text text-anchor=\"middle\" x=\"40.5\" y=\"-258.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;factor&gt;</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M46.4671,-305.9748C45.5366,-299.195 44.1326,-288.966 42.9202,-280.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"46.3645,-279.4873 41.5371,-270.0561 39.4295,-280.4392 46.3645,-279.4873\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<text text-anchor=\"middle\" x=\"27.5\" y=\"-207.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;integer&gt;</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>3&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M38.5818,-254.9748C36.8344,-248.1196 34.1882,-237.7383 31.9198,-228.8392\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"35.2877,-227.8817 29.4261,-219.0561 28.5046,-229.6108 35.2877,-227.8817\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<text text-anchor=\"middle\" x=\"27.5\" y=\"-156.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;digit&gt;</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M27.5,-203.9748C27.5,-197.195 27.5,-186.966 27.5,-178.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"31.0001,-178.0561 27.5,-168.0561 24.0001,-178.0561 31.0001,-178.0561\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<text text-anchor=\"middle\" x=\"27.5\" y=\"-105.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">1</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M27.5,-152.9748C27.5,-146.195 27.5,-135.966 27.5,-127.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"31.0001,-127.0561 27.5,-117.0561 24.0001,-127.0561 31.0001,-127.0561\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<text text-anchor=\"middle\" x=\"146.5\" y=\"-258.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;term&gt;</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>8&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M145.6476,-305.9748C145.7805,-299.195 145.9811,-288.966 146.1543,-280.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"149.655,-280.1228 146.3518,-270.0561 142.6564,-279.9855 149.655,-280.1228\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-207.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;factor&gt;</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>9&#45;&gt;10</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M139.2699,-254.9748C132.1046,-247.517 120.9294,-235.8857 111.9406,-226.53\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"114.2119,-223.8423 104.7598,-219.0561 109.1642,-228.6921 114.2119,-223.8423\"/>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>14</title>\n",
       "<text text-anchor=\"middle\" x=\"147.5\" y=\"-207.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\"> * </text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;14 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>9&#45;&gt;14</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M146.6476,-254.9748C146.7805,-248.195 146.9811,-237.966 147.1543,-229.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"150.655,-229.1228 147.3518,-219.0561 143.6564,-228.9855 150.655,-229.1228\"/>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>15</title>\n",
       "<text text-anchor=\"middle\" x=\"194.5\" y=\"-207.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;term&gt;</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;15 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>9&#45;&gt;15</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M153.5826,-254.9748C160.6017,-247.517 171.5487,-235.8857 180.3541,-226.53\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"183.0834,-228.7369 187.3884,-219.0561 177.986,-223.9393 183.0834,-228.7369\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-156.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;integer&gt;</text>\n",
       "</g>\n",
       "<!-- 10&#45;&gt;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>10&#45;&gt;11</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M97.5,-203.9748C97.5,-197.195 97.5,-186.966 97.5,-178.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.0001,-178.0561 97.5,-168.0561 94.0001,-178.0561 101.0001,-178.0561\"/>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-105.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;digit&gt;</text>\n",
       "</g>\n",
       "<!-- 11&#45;&gt;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>11&#45;&gt;12</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M97.5,-152.9748C97.5,-146.195 97.5,-135.966 97.5,-127.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.0001,-127.0561 97.5,-117.0561 94.0001,-127.0561 101.0001,-127.0561\"/>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>13</title>\n",
       "<text text-anchor=\"middle\" x=\"97.5\" y=\"-54.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">2</text>\n",
       "</g>\n",
       "<!-- 12&#45;&gt;13 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>12&#45;&gt;13</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M97.5,-101.9748C97.5,-95.195 97.5,-84.966 97.5,-76.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"101.0001,-76.0561 97.5,-66.0561 94.0001,-76.0561 101.0001,-76.0561\"/>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>16</title>\n",
       "<text text-anchor=\"middle\" x=\"194.5\" y=\"-156.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;factor&gt;</text>\n",
       "</g>\n",
       "<!-- 15&#45;&gt;16 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>15&#45;&gt;16</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M194.5,-203.9748C194.5,-197.195 194.5,-186.966 194.5,-178.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.0001,-178.0561 194.5,-168.0561 191.0001,-178.0561 198.0001,-178.0561\"/>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>17</title>\n",
       "<text text-anchor=\"middle\" x=\"194.5\" y=\"-105.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;integer&gt;</text>\n",
       "</g>\n",
       "<!-- 16&#45;&gt;17 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>16&#45;&gt;17</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M194.5,-152.9748C194.5,-146.195 194.5,-135.966 194.5,-127.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.0001,-127.0561 194.5,-117.0561 191.0001,-127.0561 198.0001,-127.0561\"/>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>18</title>\n",
       "<text text-anchor=\"middle\" x=\"194.5\" y=\"-54.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">&lt;digit&gt;</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;18 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>17&#45;&gt;18</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M194.5,-101.9748C194.5,-95.195 194.5,-84.966 194.5,-76.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.0001,-76.0561 194.5,-66.0561 191.0001,-76.0561 198.0001,-76.0561\"/>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>19</title>\n",
       "<text text-anchor=\"middle\" x=\"194.5\" y=\"-3.8\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">3</text>\n",
       "</g>\n",
       "<!-- 18&#45;&gt;19 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>18&#45;&gt;19</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M194.5,-50.9748C194.5,-44.195 194.5,-33.966 194.5,-25.1331\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"198.0001,-25.0561 194.5,-15.0561 191.0001,-25.0561 198.0001,-25.0561\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x11ed46518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "parser = EarleyParser(EXPR_GRAMMAR)\n",
    "tree, *_ = parser.parse(\"1 + 2 * 3\")\n",
    "display_tree(tree)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "1. Pick any node in the tree\n",
    "2. Produce a new expansion.\n",
    "\n",
    "We have seen this for `LangFuzzer` already, right?\n",
    "\n",
    "How about we factor this out (from the Parser notebook), and have two notebook on mutational (and genetic fuzzing):\n",
    "\n",
    "1. `LangFuzzer` – a chapter on\n",
    "    * Mutating trees (randomly)\n",
    "    * Mutating trees from a given population (the LangFuzz approach)\n",
    "    * Tree recombination (and crossover)\n",
    "    \n",
    "2. `EvoGrammarFuzzer` – a chapter on\n",
    "    * Genetic improvement (using coverage only)\n",
    "    * Genetic improvement (using a fitness function from search-based fuzzing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def mutate_tree(tree, grammar):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Grammar-Based Crossover"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Lessons Learned\n",
    "\n",
    "* _Lesson one_\n",
    "* _Lesson two_\n",
    "* _Lesson three_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Next Steps\n",
    "\n",
    "_Link to subsequent chapters (notebooks) here, as in:_\n",
    "\n",
    "* [use _mutations_ on existing inputs to get more valid inputs](MutationFuzzer.ipynb)\n",
    "* [use _grammars_ (i.e., a specification of the input format) to get even more valid inputs](Grammars.ipynb)\n",
    "* [reduce _failing inputs_ for efficient debugging](Reducer.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Background\n",
    "\n",
    "_Cite relevant works in the literature and put them into context, as in:_\n",
    "\n",
    "The idea of ensuring that each expansion in the grammar is used at least once goes back to Burkhardt \\cite{Burkhardt1967}, to be later rediscovered by Paul Purdom \\cite{Purdom1972}."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exercises\n",
    "\n",
    "_Close the chapter with a few exercises such that people have things to do.  To make the solutions hidden (to be revealed by the user), have them start with_\n",
    "\n",
    "```markdown\n",
    "**Solution.**\n",
    "```\n",
    "\n",
    "_Your solution can then extend up to the next title (i.e., any markdown cell starting with `#`)._\n",
    "\n",
    "_Running `make metadata` will automatically add metadata to the cells such that the cells will be hidden by default, and can be uncovered by the user.  The button will be introduced above the solution._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise 1: _Title_\n",
    "\n",
    "_Text of the exercise_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "cell_style": "center",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Some code that is part of the exercise\n",
    "pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    },
    "solution2": "hidden",
    "solution2_first": true
   },
   "source": [
    "_Some more text for the exercise_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "source": [
    "**Solution.** _Some text for the solution_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "cell_style": "split",
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Some code for the solution\n",
    "2 + 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    },
    "solution2": "hidden"
   },
   "source": [
    "_Some more text for the solution_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "subslide"
    },
    "solution": "hidden",
    "solution2": "hidden",
    "solution2_first": true,
    "solution_first": true
   },
   "source": [
    "### Exercise 2: _Title_\n",
    "\n",
    "_Text of the exercise_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "skip"
    },
    "solution": "hidden",
    "solution2": "hidden"
   },
   "source": [
    "**Solution.** _Solution for the exercise_"
   ]
  }
 ],
 "metadata": {
  "ipub": {
   "bibliography": "fuzzingbook.bib",
   "toc": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 2
}