docs/beta/notebooks/IllustratedCode.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "415a53f6-d901-4cdf-b544-d0b65db4fccb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Illustrated Code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46fcd836-2d4e-48be-93d0-ca9dc010be50",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Building Software in a Literate Way"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2432f801-63aa-401f-b319-f3be63f67ea8",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This notebook contains the code examples for Andreas Zeller's keynotes:\n",
    "\n",
    "* \"Illustrated Code: Building Software in a Literate Way\" at ASE 2021; and\n",
    "* \"Illustrated Code; What Software Engineering can Learn from Research Software\" at SE 2022."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7cf9d7d-39cd-40b5-ade9-a108e94f09bc",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Go and\n",
    "\n",
    "* Read the [talk slides](https://www.slideshare.net/andreas.zeller/illustrated-code-ase-2021)\n",
    "* [Interact](https://mybinder.org/v2/gh/uds-se/debuggingbook/master?urlpath=lab/tree/docs/notebooks/IllustratedCode.ipynb) with the notebook "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7b6821d-251e-4e8a-94e2-b1c00a77c360",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Talk Abstract\n",
    "\n",
    "Notebooks – rich, interactive documents that join together code, documentation, and outputs – are all the rage with data scientists. But can they be used for actual software development? In this talk, I share experiences from authoring two interactive textbooks – fuzzingbook.org and debuggingbook.org – and show how notebooks not only serve for exploring and explaining code and data, but also how they can be used as software modules, integrating self-checking documentation, tests, and tutorials all in one place. The resulting software focuses on the essential, is well-documented, highly maintainable, easily extensible, and has a much higher shelf life than the \"duct tape and wire” prototypes frequently found in research and beyond."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00a5f88b-8bf0-4bef-aa84-7a6460a15038",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<i>\n",
    "    <a href=\"https://andreas-zeller.info\">Andreas Zeller</a> is faculty at the \n",
    "    <a href=\"https://www.cispa.de\" target=_blank>CISPA Helmholtz Center for Information Security</a> and professor for Software Engineering at \n",
    "    <a href=\"https://saarland-informatics-campus.de/en/\" target=_blank>Saarland University</a>, both in Saarbrücken, Germany.  \n",
    "    His <a href=\"https://scholar.google.com/citations?user=-Qytr_YAAAAJ&hl=en&oi=ao\" target=_blank>research</a> \n",
    "    on automated debugging, mining software archives, specification mining, and security testing has proven <a href=\"/assets/ZellerCV.pdf\">highly influential</a>.  \n",
    "    Zeller is an <a href=\"https://awards.acm.org/fellows\" target=_blank>ACM Fellow</a> and holds an <a href=\"https://www.sigsoft.org/awards/outstandingResearchAward.html\" target=_blank>ACM SIGSOFT Outstanding Research Award</a>.\n",
    "    </i>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0804838c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Some Support Code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "926cf9fc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.781505Z",
     "iopub.status.busy": "2024-01-17T21:56:22.781385Z",
     "iopub.status.idle": "2024-01-17T21:56:22.812405Z",
     "shell.execute_reply": "2024-01-17T21:56:22.812121Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import bookutils.setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8f6f6d17",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.814195Z",
     "iopub.status.busy": "2024-01-17T21:56:22.814054Z",
     "iopub.status.idle": "2024-01-17T21:56:22.907021Z",
     "shell.execute_reply": "2024-01-17T21:56:22.906734Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from Tracer import Tracer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "61fcb3aa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.908856Z",
     "iopub.status.busy": "2024-01-17T21:56:22.908739Z",
     "iopub.status.idle": "2024-01-17T21:56:22.910582Z",
     "shell.execute_reply": "2024-01-17T21:56:22.910349Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from typing import Any, Callable\n",
    "from types import FrameType\n",
    "from inspect import signature, getmembers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f55d262",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sequence Diagrams with Mermaid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8946c67b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.912201Z",
     "iopub.status.busy": "2024-01-17T21:56:22.912096Z",
     "iopub.status.idle": "2024-01-17T21:56:22.915026Z",
     "shell.execute_reply": "2024-01-17T21:56:22.914763Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class SequenceDiagramTracer(Tracer):\n",
    "    def __init__(self, client='Client', server='Server'):\n",
    "        super().__init__()\n",
    "        self.lines = []\n",
    "        self.client = client\n",
    "        self.server = server\n",
    "\n",
    "    def traceit(self, frame: FrameType, event: str, arg: Any):\n",
    "        if event == 'call':\n",
    "            func = frame.f_code.co_name\n",
    "            args = frame.f_locals\n",
    "            args_line = \", \".join(reversed([var + \"=\" + repr(args[var]) \n",
    "                                            for var in args]))\n",
    "            line = f'{self.client}->>+{self.server}: {func}({args_line})'\n",
    "            self.lines.append(line)\n",
    "\n",
    "        if event == 'return':\n",
    "            line = f'{self.server}-->>-{self.client}'\n",
    "            if arg is not None:\n",
    "                line += f': {repr(arg)}'\n",
    "            self.lines.append(line)\n",
    "\n",
    "    def _repr_markdown_(self) -> str:\n",
    "        return '\\n'.join(['```mermaid'] +\n",
    "                         ['sequenceDiagram'] + \n",
    "                         self.lines + \n",
    "                         ['```'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "994b6846",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Class Diagrams with Mermaid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "11cad4bc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.916529Z",
     "iopub.status.busy": "2024-01-17T21:56:22.916422Z",
     "iopub.status.idle": "2024-01-17T21:56:22.918959Z",
     "shell.execute_reply": "2024-01-17T21:56:22.918697Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class ClassDiagram():\n",
    "    def __init__(self, cls):\n",
    "        self.cls = cls\n",
    "\n",
    "    def methods_str(self):\n",
    "        members = [(name, fun) for (name, fun) in getmembers(self.cls)\n",
    "                   if not name.startswith('_')]\n",
    "        attributes = '\\n'.join([f'  -{name} = {repr(member)}'\n",
    "                                for (name, member) in members if not callable(member)\n",
    "                                ])\n",
    "\n",
    "        methods = '\\n'.join([f'  +{name}{str(signature(member)).replace(\" -> \", \" \")}'\n",
    "                             for (name, member) in members if callable(member)\n",
    "                             ])\n",
    "        return attributes + '\\n' + methods\n",
    "\n",
    "    def _repr_markdown_(self) -> str:\n",
    "        return f\"\"\"\n",
    "```mermaid\n",
    "classDiagram\n",
    "direction TD\n",
    "class Server {{\n",
    "  {self.methods_str()}\n",
    "}}\n",
    "```\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a68dfd7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Illustrated Code: Building Software in a Literate Way"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8ebeddb-6ded-4a7f-8577-a6bbb0284835",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Jupyter Demo: Factorials\n",
    "\n",
    "We do a bit of Jupyter demo. Double-click on a cell to edit it. Press `Shift`+`Return` to execute/render it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "347da6bd-f086-4a36-893a-887ba7584f06",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`factorial(n)` computes the factorial of n, that is $n! = \\prod_{i=1}^n i = 1 \\times 2 \\times \\dots \\times n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4ff1de6d-d562-44f8-9adf-539a67e549ea",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.920669Z",
     "iopub.status.busy": "2024-01-17T21:56:22.920565Z",
     "iopub.status.idle": "2024-01-17T21:56:22.922243Z",
     "shell.execute_reply": "2024-01-17T21:56:22.922000Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def factorial(n: int) -> int:\n",
    "    return 1 if n <= 1 else n * factorial(n - 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0de2c1b0-19ad-4869-ba03-e217c7ae901e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.923727Z",
     "iopub.status.busy": "2024-01-17T21:56:22.923622Z",
     "iopub.status.idle": "2024-01-17T21:56:22.926692Z",
     "shell.execute_reply": "2024-01-17T21:56:22.926439Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factorial(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "cc9e2b4f-8bcb-4f6b-9d13-2c1393298022",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.928411Z",
     "iopub.status.busy": "2024-01-17T21:56:22.928318Z",
     "iopub.status.idle": "2024-01-17T21:56:22.930036Z",
     "shell.execute_reply": "2024-01-17T21:56:22.929772Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "assert factorial(3) == 6"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc85632b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Can Programming be Liberated from the Typewriter Style?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c8d2c00",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We define a function `middle(x, y, z)` that returns the \"middle\" of three integers $x$, $y$, and $z$ – i.e. the one that is neither the maximum nor the minimum of the three.\n",
    "\n",
    "We show how to use notebooks to\n",
    "\n",
    "* document its interface\n",
    "* provide a specification\n",
    "* provide rationales and experiments\n",
    "* include tests\n",
    "* include architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "838e022e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Interface"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c79fa4a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us define an interface for `middle()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f9afe3f4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.931621Z",
     "iopub.status.busy": "2024-01-17T21:56:22.931534Z",
     "iopub.status.idle": "2024-01-17T21:56:22.933322Z",
     "shell.execute_reply": "2024-01-17T21:56:22.933060Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def middle_i(x: int, y: int, z: int) -> int:\n",
    "    \"\"\"Return the middle of three numbers x, y, z\"\"\"\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "518e38c5-117a-4556-a1bd-0c5b889cc832",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "* Standard way of documenting things\n",
    "* No formal spec (what is \"the middle\" here?); no context; no rationale\n",
    "* No usage example\n",
    "* No implementation (yet)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c50208d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Specification\n",
    "\n",
    "Here's an (executable) specification of `middle()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9ae659b4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.934789Z",
     "iopub.status.busy": "2024-01-17T21:56:22.934709Z",
     "iopub.status.idle": "2024-01-17T21:56:22.936394Z",
     "shell.execute_reply": "2024-01-17T21:56:22.936146Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def middle_spec(x: int, y: int, z: int) -> int:\n",
    "    return sorted([x, y, z])[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30a21c16",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "This specification is executable, so we can easily include examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0f39b095",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.937868Z",
     "iopub.status.busy": "2024-01-17T21:56:22.937783Z",
     "iopub.status.idle": "2024-01-17T21:56:22.939994Z",
     "shell.execute_reply": "2024-01-17T21:56:22.939727Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_spec(5, 3, 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6c3f15b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Or just write the examples as assertions, so we can use them as tests later:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e7e2652f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.941529Z",
     "iopub.status.busy": "2024-01-17T21:56:22.941444Z",
     "iopub.status.idle": "2024-01-17T21:56:22.943175Z",
     "shell.execute_reply": "2024-01-17T21:56:22.942911Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "assert middle_spec(5, 4, 7) == 5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e49414e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Of course, your specification can also include all sorts of diagrams. (Install [jupyterlab-markup](https://opensourcelibs.com/lib/jupyterlab-markup) for this.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54bfae63",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```mermaid\n",
    "sequenceDiagram\n",
    "    Client->>+Server: middle(5, 4, 7)\n",
    "    Server-->>-Client: 5\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "406e648d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For the record, this diagram is created with five lines of Markdown:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c29aa46",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "    ```mermaid\n",
    "    sequenceDiagram\n",
    "    Client->>+Server: middle(5, 4, 7)\n",
    "    Server-->>-Client: 5\n",
    "    ```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "832b7996",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Implementation\n",
    "\n",
    "Let us now provide an efficient implementation for `middle()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6a08f669",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.944733Z",
     "iopub.status.busy": "2024-01-17T21:56:22.944650Z",
     "iopub.status.idle": "2024-01-17T21:56:22.946721Z",
     "shell.execute_reply": "2024-01-17T21:56:22.946469Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def middle(x: int, y: int, z: int) -> int:\n",
    "    \"\"\"Return the middle of three numbers x, y, z\"\"\"\n",
    "    if y < z:\n",
    "        if x < y:\n",
    "            return y\n",
    "        elif x < z:\n",
    "            return x\n",
    "    else:\n",
    "        if x > y:\n",
    "            return y\n",
    "        elif x > z:\n",
    "            return x\n",
    "    return z"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "935caee3-80a7-4827-993c-e3208f5ecdaa",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Once written, this is executable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "30138b67-8ece-4615-ab64-ecfef2ab0921",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.948199Z",
     "iopub.status.busy": "2024-01-17T21:56:22.948112Z",
     "iopub.status.idle": "2024-01-17T21:56:22.950237Z",
     "shell.execute_reply": "2024-01-17T21:56:22.949980Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle(5, 4, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ba97f27-7951-4974-8664-5fdefd1f910c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Tests and results become part of the doc!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd25ea47",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Rationale\n",
    "\n",
    "Why do we implement `middle()` as above, rather than using the much shorter `middle_spec()` code? Because it is about _twice as fast_:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "6640c751",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.951838Z",
     "iopub.status.busy": "2024-01-17T21:56:22.951753Z",
     "iopub.status.idle": "2024-01-17T21:56:22.953514Z",
     "shell.execute_reply": "2024-01-17T21:56:22.953267Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import time\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2414a718",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.955106Z",
     "iopub.status.busy": "2024-01-17T21:56:22.955020Z",
     "iopub.status.idle": "2024-01-17T21:56:22.957403Z",
     "shell.execute_reply": "2024-01-17T21:56:22.957113Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def middle_benchmark(middle_fun: Callable[[int, int, int], int],\n",
    "                     n: int = 1000000) -> float:\n",
    "    \"\"\"Return elapsed time for calling `middle_fun` `n` times\"\"\"\n",
    "    elapsed = 0.0\n",
    "\n",
    "    for i in range(n):\n",
    "        x = random.randint(0, 1000)\n",
    "        y = random.randint(0, 1000)\n",
    "        z = random.randint(0, 1000)\n",
    "        start = time.perf_counter()\n",
    "        _ = middle_fun(x, y, z)\n",
    "        end = time.perf_counter()\n",
    "        elapsed += (end - start)\n",
    "\n",
    "    return elapsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c7e7c45b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:22.958990Z",
     "iopub.status.busy": "2024-01-17T21:56:22.958901Z",
     "iopub.status.idle": "2024-01-17T21:56:24.493911Z",
     "shell.execute_reply": "2024-01-17T21:56:24.493606Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.15140068903929205"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_elapsed = middle_benchmark(middle)\n",
    "middle_elapsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "8f29aa90",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:24.495627Z",
     "iopub.status.busy": "2024-01-17T21:56:24.495510Z",
     "iopub.status.idle": "2024-01-17T21:56:26.168790Z",
     "shell.execute_reply": "2024-01-17T21:56:26.168456Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25074575236430974"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle_spec_elapsed = middle_benchmark(middle_spec)\n",
    "middle_spec_elapsed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af1cacfd-187b-458a-8ac3-2cf0425dc6ee",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The document can include all these experiments and their results as a rationale, as above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5820bfde-d95f-4026-8131-413977deec16",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The document can also discuss and evaluate more alternatives, ",
    "\n",
    "reproducing the thoughts and experiments of the original programmer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7509c7fa-6c02-42f4-91f3-5b518258a269",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can have the document check automatically whether the rationale holds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "a1864a8f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.170558Z",
     "iopub.status.busy": "2024-01-17T21:56:26.170440Z",
     "iopub.status.idle": "2024-01-17T21:56:26.172323Z",
     "shell.execute_reply": "2024-01-17T21:56:26.172045Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "assert middle_elapsed < middle_spec_elapsed, \"Inconsistent doc\"\n",
    "assert middle_elapsed * 1.2 < middle_spec_elapsed, \"Inconsistent doc\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfb977f1-d172-43a3-b11e-d982fc752cf6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This ensures consistency between text and code."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5c9d41d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Tests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9f9bb23-76af-4b85-b95a-4a597906eed0",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Tests can be written as additional examples on how the code should work:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "925fe192",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Regular Tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b41cbfc4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.174140Z",
     "iopub.status.busy": "2024-01-17T21:56:26.174011Z",
     "iopub.status.idle": "2024-01-17T21:56:26.175862Z",
     "shell.execute_reply": "2024-01-17T21:56:26.175597Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "assert middle(1, 2, 3) == 2\n",
    "assert middle(3, 1, 2) == 2\n",
    "assert middle(2, 3, 1) == 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "983eccbc-57d0-4ea5-8694-b0055cf955af",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "If a test fails, that's the same as an example failing. (And examples act as tests.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84a358b0-1d3b-4e83-9a4c-8cdda336fa20",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One can analyze (and report) test performance, again in the document – for instance, measure the coverage of our code (`*` = line is covered during testing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "bfdb7af4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.177494Z",
     "iopub.status.busy": "2024-01-17T21:56:26.177384Z",
     "iopub.status.idle": "2024-01-17T21:56:26.691259Z",
     "shell.execute_reply": "2024-01-17T21:56:26.690951Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from StatisticalDebugger import CoverageCollector, code_with_coverage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1d518e51",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.693212Z",
     "iopub.status.busy": "2024-01-17T21:56:26.693050Z",
     "iopub.status.idle": "2024-01-17T21:56:26.695030Z",
     "shell.execute_reply": "2024-01-17T21:56:26.694773Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with CoverageCollector() as c:\n",
    "    assert middle(1, 2, 3) == 2\n",
    "    assert middle(3, 1, 2) == 2\n",
    "    assert middle(2, 3, 1) == 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "eda3f55c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.696592Z",
     "iopub.status.busy": "2024-01-17T21:56:26.696468Z",
     "iopub.status.idle": "2024-01-17T21:56:26.758721Z",
     "shell.execute_reply": "2024-01-17T21:56:26.758431Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   1 * \u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x: \u001b[36mint\u001b[39;49;00m, y: \u001b[36mint\u001b[39;49;00m, z: \u001b[36mint\u001b[39;49;00m) -> \u001b[36mint\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "   2   \u001b[37m    \u001b[39;49;00m\u001b[33m\"\"\"Return the middle of three numbers x, y, z\"\"\"\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "   3 *     \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "   4 *         \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "   5 *             \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "   6 *         \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "   7               \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "   8       \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "   9 *         \u001b[34mif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "  10               \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "  11 *         \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "  12 *             \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "  13 *     \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n"
     ]
    }
   ],
   "source": [
    "code_with_coverage(middle, c.coverage())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a156f9b-e9b8-4b07-833c-5e2f9df6eae8",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It seems we need to add more tests to cover all lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "411c8caa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.760534Z",
     "iopub.status.busy": "2024-01-17T21:56:26.760421Z",
     "iopub.status.idle": "2024-01-17T21:56:26.762075Z",
     "shell.execute_reply": "2024-01-17T21:56:26.761815Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "with c:\n",
    "    assert middle(2, 1, 3) == 2\n",
    "    assert middle(3, 2, 1) == 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7a7566f-ad9a-42aa-b400-0f9f24fed9c6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "And we achieve 100% coverage:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "f2a95572",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.763763Z",
     "iopub.status.busy": "2024-01-17T21:56:26.763652Z",
     "iopub.status.idle": "2024-01-17T21:56:26.794403Z",
     "shell.execute_reply": "2024-01-17T21:56:26.794148Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   1 * \u001b[34mdef\u001b[39;49;00m \u001b[32mmiddle\u001b[39;49;00m(x: \u001b[36mint\u001b[39;49;00m, y: \u001b[36mint\u001b[39;49;00m, z: \u001b[36mint\u001b[39;49;00m) -> \u001b[36mint\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "   2   \u001b[37m    \u001b[39;49;00m\u001b[33m\"\"\"Return the middle of three numbers x, y, z\"\"\"\u001b[39;49;00m\u001b[37m\u001b[39;49;00m\n",
      "   3 *     \u001b[34mif\u001b[39;49;00m y < z:\u001b[37m\u001b[39;49;00m\n",
      "   4 *         \u001b[34mif\u001b[39;49;00m x < y:\u001b[37m\u001b[39;49;00m\n",
      "   5 *             \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "   6 *         \u001b[34melif\u001b[39;49;00m x < z:\u001b[37m\u001b[39;49;00m\n",
      "   7 *             \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "   8       \u001b[34melse\u001b[39;49;00m:\u001b[37m\u001b[39;49;00m\n",
      "   9 *         \u001b[34mif\u001b[39;49;00m x > y:\u001b[37m\u001b[39;49;00m\n",
      "  10 *             \u001b[34mreturn\u001b[39;49;00m y\u001b[37m\u001b[39;49;00m\n",
      "  11 *         \u001b[34melif\u001b[39;49;00m x > z:\u001b[37m\u001b[39;49;00m\n",
      "  12 *             \u001b[34mreturn\u001b[39;49;00m x\u001b[37m\u001b[39;49;00m\n",
      "  13 *     \u001b[34mreturn\u001b[39;49;00m z\u001b[37m\u001b[39;49;00m\n"
     ]
    }
   ],
   "source": [
    "code_with_coverage(middle, c.coverage())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71553e6b-b401-4c89-8d0f-571acc2a2547",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Assumptions about coverage can be made in the document, too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "e65bc87c-4054-4bf1-b1cc-ca6462bd9fdb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.796419Z",
     "iopub.status.busy": "2024-01-17T21:56:26.796070Z",
     "iopub.status.idle": "2024-01-17T21:56:26.798082Z",
     "shell.execute_reply": "2024-01-17T21:56:26.797823Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "assert len(c.coverage()) >= 11"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "439e65a0",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Check against Spec"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f8856da-36db-4703-9a5d-4845dd9de5a6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One can check against the spec, again in the document. Here we compare `middle()` against `middle_spec()` with 100,000 random numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "892ab4c2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.799737Z",
     "iopub.status.busy": "2024-01-17T21:56:26.799628Z",
     "iopub.status.idle": "2024-01-17T21:56:26.974985Z",
     "shell.execute_reply": "2024-01-17T21:56:26.974688Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "for i in range(100000):\n",
    "    x = random.randint(0, 1000)\n",
    "    y = random.randint(0, 1000)\n",
    "    z = random.randint(0, 1000)\n",
    "    assert middle(x, y, z) == middle_spec(x, y, z)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c951aa49-3ae5-4ab2-a793-d01c65d7191d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "All these tests can be run (and debugged) right from the document."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "650d088e-15ab-4756-b51b-3a1855b10ad2",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "### Symbolic Verification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40aac919-ae89-49ce-b0a5-1f38c53e4730",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One can also include static checks or symbolic verification. Here, we encode the path constraints from the `middle()` code for the Z3 constraint solver:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "0f75535d-10ef-40be-bc0d-b7e65ad8f7ca",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.976955Z",
     "iopub.status.busy": "2024-01-17T21:56:26.976834Z",
     "iopub.status.idle": "2024-01-17T21:56:26.989495Z",
     "shell.execute_reply": "2024-01-17T21:56:26.989195Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from z3 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "845c3604-928e-40d7-a944-cb29bcaa1538",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.991169Z",
     "iopub.status.busy": "2024-01-17T21:56:26.991082Z",
     "iopub.status.idle": "2024-01-17T21:56:26.995190Z",
     "shell.execute_reply": "2024-01-17T21:56:26.994853Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "s = Solver()  # Create a Z3 solver with four variables\n",
    "x, y, z = Int('x'), Int('y'), Int('z')\n",
    "m = Int('middle')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "f51cde26-738e-42e3-aee0-ab94727bcb2f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:26.996906Z",
     "iopub.status.busy": "2024-01-17T21:56:26.996800Z",
     "iopub.status.idle": "2024-01-17T21:56:27.004211Z",
     "shell.execute_reply": "2024-01-17T21:56:27.003882Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "s.add(Implies(And(y < z, x < y), m == y))  # Encode the middle() constraints\n",
    "s.add(Implies(And(y < z, x >= y), m == x))\n",
    "s.add(Implies(And(y >= z, x > y), m == y))\n",
    "s.add(Implies(And(y >= z, x <= y), m == x))\n",
    "s.add(Implies(And(Not(x < y), Not(x < z), Not(x > y), Not(x > z)), m == z))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d787db2b-17d4-4f53-8a26-79001499d16c",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can actually prove correctness this way: Is it possible that `middle()` returns a wrong result? (no.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "0e4fd055-5ce2-40d0-bde2-17159842d829",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:27.006057Z",
     "iopub.status.busy": "2024-01-17T21:56:27.005922Z",
     "iopub.status.idle": "2024-01-17T21:56:27.011821Z",
     "shell.execute_reply": "2024-01-17T21:56:27.011497Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>unsat</b>"
      ],
      "text/plain": [
       "unsat"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.add(And(x < y, y < z, m != y))  # This shouldn't be possible\n",
    "s.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32105d27",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc577378",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can extract architecture diagrams such as a class diagram from code, always kept up to date:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "2fcac756",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:27.013586Z",
     "iopub.status.busy": "2024-01-17T21:56:27.013453Z",
     "iopub.status.idle": "2024-01-17T21:56:27.016067Z",
     "shell.execute_reply": "2024-01-17T21:56:27.015711Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "class Server():\n",
    "    state = 42\n",
    "\n",
    "    def middle(x: int, y: int, z: int) -> int:\n",
    "        return middle(x, y, z)\n",
    "\n",
    "    def min(x: int, y: int, z: int) -> int:\n",
    "        return min(x, y, z)\n",
    "\n",
    "    def max(x: int, y: int, z: int) -> int:\n",
    "        return max(x, y, z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "beb31a29",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:27.017709Z",
     "iopub.status.busy": "2024-01-17T21:56:27.017598Z",
     "iopub.status.idle": "2024-01-17T21:56:27.019866Z",
     "shell.execute_reply": "2024-01-17T21:56:27.019604Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "\n",
       "```mermaid\n",
       "classDiagram\n",
       "direction TD\n",
       "class Server {\n",
       "    -state = 42\n",
       "  +max(x: int, y: int, z: int) int\n",
       "  +middle(x: int, y: int, z: int) int\n",
       "  +min(x: int, y: int, z: int) int\n",
       "}\n",
       "```"
      ],
      "text/plain": [
       "<__main__.ClassDiagram at 0x166fffd00>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ClassDiagram(Server)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eca215de",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can extract dynamic diagrams from executions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "f8f6f833",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:27.021483Z",
     "iopub.status.busy": "2024-01-17T21:56:27.021374Z",
     "iopub.status.idle": "2024-01-17T21:56:27.024397Z",
     "shell.execute_reply": "2024-01-17T21:56:27.024035Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "```mermaid\n",
       "sequenceDiagram\n",
       "Client->>+Server: middle(z=20, y=50, x=30)\n",
       "Server-->>-Client: 30\n",
       "Client->>+Server: __del__(self=middle)\n",
       "Client->>+Server: ref(self=<z3.z3.Context object at 0x166fff3a0>)\n",
       "Server-->>-Client: <ContextObj object at 0x166ed1540>\n",
       "Client->>+Server: ref(self=<z3.z3.Context object at 0x166fff3a0>)\n",
       "Server-->>-Client: <ContextObj object at 0x166ed1540>\n",
       "Client->>+Server: as_ast(self=middle)\n",
       "Server-->>-Client: <Ast object at 0x166ed1840>\n",
       "Client->>+Server: Z3_dec_ref(_elems=<z3.z3core.Elementaries object at 0x166f83b20>, a1=<Ast object at 0x166ed1840>, a0=<ContextObj object at 0x166ed1540>)\n",
       "Client->>+Server: from_param(obj=<ContextObj object at 0x166ed1540>)\n",
       "Server-->>-Client: <ContextObj object at 0x166ed1540>\n",
       "Client->>+Server: from_param(obj=<Ast object at 0x166ed1840>)\n",
       "Server-->>-Client: <Ast object at 0x166ed1840>\n",
       "Server-->>-Client\n",
       "Server-->>-Client\n",
       "```"
      ],
      "text/plain": [
       "<__main__.SequenceDiagramTracer at 0x166fff040>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with SequenceDiagramTracer() as tracer:\n",
    "    m = middle(30, 50, 20)\n",
    "tracer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72eee76d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One may even compare this diagram with the one in the specification and flag mismatches."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c8474c2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "902aa290-cb93-42b1-9ee8-833df996461b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The document can contain instructions on how to run things. (Of course, these would be executable too, testing the tutorial.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ae701da",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To use `middle`, you need Python 3.9 or later. First install the `debuggingbook` module, available via the Python `pip` program:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "9b0ea356",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:27.026194Z",
     "iopub.status.busy": "2024-01-17T21:56:27.026084Z",
     "iopub.status.idle": "2024-01-17T21:56:29.214154Z",
     "shell.execute_reply": "2024-01-17T21:56:29.213745Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n",
      "pytest-asyncio 0.19.0 requires pytest>=6.1.0, but you have pytest 3.10.1 which is incompatible.\r\n",
      "pytest-html 3.1.1 requires pytest!=6.0.0,>=5.0, but you have pytest 3.10.1 which is incompatible.\r\n",
      "pytest-rerunfailures 10.2 requires pytest>=5.3, but you have pytest 3.10.1 which is incompatible.\r\n",
      "pytest-xdist 2.5.0 requires pytest>=6.2.0, but you have pytest 3.10.1 which is incompatible.\u001b[0m\u001b[31m\r\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "!pip install --quiet debuggingbook"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f48314b0",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Within `debuggingbook`, the `StatisticalDebugger` provides a `middle()` function, but it is buggy (as it serves to demonstrate statistical debugging). A correct version is available as `middle_fixed()`, which you can import as `middle()` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "610af3c3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.216251Z",
     "iopub.status.busy": "2024-01-17T21:56:29.216112Z",
     "iopub.status.idle": "2024-01-17T21:56:29.227913Z",
     "shell.execute_reply": "2024-01-17T21:56:29.227606Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from debuggingbook.StatisticalDebugger import middle_fixed as middle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "63acd34c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.229726Z",
     "iopub.status.busy": "2024-01-17T21:56:29.229608Z",
     "iopub.status.idle": "2024-01-17T21:56:29.231918Z",
     "shell.execute_reply": "2024-01-17T21:56:29.231666Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "middle(5, 4, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "4c91d747",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.233377Z",
     "iopub.status.busy": "2024-01-17T21:56:29.233271Z",
     "iopub.status.idle": "2024-01-17T21:56:29.235194Z",
     "shell.execute_reply": "2024-01-17T21:56:29.234950Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from z3 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "209c1dd9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.236675Z",
     "iopub.status.busy": "2024-01-17T21:56:29.236571Z",
     "iopub.status.idle": "2024-01-17T21:56:29.239650Z",
     "shell.execute_reply": "2024-01-17T21:56:29.239412Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "s = Solver()  # Create a Z3 solver with four variables\n",
    "x, y, z = Int('x'), Int('y'), Int('z')\n",
    "m = Int('middle')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "5e682dc1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.241229Z",
     "iopub.status.busy": "2024-01-17T21:56:29.241144Z",
     "iopub.status.idle": "2024-01-17T21:56:29.248137Z",
     "shell.execute_reply": "2024-01-17T21:56:29.247862Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "s.add(Implies(And(y < z, x < y), m == y))  # Encode the middle() constraints\n",
    "s.add(Implies(And(y < z, x >= y), m == x))\n",
    "s.add(Implies(And(y >= z, x > y), m == y))\n",
    "s.add(Implies(And(y >= z, x <= y), m == x))\n",
    "s.add(Implies(And(Not(x < y), Not(x < z), Not(x > y), Not(x > z)), m == z))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f553a126-a872-4a30-be2e-725d7b484905",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can actually prove correctness this way: Is it possible that `middle()` returns a wrong result? (no.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "3209a277",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.249801Z",
     "iopub.status.busy": "2024-01-17T21:56:29.249716Z",
     "iopub.status.idle": "2024-01-17T21:56:29.255331Z",
     "shell.execute_reply": "2024-01-17T21:56:29.255065Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>unsat</b>"
      ],
      "text/plain": [
       "unsat"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.add(And(x < y, y < z, m != y))  # This shouldn't be possible\n",
    "s.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e46a17b2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Q&A\n",
    "\n",
    "The document can contain sections with questions and answers. These would be managed by the public, and continuously ensure consistency.\n",
    "\n",
    "\n",
    "* [What's wrong with middle()?](#What_s_wrong_with_middle()?)\n",
    "* [I get a syntax error](#I_get_a_syntax_error)\n",
    "* [Can we have a `largest()` and `smallest()` function too?](#Can_we_have_a_largest()_and_smallest()_function_too?)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b409779a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### What's wrong with `middle()`?\n",
    "\n",
    "**Question.** I use `middle` from the `StatisticalDebugger` module. However, it doesn't seem to return the correct value. What am I doing wrong? -- novice@python.net"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "8d32059a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.256823Z",
     "iopub.status.busy": "2024-01-17T21:56:29.256735Z",
     "iopub.status.idle": "2024-01-17T21:56:29.259005Z",
     "shell.execute_reply": "2024-01-17T21:56:29.258749Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from StatisticalDebugger import middle\n",
    "middle(2, 1, 3)  # should be 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6c64ffb",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Best Answer (+10).** You need to import `middle_fixed` instead. -- expert@debugging.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "0e2cd21a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.260682Z",
     "iopub.status.busy": "2024-01-17T21:56:29.260558Z",
     "iopub.status.idle": "2024-01-17T21:56:29.262887Z",
     "shell.execute_reply": "2024-01-17T21:56:29.262585Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from StatisticalDebugger import middle_fixed as middle\n",
    "middle(2, 1, 3)  # should be 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "016a8099",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Answer (+5)** Don't use `middle(x, y, z)` -- just write `sorted([x, y, z])[1]` -- say.no.to@libraries.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "ee0020d2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.264453Z",
     "iopub.status.busy": "2024-01-17T21:56:29.264331Z",
     "iopub.status.idle": "2024-01-17T21:56:29.266595Z",
     "shell.execute_reply": "2024-01-17T21:56:29.266353Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorted([2, 1, 3])[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30d2df2a",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Comment (-2)** Actually, `sort()` is more efficient. -- cpluspluslover@programming.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "4456cba9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.268221Z",
     "iopub.status.busy": "2024-01-17T21:56:29.268109Z",
     "iopub.status.idle": "2024-01-17T21:56:29.270452Z",
     "shell.execute_reply": "2024-01-17T21:56:29.270178Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xyz = list([2, 1, 3]); xyz.sort(); xyz[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b0cfa20",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Comment (+2)** `sort()` takes three lines of code, whereas `sorted()` takes one. Also, avoid multiple statements on a line -- haskellfan@beautifulcode.org"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ed5b4e8",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### I get a syntax error\n",
    "\n",
    "**Question.** When I run the above `middle()` code, I get a message\n",
    "`SyntaxError: invalid syntax`. What am I doing wrong? -- appleuser@mac.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "683478a8",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Best Answer (+10).** Are you using Python 2.x? Type annotations work with Python 3.6 and later -- updates@python.org"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21baa748",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Can we have a `largest()` and `smallest()` function too?\n",
    "\n",
    "**Question.** How do I get the greatest (or smallest) of $x$, $y$, and $z$?  -- novice@python.net"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f1c1973",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Best Answer (+10).** Try Python `max()` and `min()` -- guido@python.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "21fe252c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.272147Z",
     "iopub.status.busy": "2024-01-17T21:56:29.272040Z",
     "iopub.status.idle": "2024-01-17T21:56:29.274156Z",
     "shell.execute_reply": "2024-01-17T21:56:29.273884Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max(2, 1, 3)  # should be 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "71fa73f6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.275753Z",
     "iopub.status.busy": "2024-01-17T21:56:29.275583Z",
     "iopub.status.idle": "2024-01-17T21:56:29.277972Z",
     "shell.execute_reply": "2024-01-17T21:56:29.277693Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "min(2, 1, 3)  # should be 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a3d5ae",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## More Drawing UML"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89d9607b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Install [jupyterlab-markup](https://opensourcelibs.com/lib/jupyterlab-markup)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e2e50a6",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```mermaid\n",
    "graph TD;\n",
    "    A-->B;\n",
    "    A-->C;\n",
    "    B-->D;\n",
    "    C-->D;\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "210e66d5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.279570Z",
     "iopub.status.busy": "2024-01-17T21:56:29.279438Z",
     "iopub.status.idle": "2024-01-17T21:56:29.281297Z",
     "shell.execute_reply": "2024-01-17T21:56:29.281021Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.display import display, Markdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "8855d181",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-17T21:56:29.282934Z",
     "iopub.status.busy": "2024-01-17T21:56:29.282809Z",
     "iopub.status.idle": "2024-01-17T21:56:29.284915Z",
     "shell.execute_reply": "2024-01-17T21:56:29.284634Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "\n",
       "```mermaid\n",
       "graph TD;\n",
       "    A-->B;\n",
       "    B-->A;\n",
       "    A-->C;\n",
       "    B-->D;\n",
       "    C-->D;\n",
       "```"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Markdown(\"\"\"\n",
    "```mermaid\n",
    "graph TD;\n",
    "    A-->B;\n",
    "    B-->A;\n",
    "    A-->C;\n",
    "    B-->D;\n",
    "    C-->D;\n",
    "```\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0fb0267",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## More Resources\n",
    " \n",
    "* [I don't like notebooks](https://www.youtube.com/watch?v=7jiPeIFXb6U)\n",
    "* [JupyterBook](https://blog.jupyter.org/announcing-the-new-jupyter-book-cbf7aa8bc72e)\n",
    "* The [Grand Unified Theory of Documentation](https://documentation.divio.com):\n",
    "    * Tutorials\n",
    "    * How-To Guides\n",
    "    * Explanation\n",
    "    * Reference\n",
    "* [Eve: Programming for Humans](http://witheve.com)\n",
    "* [nbdev: Create Python Projects with Jupyter Notebooks](https://nbdev.fast.ai)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff090321",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Influences:\n",
    "\n",
    "* \\TeX: The Program by Donald J. Knuth\n",
    "* Operating Systems: Design and Implementation by Andrew Tanenbaum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8643f66f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Our goal: Have a _single document_ that encompasses\n",
    "\n",
    "* Code\n",
    "* Tests\n",
    "* Tutorial\n",
    "* How-To-Guides\n",
    "* Explanations\n",
    "* Reference\n",
    "\n",
    "in an _executable_ and _self-updating_ way."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c9045be",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What \"Illustrate\" stands for"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee017ad4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `I` is for `I`llustrate\n",
    "\n",
    "Every piece of code should come with an _example_ that _illustrates_ is usage."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914d5c41",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `L` is for `L`avish\n",
    "\n",
    "Go beyond typewriter text. Make use of all media modern tools have to offer – diagrams! charts! videos! Tie these to your code and examples, such that they stay in sync."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24fe90d7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `L` is for `L`og\n",
    "\n",
    "Do not just describe the final version. Discuss alternatives you have tried (and revoked, and why). This will be helpful for understanding later.\n",
    "\n",
    "Note down your ideas and plans during development + testing\n",
    "\n",
    "Also log during debugging. Keep track of your experiments and their results. Turn these into test cases later."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57846471",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `U` is for `U`pdate\n",
    "\n",
    "Make your document the single place to be updated."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f02138c4",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `S` is for `S`piralize\n",
    "\n",
    "Focus on the essentials first, and add details later. Give the reader moments where they can stay and recapitulate what they learned. Make these abstraction layers in your code, such that readers can choose what to use (and what to read, too!) Proceed step by step, illustrating one piece at a time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d78d167",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `T` is for `T`utorial\n",
    "\n",
    "Allow your readers to learn what your code is about – from a usage perspective, but also from an implementation perspective. Use quizzes, tests, exercises."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19a2fa06",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `R` is for `R`eference\n",
    "\n",
    "Allow your readers to extract and study those parts they need – interfaces, specs. And, of course, allow them to _use_ your code."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ae20aa6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `A` is for `A`ssist\n",
    "\n",
    "Have others contribute to your tutorial – e.g. by providing recipes for specific how-to questions\n",
    "Think of StackOverflow, but constantly tested and updated\n",
    "\n",
    "StackOverflow is filled with how-to questions. Yet, many of these are outdated over time. Allow for your readers to ask questions (with code that will be executed and tested as part of your work). Make your work a community effort!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8380ff3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `T` is for `T`est\n",
    "\n",
    "Having detailed examples should automatically lead to full coverage of your code. Have assertions on top – to explain what is going on, but also to ensure consistency between code and text.\n",
    "\n",
    "Save previous outputs of your examples; get notified when things change. Great way to do regression testing!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31fe1cf1",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `E` is for `E`ase\n",
    "\n",
    "Your job is coding, teaching, and testing – all in one place\n",
    "\n",
    "Ease the life of future readers (which may include you)\n",
    "\n",
    "Ease the life of people who want to use your code"
   ]
  }
 ],
 "metadata": {
  "ipub": {
   "bibliography": "fuzzingbook.bib"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": false,
  "toc-showcode": false,
  "toc-showmarkdowntxt": false,
  "toc-showtags": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}