notebooks/shared/ExpectError.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Error Handling\n",
    "\n",
    "The code in this notebook helps with handling errors.  Normally, an error in  notebook code causes the execution of the code to stop; while an infinite loop in notebook code causes the notebook to run without end.  This notebook provides two classes to help address these concerns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "**Prerequisites**\n",
    "\n",
    "* This notebook needs some understanding on advanced concepts in Python, notably \n",
    "    * classes\n",
    "    * the Python `with` statement\n",
    "    * tracing\n",
    "    * measuring time\n",
    "    * exceptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synopsis\n",
    "\n",
    "<!-- Automatically generated. Do not edit. -->\n",
    "\n",
    "\n",
    "\n",
    "The `ExpectError` class allows you to catch and report exceptions, yet resume execution.  This is useful in notebooks, as they would normally interrupt execution as soon as an exception is raised.  Its typical usage is in conjunction with a `with` clause:\n",
    "\n",
    "```python\n",
    "with ExpectError():\n",
    "    x = 1 / 0\n",
    "```\n",
    "```python\n",
    "=> Traceback (most recent call last):\n",
    "  File \"<ipython-input-14-264328004f25>\", line 2, in <module>\n",
    "    x = 1 / 0\n",
    "ZeroDivisionError: division by zero (expected)\n",
    "\n",
    "```\n",
    "The `ExpectTimeout` class allows you to interrupt execution after the specified time.  This is useful for interrupting code that might otherwise run forever.\n",
    "\n",
    "```python\n",
    "with ExpectTimeout(5):\n",
    "    long_running_test()\n",
    "```\n",
    "```python\n",
    "=> Start\n",
    "0 seconds have passed\n",
    "1 seconds have passed\n",
    "2 seconds have passed\n",
    "3 seconds have passed\n",
    "\n",
    "Traceback (most recent call last):\n",
    "  File \"<ipython-input-15-7e5136e65261>\", line 2, in <module>\n",
    "    long_running_test()\n",
    "  File \"<ipython-input-11-8d0f8e53f106>\", line 5, in long_running_test\n",
    "    print(i, \"seconds have passed\")\n",
    "  File \"<ipython-input-11-8d0f8e53f106>\", line 5, in long_running_test\n",
    "    print(i, \"seconds have passed\")\n",
    "  File \"<ipython-input-10-a28a583f0630>\", line 16, in check_time\n",
    "    raise TimeoutError\n",
    "TimeoutError (expected)\n",
    "\n",
    "```\n",
    "The exception and the associated traceback are printed as error messages.  If you do not want that, \n",
    "use these keyword options:\n",
    "\n",
    "* `print_traceback` (default True) can be set to `False` to avoid the traceback being printed\n",
    "* `mute` (default False) can be set to `True` to completely avoid any output.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Catching Errors\n",
    "\n",
    "The class `ExpectError` allows to express that some code produces an exception.  A typical usage looks as follows:\n",
    "\n",
    "```Python\n",
    "from ExpectError import ExpectError\n",
    "\n",
    "with ExpectError():\n",
    "    function_that_is_supposed_to_fail()\n",
    "```\n",
    "\n",
    "If an exception occurs, it is printed on standard error; yet, execution continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import bookutils.setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import traceback\n",
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from types import FrameType, TracebackType"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ignore\n",
    "from typing import Union, Optional, Callable, Any"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "class ExpectError:\n",
    "    \"\"\"Execute a code block expecting (and catching) an error.\"\"\"\n",
    "\n",
    "    def __init__(self, exc_type: Optional[type] = None, \n",
    "                 print_traceback: bool = True, mute: bool = False):\n",
    "        \"\"\"\n",
    "        Constructor. Expect an exception of type `exc_type` (`None`: any exception).\n",
    "        If `print_traceback` is set (default), print a traceback to stderr.\n",
    "        If `mute` is set (default: False), do not print anything.\n",
    "        \"\"\"\n",
    "        self.print_traceback = print_traceback\n",
    "        self.mute = mute\n",
    "        self.expected_exc_type = exc_type\n",
    "\n",
    "    def __enter__(self) -> Any:\n",
    "        \"\"\"Begin of `with` block\"\"\"\n",
    "        return self\n",
    "\n",
    "    def __exit__(self, exc_type: type, \n",
    "                 exc_value: BaseException, tb: TracebackType) -> Optional[bool]:\n",
    "        \"\"\"End of `with` block\"\"\"\n",
    "        if exc_type is None:\n",
    "            # No exception\n",
    "            return\n",
    "\n",
    "        if (self.expected_exc_type is not None\n",
    "            and exc_type != self.expected_exc_type):\n",
    "            raise  # Unexpected exception\n",
    "\n",
    "        # An exception occurred\n",
    "        if self.print_traceback:\n",
    "            lines = ''.join(\n",
    "                traceback.format_exception(\n",
    "                    exc_type,\n",
    "                    exc_value,\n",
    "                    tb)).strip()\n",
    "        else:\n",
    "            lines = traceback.format_exception_only(\n",
    "                exc_type, exc_value)[-1].strip()\n",
    "\n",
    "        if not self.mute:\n",
    "            print(lines, \"(expected)\", file=sys.stderr)\n",
    "        return True  # Ignore it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def fail_test() -> None:\n",
    "    # Trigger an exception\n",
    "    x = 1 / 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "with ExpectError():\n",
    "    fail_test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "with ExpectError(print_traceback=False):\n",
    "    fail_test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can specify the type of the expected exception. This way, if something else happens, we will get notified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with ExpectError(ZeroDivisionError):\n",
    "    fail_test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with ExpectError():\n",
    "    with ExpectError(ZeroDivisionError):\n",
    "        some_nonexisting_function()  # type: ignore"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Catching Timeouts\n",
    "\n",
    "The class `ExpectTimeout(seconds)` allows expressing that some code may run for a long or infinite time; execution is thus interrupted after `seconds` seconds.  A typical usage looks as follows:\n",
    "\n",
    "```Python\n",
    "from ExpectError import ExpectTimeout\n",
    "\n",
    "with ExpectTimeout(2) as t:\n",
    "    function_that_is_supposed_to_hang()\n",
    "```\n",
    "\n",
    "If an exception occurs, it is printed on standard error (as with `ExpectError`); yet, execution continues.\n",
    "\n",
    "Should there be a need to cancel the timeout within the `with` block, `t.cancel()` will do the trick.\n",
    "\n",
    "The implementation uses `sys.settrace()`, as this seems to be the most portable way to implement timeouts.  It is not very efficient, though.  Also, it only works on individual lines of Python code and will not interrupt a long-running system function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Timeout import Timeout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "class ExpectTimeout(Timeout):  # type: ignore\n",
    "    \"\"\"Execute a code block expecting (and catching) a timeout.\"\"\"\n",
    "\n",
    "    def __init__(self, timeout: Union[int, float],\n",
    "                 print_traceback: bool = True, mute: bool = False):\n",
    "        \"\"\"\n",
    "        Constructor. Interrupt execution after `seconds` seconds.\n",
    "        If `print_traceback` is set (default), print a traceback to stderr.\n",
    "        If `mute` is set (default: False), do not print anything.\n",
    "        \"\"\"\n",
    "        super().__init__(timeout)\n",
    "\n",
    "        self.print_traceback = print_traceback\n",
    "        self.mute = mute\n",
    "\n",
    "    def __exit__(self, exc_type: type,\n",
    "                 exc_value: BaseException, tb: TracebackType) -> Optional[bool]:\n",
    "        \"\"\"End of `with` block\"\"\"\n",
    "\n",
    "        super().__exit__(exc_type, exc_value, tb)\n",
    "\n",
    "        if exc_type is None:\n",
    "            return\n",
    "\n",
    "        # An exception occurred\n",
    "        if self.print_traceback:\n",
    "            lines = ''.join(\n",
    "                traceback.format_exception(\n",
    "                    exc_type,\n",
    "                    exc_value,\n",
    "                    tb)).strip()\n",
    "        else:\n",
    "            lines = traceback.format_exception_only(\n",
    "                exc_type, exc_value)[-1].strip()\n",
    "\n",
    "        if not self.mute:\n",
    "            print(lines, \"(expected)\", file=sys.stderr)\n",
    "\n",
    "        return True  # Ignore exception"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def long_running_test() -> None:\n",
    "    print(\"Start\")\n",
    "    for i in range(10):\n",
    "        time.sleep(1)\n",
    "        print(i, \"seconds have passed\")\n",
    "    print(\"End\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "with ExpectTimeout(5, print_traceback=False):\n",
    "    long_running_test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Note that it is possible to nest multiple timeouts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "with ExpectTimeout(5, print_traceback=False):\n",
    "    with ExpectTimeout(3, print_traceback=False):\n",
    "        long_running_test()\n",
    "    long_running_test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "That's it, folks – enjoy!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Synopsis\n",
    "\n",
    "The `ExpectError` class allows you to catch and report exceptions, yet resume execution.  This is useful in notebooks, as they would normally interrupt execution as soon as an exception is raised.  Its typical usage is in conjunction with a `with` clause:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with ExpectError():\n",
    "    x = 1 / 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `ExpectTimeout` class allows you to interrupt execution after the specified time.  This is useful for interrupting code that might otherwise run forever."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with ExpectTimeout(5):\n",
    "    long_running_test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The exception and the associated traceback are printed as error messages.  If you do not want that, \n",
    "use these keyword options:\n",
    "\n",
    "* `print_traceback` (default True) can be set to `False` to avoid the traceback being printed\n",
    "* `mute` (default False) can be set to `True` to completely avoid any output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": true,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Lessons Learned\n",
    "\n",
    "* With the `ExpectError` class, it is very easy to handle errors without interrupting notebook execution."
   ]
  }
 ],
 "metadata": {
  "ipub": {
   "bibliography": "fuzzingbook.bib",
   "toc": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}