tutorials/Bayesian_Logistic_Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "69bf1817-032e-457e-bfa0-e57ede210e5d",
    "tags": []
   },
   "source": [
    "## Tutorial: **Bayesian Logistic Regression**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "69bf1817-032e-457e-bfa0-e57ede210e5d"
   },
   "source": [
    "The purpose of this tutorial is to show how to build a simple Bayesian model to deduce\n",
    "the line which separates two categories of points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "b4361d0f-ed7e-494f-9270-14d364d433ad"
   },
   "source": [
    "## Problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "b4361d0f-ed7e-494f-9270-14d364d433ad"
   },
   "source": [
    "A *logistic model* is a statistical model where we have a collection of things that can\n",
    "be divided into two categories — pictures of cats or dogs, patients who are immune or\n",
    "susceptible, students who pass or fail, and so on. An assumption of the model is: the\n",
    "probability that a given thing is in a category can be computed by taking a linear\n",
    "combination of characteristics of the thing.\n",
    "\n",
    "The problem we seek to solve in this tutorial is: *when given a set of examples of each\n",
    "category, can we infer the boundary between the categories?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using the following packages within this tutorial.\n",
    "\n",
    "* [arviz](https://arviz-devs.github.io/arviz/) and\n",
    "  [bokeh](https://docs.bokeh.org/en/latest/docs/) for interactive visualizations.\n",
    "\n",
    "Let's code this in Bean Machine! Import the Bean Machine library and some fundamental\n",
    "PyTorch classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install Bean Machine in Colab if using Colab.\n",
    "import sys\n",
    "\n",
    "\n",
    "if \"google.colab\" in sys.modules and \"beanmachine\" not in sys.modules:\n",
    "    !pip install beanmachine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import warnings\n",
    "\n",
    "import arviz as az\n",
    "import beanmachine.ppl as bm\n",
    "import torch\n",
    "import torch.distributions as dist\n",
    "from beanmachine.ppl.inference.monte_carlo_samples import MonteCarloSamples\n",
    "from beanmachine.tutorials.utils import plots\n",
    "from bokeh.io import output_notebook\n",
    "from bokeh.models import ColumnDataSource, MultiLine, Span\n",
    "from bokeh.plotting import show\n",
    "from IPython.display import Markdown"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell includes convenient configuration settings to improve the notebook\n",
    "presentation as well as setting a manual seed for reproducibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Eliminate excess UserWarnings from Python.\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "# Plotting settings\n",
    "az.rcParams[\"plot.backend\"] = \"bokeh\"\n",
    "az.rcParams[\"stats.hdi_prob\"] = 0.89\n",
    "\n",
    "# Manual seed\n",
    "bm.seed(0)\n",
    "\n",
    "# Other settings for the notebook.\n",
    "smoke_test = \"SANDCASTLE_NEXUS\" in os.environ or \"CI\" in os.environ"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "e60f7e01-62ba-4c50-970b-eab5cc8f87c7"
   },
   "source": [
    "## Example: orange and blue points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "e60f7e01-62ba-4c50-970b-eab5cc8f87c7"
   },
   "source": [
    "For the purposes of this tutorial our \"things\" will be two-dimensional points between\n",
    "$(-10,-10)$ and $(10,10)$. Our two categories will be represented by $0.0$, which\n",
    "we'll render as blue, and $1.0$, which we'll render as orange.\n",
    "\n",
    "The probability that a given point $(x,y)$ will be orange is\n",
    "\n",
    "$P((x,y)\\text{ is orange})=\\Large{\\frac{1}{1+e^{-b_0-b_1x-b_2y}}}$\n",
    "\n",
    "for some coefficients $b_0,b_1,b_2$.\n",
    "\n",
    "That equation is seen to be a linear combination of properties of the point if we\n",
    "express the probability as log odds:\n",
    "\n",
    "$\\text{logit}(P((x,y)\\text{ is orange}))=b_0+b_1x+b_2y$\n",
    "\n",
    "The boundary between the two categories is the set of points where the probability is\n",
    "$0.5$ that the point is orange; the log odds of $0.5$ is $0$, so the equation of the set\n",
    "of $(x, )$ points separating the categories is a straight line:\n",
    "\n",
    "$b_0+b_1x+b_2y=0$\n",
    "\n",
    "Or, expressed as slope and intercept:\n",
    "\n",
    "$y=-\\frac{b_1}{b_2}x-\\frac{b_0}{b_2}$\n",
    "\n",
    "Our goal therefore is to infer possible values for $b_0,b_1,b_2$ from which we can\n",
    "compute the line separating the categories."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "29cd54e7-5459-44f2-8abf-2bebd919a65d"
   },
   "source": [
    "## Creating sample data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "29cd54e7-5459-44f2-8abf-2bebd919a65d"
   },
   "source": [
    "We start by creating a data set to illustrate the model. In order to make computations\n",
    "faster and easier, we will express the entire data set as a single tensor representing\n",
    "200 points. Each row of the tensor will be in the form\n",
    "$\\begin{bmatrix}1.0&x&y\\end{bmatrix}$. That way we can compute the linear combination by\n",
    "matrix-multiplying each row by $\\begin{bmatrix}b_0\\\\b_1\\\\b_2\\end{bmatrix}$ to obtain\n",
    "the log odds that the point is orange: $b_0+b_1x+b_2y$.\n",
    "\n",
    "For our synthetic dataset, we will assume the following parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "N = 200\n",
    "low = -10.0\n",
    "high = 10.0\n",
    "uniform = dist.Uniform(\n",
    "    low=torch.tensor([1.0, low, low]),\n",
    "    high=torch.tensor([1.0, high, high]),\n",
    ")\n",
    "points = torch.tensor([uniform.sample().tolist() for i in range(N)]).view(N, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  const force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  const JS_MIME_TYPE = 'application/javascript';\n",
       "  const HTML_MIME_TYPE = 'text/html';\n",
       "  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  const CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    const script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    const cell = handle.cell;\n",
       "\n",
       "    const id = cell.output_area._bokeh_element_id;\n",
       "    const server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd_clean, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            const id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd_destroy);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    const output_area = handle.output_area;\n",
       "    const output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      const bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      const script_attrs = bk_div.children[0].attributes;\n",
       "      for (let i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      const toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    const events = require('base/js/events');\n",
       "    const OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  const NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    const el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error(url) {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < css_urls.length; i++) {\n",
       "      const url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < js_urls.length; i++) {\n",
       "      const url = js_urls[i];\n",
       "      const element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n",
       "  const css_urls = [];\n",
       "  \n",
       "\n",
       "  const inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (let i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n  const css_urls = [];\n  \n\n  const inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"fdaae2b2-88f1-4169-85a2-ce4444568bfb\" data-root-id=\"1003\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  const docs_json = {\"0767360f-a44d-45ce-ad72-01958e2edaf5\":{\"defs\":[],\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1014\"}],\"center\":[{\"id\":\"1017\"},{\"id\":\"1021\"}],\"height\":500,\"left\":[{\"id\":\"1018\"}],\"outline_line_color\":\"black\",\"renderers\":[{\"id\":\"1041\"}],\"title\":{\"id\":\"1004\"},\"toolbar\":{\"id\":\"1029\"},\"width\":700,\"x_range\":{\"id\":\"1006\"},\"x_scale\":{\"id\":\"1010\"},\"y_range\":{\"id\":\"1008\"},\"y_scale\":{\"id\":\"1012\"}},\"id\":\"1003\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1052\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"black\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1040\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1053\",\"type\":\"Selection\"},{\"attributes\":{\"data\":{\"x\":[5.364436149597168,-3.851543426513672,7.92889404296875,-3.0221309661865234,-6.622820854187012,3.9533519744873047,-4.354628562927246,-2.0580015182495117,1.058140754699707,-6.295379638671875,8.640007019042969,-6.9864044189453125,8.595979690551758,0.5259151458740234,-9.336947441101074,6.309379577636719,-0.3608236312866211,3.9688215255737305,-5.888023376464844,-6.930861473083496,4.021604537963867,5.489720344543457,2.317047119140625,-7.706235885620117,8.285493850708008,1.9901456832885742,-6.256053447723389,7.603597640991211,-1.6845998764038086,3.845561981201172,5.05708122253418,-9.89735221862793,2.0930137634277344,9.407493591308594,-2.516847610473633,-7.5305891036987305,1.501450538635254,-6.085391044616699,-8.432828903198242,1.4590110778808594,0.5990018844604492,-9.592489242553711,-4.871133804321289,-2.1271514892578125,-0.5131931304931641,0.2779216766357422,6.358394622802734,9.494134902954102,-4.740772247314453,-4.970463752746582,-8.440082550048828,5.406410217285156,-7.825094223022461,-1.9261531829833984,-8.634378433227539,-4.542748928070068,-0.6748723983764648,9.030031204223633,6.326972961425781,7.996532440185547,-2.093687057495117,4.8050537109375,-2.1030545234680176,7.939781188964844,0.4459991455078125,-5.827054977416992,-0.2181720733642578,-7.559200286865234,6.999334335327148,3.6160755157470703,-1.9768152236938477,-0.07026481628417969,-5.241313934326172,-0.7180614471435547,0.2831306457519531,-7.122807502746582,-1.4808740615844727,1.2483882904052734,-6.322139739990234,-4.238251686096191,7.502281188964844,5.148235321044922,-8.269224166870117,-5.266729354858398,-2.9231953620910645,-9.911483764648438,-6.673402786254883,5.295736312866211,-3.339596748352051,8.220346450805664,-4.700774192810059,-5.611824989318848,-6.494112014770508,0.31858348846435547,7.9708251953125,-3.3524060249328613,-0.44471168518066406,3.1616592407226562,-6.164298057556152,4.096654891967773,1.7964506149291992,1.9638080596923828,4.259291648864746,-8.1978759765625,-3.648771286010742,-5.221515655517578,0.783778190612793,-7.861476898193359,9.011137008666992,4.67014217376709,-8.599210739135742,-6.912337303161621,9.996149063110352,-2.2925925254821777,5.603796005249023,4.831585884094238,2.868997573852539,-1.7994060516357422,4.254682540893555,-6.984709739685059,-9.865375518798828,5.410356521606445,-8.93045425415039,-6.494322776794434,-4.879014492034912,8.224775314331055,7.721273422241211,4.400020599365234,9.996763229370117,-3.4999217987060547,-2.3881731033325195,-7.694808006286621,-1.1381263732910156,6.394889831542969,9.190479278564453,9.570829391479492,5.621545791625977,8.491012573242188,-3.3423290252685547,0.9570064544677734,-6.1743998527526855,4.960402488708496,-7.431445121765137,-7.101302146911621,0.6558837890625,2.1836414337158203,-9.078695297241211,-2.0286331176757812,8.311870574951172,0.45602893829345703,5.384434700012207,-6.600689888000488,4.742164611816406,-9.818775177001953,-7.851617336273193,1.3930931091308594,-3.0851659774780273,5.7502899169921875,-9.810628890991211,-5.664152145385742,-4.953108787536621,-1.018951416015625,5.533426284790039,-2.17431640625,-7.005898952484131,-8.37945556640625,9.145271301269531,5.011591911315918,-5.389503479003906,-2.9549336433410645,6.182613372802734,2.5174903869628906,0.8839836120605469,3.0433387756347656,-7.2439422607421875,9.190109252929688,-4.8449554443359375,-7.541866302490234,-0.04822731018066406,-3.586721420288086,2.215825080871582,1.373854637145996,0.4643440246582031,-9.487632751464844,-6.326899528503418,-2.106570243835449,0.4779701232910156,-3.409003257751465,9.429195404052734,7.835580825805664,5.7844438552856445,-0.7440299987792969,-9.273699760437012,-2.846832275390625,2.5059938430786133,2.0213184356689453,7.763822555541992,-9.03623104095459,4.359073638916016,9.473430633544922],\"y\":[-8.230451583862305,2.6815738677978516,-0.8874406814575195,-1.9656543731689453,-4.1222310066223145,6.0002288818359375,3.632171630859375,7.483118057250977,9.05476188659668,-2.5316524505615234,-6.4817962646484375,-9.365610122680664,4.462183952331543,-5.126834869384766,-7.2256622314453125,5.863212585449219,6.395606994628906,1.3509283065795898,1.8634405136108398,-5.165835380554199,-5.923524856567383,-1.2621736526489258,6.203765869140625,-3.6646976470947266,8.702072143554688,-8.695826530456543,-9.319541931152344,-9.975279808044434,-1.6456117630004883,-5.923035144805908,7.158716201782227,-6.486968994140625,-7.800840377807617,6.738178253173828,-9.525980949401855,-7.713566780090332,-4.095302581787109,9.073699951171875,-2.4888429641723633,2.3717422485351562,-4.8792877197265625,-5.927066802978516,-3.4983339309692383,2.1375646591186523,7.15850830078125,-0.8626890182495117,9.472461700439453,-0.7232170104980469,6.809051513671875,-7.663117408752441,-2.028367519378662,-9.644318580627441,-2.114102840423584,-1.9634265899658203,-1.5647945404052734,3.7669925689697266,8.794195175170898,3.6215381622314453,-1.153944969177246,-8.080951690673828,7.14112663269043,3.5315895080566406,-8.240814208984375,6.842248916625977,-7.049341201782227,3.417449951171875,0.4206809997558594,-6.865122318267822,-3.5946502685546875,1.2662601470947266,1.2546634674072266,1.275930404663086,8.074928283691406,9.892387390136719,-8.666099548339844,-2.8386425971984863,0.10938262939453125,8.956928253173828,4.485040664672852,2.9412269592285156,-3.21915864944458,-9.670921325683594,0.13782501220703125,1.3217105865478516,-5.936941146850586,4.51393985748291,-5.761013984680176,7.67521858215332,-2.7944812774658203,2.718921661376953,-9.454689979553223,-8.91575813293457,-1.1376152038574219,-6.728816986083984,1.628387451171875,2.945554733276367,-6.090441703796387,-0.2062854766845703,6.915447235107422,-3.3625354766845703,-5.1945695877075195,-7.424894332885742,3.958310127258301,-1.5415401458740234,3.7958106994628906,0.0986185073852539,0.8353147506713867,0.7858829498291016,5.877509117126465,-4.8647966384887695,9.976097106933594,3.912425994873047,8.743949890136719,-3.5094165802001953,-6.017789840698242,5.45135498046875,-3.505624771118164,3.893125534057617,-3.398275375366211,2.258549690246582,-8.030557632446289,9.382156372070312,-6.824311256408691,6.943965911865234,-9.660913467407227,8.187601089477539,8.891864776611328,8.909303665161133,5.186629295349121,4.798274993896484,-5.63709831237793,6.7133026123046875,-5.78685998916626,0.7433404922485352,4.089406967163086,7.593738555908203,-5.681890487670898,0.4131908416748047,-2.714451789855957,9.248296737670898,0.512542724609375,-9.139385223388672,-4.266855716705322,3.717268943786621,-6.664876937866211,-7.623180389404297,-9.612933158874512,6.724328994750977,-4.0002288818359375,-9.017190933227539,9.939956665039062,8.345836639404297,-8.018291473388672,-3.894925117492676,3.1876420974731445,-6.690833568572998,4.389581680297852,-1.1260929107666016,-8.54101276397705,4.810956954956055,-8.236889839172363,7.696010589599609,0.3215608596801758,1.3290061950683594,8.392778396606445,-5.41057825088501,-9.262796401977539,5.919184684753418,3.1577577590942383,3.3465309143066406,-2.7745461463928223,3.54693603515625,5.795356750488281,-2.411870002746582,-5.880288124084473,-2.690840721130371,9.982900619506836,-8.106698989868164,-2.549038887023926,1.8893089294433594,-2.293158531188965,8.22258186340332,-3.6876773834228516,-9.5866117477417,1.9172334671020508,-2.2334775924682617,-9.736145973205566,5.032163619995117,-2.227293014526367,5.0260772705078125,-3.033538818359375,8.276374816894531,4.0992841720581055,-8.280235290527344,-0.757197380065918,3.797588348388672,-1.4968376159667969,9.336894989013672,-8.652210235595703,9.028602600097656]},\"selected\":{\"id\":\"1053\"},\"selection_policy\":{\"id\":\"1052\"}},\"id\":\"1002\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1041\"}],\"tooltips\":[[\"y\",\"@y{0.000}\"],[\"x\",\"@x{0.000}\"]]},\"id\":\"1043\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1010\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1026\",\"type\":\"ResetTool\"},{\"attributes\":{\"axis\":{\"id\":\"1018\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1021\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"black\"},\"hatch_alpha\":{\"value\":0.1},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1038\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1025\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1006\",\"type\":\"DataRange1d\"},{\"attributes\":{\"axis\":{\"id\":\"1014\"},\"coordinates\":null,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1017\",\"type\":\"Grid\"},{\"attributes\":{\"axis_label\":\"x\",\"coordinates\":null,\"formatter\":{\"id\":\"1050\"},\"group\":null,\"major_label_policy\":{\"id\":\"1051\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1015\"}},\"id\":\"1014\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"black\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1037\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1022\",\"type\":\"PanTool\"},{\"attributes\":{\"fill_color\":{\"value\":\"orange\"},\"line_width\":{\"value\":2},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1039\",\"type\":\"Circle\"},{\"attributes\":{\"overlay\":{\"id\":\"1028\"}},\"id\":\"1024\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1008\",\"type\":\"DataRange1d\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Synthetic data\"},\"id\":\"1004\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"1012\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1015\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1027\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1047\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1028\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1002\"},\"glyph\":{\"id\":\"1037\"},\"group\":null,\"hover_glyph\":{\"id\":\"1039\"},\"muted_glyph\":{\"id\":\"1040\"},\"nonselection_glyph\":{\"id\":\"1038\"},\"view\":{\"id\":\"1042\"}},\"id\":\"1041\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1048\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1050\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1051\",\"type\":\"AllLabels\"},{\"attributes\":{\"source\":{\"id\":\"1002\"}},\"id\":\"1042\",\"type\":\"CDSView\"},{\"attributes\":{\"axis_label\":\"y\",\"coordinates\":null,\"formatter\":{\"id\":\"1047\"},\"group\":null,\"major_label_policy\":{\"id\":\"1048\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1019\"}},\"id\":\"1018\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1023\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"tools\":[{\"id\":\"1022\"},{\"id\":\"1023\"},{\"id\":\"1024\"},{\"id\":\"1025\"},{\"id\":\"1026\"},{\"id\":\"1027\"},{\"id\":\"1043\"}]},\"id\":\"1029\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1019\",\"type\":\"BasicTicker\"}],\"root_ids\":[\"1003\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.2\"}};\n",
       "  const render_items = [{\"docid\":\"0767360f-a44d-45ce-ad72-01958e2edaf5\",\"root_ids\":[\"1003\"],\"roots\":{\"1003\":\"fdaae2b2-88f1-4169-85a2-ce4444568bfb\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    let attempts = 0;\n",
       "    const timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1003"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Required for visualizing in Colab.\n",
    "output_notebook(hide_banner=True)\n",
    "\n",
    "x = points[:, 1]\n",
    "y = points[:, 2]\n",
    "cds = ColumnDataSource({\"x\": x.tolist(), \"y\": y.tolist()})\n",
    "tips = [(\"y\", \"@y{0.000}\"), (\"x\", \"@x{0.000}\")]\n",
    "synthetic_data_plot = plots.scatter_plot(\n",
    "    plot_sources=cds,\n",
    "    tooltips=tips,\n",
    "    figure_kwargs={\n",
    "        \"title\": \"Synthetic data\",\n",
    "        \"x_axis_label\": \"x\",\n",
    "        \"y_axis_label\": \"y\",\n",
    "    },\n",
    "    plot_kwargs={\"fill_color\": \"black\"},\n",
    ")\n",
    "show(synthetic_data_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "a51ec7c6-587c-41cb-abfa-d2776ce03dc2",
    "showInput": false
   },
   "source": [
    "We now assign points to categories $0.0$ (blue) and $1.0$ (orange).\n",
    "\n",
    "For this example we will assign categories to points using\n",
    "$\\begin{bmatrix}-2.0\\\\0.3\\\\-0.5\\end{bmatrix}$ as our coefficients, which makes the line\n",
    "separating the categories $y=0.6x-4$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "true_coefficients = torch.tensor([-2.0, 0.3, -0.5]).view(3, 1)\n",
    "true_slope = -float(true_coefficients[1] / true_coefficients[2])\n",
    "true_intercept = -float(true_coefficients[0] / true_coefficients[2])\n",
    "\n",
    "\n",
    "def log_odds(point):\n",
    "    return point.view(1, 3).mm(true_coefficients)\n",
    "\n",
    "\n",
    "observed_categories = torch.tensor(\n",
    "    [dist.Bernoulli(logits=log_odds(point)).sample() for point in points]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "034858dd-7b08-4655-b60f-e1c4adbf153c",
    "showInput": false
   },
   "source": [
    "## Data visualization methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "034858dd-7b08-4655-b60f-e1c4adbf153c",
    "showInput": false
   },
   "source": [
    "It is useful to have helper methods to visualize the synthetic data set and the line\n",
    "which separates the two categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def categorize_points(points, categories):\n",
    "    orange_x = []\n",
    "    orange_y = []\n",
    "    blue_x = []\n",
    "    blue_y = []\n",
    "    for point, category in zip(points, categories):\n",
    "        if category == 1:\n",
    "            orange_x.append(float(point[1]))\n",
    "            orange_y.append(float(point[2]))\n",
    "        else:\n",
    "            blue_x.append(float(point[1]))\n",
    "            blue_y.append(float(point[2]))\n",
    "    return {\n",
    "        \"orange\": {\"x\": orange_x, \"y\": orange_y, \"label\": [\"orange\"] * len(orange_x)},\n",
    "        \"blue\": {\"x\": blue_x, \"y\": blue_y, \"label\": [\"blue\"] * len(blue_x)},\n",
    "    }\n",
    "\n",
    "\n",
    "def plot_line(slope, intercept, high=10, low=-10):\n",
    "    if intercept > high or intercept < low:\n",
    "        return\n",
    "    xs = [low, high]\n",
    "    ys = [slope * low + intercept, slope * high + intercept]\n",
    "    if ys[0] > high:\n",
    "        xs[0] = (high - intercept) / slope\n",
    "        ys[0] = high\n",
    "    elif ys[0] < low:\n",
    "        xs[0] = (low - intercept) / slope\n",
    "        ys[0] = low\n",
    "    if ys[1] > high:\n",
    "        xs[1] = (high - intercept) / slope\n",
    "        ys[1] = high\n",
    "    elif ys[1] < low:\n",
    "        xs[1] = (low - intercept) / slope\n",
    "        ys[1] = low\n",
    "    return xs, ys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  const force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  const JS_MIME_TYPE = 'application/javascript';\n",
       "  const HTML_MIME_TYPE = 'text/html';\n",
       "  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  const CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    const script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    const cell = handle.cell;\n",
       "\n",
       "    const id = cell.output_area._bokeh_element_id;\n",
       "    const server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd_clean, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            const id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd_destroy);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    const output_area = handle.output_area;\n",
       "    const output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      const bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      const script_attrs = bk_div.children[0].attributes;\n",
       "      for (let i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      const toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    const events = require('base/js/events');\n",
       "    const OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  const NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    const el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error(url) {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < css_urls.length; i++) {\n",
       "      const url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < js_urls.length; i++) {\n",
       "      const url = js_urls[i];\n",
       "      const element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n",
       "  const css_urls = [];\n",
       "  \n",
       "\n",
       "  const inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (let i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n  const css_urls = [];\n  \n\n  const inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"6710117a-6f1b-49b7-ba09-8492e9cefbb2\" data-root-id=\"1107\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  const docs_json = {\"2cfac753-c0ad-49d1-9524-9a3051c61419\":{\"defs\":[],\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1118\"}],\"center\":[{\"id\":\"1121\"},{\"id\":\"1125\"},{\"id\":\"1157\"}],\"height\":500,\"left\":[{\"id\":\"1122\"}],\"outline_line_color\":\"black\",\"renderers\":[{\"id\":\"1145\"},{\"id\":\"1166\"},{\"id\":\"1187\"}],\"title\":{\"id\":\"1108\"},\"toolbar\":{\"id\":\"1133\"},\"width\":700,\"x_range\":{\"id\":\"1110\"},\"x_scale\":{\"id\":\"1114\"},\"y_range\":{\"id\":\"1112\"},\"y_scale\":{\"id\":\"1116\"}},\"id\":\"1107\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.1},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1142\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1127\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"1149\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"data\":{\"label\":[\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\"],\"x\":[5.364436149597168,7.92889404296875,8.640007019042969,-6.9864044189453125,0.5259151458740234,4.021604537963867,-7.706235885620117,1.9901456832885742,7.603597640991211,-1.6845998764038086,3.845561981201172,2.0930137634277344,-2.516847610473633,-7.5305891036987305,1.501450538635254,0.5990018844604492,-4.970463752746582,5.406410217285156,-1.9261531829833984,6.326972961425781,7.996532440185547,0.4459991455078125,6.999334335327148,-1.9768152236938477,0.2831306457519531,5.148235321044922,-4.700774192810059,-5.611824989318848,3.1616592407226562,4.096654891967773,1.7964506149291992,1.9638080596923828,4.259291648864746,4.67014217376709,-2.2925925254821777,2.868997573852539,-4.879014492034912,-2.3881731033325195,-1.1381263732910156,9.190479278564453,5.621545791625977,8.491012573242188,4.960402488708496,0.6558837890625,8.311870574951172,0.45602893829345703,4.742164611816406,1.3930931091308594,-9.810628890991211,-4.953108787536621,5.533426284790039,9.145271301269531,6.182613372802734,9.190109252929688,-7.541866302490234,2.215825080871582,0.4779701232910156,9.429195404052734,5.7844438552856445,-2.846832275390625,7.763822555541992,4.359073638916016],\"y\":[-8.230451583862305,-0.8874406814575195,-6.4817962646484375,-9.365610122680664,-5.126834869384766,-5.923524856567383,-3.6646976470947266,-8.695826530456543,-9.975279808044434,-1.6456117630004883,-5.923035144805908,-7.800840377807617,-9.525980949401855,-7.713566780090332,-4.095302581787109,-4.8792877197265625,-7.663117408752441,-9.644318580627441,-1.9634265899658203,-1.153944969177246,-8.080951690673828,-7.049341201782227,-3.5946502685546875,1.2546634674072266,-8.666099548339844,-9.670921325683594,-9.454689979553223,-8.91575813293457,-0.2062854766845703,-3.3625354766845703,-5.1945695877075195,-7.424894332885742,3.958310127258301,-4.8647966384887695,-3.5094165802001953,-3.505624771118164,-9.660913467407227,-5.63709831237793,-5.78685998916626,4.089406967163086,-5.681890487670898,0.4131908416748047,-9.139385223388672,-6.664876937866211,-4.0002288818359375,-9.017190933227539,-8.018291473388672,-6.690833568572998,-8.54101276397705,-8.236889839172363,0.3215608596801758,-9.262796401977539,-2.7745461463928223,-2.690840721130371,-8.106698989868164,-2.293158531188965,-9.736145973205566,-2.227293014526367,-3.033538818359375,-8.280235290527344,-1.4968376159667969,-8.652210235595703]},\"selected\":{\"id\":\"1155\"},\"selection_policy\":{\"id\":\"1154\"}},\"id\":\"1105\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"axis_label\":\"x\",\"coordinates\":null,\"formatter\":{\"id\":\"1152\"},\"group\":null,\"major_label_policy\":{\"id\":\"1153\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1119\"}},\"id\":\"1118\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1166\"}],\"tooltips\":[[\"Category\",\"@label\"],[\"y\",\"@y{0.000}\"],[\"x\",\"@x{0.000}\"]]},\"id\":\"1181\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1177\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1150\",\"type\":\"AllLabels\"},{\"attributes\":{},\"id\":\"1200\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"label\":{\"value\":\"Category orange\"},\"renderers\":[{\"id\":\"1145\"}]},\"id\":\"1158\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1144\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1178\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1183\"},\"glyph\":{\"id\":\"1184\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"1186\"},\"nonselection_glyph\":{\"id\":\"1185\"},\"view\":{\"id\":\"1188\"}},\"id\":\"1187\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data\":{\"label\":[\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\"],\"x\":[-3.851543426513672,-3.0221309661865234,-6.622820854187012,3.9533519744873047,-4.354628562927246,-2.0580015182495117,1.058140754699707,-6.295379638671875,8.595979690551758,-9.336947441101074,6.309379577636719,-0.3608236312866211,3.9688215255737305,-5.888023376464844,-6.930861473083496,5.489720344543457,2.317047119140625,8.285493850708008,-6.256053447723389,5.05708122253418,-9.89735221862793,9.407493591308594,-6.085391044616699,-8.432828903198242,1.4590110778808594,-9.592489242553711,-4.871133804321289,-2.1271514892578125,-0.5131931304931641,0.2779216766357422,6.358394622802734,9.494134902954102,-4.740772247314453,-8.440082550048828,-7.825094223022461,-8.634378433227539,-4.542748928070068,-0.6748723983764648,9.030031204223633,-2.093687057495117,4.8050537109375,-2.1030545234680176,7.939781188964844,-5.827054977416992,-0.2181720733642578,-7.559200286865234,3.6160755157470703,-0.07026481628417969,-5.241313934326172,-0.7180614471435547,-7.122807502746582,-1.4808740615844727,1.2483882904052734,-6.322139739990234,-4.238251686096191,7.502281188964844,-8.269224166870117,-5.266729354858398,-2.9231953620910645,-9.911483764648438,-6.673402786254883,5.295736312866211,-3.339596748352051,8.220346450805664,-6.494112014770508,0.31858348846435547,7.9708251953125,-3.3524060249328613,-0.44471168518066406,-6.164298057556152,-8.1978759765625,-3.648771286010742,-5.221515655517578,0.783778190612793,-7.861476898193359,9.011137008666992,-8.599210739135742,-6.912337303161621,9.996149063110352,5.603796005249023,4.831585884094238,-1.7994060516357422,4.254682540893555,-6.984709739685059,-9.865375518798828,5.410356521606445,-8.93045425415039,-6.494322776794434,8.224775314331055,7.721273422241211,4.400020599365234,9.996763229370117,-3.4999217987060547,-7.694808006286621,6.394889831542969,9.570829391479492,-3.3423290252685547,0.9570064544677734,-6.1743998527526855,-7.431445121765137,-7.101302146911621,2.1836414337158203,-9.078695297241211,-2.0286331176757812,5.384434700012207,-6.600689888000488,-9.818775177001953,-7.851617336273193,-3.0851659774780273,5.7502899169921875,-5.664152145385742,-1.018951416015625,-2.17431640625,-7.005898952484131,-8.37945556640625,5.011591911315918,-5.389503479003906,-2.9549336433410645,2.5174903869628906,0.8839836120605469,3.0433387756347656,-7.2439422607421875,-4.8449554443359375,-0.04822731018066406,-3.586721420288086,1.373854637145996,0.4643440246582031,-9.487632751464844,-6.326899528503418,-2.106570243835449,-3.409003257751465,7.835580825805664,-0.7440299987792969,-9.273699760437012,2.5059938430786133,2.0213184356689453,-9.03623104095459,9.473430633544922],\"y\":[2.6815738677978516,-1.9656543731689453,-4.1222310066223145,6.0002288818359375,3.632171630859375,7.483118057250977,9.05476188659668,-2.5316524505615234,4.462183952331543,-7.2256622314453125,5.863212585449219,6.395606994628906,1.3509283065795898,1.8634405136108398,-5.165835380554199,-1.2621736526489258,6.203765869140625,8.702072143554688,-9.319541931152344,7.158716201782227,-6.486968994140625,6.738178253173828,9.073699951171875,-2.4888429641723633,2.3717422485351562,-5.927066802978516,-3.4983339309692383,2.1375646591186523,7.15850830078125,-0.8626890182495117,9.472461700439453,-0.7232170104980469,6.809051513671875,-2.028367519378662,-2.114102840423584,-1.5647945404052734,3.7669925689697266,8.794195175170898,3.6215381622314453,7.14112663269043,3.5315895080566406,-8.240814208984375,6.842248916625977,3.417449951171875,0.4206809997558594,-6.865122318267822,1.2662601470947266,1.275930404663086,8.074928283691406,9.892387390136719,-2.8386425971984863,0.10938262939453125,8.956928253173828,4.485040664672852,2.9412269592285156,-3.21915864944458,0.13782501220703125,1.3217105865478516,-5.936941146850586,4.51393985748291,-5.761013984680176,7.67521858215332,-2.7944812774658203,2.718921661376953,-1.1376152038574219,-6.728816986083984,1.628387451171875,2.945554733276367,-6.090441703796387,6.915447235107422,-1.5415401458740234,3.7958106994628906,0.0986185073852539,0.8353147506713867,0.7858829498291016,5.877509117126465,9.976097106933594,3.912425994873047,8.743949890136719,-6.017789840698242,5.45135498046875,3.893125534057617,-3.398275375366211,2.258549690246582,-8.030557632446289,9.382156372070312,-6.824311256408691,6.943965911865234,8.187601089477539,8.891864776611328,8.909303665161133,5.186629295349121,4.798274993896484,6.7133026123046875,0.7433404922485352,7.593738555908203,-2.714451789855957,9.248296737670898,0.512542724609375,-4.266855716705322,3.717268943786621,-7.623180389404297,-9.612933158874512,6.724328994750977,9.939956665039062,8.345836639404297,-3.894925117492676,3.1876420974731445,4.389581680297852,-1.1260929107666016,4.810956954956055,7.696010589599609,1.3290061950683594,8.392778396606445,-5.41057825088501,5.919184684753418,3.1577577590942383,3.3465309143066406,3.54693603515625,5.795356750488281,-2.411870002746582,-5.880288124084473,9.982900619506836,-2.549038887023926,1.8893089294433594,8.22258186340332,-3.6876773834228516,-9.5866117477417,1.9172334671020508,-2.2334775924682617,5.032163619995117,5.0260772705078125,8.276374816894531,4.0992841720581055,-0.757197380065918,3.797588348388672,9.336894989013672,9.028602600097656]},\"selected\":{\"id\":\"1178\"},\"selection_policy\":{\"id\":\"1177\"}},\"id\":\"1106\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1123\",\"type\":\"BasicTicker\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1105\"},\"glyph\":{\"id\":\"1141\"},\"group\":null,\"hover_glyph\":{\"id\":\"1143\"},\"muted_glyph\":{\"id\":\"1144\"},\"nonselection_glyph\":{\"id\":\"1142\"},\"view\":{\"id\":\"1146\"}},\"id\":\"1145\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1152\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"fill_color\":{\"value\":\"orange\"},\"line_width\":{\"value\":2},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1143\",\"type\":\"Circle\"},{\"attributes\":{\"source\":{\"id\":\"1105\"}},\"id\":\"1146\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1153\",\"type\":\"AllLabels\"},{\"attributes\":{\"line_alpha\":0.1,\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1185\",\"type\":\"Line\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1132\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1145\"}],\"tooltips\":[[\"Category\",\"@label\"],[\"y\",\"@y{0.000}\"],[\"x\",\"@x{0.000}\"]]},\"id\":\"1159\",\"type\":\"HoverTool\"},{\"attributes\":{\"label\":{\"value\":\"Separating line\"},\"renderers\":[{\"id\":\"1187\"}]},\"id\":\"1203\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"field\":\"label\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1141\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1201\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"field\":\"label\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1162\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1119\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1154\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"x\":[-9.999999602635718,10],\"y\":[-10,2.000000238418579]},\"selected\":{\"id\":\"1201\"},\"selection_policy\":{\"id\":\"1200\"}},\"id\":\"1183\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1184\",\"type\":\"Line\"},{\"attributes\":{\"axis\":{\"id\":\"1118\"},\"coordinates\":null,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1121\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1155\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1110\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1131\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1126\",\"type\":\"PanTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1132\"}},\"id\":\"1128\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1129\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1130\",\"type\":\"ResetTool\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1158\"},{\"id\":\"1180\"},{\"id\":\"1203\"}]},\"id\":\"1157\",\"type\":\"Legend\"},{\"attributes\":{},\"id\":\"1112\",\"type\":\"DataRange1d\"},{\"attributes\":{\"line_alpha\":0.2,\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1186\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1114\",\"type\":\"LinearScale\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.1},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1163\",\"type\":\"Circle\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1106\"},\"glyph\":{\"id\":\"1162\"},\"group\":null,\"hover_glyph\":{\"id\":\"1164\"},\"muted_glyph\":{\"id\":\"1165\"},\"nonselection_glyph\":{\"id\":\"1163\"},\"view\":{\"id\":\"1167\"}},\"id\":\"1166\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Synthetic data with categories\"},\"id\":\"1108\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1165\",\"type\":\"Circle\"},{\"attributes\":{\"tools\":[{\"id\":\"1126\"},{\"id\":\"1127\"},{\"id\":\"1128\"},{\"id\":\"1129\"},{\"id\":\"1130\"},{\"id\":\"1131\"},{\"id\":\"1159\"},{\"id\":\"1181\"}]},\"id\":\"1133\",\"type\":\"Toolbar\"},{\"attributes\":{\"label\":{\"value\":\"Category blue\"},\"renderers\":[{\"id\":\"1166\"}]},\"id\":\"1180\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1116\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis_label\":\"y\",\"coordinates\":null,\"formatter\":{\"id\":\"1149\"},\"group\":null,\"major_label_policy\":{\"id\":\"1150\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1123\"}},\"id\":\"1122\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_color\":{\"value\":\"orange\"},\"line_width\":{\"value\":2},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1164\",\"type\":\"Circle\"},{\"attributes\":{\"axis\":{\"id\":\"1122\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1125\",\"type\":\"Grid\"},{\"attributes\":{\"source\":{\"id\":\"1106\"}},\"id\":\"1167\",\"type\":\"CDSView\"},{\"attributes\":{\"source\":{\"id\":\"1183\"}},\"id\":\"1188\",\"type\":\"CDSView\"}],\"root_ids\":[\"1107\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.2\"}};\n",
       "  const render_items = [{\"docid\":\"2cfac753-c0ad-49d1-9524-9a3051c61419\",\"root_ids\":[\"1107\"],\"roots\":{\"1107\":\"6710117a-6f1b-49b7-ba09-8492e9cefbb2\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    let attempts = 0;\n",
       "    const timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1107"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Required for visualizing in Colab.\n",
    "output_notebook(hide_banner=True)\n",
    "\n",
    "points_with_categories = categorize_points(points, observed_categories)\n",
    "orange_cds = ColumnDataSource(\n",
    "    {\n",
    "        \"x\": points_with_categories[\"orange\"][\"x\"],\n",
    "        \"y\": points_with_categories[\"orange\"][\"y\"],\n",
    "        \"label\": points_with_categories[\"orange\"][\"label\"],\n",
    "    }\n",
    ")\n",
    "orange_tips = [(\"Category\", \"@label\"), (\"y\", \"@y{0.000}\"), (\"x\", \"@x{0.000}\")]\n",
    "blue_cds = ColumnDataSource(\n",
    "    {\n",
    "        \"x\": points_with_categories[\"blue\"][\"x\"],\n",
    "        \"y\": points_with_categories[\"blue\"][\"y\"],\n",
    "        \"label\": points_with_categories[\"blue\"][\"label\"],\n",
    "    }\n",
    ")\n",
    "blue_tips = [(\"Category\", \"@label\"), (\"y\", \"@y{0.000}\"), (\"x\", \"@x{0.000}\")]\n",
    "synthetic_data_with_categories_plot = plots.scatter_plot(\n",
    "    plot_sources=[orange_cds, blue_cds],\n",
    "    tooltips=[orange_tips, blue_tips],\n",
    "    figure_kwargs={\n",
    "        \"title\": \"Synthetic data with categories\",\n",
    "        \"x_axis_label\": \"x\",\n",
    "        \"y_axis_label\": \"y\",\n",
    "    },\n",
    "    legend_items=[\"Category orange\", \"Category blue\"],\n",
    "    plot_kwargs={\"fill_color\": \"label\"},\n",
    ")\n",
    "# Add the separating line.\n",
    "x, y = plot_line(true_slope, true_intercept)\n",
    "synthetic_data_with_categories_plot.line(\n",
    "    x=x,\n",
    "    y=y,\n",
    "    legend_label=\"Separating line\",\n",
    "    line_color=\"black\",\n",
    "    line_width=3,\n",
    "    line_alpha=1,\n",
    ")\n",
    "show(synthetic_data_with_categories_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "0781c3b9-4dcd-4420-b46a-7ff580eb6b7f",
    "showInput": false
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "0781c3b9-4dcd-4420-b46a-7ff580eb6b7f",
    "showInput": false
   },
   "source": [
    "We can now start building our model.\n",
    "\n",
    "The first thing we need is a prior distribution for our three coefficients.\n",
    "\n",
    "We have no reason to believe that the coefficients will be either positive or negative,\n",
    "so we should choose a prior distribution that is centered on zero. We use a matrix\n",
    "multiplication to compute the linear combination, and therefore make the prior a column\n",
    "vector of samples from a normal distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "@bm.random_variable\n",
    "def coefficients():\n",
    "    mean = torch.zeros(3, 1)\n",
    "    sigma = torch.ones(3, 1)\n",
    "    return dist.Normal(mean, sigma)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "ef0d7954-f0b2-4311-94cf-6acbbd7722c5",
    "showInput": false
   },
   "source": [
    "Our model for categories is now straightforward: each category is chosen by\n",
    "matrix-multiplying the point by the prior distribution of coefficients, and we get a set\n",
    "of categories from the Bernoulli distribution; either $0.0$ (blue) or $1.0$ (orange):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "@bm.random_variable\n",
    "def categories():\n",
    "    return dist.Bernoulli(logits=points.mm(coefficients()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "9138c92e-31ab-46f2-8428-1e2d0eea1372",
    "showInput": false
   },
   "source": [
    "## Inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "9138c92e-31ab-46f2-8428-1e2d0eea1372",
    "showInput": false
   },
   "source": [
    "We can now infer the posterior distribution of the coefficients given the observations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_samples = 2 if smoke_test else 2000\n",
    "num_adaptive_samples = 0 if smoke_test else num_samples // 2\n",
    "num_chains = 1 if smoke_test else 4\n",
    "\n",
    "observations = {categories(): observed_categories.view(N, 1)}\n",
    "queries = [coefficients()]\n",
    "mc = bm.GlobalNoUTurnSampler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fe46d8631e4b4c4ab567774f2d93ae9a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Samples collected:   0%|          | 0/3000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d39789107d1e41eaa06248bac595399d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Samples collected:   0%|          | 0/3000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d90fdeab478a4e2a995ba6e4a9c745c9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Samples collected:   0%|          | 0/3000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2d7d4a08b8b34ec1b0d74568e2debca6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Samples collected:   0%|          | 0/3000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 33.2 s, sys: 107 ms, total: 33.3 s\n",
      "Wall time: 33.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "samples = mc.infer(\n",
    "    queries=queries,\n",
    "    observations=observations,\n",
    "    num_samples=num_samples,\n",
    "    num_chains=num_chains,\n",
    "    num_adaptive_samples=num_adaptive_samples,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "sampled_coefficients = samples.get_chain()[coefficients()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "257e2dbe-845b-424d-af83-bf86678db8fa",
    "showInput": false
   },
   "source": [
    "The slopes and intercepts are computed from the sampled coefficients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "slopes = [-float(s[1] / s[2]) for s in sampled_coefficients if float(s[2]) != 0.0]\n",
    "intercepts = [-float(s[0] / s[2]) for s in sampled_coefficients if float(s[2]) != 0.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "b4fabded-f2ac-40b0-b386-4498a286580a",
    "showInput": false
   },
   "source": [
    "## Posterior results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "b4fabded-f2ac-40b0-b386-4498a286580a",
    "showInput": false
   },
   "source": [
    "A histogram of the inferred slopes should cluster near the true value, marked in red. A\n",
    "slight deviation is to be expected since there is noise in our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  const force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  const JS_MIME_TYPE = 'application/javascript';\n",
       "  const HTML_MIME_TYPE = 'text/html';\n",
       "  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  const CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    const script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    const cell = handle.cell;\n",
       "\n",
       "    const id = cell.output_area._bokeh_element_id;\n",
       "    const server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd_clean, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            const id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd_destroy);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    const output_area = handle.output_area;\n",
       "    const output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      const bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      const script_attrs = bk_div.children[0].attributes;\n",
       "      for (let i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      const toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    const events = require('base/js/events');\n",
       "    const OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  const NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    const el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error(url) {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < css_urls.length; i++) {\n",
       "      const url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < js_urls.length; i++) {\n",
       "      const url = js_urls[i];\n",
       "      const element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n",
       "  const css_urls = [];\n",
       "  \n",
       "\n",
       "  const inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (let i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n  const css_urls = [];\n  \n\n  const inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"66d63d4e-9bff-4513-a21c-d5418c0d9603\" data-root-id=\"1299\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  const docs_json = {\"38f07ddc-c7f5-4ad1-8a47-7b8cfefaf248\":{\"defs\":[],\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1308\"}],\"center\":[{\"id\":\"1311\"},{\"id\":\"1315\"},{\"id\":\"1339\"}],\"height\":500,\"left\":[{\"id\":\"1312\"}],\"outline_line_color\":\"black\",\"renderers\":[{\"id\":\"1335\"}],\"title\":{\"id\":\"1364\"},\"toolbar\":{\"id\":\"1323\"},\"width\":800,\"x_range\":{\"id\":\"1300\"},\"x_scale\":{\"id\":\"1304\"},\"y_range\":{\"id\":\"1302\"},\"y_scale\":{\"id\":\"1306\"}},\"id\":\"1299\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1298\"},\"glyph\":{\"id\":\"1331\"},\"group\":null,\"hover_glyph\":{\"id\":\"1333\"},\"muted_glyph\":{\"id\":\"1334\"},\"nonselection_glyph\":{\"id\":\"1332\"},\"view\":{\"id\":\"1336\"}},\"id\":\"1335\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1300\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1306\",\"type\":\"LinearScale\"},{\"attributes\":{\"axis\":{\"id\":\"1308\"},\"coordinates\":null,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1311\",\"type\":\"Grid\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_color\":{\"value\":\"orange\"},\"left\":{\"field\":\"left\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1333\",\"type\":\"Quad\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"steelblue\"},\"hatch_alpha\":{\"value\":0.1},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1332\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1309\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1317\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"source\":{\"id\":\"1298\"}},\"id\":\"1336\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1304\",\"type\":\"LinearScale\"},{\"attributes\":{\"coordinates\":null,\"group\":null},\"id\":\"1364\",\"type\":\"Title\"},{\"attributes\":{\"coordinates\":null,\"formatter\":{\"id\":\"1370\"},\"group\":null,\"major_label_policy\":{\"id\":\"1371\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1309\"}},\"id\":\"1308\",\"type\":\"LinearAxis\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1322\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1371\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis_label\":\"Counts\",\"coordinates\":null,\"formatter\":{\"id\":\"1367\"},\"group\":null,\"major_label_policy\":{\"id\":\"1368\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1313\"}},\"id\":\"1312\",\"type\":\"LinearAxis\"},{\"attributes\":{\"coordinates\":null,\"dimension\":\"height\",\"group\":null,\"line_color\":\"red\",\"line_width\":3,\"location\":0.6000000238418579},\"id\":\"1339\",\"type\":\"Span\"},{\"attributes\":{},\"id\":\"1372\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1302\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1319\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1321\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1373\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1316\",\"type\":\"PanTool\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"steelblue\"},\"hatch_alpha\":{\"value\":0.2},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1334\",\"type\":\"Quad\"},{\"attributes\":{\"data\":{\"bottom\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[23]},\"label\":[\"0.266 - 0.294\",\"0.294 - 0.323\",\"0.323 - 0.351\",\"0.351 - 0.380\",\"0.380 - 0.408\",\"0.408 - 0.437\",\"0.437 - 0.465\",\"0.465 - 0.494\",\"0.494 - 0.522\",\"0.522 - 0.551\",\"0.551 - 0.579\",\"0.579 - 0.608\",\"0.608 - 0.636\",\"0.636 - 0.665\",\"0.665 - 0.693\",\"0.693 - 0.722\",\"0.722 - 0.751\",\"0.751 - 0.779\",\"0.779 - 0.808\",\"0.808 - 0.836\",\"0.836 - 0.865\",\"0.865 - 0.893\",\"0.893 - 0.922\"],\"left\":[0.2657434344291687,0.2942587007646975,0.3227739671002264,0.3512892334357552,0.37980449977128405,0.40831976610681286,0.4368350324423417,0.4653502987778705,0.49386556511339935,0.5223808314489282,0.550896097784457,0.5794113641199858,0.6079266304555146,0.6364418967910435,0.6649571631265723,0.6934724294621012,0.72198769579763,0.7505029621331588,0.7790182284686876,0.8075334948042165,0.8360487611397454,0.8645640274752742,0.893079293810803],\"right\":[0.2942587007646975,0.3227739671002264,0.3512892334357552,0.37980449977128405,0.40831976610681286,0.4368350324423417,0.4653502987778705,0.49386556511339935,0.5223808314489282,0.550896097784457,0.5794113641199858,0.6079266304555146,0.6364418967910435,0.6649571631265723,0.6934724294621012,0.72198769579763,0.7505029621331588,0.7790182284686876,0.8075334948042165,0.8360487611397454,0.8645640274752742,0.893079293810803,0.9215945601463318],\"top\":[1,0,3,11,15,50,92,123,191,268,266,277,214,187,134,85,37,25,14,3,1,2,1]},\"selected\":{\"id\":\"1373\"},\"selection_policy\":{\"id\":\"1372\"}},\"id\":\"1298\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"overlay\":{\"id\":\"1322\"}},\"id\":\"1318\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1335\"}],\"tooltips\":[[\"Counts\",\"@top\"],[\"Bin\",\"@label\"]]},\"id\":\"1337\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1313\",\"type\":\"BasicTicker\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"steelblue\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1331\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1367\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1370\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis\":{\"id\":\"1312\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1315\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1368\",\"type\":\"AllLabels\"},{\"attributes\":{\"tools\":[{\"id\":\"1316\"},{\"id\":\"1317\"},{\"id\":\"1318\"},{\"id\":\"1319\"},{\"id\":\"1320\"},{\"id\":\"1321\"},{\"id\":\"1337\"}]},\"id\":\"1323\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1320\",\"type\":\"ResetTool\"}],\"root_ids\":[\"1299\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.2\"}};\n",
       "  const render_items = [{\"docid\":\"38f07ddc-c7f5-4ad1-8a47-7b8cfefaf248\",\"root_ids\":[\"1299\"],\"roots\":{\"1299\":\"66d63d4e-9bff-4513-a21c-d5418c0d9603\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    let attempts = 0;\n",
       "    const timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1299"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Required for visualizing in Colab.\n",
    "output_notebook(hide_banner=True)\n",
    "\n",
    "slope_hist_plot = plots.histogram_plot(slopes)\n",
    "# Add a line showing the true slope.\n",
    "span = Span(\n",
    "    location=true_slope,\n",
    "    dimension=\"height\",\n",
    "    line_color=\"red\",\n",
    "    line_width=3,\n",
    ")\n",
    "slope_hist_plot.add_layout(span)\n",
    "show(slope_hist_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "61a761a5-509a-4be1-9488-cb89a246bc7e"
   },
   "source": [
    "And similarly for the intercepts. Our model actually predicts a higher median intercept\n",
    "to capture the orange points above the slope in our ground truth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  const force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  const JS_MIME_TYPE = 'application/javascript';\n",
       "  const HTML_MIME_TYPE = 'text/html';\n",
       "  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  const CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    const script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    const cell = handle.cell;\n",
       "\n",
       "    const id = cell.output_area._bokeh_element_id;\n",
       "    const server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd_clean, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            const id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd_destroy);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    const output_area = handle.output_area;\n",
       "    const output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      const bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      const script_attrs = bk_div.children[0].attributes;\n",
       "      for (let i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      const toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    const events = require('base/js/events');\n",
       "    const OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  const NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    const el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error(url) {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < css_urls.length; i++) {\n",
       "      const url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < js_urls.length; i++) {\n",
       "      const url = js_urls[i];\n",
       "      const element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n",
       "  const css_urls = [];\n",
       "  \n",
       "\n",
       "  const inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (let i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n  const css_urls = [];\n  \n\n  const inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"5e6c2a35-603d-4d4a-8c72-5baad2fbd577\" data-root-id=\"1431\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  const docs_json = {\"db9c0893-2142-4fd2-a851-5c919e4ac96e\":{\"defs\":[],\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1440\"}],\"center\":[{\"id\":\"1443\"},{\"id\":\"1447\"},{\"id\":\"1471\"}],\"height\":500,\"left\":[{\"id\":\"1444\"}],\"outline_line_color\":\"black\",\"renderers\":[{\"id\":\"1467\"}],\"title\":{\"id\":\"1507\"},\"toolbar\":{\"id\":\"1455\"},\"width\":800,\"x_range\":{\"id\":\"1432\"},\"x_scale\":{\"id\":\"1436\"},\"y_range\":{\"id\":\"1434\"},\"y_scale\":{\"id\":\"1438\"}},\"id\":\"1431\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"1516\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"1432\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1436\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1434\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1438\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"1451\",\"type\":\"SaveTool\"},{\"attributes\":{\"coordinates\":null,\"group\":null},\"id\":\"1507\",\"type\":\"Title\"},{\"attributes\":{\"axis_label\":\"Counts\",\"coordinates\":null,\"formatter\":{\"id\":\"1510\"},\"group\":null,\"major_label_policy\":{\"id\":\"1511\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1445\"}},\"id\":\"1444\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1449\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"overlay\":{\"id\":\"1454\"}},\"id\":\"1450\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"steelblue\"},\"hatch_alpha\":{\"value\":0.1},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1464\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1448\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1453\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"1441\",\"type\":\"BasicTicker\"},{\"attributes\":{\"source\":{\"id\":\"1430\"}},\"id\":\"1468\",\"type\":\"CDSView\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1467\"}],\"tooltips\":[[\"Counts\",\"@top\"],[\"Bin\",\"@label\"]]},\"id\":\"1469\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1510\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1430\"},\"glyph\":{\"id\":\"1463\"},\"group\":null,\"hover_glyph\":{\"id\":\"1465\"},\"muted_glyph\":{\"id\":\"1466\"},\"nonselection_glyph\":{\"id\":\"1464\"},\"view\":{\"id\":\"1468\"}},\"id\":\"1467\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"coordinates\":null,\"dimension\":\"height\",\"group\":null,\"line_color\":\"red\",\"line_width\":3,\"location\":-4.0},\"id\":\"1471\",\"type\":\"Span\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_color\":{\"value\":\"orange\"},\"left\":{\"field\":\"left\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1465\",\"type\":\"Quad\"},{\"attributes\":{},\"id\":\"1511\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis\":{\"id\":\"1440\"},\"coordinates\":null,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1443\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1513\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"axis\":{\"id\":\"1444\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1447\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1445\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"1514\",\"type\":\"AllLabels\"},{\"attributes\":{\"coordinates\":null,\"formatter\":{\"id\":\"1513\"},\"group\":null,\"major_label_policy\":{\"id\":\"1514\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1441\"}},\"id\":\"1440\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1452\",\"type\":\"ResetTool\"},{\"attributes\":{\"tools\":[{\"id\":\"1448\"},{\"id\":\"1449\"},{\"id\":\"1450\"},{\"id\":\"1451\"},{\"id\":\"1452\"},{\"id\":\"1453\"},{\"id\":\"1469\"}]},\"id\":\"1455\",\"type\":\"Toolbar\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"value\":\"steelblue\"},\"hatch_alpha\":{\"value\":0.2},\"left\":{\"field\":\"left\"},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1466\",\"type\":\"Quad\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1454\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"bottom\":{\"field\":\"bottom\"},\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"steelblue\"},\"left\":{\"field\":\"left\"},\"line_color\":{\"value\":\"white\"},\"right\":{\"field\":\"right\"},\"top\":{\"field\":\"top\"}},\"id\":\"1463\",\"type\":\"Quad\"},{\"attributes\":{\"data\":{\"bottom\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"order\":\"little\",\"shape\":[23]},\"label\":[\"-5.634 - -5.494\",\"-5.494 - -5.355\",\"-5.355 - -5.215\",\"-5.215 - -5.075\",\"-5.075 - -4.935\",\"-4.935 - -4.796\",\"-4.796 - -4.656\",\"-4.656 - -4.516\",\"-4.516 - -4.376\",\"-4.376 - -4.237\",\"-4.237 - -4.097\",\"-4.097 - -3.957\",\"-3.957 - -3.817\",\"-3.817 - -3.678\",\"-3.678 - -3.538\",\"-3.538 - -3.398\",\"-3.398 - -3.258\",\"-3.258 - -3.119\",\"-3.119 - -2.979\",\"-2.979 - -2.839\",\"-2.839 - -2.699\",\"-2.699 - -2.559\",\"-2.559 - -2.420\"],\"left\":[-5.634122848510742,-5.49436616897583,-5.354609489440918,-5.214852809906006,-5.075096130371094,-4.935339450836182,-4.7955827713012695,-4.655826091766357,-4.516069412231445,-4.376312732696533,-4.236556053161621,-4.096799373626709,-3.957042694091797,-3.8172860145568848,-3.6775293350219727,-3.5377726554870605,-3.3980159759521484,-3.2582592964172363,-3.118502616882324,-2.978745937347412,-2.8389892578125,-2.699232578277588,-2.559475898742676],\"right\":[-5.49436616897583,-5.354609489440918,-5.214852809906006,-5.075096130371094,-4.935339450836182,-4.7955827713012695,-4.655826091766357,-4.516069412231445,-4.376312732696533,-4.236556053161621,-4.096799373626709,-3.957042694091797,-3.8172860145568848,-3.6775293350219727,-3.5377726554870605,-3.3980159759521484,-3.2582592964172363,-3.118502616882324,-2.978745937347412,-2.8389892578125,-2.699232578277588,-2.559475898742676,-2.4197192192077637],\"top\":[6,6,14,18,23,37,45,96,111,151,210,232,220,213,176,156,120,70,49,20,12,7,8]},\"selected\":{\"id\":\"1516\"},\"selection_policy\":{\"id\":\"1515\"}},\"id\":\"1430\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1515\",\"type\":\"UnionRenderers\"}],\"root_ids\":[\"1431\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.2\"}};\n",
       "  const render_items = [{\"docid\":\"db9c0893-2142-4fd2-a851-5c919e4ac96e\",\"root_ids\":[\"1431\"],\"roots\":{\"1431\":\"5e6c2a35-603d-4d4a-8c72-5baad2fbd577\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    let attempts = 0;\n",
       "    const timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1431"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Required for visualizing in Colab.\n",
    "output_notebook(hide_banner=True)\n",
    "\n",
    "intercept_hist_plot = plots.histogram_plot(intercepts)\n",
    "# Add a line showing the true slope.\n",
    "span = Span(\n",
    "    location=true_intercept,\n",
    "    dimension=\"height\",\n",
    "    line_color=\"red\",\n",
    "    line_width=3,\n",
    ")\n",
    "intercept_hist_plot.add_layout(span)\n",
    "show(intercept_hist_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "413d1834-8af4-404c-8658-af43520a0d5b"
   },
   "source": [
    "Inference typically finds a reasonable distribution of possible lines that separate\n",
    "these two categories."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "ce592651-6157-4246-a1ff-5972639ef192"
   },
   "source": [
    "## Diagnostics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "ce592651-6157-4246-a1ff-5972639ef192"
   },
   "source": [
    "In addition to visualizing the lines directly, we can also use the diagnostics\n",
    "capabilities of Arviz to examine the diagnostics of our variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "|                     |   mean |    sd |   hdi_5.5% |   hdi_94.5% |   mcse_mean |   mcse_sd |   ess_bulk |   ess_tail |   r_hat |\n",
       "|:--------------------|-------:|------:|-----------:|------------:|------------:|----------:|-----------:|-----------:|--------:|\n",
       "| coefficients()[0,0] | -1.774 | 0.294 |     -2.245 |      -1.313 |       0.005 |     0.004 |    3561.2  |    5042.12 |   1.001 |\n",
       "| coefficients()[1,0] |  0.262 | 0.048 |      0.185 |       0.336 |       0.001 |     0.001 |    3543.11 |    4425.9  |   1     |\n",
       "| coefficients()[2,0] | -0.452 | 0.064 |     -0.548 |      -0.342 |       0.001 |     0.001 |    3137.52 |    4031.13 |   1.001 |"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filtered_samples = {k: v for k, v in samples.items() if k == coefficients()}\n",
    "az_data = MonteCarloSamples(filtered_samples).to_inference_data()\n",
    "summary_df = az.summary(az_data, round_to=3).to_markdown()\n",
    "Markdown(summary_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "originalKey": "e3ccd720-144e-43f6-9adb-3dee890e1832"
   },
   "source": [
    "As you can see, the average and median (50%) values inferred are reasonably close to the\n",
    "true values but there is a large spread. Remember that *any constant multiple of the\n",
    "true values would produce the same line*; if we got $-4.0$, $0.6$ and $-1.2$ then the\n",
    "slope and intercept would be the same and only the amount of \"mixing\" near the line\n",
    "would change, so we can expect that this inference problem may have some fairly large\n",
    "variance.\n",
    "\n",
    "The `n_eff` column is the effective sample size, which indicates how correlated the\n",
    "posterior samples are to each other; higher is better. This effective sample size is a\n",
    "little low.\n",
    "\n",
    "Notice that we generated the data by separating points along a line with slope $0.6$ and\n",
    "intercept $-4.0$, but were we to observe the *specific* set of 200 points we classified\n",
    "without knowing the true parameters ahead of time, we would deduce that the separating\n",
    "line had a slightly lower slope than $0.6$ and an intercept around $-3.0$.\n",
    "\n",
    "To more clearly illustrate the accuracy of the inference, we can take a random selection\n",
    "of the inferred lines and plot them on the data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(root) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  const force = true;\n",
       "\n",
       "  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n",
       "    root._bokeh_onload_callbacks = [];\n",
       "    root._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "  const JS_MIME_TYPE = 'application/javascript';\n",
       "  const HTML_MIME_TYPE = 'text/html';\n",
       "  const EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n",
       "  const CLASS_NAME = 'output_bokeh rendered_html';\n",
       "\n",
       "  /**\n",
       "   * Render data to the DOM node\n",
       "   */\n",
       "  function render(props, node) {\n",
       "    const script = document.createElement(\"script\");\n",
       "    node.appendChild(script);\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when an output is cleared or removed\n",
       "   */\n",
       "  function handleClearOutput(event, handle) {\n",
       "    const cell = handle.cell;\n",
       "\n",
       "    const id = cell.output_area._bokeh_element_id;\n",
       "    const server_id = cell.output_area._bokeh_server_id;\n",
       "    // Clean up Bokeh references\n",
       "    if (id != null && id in Bokeh.index) {\n",
       "      Bokeh.index[id].model.document.clear();\n",
       "      delete Bokeh.index[id];\n",
       "    }\n",
       "\n",
       "    if (server_id !== undefined) {\n",
       "      // Clean up Bokeh references\n",
       "      const cmd_clean = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n",
       "      cell.notebook.kernel.execute(cmd_clean, {\n",
       "        iopub: {\n",
       "          output: function(msg) {\n",
       "            const id = msg.content.text.trim();\n",
       "            if (id in Bokeh.index) {\n",
       "              Bokeh.index[id].model.document.clear();\n",
       "              delete Bokeh.index[id];\n",
       "            }\n",
       "          }\n",
       "        }\n",
       "      });\n",
       "      // Destroy server and session\n",
       "      const cmd_destroy = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n",
       "      cell.notebook.kernel.execute(cmd_destroy);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  /**\n",
       "   * Handle when a new output is added\n",
       "   */\n",
       "  function handleAddOutput(event, handle) {\n",
       "    const output_area = handle.output_area;\n",
       "    const output = handle.output;\n",
       "\n",
       "    // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n",
       "    if ((output.output_type != \"display_data\") || (!Object.prototype.hasOwnProperty.call(output.data, EXEC_MIME_TYPE))) {\n",
       "      return\n",
       "    }\n",
       "\n",
       "    const toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n",
       "\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n",
       "      toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n",
       "      // store reference to embed id on output_area\n",
       "      output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n",
       "    }\n",
       "    if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n",
       "      const bk_div = document.createElement(\"div\");\n",
       "      bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n",
       "      const script_attrs = bk_div.children[0].attributes;\n",
       "      for (let i = 0; i < script_attrs.length; i++) {\n",
       "        toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n",
       "        toinsert[toinsert.length - 1].firstChild.textContent = bk_div.children[0].textContent\n",
       "      }\n",
       "      // store reference to server id on output_area\n",
       "      output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function register_renderer(events, OutputArea) {\n",
       "\n",
       "    function append_mime(data, metadata, element) {\n",
       "      // create a DOM node to render to\n",
       "      const toinsert = this.create_output_subarea(\n",
       "        metadata,\n",
       "        CLASS_NAME,\n",
       "        EXEC_MIME_TYPE\n",
       "      );\n",
       "      this.keyboard_manager.register_events(toinsert);\n",
       "      // Render to node\n",
       "      const props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n",
       "      render(props, toinsert[toinsert.length - 1]);\n",
       "      element.append(toinsert);\n",
       "      return toinsert\n",
       "    }\n",
       "\n",
       "    /* Handle when an output is cleared or removed */\n",
       "    events.on('clear_output.CodeCell', handleClearOutput);\n",
       "    events.on('delete.Cell', handleClearOutput);\n",
       "\n",
       "    /* Handle when a new output is added */\n",
       "    events.on('output_added.OutputArea', handleAddOutput);\n",
       "\n",
       "    /**\n",
       "     * Register the mime type and append_mime function with output_area\n",
       "     */\n",
       "    OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n",
       "      /* Is output safe? */\n",
       "      safe: true,\n",
       "      /* Index of renderer in `output_area.display_order` */\n",
       "      index: 0\n",
       "    });\n",
       "  }\n",
       "\n",
       "  // register the mime type if in Jupyter Notebook environment and previously unregistered\n",
       "  if (root.Jupyter !== undefined) {\n",
       "    const events = require('base/js/events');\n",
       "    const OutputArea = require('notebook/js/outputarea').OutputArea;\n",
       "\n",
       "    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n",
       "      register_renderer(events, OutputArea);\n",
       "    }\n",
       "  }\n",
       "\n",
       "  \n",
       "  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n",
       "    root._bokeh_timeout = Date.now() + 5000;\n",
       "    root._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  const NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    const el = document.getElementById(null);\n",
       "    if (el != null) {\n",
       "      el.textContent = \"BokehJS is loading...\";\n",
       "    }\n",
       "    if (root.Bokeh !== undefined) {\n",
       "      if (el != null) {\n",
       "        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n",
       "      }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "\n",
       "  function run_callbacks() {\n",
       "    try {\n",
       "      root._bokeh_onload_callbacks.forEach(function(callback) {\n",
       "        if (callback != null)\n",
       "          callback();\n",
       "      });\n",
       "    } finally {\n",
       "      delete root._bokeh_onload_callbacks\n",
       "    }\n",
       "    console.debug(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(css_urls, js_urls, callback) {\n",
       "    if (css_urls == null) css_urls = [];\n",
       "    if (js_urls == null) js_urls = [];\n",
       "\n",
       "    root._bokeh_onload_callbacks.push(callback);\n",
       "    if (root._bokeh_is_loading > 0) {\n",
       "      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    root._bokeh_is_loading = css_urls.length + js_urls.length;\n",
       "\n",
       "    function on_load() {\n",
       "      root._bokeh_is_loading--;\n",
       "      if (root._bokeh_is_loading === 0) {\n",
       "        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n",
       "        run_callbacks()\n",
       "      }\n",
       "    }\n",
       "\n",
       "    function on_error(url) {\n",
       "      console.error(\"failed to load \" + url);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < css_urls.length; i++) {\n",
       "      const url = css_urls[i];\n",
       "      const element = document.createElement(\"link\");\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.rel = \"stylesheet\";\n",
       "      element.type = \"text/css\";\n",
       "      element.href = url;\n",
       "      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n",
       "      document.body.appendChild(element);\n",
       "    }\n",
       "\n",
       "    for (let i = 0; i < js_urls.length; i++) {\n",
       "      const url = js_urls[i];\n",
       "      const element = document.createElement('script');\n",
       "      element.onload = on_load;\n",
       "      element.onerror = on_error.bind(null, url);\n",
       "      element.async = false;\n",
       "      element.src = url;\n",
       "      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.head.appendChild(element);\n",
       "    }\n",
       "  };\n",
       "\n",
       "  function inject_raw_css(css) {\n",
       "    const element = document.createElement(\"style\");\n",
       "    element.appendChild(document.createTextNode(css));\n",
       "    document.body.appendChild(element);\n",
       "  }\n",
       "\n",
       "  \n",
       "  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n",
       "  const css_urls = [];\n",
       "  \n",
       "\n",
       "  const inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "    \n",
       "    \n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if (root.Bokeh !== undefined || force === true) {\n",
       "      \n",
       "    for (let i = 0; i < inline_js.length; i++) {\n",
       "      inline_js[i].call(root, root.Bokeh);\n",
       "    }\n",
       "    } else if (Date.now() < root._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!root._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      root._bokeh_failed_load = true;\n",
       "    } else if (force !== true) {\n",
       "      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (root._bokeh_is_loading === 0) {\n",
       "    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(css_urls, js_urls, function() {\n",
       "      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(window));"
      ],
      "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n\n  if (typeof root._bokeh_onload_callbacks === \"undefined\" || force === true) {\n    root._bokeh_onload_callbacks = [];\n    root._bokeh_is_loading = undefined;\n  }\n\n  \n\n  \n  if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  const NB_LOAD_WARNING = {'data': {'text/html':\n     \"<div style='background-color: #fdd'>\\n\"+\n     \"<p>\\n\"+\n     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n     \"</p>\\n\"+\n     \"<ul>\\n\"+\n     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n     \"</ul>\\n\"+\n     \"<code>\\n\"+\n     \"from bokeh.resources import INLINE\\n\"+\n     \"output_notebook(resources=INLINE)\\n\"+\n     \"</code>\\n\"+\n     \"</div>\"}};\n\n  function display_loaded() {\n    const el = document.getElementById(null);\n    if (el != null) {\n      el.textContent = \"BokehJS is loading...\";\n    }\n    if (root.Bokeh !== undefined) {\n      if (el != null) {\n        el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(display_loaded, 100)\n    }\n  }\n\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n\n    root._bokeh_onload_callbacks.push(callback);\n    if (root._bokeh_is_loading > 0) {\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    }\n    if (js_urls == null || js_urls.length === 0) {\n      run_callbacks();\n      return null;\n    }\n    console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n    root._bokeh_is_loading = css_urls.length + js_urls.length;\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n\n    function on_error(url) {\n      console.error(\"failed to load \" + url);\n    }\n\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }\n\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error.bind(null, url);\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  \n  const js_urls = [\"https://cdn.bokeh.org/bokeh/release/bokeh-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.4.2.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-mathjax-2.4.2.min.js\"];\n  const css_urls = [];\n  \n\n  const inline_js = [\n    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\n    function(Bokeh) {\n    \n    \n    }\n  ];\n\n  function run_inline_js() {\n    \n    if (root.Bokeh !== undefined || force === true) {\n      \n    for (let i = 0; i < inline_js.length; i++) {\n      inline_js[i].call(root, root.Bokeh);\n    }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    } else if (force !== true) {\n      const cell = $(document.getElementById(null)).parents('.cell').data().cell;\n      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n    }\n\n  }\n\n  if (root._bokeh_is_loading === 0) {\n    console.debug(\"Bokeh: BokehJS loaded, going straight to plotting\");\n    run_inline_js();\n  } else {\n    load_libs(css_urls, js_urls, function() {\n      console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n      run_inline_js();\n    });\n  }\n}(window));"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "  <div class=\"bk-root\" id=\"ade652e3-4cd6-4284-a538-795c26d76b36\" data-root-id=\"1573\"></div>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "(function(root) {\n",
       "  function embed_document(root) {\n",
       "    \n",
       "  const docs_json = {\"d4b08183-767e-49f9-b1a7-9e84fcbbd0b0\":{\"defs\":[],\"roots\":{\"references\":[{\"attributes\":{\"below\":[{\"id\":\"1584\"}],\"center\":[{\"id\":\"1587\"},{\"id\":\"1591\"},{\"id\":\"1623\"}],\"height\":500,\"left\":[{\"id\":\"1588\"}],\"outline_line_color\":\"black\",\"renderers\":[{\"id\":\"1611\"},{\"id\":\"1632\"},{\"id\":\"1651\"},{\"id\":\"1657\"}],\"title\":{\"id\":\"1574\"},\"toolbar\":{\"id\":\"1599\"},\"width\":700,\"x_range\":{\"id\":\"1576\"},\"x_scale\":{\"id\":\"1580\"},\"y_range\":{\"id\":\"1578\"},\"y_scale\":{\"id\":\"1582\"}},\"id\":\"1573\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.1},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1629\",\"type\":\"Circle\"},{\"attributes\":{\"label\":{\"value\":\"Separating line\"},\"renderers\":[{\"id\":\"1657\"}]},\"id\":\"1675\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"1595\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"1596\",\"type\":\"ResetTool\"},{\"attributes\":{\"data\":{\"label\":[\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\",\"orange\"],\"x\":[5.364436149597168,7.92889404296875,8.640007019042969,-6.9864044189453125,0.5259151458740234,4.021604537963867,-7.706235885620117,1.9901456832885742,7.603597640991211,-1.6845998764038086,3.845561981201172,2.0930137634277344,-2.516847610473633,-7.5305891036987305,1.501450538635254,0.5990018844604492,-4.970463752746582,5.406410217285156,-1.9261531829833984,6.326972961425781,7.996532440185547,0.4459991455078125,6.999334335327148,-1.9768152236938477,0.2831306457519531,5.148235321044922,-4.700774192810059,-5.611824989318848,3.1616592407226562,4.096654891967773,1.7964506149291992,1.9638080596923828,4.259291648864746,4.67014217376709,-2.2925925254821777,2.868997573852539,-4.879014492034912,-2.3881731033325195,-1.1381263732910156,9.190479278564453,5.621545791625977,8.491012573242188,4.960402488708496,0.6558837890625,8.311870574951172,0.45602893829345703,4.742164611816406,1.3930931091308594,-9.810628890991211,-4.953108787536621,5.533426284790039,9.145271301269531,6.182613372802734,9.190109252929688,-7.541866302490234,2.215825080871582,0.4779701232910156,9.429195404052734,5.7844438552856445,-2.846832275390625,7.763822555541992,4.359073638916016],\"y\":[-8.230451583862305,-0.8874406814575195,-6.4817962646484375,-9.365610122680664,-5.126834869384766,-5.923524856567383,-3.6646976470947266,-8.695826530456543,-9.975279808044434,-1.6456117630004883,-5.923035144805908,-7.800840377807617,-9.525980949401855,-7.713566780090332,-4.095302581787109,-4.8792877197265625,-7.663117408752441,-9.644318580627441,-1.9634265899658203,-1.153944969177246,-8.080951690673828,-7.049341201782227,-3.5946502685546875,1.2546634674072266,-8.666099548339844,-9.670921325683594,-9.454689979553223,-8.91575813293457,-0.2062854766845703,-3.3625354766845703,-5.1945695877075195,-7.424894332885742,3.958310127258301,-4.8647966384887695,-3.5094165802001953,-3.505624771118164,-9.660913467407227,-5.63709831237793,-5.78685998916626,4.089406967163086,-5.681890487670898,0.4131908416748047,-9.139385223388672,-6.664876937866211,-4.0002288818359375,-9.017190933227539,-8.018291473388672,-6.690833568572998,-8.54101276397705,-8.236889839172363,0.3215608596801758,-9.262796401977539,-2.7745461463928223,-2.690840721130371,-8.106698989868164,-2.293158531188965,-9.736145973205566,-2.227293014526367,-3.033538818359375,-8.280235290527344,-1.4968376159667969,-8.652210235595703]},\"selected\":{\"id\":\"1155\"},\"selection_policy\":{\"id\":\"1154\"}},\"id\":\"1105\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"1670\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data\":{\"xs\":[[-10,10],[-10,10],[-9.327946198498571,10],[-9.9044180288823,10],[-10,10],[-9.472003468509621,10],[-10,10],[-10,10],[-10,10],[-10,10],[-10,10],[-10,10],[-10,10],[-9.966026161311154,10],[-10,10],[-10,10],[-10,10],[-10,10],[-10,10],[-10,10],[-8.168714726276269,10],[-10,10],[-10,10],[-8.768089301840126,10],[-10,10]],\"ys\":[[-9.796119689941406,1.3845195770263672],[-9.918907165527344,1.7569799423217773],[-10,2.0514097213745117],[-10,0.14649081230163574],[-8.748709261417389,0.3776755928993225],[-10,1.5429437160491943],[-9.721304416656494,2.9314122200012207],[-9.511227130889893,1.4006333351135254],[-9.966621160507202,1.4182546138763428],[-9.955542802810669,2.4747650623321533],[-7.490387797355652,0.45260941982269287],[-9.089827418327332,1.9529403448104858],[-8.556718945503235,1.0354958772659302],[-10,1.480668306350708],[-9.553393840789795,2.0263891220092773],[-9.106978178024292,0.9051454067230225],[-9.276813983917236,2.6477293968200684],[-9.005211472511292,2.3687268495559692],[-9.767488598823547,3.0922099351882935],[-8.747766852378845,2.132804751396179],[-10,1.8859052658081055],[-9.06132173538208,2.0376782417297363],[-8.154603779315948,1.1638010144233704],[-10,2.48429799079895],[-9.647748112678528,2.8430382013320923]]},\"selected\":{\"id\":\"1671\"},\"selection_policy\":{\"id\":\"1670\"}},\"id\":\"1649\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1631\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"field\":\"label\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1628\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1585\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_color\":{\"value\":\"orange\"},\"line_width\":{\"value\":2},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1630\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1177\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1106\"},\"glyph\":{\"id\":\"1628\"},\"group\":null,\"hover_glyph\":{\"id\":\"1630\"},\"muted_glyph\":{\"id\":\"1631\"},\"nonselection_glyph\":{\"id\":\"1629\"},\"view\":{\"id\":\"1633\"}},\"id\":\"1632\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1671\",\"type\":\"Selection\"},{\"attributes\":{\"axis_label\":\"x\",\"coordinates\":null,\"formatter\":{\"id\":\"1618\"},\"group\":null,\"major_label_policy\":{\"id\":\"1619\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1585\"}},\"id\":\"1584\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1611\"}],\"tooltips\":[[\"Category\",\"@label\"],[\"y\",\"@y{0.000}\"],[\"x\",\"@x{0.000}\"]]},\"id\":\"1625\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1178\",\"type\":\"Selection\"},{\"attributes\":{\"axis\":{\"id\":\"1584\"},\"coordinates\":null,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1587\",\"type\":\"Grid\"},{\"attributes\":{\"data\":{\"label\":[\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\",\"blue\"],\"x\":[-3.851543426513672,-3.0221309661865234,-6.622820854187012,3.9533519744873047,-4.354628562927246,-2.0580015182495117,1.058140754699707,-6.295379638671875,8.595979690551758,-9.336947441101074,6.309379577636719,-0.3608236312866211,3.9688215255737305,-5.888023376464844,-6.930861473083496,5.489720344543457,2.317047119140625,8.285493850708008,-6.256053447723389,5.05708122253418,-9.89735221862793,9.407493591308594,-6.085391044616699,-8.432828903198242,1.4590110778808594,-9.592489242553711,-4.871133804321289,-2.1271514892578125,-0.5131931304931641,0.2779216766357422,6.358394622802734,9.494134902954102,-4.740772247314453,-8.440082550048828,-7.825094223022461,-8.634378433227539,-4.542748928070068,-0.6748723983764648,9.030031204223633,-2.093687057495117,4.8050537109375,-2.1030545234680176,7.939781188964844,-5.827054977416992,-0.2181720733642578,-7.559200286865234,3.6160755157470703,-0.07026481628417969,-5.241313934326172,-0.7180614471435547,-7.122807502746582,-1.4808740615844727,1.2483882904052734,-6.322139739990234,-4.238251686096191,7.502281188964844,-8.269224166870117,-5.266729354858398,-2.9231953620910645,-9.911483764648438,-6.673402786254883,5.295736312866211,-3.339596748352051,8.220346450805664,-6.494112014770508,0.31858348846435547,7.9708251953125,-3.3524060249328613,-0.44471168518066406,-6.164298057556152,-8.1978759765625,-3.648771286010742,-5.221515655517578,0.783778190612793,-7.861476898193359,9.011137008666992,-8.599210739135742,-6.912337303161621,9.996149063110352,5.603796005249023,4.831585884094238,-1.7994060516357422,4.254682540893555,-6.984709739685059,-9.865375518798828,5.410356521606445,-8.93045425415039,-6.494322776794434,8.224775314331055,7.721273422241211,4.400020599365234,9.996763229370117,-3.4999217987060547,-7.694808006286621,6.394889831542969,9.570829391479492,-3.3423290252685547,0.9570064544677734,-6.1743998527526855,-7.431445121765137,-7.101302146911621,2.1836414337158203,-9.078695297241211,-2.0286331176757812,5.384434700012207,-6.600689888000488,-9.818775177001953,-7.851617336273193,-3.0851659774780273,5.7502899169921875,-5.664152145385742,-1.018951416015625,-2.17431640625,-7.005898952484131,-8.37945556640625,5.011591911315918,-5.389503479003906,-2.9549336433410645,2.5174903869628906,0.8839836120605469,3.0433387756347656,-7.2439422607421875,-4.8449554443359375,-0.04822731018066406,-3.586721420288086,1.373854637145996,0.4643440246582031,-9.487632751464844,-6.326899528503418,-2.106570243835449,-3.409003257751465,7.835580825805664,-0.7440299987792969,-9.273699760437012,2.5059938430786133,2.0213184356689453,-9.03623104095459,9.473430633544922],\"y\":[2.6815738677978516,-1.9656543731689453,-4.1222310066223145,6.0002288818359375,3.632171630859375,7.483118057250977,9.05476188659668,-2.5316524505615234,4.462183952331543,-7.2256622314453125,5.863212585449219,6.395606994628906,1.3509283065795898,1.8634405136108398,-5.165835380554199,-1.2621736526489258,6.203765869140625,8.702072143554688,-9.319541931152344,7.158716201782227,-6.486968994140625,6.738178253173828,9.073699951171875,-2.4888429641723633,2.3717422485351562,-5.927066802978516,-3.4983339309692383,2.1375646591186523,7.15850830078125,-0.8626890182495117,9.472461700439453,-0.7232170104980469,6.809051513671875,-2.028367519378662,-2.114102840423584,-1.5647945404052734,3.7669925689697266,8.794195175170898,3.6215381622314453,7.14112663269043,3.5315895080566406,-8.240814208984375,6.842248916625977,3.417449951171875,0.4206809997558594,-6.865122318267822,1.2662601470947266,1.275930404663086,8.074928283691406,9.892387390136719,-2.8386425971984863,0.10938262939453125,8.956928253173828,4.485040664672852,2.9412269592285156,-3.21915864944458,0.13782501220703125,1.3217105865478516,-5.936941146850586,4.51393985748291,-5.761013984680176,7.67521858215332,-2.7944812774658203,2.718921661376953,-1.1376152038574219,-6.728816986083984,1.628387451171875,2.945554733276367,-6.090441703796387,6.915447235107422,-1.5415401458740234,3.7958106994628906,0.0986185073852539,0.8353147506713867,0.7858829498291016,5.877509117126465,9.976097106933594,3.912425994873047,8.743949890136719,-6.017789840698242,5.45135498046875,3.893125534057617,-3.398275375366211,2.258549690246582,-8.030557632446289,9.382156372070312,-6.824311256408691,6.943965911865234,8.187601089477539,8.891864776611328,8.909303665161133,5.186629295349121,4.798274993896484,6.7133026123046875,0.7433404922485352,7.593738555908203,-2.714451789855957,9.248296737670898,0.512542724609375,-4.266855716705322,3.717268943786621,-7.623180389404297,-9.612933158874512,6.724328994750977,9.939956665039062,8.345836639404297,-3.894925117492676,3.1876420974731445,4.389581680297852,-1.1260929107666016,4.810956954956055,7.696010589599609,1.3290061950683594,8.392778396606445,-5.41057825088501,5.919184684753418,3.1577577590942383,3.3465309143066406,3.54693603515625,5.795356750488281,-2.411870002746582,-5.880288124084473,9.982900619506836,-2.549038887023926,1.8893089294433594,8.22258186340332,-3.6876773834228516,-9.5866117477417,1.9172334671020508,-2.2334775924682617,5.032163619995117,5.0260772705078125,8.276374816894531,4.0992841720581055,-0.757197380065918,3.797588348388672,9.336894989013672,9.028602600097656]},\"selected\":{\"id\":\"1178\"},\"selection_policy\":{\"id\":\"1177\"}},\"id\":\"1106\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"label\":{\"value\":\"Category orange\"},\"renderers\":[{\"id\":\"1611\"}]},\"id\":\"1624\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom_units\":\"screen\",\"coordinates\":null,\"fill_alpha\":0.5,\"fill_color\":\"lightgrey\",\"group\":null,\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":1.0,\"line_color\":\"black\",\"line_dash\":[4,4],\"line_width\":2,\"right_units\":\"screen\",\"syncable\":false,\"top_units\":\"screen\"},\"id\":\"1598\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1582\",\"type\":\"LinearScale\"},{\"attributes\":{\"source\":{\"id\":\"1105\"}},\"id\":\"1612\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1597\",\"type\":\"HelpTool\"},{\"attributes\":{\"tools\":[{\"id\":\"1592\"},{\"id\":\"1593\"},{\"id\":\"1594\"},{\"id\":\"1595\"},{\"id\":\"1596\"},{\"id\":\"1597\"},{\"id\":\"1625\"},{\"id\":\"1647\"}]},\"id\":\"1599\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"1672\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"magenta\"},\"xs\":{\"field\":\"xs\"},\"ys\":{\"field\":\"ys\"}},\"id\":\"1650\",\"type\":\"MultiLine\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1105\"},\"glyph\":{\"id\":\"1607\"},\"group\":null,\"hover_glyph\":{\"id\":\"1609\"},\"muted_glyph\":{\"id\":\"1610\"},\"nonselection_glyph\":{\"id\":\"1608\"},\"view\":{\"id\":\"1612\"}},\"id\":\"1611\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"1106\"}},\"id\":\"1633\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"1673\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"text\":\"Synthetic data with categories\"},\"id\":\"1574\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"field\":\"label\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1607\",\"type\":\"Circle\"},{\"attributes\":{\"data\":{\"x\":[-9.999999602635718,10],\"y\":[-10,2.000000238418579]},\"selected\":{\"id\":\"1673\"},\"selection_policy\":{\"id\":\"1672\"}},\"id\":\"1653\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"overlay\":{\"id\":\"1598\"}},\"id\":\"1594\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"1154\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"1615\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"fill_color\":{\"value\":\"orange\"},\"line_width\":{\"value\":2},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1609\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1155\",\"type\":\"Selection\"},{\"attributes\":{\"coordinates\":null,\"group\":null,\"items\":[{\"id\":\"1624\"},{\"id\":\"1646\"},{\"id\":\"1675\"}]},\"id\":\"1623\",\"type\":\"Legend\"},{\"attributes\":{\"callback\":null,\"renderers\":[{\"id\":\"1632\"}],\"tooltips\":[[\"Category\",\"@label\"],[\"y\",\"@y{0.000}\"],[\"x\",\"@x{0.000}\"]]},\"id\":\"1647\",\"type\":\"HoverTool\"},{\"attributes\":{},\"id\":\"1618\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"source\":{\"id\":\"1649\"}},\"id\":\"1652\",\"type\":\"CDSView\"},{\"attributes\":{\"label\":{\"value\":\"Category blue\"},\"renderers\":[{\"id\":\"1632\"}]},\"id\":\"1646\",\"type\":\"LegendItem\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1649\"},\"glyph\":{\"id\":\"1650\"},\"group\":null,\"hover_glyph\":null,\"view\":{\"id\":\"1652\"}},\"id\":\"1651\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1616\",\"type\":\"AllLabels\"},{\"attributes\":{\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1654\",\"type\":\"Line\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.2},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.2},\"line_alpha\":{\"value\":0.2},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1610\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"field\":\"label\"},\"hatch_alpha\":{\"value\":0.1},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"white\"},\"size\":{\"value\":10},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1608\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"1593\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"1619\",\"type\":\"AllLabels\"},{\"attributes\":{\"axis_label\":\"y\",\"coordinates\":null,\"formatter\":{\"id\":\"1615\"},\"group\":null,\"major_label_policy\":{\"id\":\"1616\"},\"minor_tick_line_color\":\"grey\",\"ticker\":{\"id\":\"1589\"}},\"id\":\"1588\",\"type\":\"LinearAxis\"},{\"attributes\":{\"source\":{\"id\":\"1653\"}},\"id\":\"1658\",\"type\":\"CDSView\"},{\"attributes\":{\"axis\":{\"id\":\"1588\"},\"coordinates\":null,\"dimension\":1,\"grid_line_alpha\":0.2,\"grid_line_color\":\"grey\",\"grid_line_width\":0.2,\"group\":null,\"ticker\":null},\"id\":\"1591\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1589\",\"type\":\"BasicTicker\"},{\"attributes\":{\"coordinates\":null,\"data_source\":{\"id\":\"1653\"},\"glyph\":{\"id\":\"1654\"},\"group\":null,\"hover_glyph\":null,\"muted_glyph\":{\"id\":\"1656\"},\"nonselection_glyph\":{\"id\":\"1655\"},\"view\":{\"id\":\"1658\"}},\"id\":\"1657\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.2,\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1656\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1576\",\"type\":\"DataRange1d\"},{\"attributes\":{\"line_alpha\":0.1,\"line_width\":3,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1655\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1578\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"1592\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1580\",\"type\":\"LinearScale\"}],\"root_ids\":[\"1573\"]},\"title\":\"Bokeh Application\",\"version\":\"2.4.2\"}};\n",
       "  const render_items = [{\"docid\":\"d4b08183-767e-49f9-b1a7-9e84fcbbd0b0\",\"root_ids\":[\"1573\"],\"roots\":{\"1573\":\"ade652e3-4cd6-4284-a538-795c26d76b36\"}}];\n",
       "  root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "\n",
       "  }\n",
       "  if (root.Bokeh !== undefined) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    let attempts = 0;\n",
       "    const timer = setInterval(function(root) {\n",
       "      if (root.Bokeh !== undefined) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else {\n",
       "        attempts++;\n",
       "        if (attempts > 100) {\n",
       "          clearInterval(timer);\n",
       "          console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\");\n",
       "        }\n",
       "      }\n",
       "    }, 10, root)\n",
       "  }\n",
       "})(window);"
      ],
      "application/vnd.bokehjs_exec.v0+json": ""
     },
     "metadata": {
      "application/vnd.bokehjs_exec.v0+json": {
       "id": "1573"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Required for visualizing in Colab.\n",
    "output_notebook(hide_banner=True)\n",
    "\n",
    "# Replicate the original data separating plot above.\n",
    "randomly_selected_lines_plot = plots.scatter_plot(\n",
    "    plot_sources=[orange_cds, blue_cds],\n",
    "    tooltips=[orange_tips, blue_tips],\n",
    "    figure_kwargs={\n",
    "        \"title\": \"Synthetic data with categories\",\n",
    "        \"x_axis_label\": \"x\",\n",
    "        \"y_axis_label\": \"y\",\n",
    "    },\n",
    "    legend_items=[\"Category orange\", \"Category blue\"],\n",
    "    plot_kwargs={\"fill_color\": \"label\"},\n",
    ")\n",
    "\n",
    "# Add randomly selected sampled separating lines.\n",
    "num_lines = 25\n",
    "sampled_indices = torch.randint(0, len(slopes), (num_lines,)).tolist()\n",
    "xs = []\n",
    "ys = []\n",
    "for sampled_index in sampled_indices:\n",
    "    sampled_slope = slopes[sampled_index]\n",
    "    sampled_intercept = intercepts[sampled_index]\n",
    "    x, y = plot_line(sampled_slope, sampled_intercept)\n",
    "    xs.append(x)\n",
    "    ys.append(y)\n",
    "cds = ColumnDataSource({\"xs\": xs, \"ys\": ys})\n",
    "glyph = MultiLine(xs=\"xs\", ys=\"ys\", line_color=\"magenta\", line_alpha=0.2)\n",
    "randomly_selected_lines_plot.add_glyph(cds, glyph)\n",
    "\n",
    "# Add the separating line.\n",
    "x, y = plot_line(true_slope, true_intercept)\n",
    "randomly_selected_lines_plot.line(\n",
    "    x=x,\n",
    "    y=y,\n",
    "    legend_label=\"Separating line\",\n",
    "    line_color=\"black\",\n",
    "    line_width=3,\n",
    "    line_alpha=1,\n",
    ")\n",
    "\n",
    "show(randomly_selected_lines_plot)"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}