examples/user_guide/Performance_and_Debugging.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import panel as pn\n",
    "pn.extension('terminal')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When developing applications that are to be used by multiple users and which may process a lot of data it is important to ensure the application is well optimized. Additionally complex applications may have very complex callbacks which are difficult to trace and debug. In this user guide section we will walk you some of the best practices to debug your applications and profile your application to maximize performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Caching\n",
    "\n",
    "The Panel architecture ensures that multiple user sessions can run in the same process and therefore have access to the same global state. This means that we can cache data in Panel's global `state` object, either by directly assigning to the `pn.state.cache` dictionary object or by using the `pn.state.as_cached` helper function.\n",
    "\n",
    "To assign to the cache manually, simply put the data load or expensive calculation in an `if`/`else` block which checks whether the custom key is already present: \n",
    "\n",
    "```python\n",
    "if 'data' in pn.state.cache:\n",
    "    data = pn.state.cache['data']\n",
    "else:\n",
    "    pn.state.cache['data'] = data = ... # Load some data or perform an expensive computation\n",
    "```\n",
    "\n",
    "The `as_cached` helper function on the other hand allows providing a custom key and a function and automatically caching the return value. If provided the `args` and `kwargs` will also be hashed making it easy to cache (or memoize) on the arguments to the function: \n",
    "\n",
    "```python\n",
    "def load_data(*args, **kwargs):\n",
    "    return  ... # Load some data\n",
    "\n",
    "data = pn.state.as_cached('data', load_data, *args, **kwargs)\n",
    "```\n",
    "\n",
    "The first time the app is loaded the data will be cached and subsequent sessions will simply look up the data in the cache, speeding up the process of rendering. If you want to warm up the cache before the first user visits the application you can also provide the `--warm` argument to the `panel serve` command, which will ensure the application is initialized as soon as it is launched. If you want to populate the cache in a separate script from your main application you may also provide the path to a setup script using the `--setup` argument to `panel serve`. If you want to periodically update the cache look into the ability to [schedule tasks](Deploy_and_Export.ipynb#Scheduling-task-with-pn.state.schedule_task)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Layouts\n",
    "\n",
    "Panel is built on the Bokeh library which was originally written as a plotting library but grew into a more general framework for building data apps and dashboards. This history shows up in some architectural decisions and the main one is the way layouts are handled by Bokeh. Instead of relying on CSS to configure and compute the layouts Bokeh (and therefore Panel) rely on a layout engine which computes the sizes of different components to apply fixed positioning on the page. This works well but is an expensive operation and scales imperfectly the more components are added to a layout. In future, i.e. with the upcoming Bokeh 3.0 and Panel 1.0 releases the layout engine will be replaced with a CSS based framework. Until that time there are a few tricks to apply to avoid the layout engine degrading the performance of your application:\n",
    "\n",
    "The main approach to avoid bogging down the layout engine is to use [Templates](./Templates.ipynb) and add components as separate \"roots\" to the page. This means that instead of adding components to a layout like this:\n",
    "\n",
    "```python\n",
    "pn.Column(a, b, c, ...)\n",
    "```\n",
    "\n",
    "You add them as separate roots to the template:\n",
    "\n",
    "```python\n",
    "template = pn.template.BootstrapTemplate()\n",
    "\n",
    "template.main.append(a)\n",
    "template.main.append(b)\n",
    "template.main.append(c)\n",
    "```\n",
    "\n",
    "Doing this will ensure that the Bokeh.js layout engine considers each component separately and will speed up rendering a lot if `a`, `b` and `c` are deeply nested components."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Throttling\n",
    "\n",
    "One of the simplest ways to avoid slowing down your application is simply to control how often events from the frontend trigger code execution in Python. Particularly when using sliders this can be a problem. To solve this isssue sliders offer `value_throttled` parameters which are updated only when the user releases the slider unlike the `value` parameter which is updated continuously as the slider is dragged. If you are building apps using the reactive `pn.bind` or `pn.depends` functions you can depend on the `value_throttled` parameter directly:\n",
    "\n",
    "```python\n",
    "slider = pn.widgets.IntSlider()\n",
    "\n",
    "def output(value):\n",
    "    return ...\n",
    "\n",
    "pn.bind(output, slider.param.value_throttled)\n",
    "```\n",
    "\n",
    "Alternatively you can also ensure that all sliders only update on mouse release if you set `pn.config.throttled = True`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Concurrent processing\n",
    "\n",
    "When deploying a Panel application to be accessed by multiple users they will often access the same server simultaneously. To maintain responsiveness of the application when multiple users are interacting with it at the same time there are multiple approaches to concurrency, each with their own drawbacks and advantages:\n",
    "\n",
    "1. `Load balancing`: A load balancer launches multiple instances of the Panel application and distributes network traffic between them. This is ensures that users the load is distributed across multiple servers but also requires a lot configuration and resources.\n",
    "2. `Multiple processes`: Launches multiple processes on a single machine, effectively load balancing across the processes. Much simpler to set up than a load balancer but you are limited by the compute and memory resources on one machine.\n",
    "2. `Threading`: Attempts to distribute processing across multiple threads. Effectiveness depends on the operations being performed, I/O bound and CPU bound operations that release the GIL can easily be made concurrent in this way. \n",
    "3. `AsyncIO`: Allows asynchronously processing I/O bound operations. Effective for many concurrent I/O operations but requires rewriting your application and callbacks to make use of `async`/`await` paradigm.\n",
    "\n",
    "### Scaling across processes\n",
    "\n",
    "Both load balancing and starting multiple processes effectively spin up multiple copies of the same application and distribute the load across the processes. This results in duplication and therefore significantly higher overhead (basically scaling linearly with the number of processes). In applications where you are relying on global state (e.g. the `pn.state.cache`) this can introduce significant challenges to ensure that application state stays synchronized.\n",
    "\n",
    "#### Load balancing\n",
    "\n",
    "Setting up load balancing is a huge topic dependent on the precise system you are using so we won't go into any specific implementation here. In most cases you set up a reverse proxy (like NGINX) to distribute the load across multiple application servers. If you are using a system like Kubernetes it will also handle spinning up the servers for you and can even do so dynamically depending on the amount of concurrent users to ensure that you are not wasting resources when there are fewer users.\n",
    "\n",
    "<figure>\n",
    "<img src=\"https://www.nginx.com/wp-content/uploads/2014/07/what-is-load-balancing-diagram-NGINX-1024x518.png\" width=\"768\"></img>\n",
    "<figcaption>Diagram showing concept of load balacing (NGINX)</figcaption>\n",
    "</figure>\n",
    "\n",
    "Load balancing is the most complex approach to set up but is guaranteed to improve concurrent usage of your application since different users are not contending for access to the same process or even necessarily the same physical compute and memory resources. At the same time it is more wasteful of resources since it potentially occupies multiple machines and since each process is isolated there is no sharing of cached data or global state. \n",
    "\n",
    "#### Multiple processes\n",
    "\n",
    "Launching a Panel application on multiple processes is a effectively a simpler version of load balancing with many of the same advantages and drawbacks. One major advantage is that it is easy to set up, when deploying your application with `panel serve` simply configure `--num-procs N`, where N is the number of processes. Generally choose an `N` that is no larger than the number of processors on your machine. This still uses significantly more resources since each process has the same overhead and all processes will be contending for the same memory and compute resources. However if your application is single-threaded and you have sufficient memory this is a simple way to make your application scale.\n",
    "\n",
    "### Scaling within a single process\n",
    "\n",
    "Threading and async are both approaches to speed up processing in Python using concurrency in a single Python process. Since we can't provide a complete primer on either threading or asynchronous processing here, if you are not familiar with these concepts we recommend reading up on them before continuing. Read about [threading in Python here](https://realpython.com/intro-to-python-threading/) and [AsyncIO here](https://realpython.com/async-io-python/).\n",
    "\n",
    "When to use which approach cannot be answered easily and is never completely clear cut. As a general guide however use `asyncio` can scale almost arbitrarily allowing you to perform thousands or even millions of IO bound operations concurrently, while threading  limits you to the number of available threads. In practice this may never actually become relevant so the other main differences are that `async` coroutines are significantly more lightweight but that you have to carefully consider accessing shared objects across threads. Using `async` coroutines makes it very clear where concurrency occurs and therefore can make it easier to avoid race conditions and avoid having to think about locking a thread to access shared objects. However, in some situations threading can also be useful for CPU intensive operations where the operation being executed [releases the GIL](https://realpython.com/python-gil/), this includes many NumPy, Pandas and Numba functions.\n",
    "\n",
    "### Threading\n",
    "\n",
    "Using threading in Panel can either be enabled manually, e.g. by managing your own thread pool and dispatching concurrent tasks to it, or it can be managed by Panel itself by setting the `config.nthreads` parameter (or equivalently by setting it with `pn.extension(nthreads=...)`. This will start a `ThreadPoolExecutor` with the specified number of threads (or if set to `0` it will set the number of threads based on your system, i.e. `min(32, os.cpu_count() + 4)`). \n",
    "\n",
    "Whenever an event is generated or a periodic callback fires Panel will then automatically dispatch the event to the executor. An event in this case refers to any action generated on the frontend such as the manipulation of a widget by a user or the interaction with a plot. If you are launching an application with `panel serve` you should enable this option configure this option on the CLI by setting `--num-threads`.\n",
    "\n",
    "To demonstrate the effect of enabling threading take this example below:\n",
    "\n",
    "```python\n",
    "import panel as pn\n",
    "\n",
    "pn.extension(nthreads=2)\n",
    "\n",
    "def button_click(event):\n",
    "    print('Button clicked for the {event.new}th time.')\n",
    "    time.sleep(2) # Simulate long running operation\n",
    "    print('Finished processing {event.new}th click.')\n",
    "    \n",
    "button = pn.widgets.Button(name='Click me!')\n",
    "\n",
    "button.on_click(button_click)\n",
    "```\n",
    "\n",
    "When we click the button twice successively in a single-threaded context we will see the following output:\n",
    "\n",
    "```\n",
    "> Button clicked for the 1th time.\n",
    "... 2 second wait\n",
    "> Finished processing 1th click.\n",
    "> Button clicked for the 2th time.\n",
    "... 2 second wait\n",
    "> Finished processing 2th click.\n",
    "```\n",
    "\n",
    "In a threaded context on the other hand the two clicks will be processed concurrently:\n",
    "\n",
    "```\n",
    "> Button clicked for the 1th time.\n",
    "> Button clicked for the 2th time.\n",
    "... 2 second wait\n",
    "> Finished processing 1th click.\n",
    "> Finished processing 2th click.\n",
    "```\n",
    "\n",
    "### AsyncIO\n",
    "\n",
    "When using Python>=3.8 you can use async callbacks wherever you would ordinarily use a regular synchronous function. For instance you can use `pn.bind` on an async function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import aiohttp\n",
    "\n",
    "widget = pn.widgets.IntSlider(start=0, end=10)\n",
    "\n",
    "async def get_img(index):\n",
    "    async with aiohttp.ClientSession() as session:\n",
    "        async with session.get(f\"https://picsum.photos/800/300?image={index}\") as resp:\n",
    "            return pn.pane.JPG(await resp.read())\n",
    "            \n",
    "pn.Column(widget, pn.bind(get_img, widget))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example Panel will invoke the function and update the output when the function returns while leaving the process unblocked for the duration of the `aiohttp` request. \n",
    "\n",
    "Similarly you can attach asynchronous callbacks using `.param.watch`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "widget = pn.widgets.IntSlider(start=0, end=10)\n",
    "\n",
    "image = pn.pane.JPG()\n",
    "\n",
    "async def update_img(event):\n",
    "    async with aiohttp.ClientSession() as session:\n",
    "        async with session.get(f\"https://picsum.photos/800/300?image={event.new}\") as resp:\n",
    "            image.object = await resp.read()\n",
    "            \n",
    "widget.param.watch(update_img, 'value')\n",
    "widget.param.trigger('value')\n",
    "            \n",
    "pn.Column(widget, image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example Param will await the asynchronous function and the image will be updated when the request completes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Admin Panel\n",
    "\n",
    "The `/admin` panel provides an overview of the current application and provides tools for debugging and profiling. It can be enabled by passing the ``--admin`` argument to the `panel serve` command."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Overview\n",
    "\n",
    "The overview page provides some details about currently active sessions, running versions and resource usage (if `psutil` is installed).\n",
    "\n",
    "<img src=\"../assets/admin_overview.png\" width=\"70%\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Launch Profiler\n",
    "\n",
    "The launch profiler profiles the execution time of the initialization of a particular application. It can be enabled by setting a profiler using the commandline ``--profiler`` option. Available profilers include:\n",
    "\n",
    "- [`pyinstrument`](https://pyinstrument.readthedocs.io): A statistical profiler with nice visual output\n",
    "- [`snakeviz`](https://jiffyclub.github.io/snakeviz/): SnakeViz is a browser based graphical viewer for the output of Python’s cProfile module and an alternative to using the standard library pstats module.\n",
    "\n",
    "Once enabled the launch profiler will profile each application separately and provide the profiler output generated by the selected profiling engine.\n",
    "\n",
    "<img src=\"../assets/launch_profiler.png\" width=\"80%\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User profiling\n",
    "\n",
    "In addition to profiling the launch step of an application it is often also important to get insight into the interactive performance of an application. For that reason Panel also provides the `pn.io.profile` decorator that can be added to any callback and will report the profiling results in the `/admin` panel. The `profile` helper takes to arguments, the name to record the profiling results under and the profiling `engine` to use.\n",
    "\n",
    "```python\n",
    "@pn.io.profile('clustering', engine='snakeviz')\n",
    "def get_clustering(event):\n",
    "    # some expensive calculation\n",
    "    ...\n",
    "    \n",
    "widget.param.watch(my_callback, 'value')\n",
    "```\n",
    "\n",
    "<img src=\"../assets/user_profiling.png\" width=\"80%\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user profiling may also be used in an interactive session, e.g. we might decorate a simple callback with the `profile` decorator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "slider = pn.widgets.FloatSlider(name='Test')\n",
    "\n",
    "@pn.depends(slider)\n",
    "@pn.io.profile('formatting')\n",
    "def format_value(value):\n",
    "    time.sleep(1)\n",
    "    return f'Value: {value+1}'\n",
    "\n",
    "pn.Row(slider, format_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can request the named profile 'formatting' using the `pn.state.get_profile` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pn.state.get_profile('formatting')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logs\n",
    "\n",
    "The Logs page provides a detailed breakdown of the user interaction with the application. Additionally users may also log to this logger using the `pn.state.log` function, e.g. in this example we log the arguments to the clustering function:\n",
    "\n",
    "```python\n",
    "def get_clusters(x, y, n_clusters):\n",
    "    pn.state.log(f'clustering {x!r} vs {y!r} into {n_clusters} clusters.')\n",
    "    ...\n",
    "    return ...\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "<img src=\"../assets/admin_logs.png\" width=\"80%\"></img>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logging terminal may also be used interactively, however you have to ensure that the 'terminal' extension is loaded with `pn.extension('terminal')`. If the extension is initialized it can be rendered by accessing it on `pn.state.log_terminal`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "slider = pn.widgets.FloatSlider(name='Test')\n",
    "\n",
    "@pn.depends(slider)\n",
    "def format_value(value):\n",
    "    pn.state.log(f'formatting value {value}')\n",
    "    return f'Value: {value+1}'\n",
    "\n",
    "pn.Column(\n",
    "    pn.Row(slider, format_value),\n",
    "    pn.state.log_terminal,\n",
    "    sizing_mode='stretch_both'\n",
    ")\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}