notebooks/13_Deploying_Bokeh_Apps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.container { width: 100% }</style>\n",
    "<img style=\"float:left;  vertical-align:text-bottom;\" height=\"65\" width=\"172\" src=\"assets/PyViz_logo_wm_line.png\" />\n",
    "<div style=\"float:right; vertical-align:text-bottom;\"><h2>Tutorial 13. Deploying Bokeh Apps</h2></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous sections we discovered how to use a ``HoloMap`` to build a Jupyter notebook with interactive visualizations that can be exported to a standalone HTML file, as well as how to use ``DynamicMap`` and ``Streams`` to set up dynamic interactivity backed by the Jupyter Python kernel. However, frequently we want to package our visualization or dashboard for wider distribution, backed by Python but run outside of the notebook environment. Bokeh Server provides a flexible and scalable architecture to deploy complex interactive visualizations and dashboards, integrating seamlessly with Bokeh and with HoloViews.\n",
    "\n",
    "Bokeh server apps can be used for a wide range of applications, but here we will show how to use them with Datashader and related libraries:\n",
    "\n",
    "<div style=\"margin: 10px\">\n",
    "<a href=\"http://holoviews.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"./assets/holoviews.png\"/></a>\n",
    "<a href=\"http://geoviews.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"./assets/geoviews.png\"/></a>\n",
    "<a href=\"http://bokeh.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"./assets/bokeh.png\"/></a>\n",
    "<a href=\"http://ioam.github.io/param\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"./assets/param.png\"/></a><br><br>\n",
    "<a href=\"http://datashader.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:150px\" src=\"./assets/datashader.png\"/></a>\n",
    "<a href=\"http://dask.pydata.org\"><img style=\"margin:8px; display:inline; object-fit:scale-down; max-height:140px\" src=\"./assets/dask.png\"/></a>\n",
    "</div>\n",
    "\n",
    "For a detailed background on Bokeh Server see [the Bokeh user guide](http://bokeh.pydata.org/en/latest/docs/user_guide/server.html). In this tutorial we will discover how to deploy the visualizations we have created so far as a standalone Bokeh Server app, and how to flexibly combine HoloViews and ParamBokeh to build complex apps. We will also reuse a lot of what we have learned so far---loading large, tabular datasets, applying Datashader operations to them, and adding linked Streams to our app.\n",
    "\n",
    "## A simple Bokeh app\n",
    "\n",
    "The preceding sections of this tutorial focused solely on the Jupyter notebook, but now let's look at a bare Python script that can be deployed using Bokeh Server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('../apps/server_app.py', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 1 of this app should be very familiar by now -- declare that we are using Bokeh to render plots, load some taxi dropoff locations, declare a Points object, Datashade them, and set some plot options.\n",
    "\n",
    "At this point, if we were working with this code in a notebook, we would simply type ``shaded`` and let Jupyter's rich display support take over, rendering the object into a Bokeh plot and displaying it inline.  Here, step 2 adds the code necessary to do those steps explicitly:\n",
    "\n",
    "- get a handle on the Bokeh renderer object using ``hv.renderer``\n",
    "- create a Bokeh document from ``shaded`` by passing it to the renderer's ``server_doc`` method\n",
    "- optionally, change some properties of the Bokeh document like the title.\n",
    "\n",
    "This simple chunk of boilerplate code can be added to turn any HoloViews object into a fully functional, deployable Bokeh app!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploying the app\n",
    "\n",
    "Assuming that you have a terminal window open with the ``pyviz`` environment activated, in the ``../apps/`` directory, you can launch this app using Bokeh Server:\n",
    "\n",
    "```\n",
    "bokeh serve --show server_app.py\n",
    "```\n",
    "\n",
    "If you don't already have a favorite way to get a terminal, one way is to [open it from within Jupyter](../terminals/1), then make sure you are in the ``../apps`` directory, and make sure you are in the right Conda environment if you created one (activating it using ``source activate pyviz`` (or ``activate pyviz`` on Windows))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Modify the app to display the pickup locations and add a tilesource, then run the app with bokeh serve\n",
    "# Tip: Refer to the previous notebook\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building an app with custom widgets\n",
    "\n",
    "The above app script can be built entirely without using Jupyter, though we displayed it here using Jupyter for convenience in the tutorial.  Jupyter notebooks are also often helpful when initially developing such apps, allowing you to quickly iterate over visualizations in the notebook, deploying it as a standalone app only once we are happy with it. In this section we will combine everything we have learned so far including declaring of various parameters to control our visualization using a set of widgets.\n",
    "\n",
    "We begin as usual with a set of imports:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import holoviews as hv, geoviews as gv, param, parambokeh, dask.dataframe as dd\n",
    "\n",
    "from colorcet import cm_n\n",
    "from bokeh.document import Document\n",
    "from holoviews.operation.datashader import datashade\n",
    "from holoviews.streams import RangeXY\n",
    "from cartopy import crs\n",
    "\n",
    "hv.extension('bokeh', logo=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we once again load the Taxi dataset and define a tile source:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "usecols = ['dropoff_x', 'dropoff_y', 'pickup_x', 'pickup_y', 'dropoff_hour']\n",
    "df = dd.read_parquet('../data/nyc_taxi_wide.parq')\n",
    "df = df[usecols].persist()\n",
    "\n",
    "url='https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'\n",
    "tiles = gv.WMTS(url, crs=crs.GOOGLE_MERCATOR)\n",
    "tile_options = dict(width=600,height=400,xaxis=None,yaxis=None,bgcolor='black',show_grid=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we will put together a complete dashboard with a number of parameters controlling our visualization, including controls over the alpha level of the tiles and the colormap as well as the hour of day and whether to plot dropoff or pickup location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NYCTaxiExplorer(hv.streams.Stream):\n",
    "    alpha      = param.Magnitude(default=0.75, doc=\"Alpha value for the map opacity\")\n",
    "    colormap   = param.ObjectSelector(default=cm_n[\"fire\"], objects=cm_n.values())\n",
    "    hour       = param.Range(default=(0, 24), bounds=(0, 24))\n",
    "    location   = param.ObjectSelector(default='dropoff', objects=['dropoff', 'pickup'])\n",
    "\n",
    "    def make_view(self, x_range, y_range, **kwargs):\n",
    "        map_tiles = tiles.opts(style=dict(alpha=self.alpha), plot=tile_options)\n",
    "        points = hv.Points(df, kdims=[self.location+'_x', self.location+'_y'], vdims=['dropoff_hour'])\n",
    "        if self.hour != (0, 24): points = points.select(dropoff_hour=self.hour)\n",
    "        taxi_trips = datashade(points, x_sampling=1, y_sampling=1, cmap=self.colormap,\n",
    "                               dynamic=False, x_range=x_range, y_range=y_range, width=600, height=400)\n",
    "        return map_tiles * taxi_trips\n",
    "\n",
    "explorer = NYCTaxiExplorer(name=\"NYC Taxi Trips\")\n",
    "dmap = hv.DynamicMap(explorer.make_view, streams=[explorer, RangeXY()])\n",
    "plot = hv.renderer('bokeh').get_plot(dmap, doc=Document())\n",
    "parambokeh.Widgets(explorer, view_position='right', callback=explorer.event, plots=[plot])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's open the [text editor](../edit/apps/nyc_taxi/main.py) again and make this edit to a separate app, which we can then launch using Bokeh Server from the [terminal](../terminals/1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Note the differences between the server app and the app defined above\n",
    "#           then add an additional parameter and plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exercise: Click the link below and edit the Jinja2 template to customize the app "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Edit the template](../edit/apps/nyc_taxi/templates/index.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining HoloViews with bokeh models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now for a last hurrah let's put everything we have learned to good use and create a bokeh app with it. This time we will go straight to a [Python script containing the app](../edit/apps/player_app.py). If you run the app with ``bokeh serve --show ./apps/player_app.py`` from [your terminal](../terminals/1) you should see something like this:\n",
    "\n",
    "<img src=\"./assets/tutorial_app.gif\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This more complex app consists of several components:\n",
    "\n",
    "1. A datashaded plot of points for the indicated hour of the daty (in the slider widget)\n",
    "2. A linked ``PointerX`` stream,  to compute a cross-section\n",
    "3. A set of custom Bokeh widgets linked to the hour-of-day stream\n",
    "\n",
    "We have already covered 1. and 2. so we will focus on 3., which shows how easily we can combine a HoloViews plot with custom Bokeh models. We will not look at the precise widgets in too much detail; instead let's have a quick look at the callback defined for slider widget updates:\n",
    "\n",
    "```python\n",
    "def slider_update(attrname, old, new):\n",
    "    stream.event(hour=new)\n",
    "```\n",
    "\n",
    "Whenever the slider value changes this will trigger a stream event updating our plots. The second part is how we combine HoloViews objects and Bokeh models into a single layout we can display. Once again we can use the renderer to convert the HoloViews object into something we can display with Bokeh:\n",
    "\n",
    "```python\n",
    "renderer = hv.renderer('bokeh')\n",
    "plot = renderer.get_plot(hvobj, doc=curdoc())\n",
    "```\n",
    "\n",
    "The ``plot`` instance here has a ``state`` attribute that represents the actual Bokeh model, which means we can combine it into a Bokeh layout just like any other Bokeh model:\n",
    "\n",
    "```python\n",
    "layout = layout([[plot.state], [slider, button]], sizing_mode='fixed')\n",
    "curdoc().add_root(layout)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Advanced Exercise: Add a histogram to the bokeh layout next to the datashaded plot\n",
    "# Hint: Declare the histogram like this: hv.operation.histogram(aggregated, bin_range=(0, 20))\n",
    "#       then use renderer.get_plot and hist_plot.state and add it to the layout\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1-billion-point example\n",
    "\n",
    "If you have a machine with at least 16GB of memory, you can test its limits by trying out an app similar to the above ones, but using a dataset with more than 1 billion points.  Just download [osm-1billion.snappy.parq.zip](http://s3.amazonaws.com/datashader-data/osm-1billion.snappy.parq.zip), unzip it into ../data/, and run the app:\n",
    "\n",
    "```bash\n",
    "cd apps\n",
    "bokeh serve osm-1billion.py\n",
    "```\n",
    "\n",
    "The app will print out an address that you can paste into your browser, and when you load or reload that page the script will be executed (which could take a minute to load all the data, depending on your system's I/O performance).  You can then explore the dataset in your browser as for the smaller datasets above.  See the [OSM Datashader example](http://datashader.org/topics/osm-1billion.html) for more details.\n",
    "\n",
    "\n",
    "# Onwards\n",
    "\n",
    "Although some of the app code shown above is more complex than in previous tutorials, it's providing a huge range of custom types of interactivity, which if implemented in Bokeh alone would have required far more than a notebook cell of code.  Hopefully it is clear that arbitrarily complex collections of visualizations and interactive controls can be built from the components provided by HoloViews and the Param libraries, allowing you to make simple analyses very easily and making it practical to make even quite complex apps when needed.  The [user guide](http://holoviews.org/user_guide), [gallery](http://holoviews.org/gallery/index.html), and [reference gallery](http://holoviews.org/reference) should have all the information you need to get started with all this power on your own datasets and tasks.  Good luck!"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}