Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save Bokeh Plot as Bokeh Plot #5231

Closed
mrocklin opened this issue Sep 25, 2016 · 15 comments
Closed

Save Bokeh Plot as Bokeh Plot #5231

mrocklin opened this issue Sep 25, 2016 · 15 comments

Comments

@mrocklin
Copy link
Contributor

@mrocklin mrocklin commented Sep 25, 2016

I would like to save a snapshot of Bokeh plot created by a Bokeh server application in all of the following ways:

  1. Download the contents of the ColumnDataSource as JSON/msgpack/whatever
  2. Download a static HTML file of the Bokeh plot
  3. Publish that bokeh plot as a gist

Dask's use case

For example with Dask.distributed, the current best way to have a discussion about performance is by referring to our task-stream diagnostic Bokeh plot. This occurs when I write blogposts discussing algorithms and when users have performance questions. If there was a Bokeh Tool that let users publish a static view of their task-stream plot as a gist from within the Dask Dashboard then it would elevate the level of conversation significantly (and produce a lot of cool looking Bokeh images). As a pleasant side effect, I would start including Bokeh plots way more often when writing blogposts and documentation.

How to do this?

When I briefly mentioned this to @birdsarah she recommended building a custom tool like the current SaveTool for Dask which would dump to one of the above forms rather than to png. However, looking at this a bit more I don't think that this functionality necessarily needs to be Dask-specific. This may be of general utility and would, I think, encourage people to embed Bokeh plots more often within broadcast publications. For example if they built a Bokeh plot in a notebook but wanted to include it in a blogpost then this might help.

@birdsarah birdsarah added this to the 0.12.4 milestone Sep 25, 2016
@gryBox
Copy link
Contributor

@gryBox gryBox commented Sep 26, 2016

This is something we are tried/trying to jerry rig. Some of the issues we ran into were (1) legend placement (2) combining multiple widgets with widgets that were not on a canvas (3) data tables.

Download the contents of the ColumnDataSource as JSON/msgpack/whatever

That was trivial, since ColumnDataSource (for us) originates as a dataframe. Personally not a fan of msgpack for this user case. We ran into datetime datatype issues when we wanted to read the file back for debugging.

Publish that bokeh plot as a gist

Yes!

I for one would love to have all three functions proposed by @mrocklin

@mattpap
Copy link
Contributor

@mattpap mattpap commented Sep 26, 2016

Download the contents of the ColumnDataSource as JSON/msgpack/whatever

As part of "whatever" we could consider CSV as proposed in PR #4228. Allowing JSON could be useful as well.

@gryBox
Copy link
Contributor

@gryBox gryBox commented Sep 26, 2016

Using the Bokeh stock example as a simple use case to illustrate some problems we encountered.

image

Problem(s)

There are plenty of times where the user may want to save only certain widgets/figures.
Example:

  1. A debug pretext window under all the plots - Rarely needs to get saved
  2. In the example above, maybe only the figures need to be saved.
  3. Data tables:
  • Saving bokeh pretext and data tables is not currently supported out of the box. Is this a separate PR? My hope was to eventually make legends out of data tables -
  • They can be 'to large to save.' I have seen comments on the bokeh board where people are loading copious amounts of data into tables. Imagine the pretext description in the doc above. What if that was a table with ~100,000 records. Does it get saved as part of the HTML or only what's 'being viewed'?

Since every Bokeh doc is unique it becomes hard to generalize what the user wants saved or not.

Proposed Solution

Creating a tools for a bokeh doc. It's first widget would be the SaveTool with the behavior of the HoverTool checkbox. Where each checkbox is mapped to the grid in `layout'.

# set up layout
widgets = column(ticker1, ticker2, stats)
main_row = row(corr, widgets)
series = column(ts1, ts2)
layout = column(main_row, series)

# List of checkboxes that magically appear in bokeh SaveTool
SaveTool(... ,components=[layout, main_row, series, corr, widgets, ticker1, ticker2, stats])

# Or a dictionary that can accept default values
SaveTool(... ,components= {layout : False, main_row: True ...})
@gryBox
Copy link
Contributor

@gryBox gryBox commented Sep 26, 2016

Download the contents of the ColumnDataSource as JSON/msgpack/whatever

@mrocklin Would odo work?
image

@mrocklin
Copy link
Contributor Author

@mrocklin mrocklin commented Nov 4, 2016

@mrocklin Would odo work?

Probably not, no

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Nov 4, 2016

All the current tools apply only to the plot they are on, and I don't think that should change. That is, an export tool that was on a particular plot should save information only for recreating that particular plot. If the intent is to save an entire layout or document, I think that should be handled differently. For example, a button, outside of any particular plot, could be configured with what to save. Although, I think no-configuration would be easier for users and developing alike: a button callback that just exports the current document as-is in its current state would be my definite preference.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Nov 4, 2016

There are two possible routes to go that I can think of:

  • export directly from BokehJS

    pros: works for any document, server or standalone
    cons: a fair amount of BokehJS code to write, no doc->json code exists currently in BokehJS (only the converse, json->doc exist)

  • export from a python server callback

    pros: probably easiest to get something up and running for the "whole-document" case, Bokeh python side already knows how to export docs to JSON
    cons: only works for server apps

@mrocklin
Copy link
Contributor Author

@mrocklin mrocklin commented Nov 4, 2016

Personally I care more about saving single plots than entire documents.
I'm also obviously biased towards bokeh server apps.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Nov 4, 2016

Personally I care more about saving single plots than entire documents.

That's probably actually a fairly harder case. Basically individual models can't be meaningfully or faithfully serialized outside the context of the entire document they inhabit. What if the plot has a CustomJS callback and the callback is configured with models or data sources for/from other plots? Things can't really be taken in isolation like that. The more I think about it, for a general tool, "whole-document" export is probably the only realistic option.

@mrocklin this is another reason I think you might be better off with a custom tool. A custom tool can bake in simplifying assumptions (in your case about the structure of the Dask dashboard app, and what is the exact and "minimum" set of things that could be surgically exported) that a general tool can't make.

@mrocklin
Copy link
Contributor Author

@mrocklin mrocklin commented Nov 4, 2016

OK, sounds like I should pursue this through a custom tool. There is really just one major plot that I care about. I'm ok to close this.

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Nov 20, 2016

Here is a reference implementation of whole document export:

import numpy as np
from numpy import pi

from bokeh.core.properties import Int, String
from bokeh.driving import cosine
from bokeh.embed import file_html
from bokeh.io import curdoc
from bokeh.models import Tool
from bokeh.plotting import figure
from bokeh.resources import CDN


JS_CODE = """
p = require "core/properties"
ActionTool = require "models/tools/actions/action_tool"


class ExportToolView extends ActionTool.View

  initialize: (options) ->
    super(options)
    @listenTo(@model, 'change:content', @export)

  do: () ->
    # This is just to trigger an event a python callback can respond to
    @model.event = @model.event + 1

  export: () ->
    if @model.content?
      x = window.open()
      x.document.open()
      x.document.write(@model.content)
      x.document.close()

class ExportTool extends ActionTool.Model
  default_view: ExportToolView
  type: "ExportTool"
  tool_name: "Export"
  icon: "bk-tool-icon-save"

  @define {
    event:   [ p.Int,   0 ]
    content: [ p.String   ]
  }

module.exports =
  Model: ExportTool
  View: ExportToolView
"""


class ExportTool(Tool):
    __implementation__ = JS_CODE
    event   = Int(default=0)
    content = String()

x = np.linspace(0, 4*pi, 80)
y = np.sin(x)

def export_callback(attr, old, new):
  # really, export the doc as JSON
  export_tool.content = None
  html = file_html(p, CDN, "exported plot")
  export_tool.content = html

export_tool = ExportTool()
export_tool.on_change('event', export_callback)

p = figure(tools=[export_tool])
r1 = p.line([0, 4*pi], [-1, 1], color="firebrick")
r2 = p.line(x, y, color="navy", line_width=4)

@cosine(w=0.03)
def update(step):
    r2.data_source.data["y"] = y * step
    r2.glyph.line_alpha = 1 - 0.8 * abs(step)

curdoc().add_root(p)
curdoc().add_periodic_callback(update, 50)
@bryevdv bryevdv modified the milestones: 0.12.5, 0.12.4 Nov 30, 2016
@bryevdv bryevdv modified the milestones: 0.12.6, 0.12.5 Feb 1, 2017
@bryevdv bryevdv modified the milestones: short-term, 0.12.6 Mar 23, 2017
@oarcher
Copy link

@oarcher oarcher commented Jun 22, 2018

This would be a nice feature !

One user working on a bokeh standalone html file with widgets may want to share his analysis with an other user. Currently, he has to say to the other user how to adjust sliders, etc ... to have the same plots.

Currently, using browser 'save as' feature produce an half working html. From the slider example, if One change sliders, and save the page to html, the document is duplicated in two parts:

  1. no plots, and sliders positioned to their good state, but non movable.
  2. a fully working document, but with sliders reset to their init state.

image

@andersy005
Copy link

@andersy005 andersy005 commented Aug 9, 2018

@mrocklin, I am trying to export bokeh plots from dask dashboard to html, and I was wondering if you could share the method you use to save plots (like the ones you have in your blog articles) in static html files? For instance, http://matthewrocklin.com/blog/work/2017/03/22/dask-glm-1. Thanks!

@bryevdv
Copy link
Member

@bryevdv bryevdv commented Aug 9, 2018

FYI I am soon working on making "export to JSON, load from JSON" simpler to do. It is possible to do this now:

from collections import OrderedDict
import json

from bokeh.embed.standalone import _ModelInDocument
from bokeh.embed.util import standalone_docs_json_and_render_items
from bokeh.plotting import figure
from bokeh.sampledata.iris import flowers

from flask import Flask
from jinja2 import Template

app = Flask(__name__)

page = Template("""
<!DOCTYPE html>
<html lang="en">
<head>
 <link href="https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.css" rel="stylesheet" type="text/css">
 <script src="https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.js"></script>
</head>

<body>
 <div class="bk-root" id="myplot" />

 <script>
   fetch('/plot').then(function(response) {
     return response.json();
   })
   .then(function(spec) {
     Bokeh.embed.embed_items(spec.docs_json, spec.render_items);
   })
 </script>
</body>

""")


@app.route('/')
def root():
   return page.render()

colormap = {'setosa': 'red', 'versicolor': 'green', 'virginica': 'blue'}
colors = [colormap[x] for x in flowers['species']]

@app.route('/plot')
def plot():
   p = figure(title = "Iris Morphology")
   p.xaxis.axis_label = 'Petal Length'
   p.yaxis.axis_label = 'Petal Width'

   p.circle(flowers["petal_length"], flowers["petal_width"],
            color=colors, fill_alpha=0.2, size=10)

   # Make this simple
   with _ModelInDocument([p]):
       (docs_json, [render_item]) = standalone_docs_json_and_render_items([p])

   roots = list(list(render_item.roots._roots.items())[0])
   roots[1] = "myplot"
   render_item.roots._roots = OrderedDict([roots])

   response = dict(docs_json=docs_json, render_items=[render_item.to_json()])
   return json.dumps(response)

if __name__ == '__main__':
   app.run()

Now the above only specifically concerned exporting to JSON from python more easily from python but the JS "embed" function needs to be made simpler too, and I will see about rounding things out with an easy way to also export from BokehJS. Combined with the new CustomAction custom toolbar button it would then be trivial to make a toolbar button to export to json and do whatever you liked with it.

I will use this issue for this work.

@bryevdv bryevdv modified the milestones: short-term, 1.0 Aug 9, 2018
@bryevdv
Copy link
Member

@bryevdv bryevdv commented Aug 13, 2018

Noting that as part of this work, BOKEH_SIMPLE_IDS=1 should be made the default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

7 participants
You can’t perform that action at this time.