Use JavaScript output and save static PNG as well in the notebook #32
Conversation
|
Here is example of the output that this PR will produce. First, it sends an HTML output to the notebook with an empty div that will be used for the rendering and some inline CSS: <div class="vega-embed" id="cea3f6f3-620b-4c66-8372-60aa3a9a296e"></div>
<style>
.vega-embed svg, .vega-embed canvas {
border: 1px dotted gray;
}
.vega-embed .vega-actions a {
margin-right: 6px;
}
</style>Then it sends a JavaScript output that requires the nbextension and then renders the visualization into the div: var spec = {"data": {"values": [{"b": 28, "a": "A"}, {"b": 55, "a": "B"}, {"b": 43, "a": "C"}, {"b": 91, "a": "D"}, {"b": 81, "a": "E"}, {"b": 53, "a": "F"}, {"b": 19, "a": "G"}, {"b": 87, "a": "H"}, {"b": 52, "a": "I"}]}, "encoding": {"x": {"type": "ordinal", "field": "a"}, "y": {"type": "quantitative", "field": "b"}}, "mark": "bar", "config": {"cell": {"width": 500, "height": 350}}};
var selector = "#cea3f6f3-620b-4c66-8372-60aa3a9a296e";
var type = "vega-lite";
var output_area = this;
require(['nbextensions/vega/index'], function(vega) {
vega.render(selector, spec, type, output_area);
}); |
| if type(df.index) == pd.core.index.MultiIndex: | ||
| raise ValueError('Hierarchical indices not supported') | ||
| if type(df.columns) == pd.core.index.MultiIndex: | ||
| raise ValueError('Hierarchical indices not supported') |
There was a problem hiding this comment.
Maybe use an isinstance check here to catch derived classes as well.
|
Looks really good so far! I like the approach to defaults here. I've not yet had a chance to download and see the vega display in action, but I'll do that when you think it's ready. |
| <div class="vega-embed" data-type="vega-lite">{spec}</div> | ||
| <div class="vega-embed" id="{id}"></div> | ||
|
|
||
| <style> |
There was a problem hiding this comment.
Did you do this so that we don't need the js file for styling?
There was a problem hiding this comment.
Yeah, seemed a tad simpler, but I don't mind going back...
On Tue, May 17, 2016 at 3:41 PM, Dominik Moritz notifications@github.com
wrote:
In vega/static/vega-lite.html
#32 (comment):@@ -1 +1,11 @@
-{spec}
+
+
+<style>Did you do this so that we don't need the js file for styling?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/vega/ipyvega/pull/32/files/8fe2e0feea95138022e230298c9c4f4efbf0252e#r63617094
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com
There was a problem hiding this comment.
I think it totally makes sense for github where we can't use js.
|
Jake, can you successfully run "npm install && npm run build"? I still On Tue, May 17, 2016 at 3:32 PM, Jake Vanderplas notifications@github.com
Brian E. Granger |
|
|
||
| import pandas as pd | ||
|
|
||
|
|
||
| def update(d, u): | ||
| """Update dictionary.""" | ||
| def update(d, u, overwrite=True): |
There was a problem hiding this comment.
Instead of adding this option, why don't we just reorder the arguments in the call?
I'm getting the same error as Brian: |
|
OK, I think the code should work now, but I am still stuck at building the webpack bundle. |
| console.log(spec); | ||
| console.log(selector); | ||
| console.log(type); | ||
|
|
There was a problem hiding this comment.
Do you mean for these to stay in here? Some sort of permanent console message might actually be useful to keep.
There was a problem hiding this comment.
They are really helpful for now. I agree that some message in the console
would be helpful, but I probably wouldn't log the entire spec with the data
though. Maybe just the non-data part of the spec?
On Tue, May 17, 2016 at 4:20 PM, Jake Vanderplas notifications@github.com
wrote:
In vega/static/vega-lite.js
#32 (comment):@@ -2,6 +2,10 @@ var spec = {spec};
var selector = "{selector}";
var type = "{type}";+console.log(spec);
+console.log(selector);
+console.log(type);
+Do you mean for these to stay in here? Some sort of permanent console
message might actually be useful to keep.—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/vega/ipyvega/pull/32/files/d4541c081fe73ace7618ab34c6921f85cc63d86b..15e0a6819c108f72b5ef3dc647846de13bea216d#r63621113
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com
There was a problem hiding this comment.
Typically all console.log statements should be removed. I think they are only relevant for debugging.
|
@domoritz hmmm, I am looking at the build problems. The vega-embed |
|
I am making progress by just adding js to the list of dependencies in ipyvega. Almost have it working now... |
|
I use vega-embed form npm (nothing special). Yes, vega-embed does not require d3 directly, but vega does. See https://github.com/vega/vega/blob/master/package.json#L39 When you install d3 manually ( |
|
This is the content of my |
|
Do you see the same issues with master? |
* List d3 in dependencies * Debug everything else...
|
Yes, I see the same thing in master. I got it to work (see last commit) by just listing d3 in the package.json of this project. Here is proof it is working in this branch :-)
|
|
What method do I call on the view or spec that is returned by render to grab the base64 encoded image data? |
|
Very odd. |
|
I'm getting errors with this branch. Let me know when I can try it. |
|
This gives you the image data: result.view.toImageURL()It looks like this even works when the renderer is SVG. |
|
Hmm, trying to track down that last traceback. Can you post the code that On Tue, May 17, 2016 at 6:09 PM, Dominik Moritz notifications@github.com
Brian E. Granger |
|
I just launched the notebook from this repo and re-ran all cells. |
|
Very nice! One minor thing: on nbviewer there is a side-effect of an error being printed in the console when the vega plugin is loaded. Probably not a big deal, but it may cause someone unnecessary worry. Is there an easy way in JS to catch that exception somehow? |
|
Also – I noticed that while the vegalite functionality works, the vega functionality does not. How hard would it be to adapt what you've done for that as well? Or perhaps we should plan to merge this and update that separately? |
* Common base for vega and vegalite capabilities. * Simplified display. * Top-level imports. * Working notebooks.
ellisonbg
changed the title
|
OK this is ready for review. Everything should be ready, including vega functionality... |
|
Here are working notebooks that show vega and vegalite on GitHub and nbviewer: |
|
Do you think the console errors on nbviewer are worth worrying about? |
| if data is '' and 'data' not in spec: | ||
| raise ValueError('No data provided') | ||
| # Data provided, prepared it and put it into the spec | ||
| if not data is '': |
There was a problem hiding this comment.
probably == rather than is here. I'm not sure whether empty strings are pooled.
|
One thought: what if we handle the pandas encoding using a custom JSON encoder? i.e. something like import json
class JSONEncoder(json.JSONEncoder):
def default(self, obj):
if hasattr(obj,'to_json'):
return obj.to_json()
return json.JSONEncoder.default(self, obj)
spec = json.dumps(spec_dict, cls=JSONEncoder)It would simplify the logic within the classes, and also delegate encoding of dataframes to pandas. |
|
We have to avoid comparing a DataFrame to None as it raises a warning in We could use a custom JSONEncoder, but there are still some transformations On Wed, May 18, 2016 at 4:14 PM, Jake Vanderplas notifications@github.com
Brian E. Granger |
Sounds good |
Need to add tests for this nastiness!
|
OK I have fixed a few more things and addressed comments. Master of altair is also now working with this and I have updated its (non autogenerated) examples: https://github.com/ellisonbg/altair/tree/master/notebooks All render on GitHub! @jakevdp this was just the very basic of cleaning up the examples. If you want to start to work on a nicer set of example and tutorial notebooks for altair that would be great! |
|
@domoritz can we merge this now and iterate in further PRs? |
|
I started reviewing half an hour ago and rewrite some of the js. Will merge when I'm done testing. Looks awesome so far! |
|
See WIP at https://github.com/vega/ipyvega/tree/newrender. I'm just making sure that things sill work on nbviewer + github. |
|
@ellisonbg Is there a way to tell nbviewer to pull a new Version from github. I'm trying to simplify the js code but can't get new versions into nbviewer. |
|
🎉 |
|
Awesome, thanks for the review and merge! On Wed, May 18, 2016 at 6:51 PM, Dominik Moritz notifications@github.com
Brian E. Granger |
|
Add |
This is a work in progress...
This PR changes how output is rendered in Jupyter notebooks. The goal here is get ipyvega to work in the following three contexts:
This is accomplished by having a JavaScript output that works on the live notebook and having that JS also save a static PNG in the notebook output that will be used on GitHub and nbviewer.