Fake requirejs & JS transform specific to electron frontends #1714

rgbkrk · 2017-05-24T16:48:31Z

This is the beginning of custom transforms for electron based frontends to handle "Jupyter-style" JS and HTML in a secure and friendly way. Secure because we could lock it down. Friendly because we'll try to match some invariants:

Access to a partially limited requirejs as require -- will likely use the vm module for js runtime
- We'll provide some faked access to more common require paths, including vega and others since we already support vega
All the output from a single execution request (cell) should get rendered in the same webview

There's no code here yet, figured I should post where I'm headed with it.

After this PR I'm going to make a version that is web specific that will use the requirejs helper code.

Libraries to support

nbextensions/jupyter-vega/index
d3

Libraries definitely not going to be supported as part of this:

ipywidgets
pythreejs (relies on widgets)
the jupyter and ipython js APIs from the classic notebook
plotly - we already handle the plotly mimetype, we won't support the text/html version from them
holoviews
bokeh

Related PR: #1709

codecov-io · 2017-05-24T16:48:35Z

Codecov Report

Merging #1714 into master will increase coverage by 1.1%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #1714     +/-   ##
=========================================
+ Coverage   79.04%   80.15%   +1.1%     
=========================================
  Files          62       60      -2     
  Lines        2047     1990     -57     
=========================================
- Hits         1618     1595     -23     
+ Misses        429      395     -34

rgbkrk · 2017-05-24T23:06:58Z

It works!

Time to push the code.

rgbkrk · 2017-05-25T00:03:28Z

Ready for feedback @lgeiger, primarily for the sake of Hydrogen. I started with "just make it work" and haven't adapted the webview stuff yet. Outline a plan and I'll tackle it. 😄 I'm still iterating a bit with the requirejs setup, as I want to use something similar in #1709 for the web based frontends too.

jdetle · 2017-05-25T18:57:20Z

packages/transforms-electron/__tests__/index.test.js

+    expect(component.shouldComponentUpdate({ data: "<b>woo</b>" })).toBeFalsy();
+    expect(
+      component.shouldComponentUpdate({ data: "<b>womp</b>" })
+    ).toBeTruthy();


Didn't realize that Stephen Colbert wrote enzyme.

rgbkrk · 2017-05-26T18:04:55Z

Full query across all public notebooks for use of requirejs calls: https://bigquery.cloud.google.com/savedquery/300830567303:6116513b17ca4a77b58fec869fe5a846

😄

rgbkrk · 2017-05-26T18:27:34Z

As a result of doing these queries, the modules that we should support include:

d3
underscore / lodash
plotly
nbextensions/jupyter-vega/index
jquery
pivot
datatables
codemirror/mode/clike/clike...
crossfilter

Possible contenders:

topojson
escher
cartogram
krisk
slick_grid
data_grid
echarts
c3
brunel

Other large uses, haven't diagnosed much about them:

//dato.com/files/canvas/1.6.1/js/ipython_app.js

rgbkrk · 2017-05-26T18:44:03Z

Here are all the modules in public repos. These are uniqued yet not counted very well...:

  268 - //dato.com/files/canvas/1.6.1/js/ipython_app.js
  174 - //dato.com/files/canvas/1.5.2/js/ipython_app.js
  153 - plotly
  145 - nbextensions/jupyter-vega/index
   89 - d3
   81 - https://static.turi.com/products/graphlab-create/2.1/canvas/js/ipython_app.js
   69 - downloadable
   58 - base/js/utils
   49 - //dato.com/files/canvas/1.8.5/js/ipython_app.js
   38 - echarts
   37 - vintage
   37 - shine
   37 - roma
   37 - macarons
   37 - infographic
   37 - dark
   36 - //dato.com/files/canvas/1.7.1/js/ipython_app.js
   35 - style!/static/extensions/charting.css
   35 - extensions/charting
   33 - JSRootCore
   33 - //dato.com/files/canvas/1.8.4/js/ipython_app.js
   31 - imolecule
   29 - matta
   26 - cytoscape
   22 - https://static.turi.com/products/graphlab-create/2.0/canvas/js/ipython_app.js
   22 - //dato.com/files/canvas/1.6/js/ipython_app.js
   16 - //dato.com/files/canvas/1.9/js/ipython_app.js
   15 - escher
   13 - notebook
   11 - style!/static/extensions/bigquery.css
   11 - https://root.cern.ch/js/3.4/scripts/d3.v3.min.js
   11 - https://root.cern.ch/js/3.4/scripts/JSRootPainter.min.js
   11 - https://root.cern.ch/js/3.4/scripts/JSRootCore.min.js
   11 - extensions/bigquery
   11 - //turi.com/files/canvas/1.8.5/js/ipython_app.js
   10 - music21
   10 - codemirror/mode/clike/clike
   10 - //turi.com/files/canvas/1.6.911/js/ipython_app.js
    9 - //dato.com/files/canvas/1.8.3/js/ipython_app.js
    8 - igraph
    7 - ipnm
    7 - https://static.turi.com/products/graphlab-create/1.10/canvas/js/ipython_app.js
    6 - topojson
    6 - jgraph
    6 - //dato.com/files/canvas/1.10.1/js/ipython_app.js
    5 - widgets/js/widget
    5 - slick_grid
    5 - https://static.turi.com/products/graphlab-create/2.0.1/canvas/js/ipython_app.js
    5 - data_grid
    5 - base/js/dialog
    4 - widgets/js/manager
    4 - cloud
    4 - cartodblib
    4 - //turi.com/files/canvas/1.6.1/js/ipython_app.js
    3 - candela
    3 - c3
    3 - BrunelEventHandlers
    3 - BrunelData
    3 - BrunelD3
    3 - //turi.com/files/canvas/1.6.916/js/ipython_app.js
    3 - //dato.com/files/canvas/1.8.2/js/ipython_app.js
    2 - vis
    2 - three
    2 - sankey
    2 - pivot
    2 - parsets
    2 - mapboxgl
    2 - js/stacked.js
    2 - jquery
    2 - https://cdnjs.cloudflare.com/ajax/libs/Chart.js/1.0.2/Chart.min.js
    2 - http://fperucic.github.io/treant-js/vendor/raphael.js
    2 - dc
    2 - crossfilter
    2 - cola
    2 - cartogram
    2 - bootstrap
    2 - base/js/namespace
    2 - JSRootPainter
    2 - //turi.com/files/canvas/1.6/js/ipython_app.js
    2 - //dato.com/files/canvas/1.6.908/js/ipython_app.js

Decent survey though. Note that I filtered out any occurences that were less than 2.

rgbkrk · 2017-05-26T20:42:36Z

Note that dato / graphlab / turi come up often in the results as well, as part of a coursera course. I can't tell if they're well supported any more -- I'd like to use a canonical version as much as possible.

rgbkrk · 2017-05-26T21:32:15Z

Ok now that I've had a go at this for a bit, it's definitely seeming like we'd need to approach parts of this differently:

Long term plan

JS output and HTML output should have the same output context / area. This likely means doing a unified iframed (webview for electron) display area. We'd still inject dependencies (vega, plotly, etc.)

Short term plan

Provide document and window to the "sandbox" so we don't break current functionality. The only thing that's changing here is that we're providing a requirejs "shim" of sorts and for Atom, we'll be using vm. We'll also be able to provide better error messages for use of objects we don't have (e.g. IPython, Jupyter).

* Support the specific require path for both of these * Includes vega-embed and d3 in deps

rgbkrk · 2017-05-26T22:06:29Z

Once this passes, it's ready to go and provides us a nice path for supporting vega/altair, d3, and informing users in a much richer way about what we can and can't support.

lgeiger · 2017-05-28T21:35:07Z

Just tried out this branch 🎉

I'm hitting a strange bug when trying a simple bokeh example:

from bokeh.io import push_notebook, show, output_notebook
from bokeh.layouts import row, gridplot
from bokeh.plotting import figure, show, output_file
output_notebook()

import numpy as np

x = np.linspace(0, 4*np.pi, 100)
y = np.sin(x)
TOOLS = "pan,wheel_zoom,box_zoom,reset,save,box_select"

p1 = figure(title="Legend Example", tools=TOOLS)
p1.circle(x,   y, legend="sin(x)")
p1.circle(x, 2*y, legend="2*sin(x)", color="orange")
p1.circle(x, 3*y, legend="3*sin(x)", color="green")
show(p1)

This used to work in previous versions of nteract, but now the app completely crashes without any debug output.

rgbkrk · 2017-05-29T16:46:18Z

Whoa! Ok I'll check that out.

rgbkrk · 2017-08-10T23:55:20Z

I'm beginning to think that HTML output should end up in an iframe that we provide certain libraries into, while also handling the resize. I'm going to try out a little mockup.

rgbkrk · 2017-08-15T22:47:33Z

Will come back to this again later. Cleaning up PRs for now.

lock · 2018-04-02T15:28:44Z

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.

rgbkrk force-pushed the electron-transforms branch from 804d36d to 6f98be2 Compare May 24, 2017 16:53

rgbkrk requested a review from lgeiger May 25, 2017 00:01

rgbkrk force-pushed the electron-transforms branch 2 times, most recently from b0affc3 to 2756395 Compare May 25, 2017 01:04

jdetle reviewed May 25, 2017

View reviewed changes

rgbkrk force-pushed the electron-transforms branch from ad0657b to 6290cac Compare May 26, 2017 17:46

feat: transforms only for electron based frontends

14ec0b8

rgbkrk force-pushed the electron-transforms branch from 6290cac to 1ed9390 Compare May 26, 2017 20:21

rgbkrk force-pushed the electron-transforms branch from cd28ad4 to bdee152 Compare May 26, 2017 21:42

rgbkrk changed the title ~~[WIP] HTML and JS transforms specific to electron frontends~~ Fake requirejs & JS transform specific to electron frontends May 26, 2017

rgbkrk added 2 commits May 26, 2017 15:04

feat: support vega nbextension

1b55fec

* Support the specific require path for both of these * Includes vega-embed and d3 in deps

test: copy over the old html transform test

eb2aa80

rgbkrk force-pushed the electron-transforms branch from c2fa1f0 to eb2aa80 Compare May 26, 2017 22:04

rgbkrk mentioned this pull request May 26, 2017

[WIP] Start faking requirejs #1709

Closed

3 tasks

expand what we provide access to

48239fd

rgbkrk closed this Aug 15, 2017

rgbkrk deleted the electron-transforms branch August 15, 2017 22:47

lock bot locked and limited conversation to collaborators Apr 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fake requirejs & JS transform specific to electron frontends #1714

Fake requirejs & JS transform specific to electron frontends #1714

rgbkrk commented May 24, 2017 •

edited

Loading

codecov-io commented May 24, 2017 •

edited

Loading

rgbkrk commented May 24, 2017

rgbkrk commented May 25, 2017

jdetle May 25, 2017

rgbkrk commented May 26, 2017

rgbkrk commented May 26, 2017 •

edited

Loading

rgbkrk commented May 26, 2017 •

edited

Loading

rgbkrk commented May 26, 2017

rgbkrk commented May 26, 2017 •

edited

Loading

rgbkrk commented May 26, 2017

lgeiger commented May 28, 2017

rgbkrk commented May 29, 2017

rgbkrk commented Aug 10, 2017

rgbkrk commented Aug 15, 2017

lock bot commented Apr 2, 2018

Fake requirejs & JS transform specific to electron frontends #1714

Fake requirejs & JS transform specific to electron frontends #1714

Conversation

rgbkrk commented May 24, 2017 • edited Loading

codecov-io commented May 24, 2017 • edited Loading

Codecov Report

rgbkrk commented May 24, 2017

rgbkrk commented May 25, 2017

jdetle May 25, 2017

Choose a reason for hiding this comment

rgbkrk commented May 26, 2017

rgbkrk commented May 26, 2017 • edited Loading

rgbkrk commented May 26, 2017 • edited Loading

rgbkrk commented May 26, 2017

rgbkrk commented May 26, 2017 • edited Loading

Long term plan

Short term plan

rgbkrk commented May 26, 2017

lgeiger commented May 28, 2017

rgbkrk commented May 29, 2017

rgbkrk commented Aug 10, 2017

rgbkrk commented Aug 15, 2017

lock bot commented Apr 2, 2018

rgbkrk commented May 24, 2017 •

edited

Loading

codecov-io commented May 24, 2017 •

edited

Loading

rgbkrk commented May 26, 2017 •

edited

Loading

rgbkrk commented May 26, 2017 •

edited

Loading

rgbkrk commented May 26, 2017 •

edited

Loading