Proper method for including external JS libraries?

[randyzwitch]

(cc: @Carreau from our Twitter conversation)

Not sure if this is really IJulia or Jupyter, but it's for Vega.jl...

What's the proper way of including JavaScript libraries? Vega.jl has 5 JavaScript libraries; when I pop open a new webpage see code here, this obviously works because it's valid HTML.

What's the correct way to have these libraries available within in Jupyter Notebook, after using Vega gets called? I used to be able to have success with this code, just dumping the JS using readall. But this can't be the right way to do this....in fact, it doesn't work with the new version of the Vega libraries.

Any help is greatly appreciated!

[Carreau]

Quickly, and I'll try to re-respond later is to drop a require dependencies in the nbextension/... directory, that if your library just use require(['nbextension/dep'], function(dep){...}) it should load at the right time, and avoid the need to re-init, the libs.

We are pondering (or is it already implemented ?) a per-kernel nbextension or folder when kernel can put js libraries.

I understant that it might need an extra non-obvious step to drop the libs in the right places. We are working on a way to have kernel be able to "expose" javascript in a better way, but we miss manpower as usual to work on that.

[randyzwitch]

I'm not sure I'm following here, as other libraries seem to be able to plot inline in Jupyter Notebook without having anything in the nbextension as far as I know.

If I were to put the Vega/d3 libraries in the nbextension folder, are you saying that I could call that folder as it's own library of sorts and it would load the JavaScript? Ideally, the libraries would live in my package folder, so that I could update them as Vega gets updated, instead of having to write out to the nbextensions folder each time.

This blog post shows that that in Python, you can load JavaScript via a magic; does Julia have anything like this?

[stevengj]

You can just use display("text/html", "....") to output HTML text (potentially including Javascript) to be inserted into the notebook, although this has the issues described in jupyter/notebook#319. @Carreau, is the IPython %javascript magic doing anything different from this?

[Carreau]

Nop, that's exactly what it does. Here is the full source:

    @cell_magic
    def javascript(self, line, cell):
        """Run the cell block of Javascript code"""
        display(Javascript(cell))

[randyzwitch]

Thanks @stevengj, that confirms what I was doing.

I have this code in my module; when calling using Vega, the 5 <script> values are the result in the Jupyter cell:

  display("text/html", "<script src=\"http://vega.github.io/vega-editor/vendor/d3.min.js\" charset=\"utf-8\"></script>")
  display("text/html", "<script src=\"http://vega.github.io/vega-editor/vendor/d3.geo.projection.min.js\" charset=\"utf-8\"></script>")
  display("text/html", "<script src=\"http://vega.github.io/vega-editor/vendor/topojson.js\" charset=\"utf-8\"></script>")
  display("text/html", "<script src=\"http://vega.github.io/vega-editor/vendor/d3.layout.cloud.js\" charset=\"utf-8\"></script>")
  display("text/html", "<script src=\"http://vega.github.io/vega/vega.min.js\" charset=\"utf-8\"></script>")

Where I'm getting stuck is that I can see these script references in the DOM in the div.output_subarea.output_html.rendered_html div . Additionally, d3 gets created in the global scope. However, vg does not, which is what I actually need to reference in order to render Vega graphs. When I create a stand-alone HTML file, vg and d3 are both available at the JavaScript console.

So, if anyone could help me with this, I'd be greatly appreciated. The Vega folks already tried to help me, but I'm pretty convinced that this is just some general JavaScript problem. I don't know JavaScript, so I'm just trying everything and haven't been successful thus far :(

[randyzwitch]

I finally have gotten over the hurdle, realizing the issue is with the use of requirejs (which, having no front-end dev experience confuses the heck out of me.

Anyone running into this in the future, and assuming Jupyter doesn't change, this is vaguely the answer. I still have some bugs to work out, but running this code will eventually plot JS charts inline in the Notebook.

import Base.writemime
function writemime(io::IO, ::MIME"text/html", v::VegaVisualization)

        spec = JSON.json(tojs(v))
        divid = "vg" * randstring(3)

        display("text/html", """

              <body>
                <div id=\"$divid\"></div>
              </body>

              <script type="text/javascript">
              require.config({paths: {vega: "https://vega.github.io/vega/vega.min"}});

              require(["vega"], function(vg) {

                  vg.parse.spec($spec, function(chart) { chart({el:\"#$divid\"}).update(); });

              });
              </script>


              """)
end

[Carreau]

require.config({paths: {vega: "http://vega.github.io/vega/vega.min"}});

Boom, Man in the middle. -> https please, or you have remote code execution vulnerability. (edited your snipets).
And also some browser might just plainly refuse to load that in authenticated context.

you should be able to install vaga.min in nbextension and require nbextension/vega to have it working with no internet connexion.

Happy you figured it out.

[randyzwitch]

Thanks for noticing the https issue @Carreau, a silly oversight on my part.

When you say "install in nbextension", is it literally dropping a folder ~/.ipython/nbextensions/vega, then using similar requirejs code to load the libraries? You hit on the main thing I still haven't accomplished, which is that I don't want to require Internet.

[Carreau]

yes, if you install in ~/.ipython/nbextensions or ~/.jupyter/nbextensions or global /usr/local/jupyter.. etc path you can require nbextension/vega/vega.min. The exact folder I'm don't remember, but on 4.0 you coud shell out to jupyter --path [--json] to get the various location.

There is also a nbextension install command to which you can give a folder and it should copy/link it into the right place.

[randyzwitch]

Great, thanks so much. Hopefully I can solve this today before the holiday weekend!

[aishfenton]

@randyzwitch isn't there still a problem with the solution you sketch out above with the order that external scripts are loaded.

Doesn't it need to be something like the following to ensure that dependancies between d3, vega, and vega-embed are loaded in the right order?

  require.config({
    paths: {
      d3 : '//d3js.org/d3.v3.min.js?noext',
      vg : '//vega.github.io/vega/vega.js?noext',
      vl: '//vega.github.io/vega-lite/vega-lite.js?noext',
      vg_embed : '//vega.github.io/vega-editor/vendor/vega-embed.js?noext'
    },
    shim: {
      vg_embed: { deps: ["vg", "vl"] },
      vl: {deps: ["vg"]},
      vg: {deps: ["d3"]}
    }
  })

Although having said that. I can't get the above to work either. I get a util not defined error, which I haven't figured it out yet. Any suggestions much appreciated.

[aishfenton]

Same issue mentioned here.

[randyzwitch]

I don't use vega-embed, but here's how I render Vega inline for Jupyter Notebook.

https://github.com/johnmyleswhite/Vega.jl/blob/master/src/render.jl#L45-#L74

Not saying it's what a JavaScript developer might do in terms of elegance, but it has worked for many months now.

[aishfenton]

Thanks @randyzwitch. I also ended up getting around it by switching to an iFrame with srcdoc (which means you can just add the scripts tags as if it were a standalone HTML page), but that's also probably not particularly elegant.

[joshday]

Any chance there's an update for 2023 that makes this easier?