How do you engineer a JupyterLab React Component?

[bionicles]

Do you like interactive data? Many do.

We should make a way to import JupyterLab into React and sync state between the two. Just imagine we integrate the interactive coding and remote kernels of JupyterLab with a bajillion existing React components already out there. There's an awesome-react-components repo with almost 18k stars plus react is learnable quickly and in use by bigcos like facebook, palantir, atlassian, alibaba. If jupyterlab works in react then we can make interactive dashboards, data grids, scriptable visualizations, documents, etc, connect multiple kernels within a shared web app, etc etc

Challenges:

1. How do you make JupyterLab fit inside a div in a react component? (partially solved in linked repo)
2. How do you tell JupyterLab to connect to the desired kernel? (probably easy)
3. How do you sync state between React and JupyterLab? (many solutions)

here is progress on issue 1:

Tomorrow I'll try to fix the kernel connection by running a separate jupyter lab server and connecting the app to that... but ideally we can use jupyterhub remotely.

Re: state, we prefer useReducer hook from react 16.8. Assuming jupyter has internal data, we just need to figure out how that works and hook it up. We could make helpers in various languages so different kernels (python, julia, R, scala, node) can connect to the state of the parent app.

[vidartf]

I don't know enough about the react eco-system, so I will ask a "dumb" question: Are there anything you can do do with "lab-in-react" that you cannot do with "react-in-lab"? What are the inherent differences? From the lab side, we've mostly imagined react for use in the leaf nodes on the DOM, and I don't really have a good understanding of use cases for the opposite approach.

[bionicles]

not sure, let's play with both, one benefit to lab-in-react would be, you can drop the lab as an addon to data science sites and stuff already in production; react-in-lab could be fun to bring components into notebooks for data visualization

here's a shopped mockup of a graph database connected to jupyterlab:

[jasongrout]

Thanks for consolidating this discussion!

For completeness, here's @saulshanabrook's comment from gitter:

bion howard @bionicles May 18 03:02
Hi, going to start scrolling up and searching but figured i'd ask... how can I make a react component for JupyterLab? JupyterLab application is built on phosphor application is built on phosphor widget. All I want to do is make something like react-jupyterlab so I can embed JupyterLab inside a React app and pass State and Dispatch to that component as props, so the JupyterLab computations can update the react state. Just embedding Jupyter Lab inside a web app. How?

Saul Shanabrook @saulshanabrook May 18 11:36
@bionicles One short answer: You can't. Not only is JupyterLab not built with React it doesn't have global state storage.

Longer answer: You could wrap JupyterLab in a React component, but you would have to manually link the state to any changes in JupyterLab's application. We have been transitioning some of JupyterLab to use React internally and if you are interested in solving this problem long term, we would love to have your help with that. If that is something you are interested in, I am happy to help get you started there.

Which parts of JupyterLab are you looking to control externally?

[jasongrout]

@saulshanabrook's comments about JLab state help give context about your point 3. There isn't one central store of state in JLab (right now). State is scattered all across different components of JLab. You can individually hook those up with their various APIs, but it may be rather difficult if you want to capture all of the state. In the future we'd like to consolidate at least some of the state to help support real time collaboration, but that is a long term project.

[bionicles]

Thanks Jason. I'm feeling a bit stumped about how to make this work; I don't know how to tell the jupyterlab application running on the react server at localhost:3000 to send api calls to the jupyterlab server on localhost:8888. Perhaps I need to pass an option with the desired port, but, where?

if we tell the jupyter lab --port 3000 then we can't access the react site

[jasongrout]

The jupyterlab app should be served from the jupyterlab server. The jupyterlab server sets the app configuration in a script tag on the page. You can see this as one of the first script tags on the jupyterlab page. That tells the app where the api servers live.

[bionicles]

Seems harder to make a react component if the view and the server are not modular. If it were possible to draw the app with html/js/css and then have the app connect to the server, then it could work, and even work with no server at all

[jasongrout]

They are modular, however there is a question of how to get server information to the app. Right now the app gets information about the server (like the API URLs to use) from a script tag on the page (see https://github.com/jupyterlab/jupyterlab/blob/master/packages/coreutils/src/pageconfig.ts).

It sounds like an excellent project would be to refactor that to be customized in an app, so you could pass in state to the app for those values, and each JLab extension could ask the app for that information instead of importing the reference implementation in @jupyterlab/coreutils

[bionicles]

on the example app template html file it reads:

    <!-- <script id='jupyter-config-data' type="application/json">{
      {% for key, value in page_config.items() -%}
      "{{ key }}": "{{ value }}",
      {% endfor -%}
      "baseUrl": "{{base_url}}",
      "wsUrl": "{{ws_url}}"
      }</script> -->

But it's unclear:
where does page_config come from?
how do you pass the baseUrl?
what do you pass as wsUrl?

What is the reason to extract the paths config from the HTML when we could just pass it to the constructor?

  var lab = new JupyterLab({
    name: "JupyterLab Example",
    namespace: "lab-example",
    version: require("../package.json").version,
    paths: {
      urls: {
        baseUrl: "localhost:8888"
      }
    }
  });

^^^ doesn't seem to work... the server connection continues to make requests to the wrong location.

https://github.com/jupyterlab/jupyterlab/blob/master/packages/application/src/lab.ts#L50
reads const optionURLs = (options.paths && options.paths.urls) || {}; which assigns optionUrls as a boolean instead of actually assigning the urls object. If you pass options.paths.urls, then the left side is "true" and it becomes const optionUrls = true;

shouldn't this instead read,

  if (options.paths) {
    optionUrls = options.paths.urls;
  } else {
    optionUrls = {};
  }

re: docs improvement, https://jupyterlab.github.io/jupyterlab/ has no overall search feature so it's not possible to search all the api at once, you have to click into one package at a time and search it. If you don't know which package contains what you're looking for, then you have to potentially click into and search multiple different packages. How do you add root level search to the jupyterlab api docs to search all package APIs at once?

edit: also, yarn add @jupyterlab/application adds an old version 0.19.1 and it is unclear how old that is, but it's missing the application/src/lab file so I just spent 2hrs reading github code which is not the same as the code in npm :(

[bionicles]

I really want to do this but I'm in the middle of a sprint, I can't slow down to study this huge codebase so I'm going to try instead to do it with thebelab and come back to this idea later. Please feel free to use my repo as a starting point.

[jasongrout]

I really want to do this but I'm in the middle of a sprint, I can't slow down to study this huge codebase so I'm going to try instead to do it with thebelab and come back to this idea later.

Thanks. We're also in middle of a sprint to release 1.0, so can't devote a lot of cycles to this. However, when you're ready to pick it back up, we can help answer questions, etc.

[jasongrout]

where does page_config come from?
how do you pass the baseUrl?
what do you pass as wsUrl?

This is a jinja template populated from the python tornado server.

What is the reason to extract the paths config from the HTML when we could just pass it to the constructor?

It's easier to pass in state via a JSON dict in a script tag than to template js code.

Edit: However, best of both worlds is reading that script tag in js, then passing it in as an argument.

shouldn't this instead read,

options.paths && options.paths.urls returns a dictionary (and I just checked in a js console to make sure): let options={paths: {urls: {test: 1}}}; options.paths && options.paths.urls returns {test: 1}.

How do you add root level search to the jupyterlab api docs to search all package APIs at once?

Thanks for pointing that out. That's a significant regression from how we used to expose API docs (where all packages were in one searchable interface). I've opened #6387 to address this.