IPyWidget Architecture Challenges

IPyWidgets in VS Code

We have directly adopted one of the official sample Embedding Jupyter Widgets in other Contexts.

Starting Kernels in VS Code (extension host, node.js)

• Look for kernels in known locations on disc
• Spawn the process using the arguments defined in the kernel.json file
• Setup a zmq connection and an IKernelConnection for communications with the Kernel
• Use @jupyterlab/services for all communication with the kernel

Displaying Widgets in VS Code (webview)

• Setup a new IKernelConnection that mirrors the IKernelConnection in the extension host
• Synchronize the kernel messages to ensure messages received from the backend kernel are received by both IKernelConnections
• Setup a WidgetManager in the WebView and provide the mirrored IKernelConnection

Mirroring messages

• Messages sent from the backend kernel are received by the fake WebSockets in node.js and webviews,
  Hence both IKernelConnections receive the messages.
• Hooks added by WidgetManager in IKernelConnection in the WebView result in corresponding hooks added to the corresponding IKernelConnection.

Challenges in adding support for IPyWidgets in VS Code

1. Embedding Jupyter Widgets in other contexts, BUT without access to the Kernel Web Socket connection

The current samples for embedding Jupyter Widgets require a direct access to the Kernel WebSocket connection.

VS Code spawns the kernel process and communicate with the kernel process directly over zmq (without the need to install Jupyter or Python). The IKernelConnection is constructed in the node.js process. However widgets are rendered within a webview that do not have access to this IKernelConnection.
We have managed to create a mirror IKernelConnection in the webview (by synchronizing the kernel messages between the two IKernelConnections).

Complications with this approach:

• This works for the most part, however this is fraught with issues when dealing with message hooks (IKernelConnection.registerMessageHook).
  The Output widget is one such widget that makes heavy use of these hooks.

Asks:

• Provide a sample to embed Jupyter Widgets in another context without access to the Kernel Web Socket connection.

Possible solutions:

• Provide a standard library that can be used to perform the necessary synchronization of these messages.
• Re-visit exposing the IKernelConnection to Widgets.
  I believe the latter will be more challenging as widgets out there today access the KernelConnection and directly communicate with the Kernels via the IKernelConnection.
• Re-visit message hooks.

2. Fetching/Loading Widget Scripts

VS Code supports two types of kernel connections:

• Connecting to a Kernel by manually spawning the kernel process and communicating via zmq
• Connecting to a Kernel on a remote Jupyter Server (using the @jupyterlab/services npm)

We have added the necessary hooks into the Widget Manager to load the widgets from the following 3 locations:

• CDN (used as the primary source of widgets for all types of connections)
• Local Disk (<Python Environment>/share/jupyter/nbextensions)
• Remote Jupyter Server (used when connecting to a remote jupyter server)

Complications:

• Some widgets end up rendering outputs with simple text/html mime types.
  These contain js code that end up assuming the widgets have been loaded into the current webview context.
• Not all widgets contribute scripts to a CDN
• No API in Jupyter Notebook/Lab to get all of the Widget Scripts
• Widgets in nbextension folders end up defining their entry points in extension.js file which cannot be loaded in WebViews (outside Jupyter Application)

Asks:

• Clear documentation of expected requirements of the WebView state where WidgetManager is loaded
  • E.g. require.js, jQuery need to be loaded
  • Or Widget scripts need to be loaded into the webview.
  • Other global variables need to be initialized, etc.
• Change the build process to build and package CDN scripts along with the other scripts. Either define them as cdn.js or the like or define them in package.json, so VS Code and others can load these scripts.
  • Optionally make it easier to identify Widget entry points for widgets on disc.
• Provide an API from Jupyter application to list the path to the widget scripts.