Support for ipycytoscape

[rchurt]

Bug report for Colab: http://colab.research.google.com/. Related to #60.

• Describe the expected behavior:
  I'd love to use ipycytoscape to work with network graphs in Colab. I'm able to install it without a problem, but when I try to display a network graph, nothing shows up. Reminiscent of trying to display a HoloViews plot without having run hv.extension('bokeh') in the cell.

• The web browser you are using (Chrome, Firefox, Safari, etc.):
  Chrome 84.0.4147.135

• Link (not screenshot!) to a minimal, public, self-contained notebook that
  reproduces this issue (click the Share button, then Get Shareable Link):

Here's a self-contained example from their website (you can find it and others here). After running the last line I would expect to see a network graph, but nothing shows up.

!pip install ipycytoscape
from ipycytoscape import *
import pandas as pd

df = pd.read_csv("https://raw.githubusercontent.com/nextstrain/ncov/fe5a7ed1f92af63d6d1d43d0307e4b2620108aaa/data/metadata.tsv", sep = '\t')
df.columns

cytoscapeobj = CytoscapeWidget()
cytoscapeobj.set_tooltip_source('name')
cytoscapeobj.graph.add_graph_from_df(df[:30], ['country'], ['age', 'virus'])
cytoscapeobj

They provide some installation tips here that might be helpful, including how to run tests locally.

[blois]

More specifically this is #498.

Colab recently started exposing Jupyter Comms channels so custom visualizations can use them for communicating with the kernel (see https://colab.sandbox.google.com/notebooks/snippets/advanced_outputs.ipynb#scrollTo=YOd4khppTskD) but ipycytoscape requires an extension to be installed to view. I'd be happy to give pointers. This may be tangentially related to cytoscape/ipycytoscape#80.

[rchurt]

Great, happy to hear it can be done! I'm still pretty new to coding and this JS stuff is way over my head, but here's my interpretation of what I need to do:

1. Install the jupyter-cytoscape extension (and the dependencies required to install it). I tried to do this with the following code:

!pip install jupyterlab
!pip install nodejs
!jupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter-cytoscape

But I got an error ValueError: Please install nodejs >=10.0.0 before continuing.. I also tried installing nodejs with !sudo apt install nodejs, but the version of nodejs that was installed is 8.10.0~dfsg-2ubuntu0.4. Does that mean I should install nodejs from a binary here, or do you have another way of doing it in Colab?

2. Paste the code you linked here--namely the code in the sections "Establish a comm channel from client to kernel" and "Establish a comm channel from kernel to client"--into the notebook.

3. That's it?

Sorry for such basic questions, and thanks very much for helping!

[blois]

Unfortunately it's a bit more involved than that- jupyter-cytoscape is an extension specifically for JupyterLab and the code needs to be adapted to work in other contexts. We've tried to expose some of the underlying pieces to make this possible, and ideally in a manner that would be compatible with other notebooks, but it still requires a decent amount of change to the code.

[rchurt]

I see, so that means that at this point it's entirely an ipycytoscape issue? In that case I'll open an issue on their GitHub for this and CC you. Just to clarify, specifically what I should be asking is for them to modify the jupyter-cytoscape extension to be compatible with Colab (or make a new version colab-cytoscape), correct? Thanks for being willing to help!

[timkpaine]

Just wanted to pop over here and say this thread is incorrect. Widget based extensions that don't have specific JLab dependencies (e.g. they dont include a jupyterlab mimerenderer or interact with JupyterLab app or leverage the windowing system, etc) work in all widget contexts.

[blois]

@timkpaine - there are a couple of points of difficulty here-

1. We need a mechanism to map widget module name to JS module, without an explicit Colab dependency on external services such as unpkg.
2. The versioning scheme for widgets has not-infrequent changes to major versions of the widgets or dependencies. For a hosted service this is problematic as a widget which worked one day may break the next as a daily release is rolled out.

I believe it should be possible to create a custom widget manager on top of the stable comms APIs Colab exposes. With this each custom widget could declare its own widget manager with info about where to load the module from and control the versioning itself. If we took this approach and provided a widget manager which could be used by custom widgets, how many widgets would take advantage of it? Should we instead invest in addressing more of the versioning and packaging issues?

[timkpaine]

1. The widget cookiecutter ships with a facility to webpack static assets into package/nbextension. So assuming you have client packages open sourced and published to work with your service, it should be straightforward enough to make integrating with these packages part of the standard procedure without external runtime dependency on e.g. unpkg

2. This is true of every hosted service everywhere. I ran one, we solved it be locking down what could and could not be installed. If you try to install a different version of the package that is incompatible with the frontend assets, you should put this check into your open source client assets and wrap up a nice error for the end user (which is something JupyterLab does, but Jupyter does not).

Getting every end package to integrate with proprietary APIs is unlikely to be successful. Open source a widget manager and integrate that widget manager to export frontend assets into the open source cookiecutters (e.g. here). Ideally, make such a manager completely natively compatible with widgets (e.g. Voila's manager here).

[blois]

To be clear- the goal has been to not have proprietary APIs but instead to work towards a common stable API that works in more environments.

If you try to install a different version of the package that is incompatible with the frontend assets

In our context there are no frontend assets at all. Aside from the minimal public API visualizations need to include all of their dependencies.

What I'm basically saying above is essentially a widget manager who's API to the host is limited to just something along the lines of https://github.com/Quansight-Labs/jupyter-output-spec, then either loading that before the custom widget (initiated by user code), or just bundling that with the custom widget in a standalone binary which would have to be >800K.

In my view the widget manager and widget are tightly coupled due to the significant API surface area of widgets and their requirement of a common build tool (when using webpack's module federation), thus they should be distributed together. Colab already uses a widget manager per cell execution using browser broadcast channels to coordinate, I don't think it would be impossible to do a widget manager per widget? Then just include some esbuild created binary of the standalone binary in the widget cookie cutter and be free of the versioning issues? (I'm sure I'm oversimplifying here). We had hoped 1) to get consensus on the stable APIs necessary to support this so it could be used by other hosts and 2) to have a more minimal API to remove that large binary size.

[blois]

These should now be working in Colab, see #498 (comment).

An example is https://colab.research.google.com/gist/blois/665fb53f06c09d6dd380b934eaea15f4/ipycytoscape.ipynb.