Neo4j graph visualization or plugin for Jupyter Lab

[pwrose]

A Jupyter Lab plugin is urgently needed to visualize and interactively explore Neo4j KG.

Here is a list of JS libraries:
https://neo4j.com/developer/tools-graph-visualization/

See also:
https://github.com/neo4j-graph-analytics/graph-algorithms-notebooks/blob/master/notebooks/DegreeCentrality.ipynb

https://ipython-cypher.readthedocs.io/en/latest/

https://nicolewhite.github.io/neo4j-jupyter/hello-world.html

[martinRenou]

@jtpio Do you think we could help with this one? You definitely know how to make a JupyterLab plugin. And I'm happy to help as well.

[martinRenou]

cc. @wolfv as you said you were interested in this

This might be helpful: https://github.com/neo4j/neo4j-javascript-driver but I guess it requires back-end logic.

[jtpio]

That should definitely be possible by wrapping one of the existing graph libraries.

Wondering whether this should be a widget or a pure lab extension, since there is some authentication to access the Neo4j database.

[pwrose]

I think you can assume that Neo4j has been authenticated and you have access to the graph object. See ExampleQueries.ipynb.

@jtpio @martinRenou @wolfv It would be great if you could help with this project. I'll post some requirements for the graph visualization soon.

If you can, sign up as a GraphHack contributor (https://medium.com/neo4j/graphhackers-lets-unite-to-help-save-the-world-graphs4good-2020-fed53562b41f) and join the kick-off meeting:

Friday, March 27th at 11am ET | 15:00 GMT to Zoom.
Link: https://neo4j.zoom.us/j/4601817416
Meeting ID: 460 181 7416
Find your local number: https://neo4j.zoom.us/u/abpGfRX3KS

[seangrant82]

This doesn't have all the functionality of D3 but could be a good start at least: http://compbio.ucsd.edu/bringing-interactivity-network-visualization-jupyter-notebooks-visjs2jupyter/

[pwrose]

@seangrant82 visjs2jupyter cannot display multiple relationships with opposite directions. I haven't tried it on Jupyter Lab.

[pwrose]

What are the minimal requirements for a graph vis. in Juypter Lab?

Here are some thoughts:

• Render graph with node and relationship names or any of their properties
• Set color and size by node type
• Set relationship color and line thickness and style by relationship type
• Render only a specified list of node types (hiding all others)
• Rearrange nodes interactively
• Save graph image, e.g., png

Nice to have:

• Clickable URLs in nodes
• Applying an action when a node is clicked
• Images in nodes

[wolfv]

Hi @pwrose we have worked on a graph visualization tool using ipywidgets (so that it works in Jupyter and Jupyter Lab).

It's based on cytoscape.js (https://github.com/quantstack/ipycytoscape) I am sure we could extend it to support all functionality you desire. Let me know if you want to chat about this cc @martinRenou @marimeireles

[sonygreen]

@pwrose I think we can help with this too. Our GraphXR software covers all of the need to haves (and 2 out of 3 nice to haves) that you listed.

[pwrose]

Hi @wolfv @martinRenou @marimeireles, I'd be happy to chat. I can set up a zoom call for Tue. Contact me at pwrose@ucsd.edu.

[pwrose]

@sonygreen do you want to add an example notebook to this repo to show how it works? Note, we are interested in open source solutions that can be installed from conda or pypi.

[sonygreen]

@pwrose Ah, my apologies. GraphXR is free to use but not open source

[wdyang]

@pwrose GraphxR is not opensource. However, we made it free for people to use with minimum restrictions. My workflow often involves using Python+Jupyter for data preparation, analysis, and bring to GraphXR for doing things you listed.

As a dedicated visual analysis tool, it can handle much more nodes, much easier to shift perspectives in visualization, as well as integration with other tools through API. Let me know if opensource is a must have for you. If you are open to use tools that are available for free, I'd love to chat with you.

[pwrose]

@sonygreen and @wdyang, let's chat in a week or two once we have some initial data in our KG and then see how GraphXR could be used for visuals analysis. It sounds like it would be complementary to what we do with Jupyter Notebooks.

[marimeireles]

Hey @pwrose so I'll be testing locally until we have release the new version of ipycytoscape and you'll be able to test it locally too. (Should take a couple of days, maybe this Friday).

To have a minimal working version:

• We'd like to open graphs with networkx
• Add label to edges

It'd be nice if we could:

• Open graphs with dataframes

Plus all of the things you mentioned in the #1 (comment) and we still don't have

Regarding the interactive aspects like:

• Render graph with node and relationship names or any of their properties
• Set line thickness and style by relationship type
• Render only a specified list of node types (hiding all others)

It's currently doable, the "materials" are all there, but there is not an API to do it. So we have two options here, either we implement something specific for your user case or try to create something more generic and add it to ipycytoscape. I think the later is doable and would be actually pretty cool, but might take a little longer and it's a bit more complicated. If anyone wants to help here and wants to discuss, I'm open to ideas.

As I said in the meeting I'm not really sure if we can add specific colors to nodes in graphs with the cytoscape API. If anyone has an idea on how to do that, it'll be welcomed =)

[marimeireles]

@wolfv @jtpio did I miss something?

[pwrose]

Also, since the Cytoscape team is also here at UCSD, we can connect with them for specific questions or feature requests as well.

…

On Thu, Apr 2, 2020 at 9:36 AM Mariana Meireles ***@***.***> wrote: @wolfv <https://github.com/wolfv> @jtpio <https://github.com/jtpio> did I miss something? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA53AEH3GEH4TD5R35HMDC3RKS5ILANCNFSM4LRFYWOA> .

[pwrose]

Here is an example of how to get the data for a graph object:
https://github.com/nicolewhite/neo4j-jupyter/blob/master/scripts/vis.py

[marimeireles]

@pwrose if it's this Cytoscape team they may not be aware of what's going on on CytoscapeJS which is the implementation I'm basing the widget in.

[marimeireles]

Hey @pwrose, I remember seeing some examples in your screen in the meeting where you showed some uses of networx, but I can't seem to find them anymore. Could you please link it to me?
Thanks!

[pwrose]

@marimeireles The examples are here:
https://nicolewhite.github.io/neo4j-jupyter/hello-world.html

[marimeireles]

Oh, thanks! Was looking in this repo.

[pwrose]

@marimeireles if you have any prototypes you want me to review, please let me know.

[marimeireles]

Hi @pwrose, sorry for taking some time for getting back to you.
I prepared a small reproducible notebook example using the same database that you're using. This one.

I also found out that there is a straightforward way of changing nodes color, apparently you just have to add a style to it. I think you could iterate through a column of the df and change the colors of them to make a group. But I haven't tested it yet, lemme know if you need any help to get this working.

I really couldn't think of many different examples on the kind of data visualization you want to have. If you're struggling with anything let me know and we can build a solution together.

I also tested ipycytoscape with neo4j stuff, including the outputs of Cypher. I don't have an example right now, but I can write one for you if you need it, the results from NetworkX extracted from Cypher are pretty broken though, they're not able to form connections out of the box with the data that they're used in the repo you linked here.

To have access to this new features please download the new version of ipycytoscape, 0.2.0, make sure you have it as the latest in your environment =)
A pip install ipycytoscape --upgrade should do it or conda install -c conda-forge ipycytoscape. You can check with pip list or conda list if the version is correct.

[pwrose]

Thanks @marimeireles !

I ran into a couple of issues:

1. ipycytoscape is not on conda-forge, yet, so I used the version from PyPi
2. It works in Jupyter notebook, but in Jupyter lab I get this error: Error displaying widget: model not found
3. To reproduce this, I created the following repo:
   https://github.com/pwrose/neo4j-ipycytoscape
   There are two Binder links to try this out, one for Jupyter Notebook and one for Jupyter Lab. I added the Jupyter Lab extension here:
   https://github.com/pwrose/neo4j-ipycytoscape/blob/master/binder/postBuild#L19
4. I created a fully functional deployment of Neo4j for Binder. Launch the Neo4j notebook to see how to create a small graph and list the nodes and edges. Can you add the graph visualization there? I put some notes there:
   https://github.com/pwrose/neo4j-ipycytoscape/blob/master/notebooks/Neo4j.ipynb

Please let me know if you have any questions.

[marimeireles]

@pwrose

1. Was fixed, thanks.
2. Could you please try with the new version? It's working for me in a new environment.
   I'll update the Readme as I find it a bit confusing.
3. Okay, I gave a look at your notebook examples and they're using the input directly from neo4j. The alterations I did were based in other examples you sent me, they were manipulating the data from dataframes, and that's what the new function I added here is doing.
   I made a fast google search and apparently it's possible to make this conversion without much work. I'll work on an example for you.

Thanks for the repo with examples, they made it much clearer for me.

[pwrose]

@marimeireles I've updated the environment.yml file to use ipycytoscape 0.2.1 from conda-forge, see: https://github.com/pwrose/neo4j-ipycytoscape. Click the launch binder link for Jupyter Lab and you'll see the error: Error displaying widget: model not found

[pwrose]

@ery010 is now working on this issue.

[marimeireles]

I'm so sorry I'm not making progress on this!
I'll keep you updated when I finish what you sent me by email. Good luck!

[pwrose]

Added a prototype using [ipycytoscape]
(https://github.com/covid-19-net/covid-19-community/blob/master/notebooks/analyses/GraphVisualization.ipynb)