Create a D3 implementation of the C. elegans connectome HivePlot

[slarson]

Building on the work of the previous issue #39 , open the source for the D3.js implementation of a hive plot and load up the c. elegans connectome data set in it so it can be interactively explored in a browser.

More guidance on how to go about this may be found on the d3 mailing list

[GasGen]

Hi,
What's the idea?, take the connectome of c.elegans and create a visualization like the first dataviz, but with the "implementation of hive a plot" design?

[slarson]

Basically if you click on the link under "implementation of a hive plot" -- we want to put the C. elegans connectome data into that interface.

[slarson]

@msasinski said:

Hi Stephen,
I'm working on the DataViz D3 Hive Plot and I have one question you
probably can answer - in the connectome tab under in the spreadsheet
file there are only two types - Send and GapJunction. Does it mean that
we should have only two axes? Or should I use one of the other columns,
i.e Neurotransmitter for that?

---

Hi Mariusz,

Thanks for looking into this! In a Hive Plot, the axes are defined by the type of node that should go on them. That's why the python script here:

https://github.com/openworm/data-viz/blob/master/HivePlots/worm2hive.py#L23

Is parsing out nodes based on whether the neuron is 'sensory', 'motor' or 'interneuron'. The Send and GapJunction fields have to do with the type of connection between the axes, the vertices. That's the difference between the blue and orange lines here:

https://github.com/openworm/data-viz/blob/master/HivePlots/ntype%20-%20orange%20gap%20junctions.png

Orange are the gap junctions and blue are the synapses. Does that make sense?

Thanks,
Stephen

[msasinski]

Yes, it makes sense now. Thanks!

[ashishkadam]

Is anyone still working on this?

[slarson]

Haven't seen an update in a long time. Want to jump in on it?

On Saturday, June 21, 2014, ashishkadam notifications@github.com wrote:

Is anyone still working on this?

—
Reply to this email directly or view it on GitHub
#89 (comment).

[tarelli]

Did this happen in the meanwhile http://wormweb.org/neuralnet#c=DVC&m=1? Not a hive plot but interesting nonetheless.

Update: That's actually Nikhil's one, forgot about it!

[slarson]

No, that's Nikhil Bhatla's connection browser that pre-dates even the
OpenWorm project :)

On Mon, Jun 23, 2014 at 3:05 PM, Matteo Cantarelli <notifications@github.com

wrote:

Did this happen in the meanwhile http://wormweb.org/neuralnet#c=DVC&m=1?
Not a hive plot but interesting nonetheless.

—
Reply to this email directly or view it on GitHub
#89 (comment).

[ashishkadam]

Yes! I could work on this to start with.. :)

[slarson]

Can I help out at all with this? Would you like to get a chance to chat
about it?

On Thu, Jun 26, 2014 at 8:35 AM, ashishkadam notifications@github.com
wrote:

Yes! I could work on this to start with.. :)

—
Reply to this email directly or view it on GitHub
#89 (comment).

[slarson]

Also relevant to this issue is this presentation on this work.

In particular check out the images on slide 22 and slide 23. If we could reproduce exactly those two they would be some of the most significant visualizations to start with.

[ashishkadam]

Hello, could someone please reply to my query which I had posted on the mailing list.
Its about where do I get the raw data for this visualization and some other stuff.

https://groups.google.com/forum/#!topic/openworm-discuss/aRtJGdSj0c4

[borismarin]

HivePlots have reemerged in the context of Geppetto/OSB network visualization, and it would be great to join efforts and get things done. Here's a brief outline of a possible line of attack:

Connectivity widget for Geppetto

• implement the Geppetto model for connections, or the 'edge' side of things (jrmartin @37d07e265b1bbbe9257beecb198d6ecd1620b073 is currently working on that)
• Create Geppetto objects (mentioned above) with info extracted from NeuroML files ( @adrianq and tarelli@e87fd7db38c841f16d549c6a0a6854da74f44696 are working on that but from the 'node' side of things -- i.e. scraping NeuroML for info on variables/dynamics/parameters)
• Bubble d3.js up from the ThreeVisualizer widget into Geppetto (similarly to three.js which once existed under both the Scatter3d and at the root) - @tarelli, is that a good idea?
• Implement d3 infrastructure for creating hiveplots & co. from Geppetto objects. I have some experiments running locally (without geppetto) where such plots are produced from NeuroML (without using any Geppetto-based intermediate format). It is then a matter of tying things up.

I thus think it's a good time to add a Waffle card to join all those things together, under the auspices of Geppetto.

[borismarin]

btw, just to keep you informed:

Here's an example of our current (untweaked, proof-of-concept, ugly and useless) d3 mvc capabilities. All plots were created using d3, from the same data (i.e., different "views" of the same data).
Data currently preparsed from https://github.com/openworm/CElegansNeuroML/blob/master/CElegansNeuronTables.xls?raw=true

Hive Plot

Force layout

Connectivity Matrix

All static figures displayed above are actually dynamic svgs (unfortunately, git doesn't allow embedding svg in comments): we can change colours, reorder entries in the matrix according to predefined criteria - in other words: lots of fun (and lots of things to be done).

See you at the Worm Retreat next week!

[slarson]

@borismarin BEAUTIFUL! Are there additional Github issues that are following up on this?

[borismarin]

I've just had a chat with @adrianq, and we have decided to organize all those things into a Geppetto widget during the OW hackaton. Progress here.

[slarson]

@borismarin -- I saw the connectivity widget demoed at the last Geppetto meeting --very cool! Did the hive plot part get in there too yet?

[borismarin]

@slarson the infrastructure is there, but we need a more systematic way of
mapping qualitative domain specific info (e.g. how to determine if a cell
is inhibitory) and map that to visual representations in geppetto (e.g. map
a synaptic weight onto a line thickness). @adrianq and I will have a
meeting with @tarelli and @gidilli to discuss these issues next week. For
updates, see #235, #238, #323 .
If you want to play with a prototype, I (or @adrianq) can probably set up a
branch with our c302 / acnet specific code.

On 10 January 2015 at 23:57, Stephen Larson notifications@github.com
wrote:

@borismarin https://github.com/borismarin -- I saw the connectivity
widget demoed at the last Geppetto meeting --very cool! Did the hive plot
part get in there too yet?

Reply to this email directly or view it on GitHub
#89 (comment).

[borismarin]

Just saw this issue again while organising my waffle cards... I'd like to point out that the connectivity widget in geppetto has had four "incarnations" for quite a long time now (force, chord, hive, matrix), which look like that:

(cortical layers 2/3 from Traub's model, not C. elegans!!)

The only thing preventing this issue from being closed is finding a way of mapping the cells in c302 into categories (sensory, motor, ...), which is not captured in neuroml2 systematically yet. In any case, it is indeed possible to hardcode that info via the conn. widget settings object (which is an obscene thing to do, as we are adding data via visualisation...), and get something like that

for the c302 example in live.geppetto.org. Bottomline, I guess this issue can be closed and more specific ones added to address 1) qualitative info in nml2 2)specific settings for "useful" hive plots.

[gidili]

@borismarin looks AWESOME 🍻

[borismarin]

BTW, does that small motor->sensory projection make sense (thin orange band connecting [green slice]->[orange slice], or is that a bug in my code? (Notice the convention: chords emerge from the presynaptic category, but finish before "touching" the postsynaptic category, and are coloured according to the target)

[slarson]

@borismarin Let me echo @gidili here and say this is SICK (in a good way :) :+1

It hadn't been obvious this has been working. I've noticed that in the latest version of the "user" docs, there is no mention of the widgets or how to use them anymore, so it seems easy to overlook.

I had to do a search to find the old using widgets page, which finally does link down to showing those pictures. But it seems we have lost the link unless I am just not seeing it on the user docs.

Do you know if this is on purpose, or should I submit a pull request to re-work the widgets back into the user documentation? I think even better would be to pull some of these images up into the user docs page from the beginning, honestly, to highlight that this is possible.

After this is settled, I'd agree we should close this and re-open some more targeted derivative issues for connecting the c302 pieces in here.

Also, if I understand your question correctly, yes the small motor to sensory projection makes sense. You can compare with the ones on slide 22 and slide 23 that I generated long ago.

[slarson]

Ah, now that I did a direct search, I do see that there are links to the using connectivity widgets page.

OK I will make a pull request to make this more obvious as it could be much clearer. (openworm/org.geppetto.docs#31)

[slarson]

OK, this prompted me also to ask about how to improve the connectivity widget docs to better serve the closing of this one: openworm/org.geppetto.docs#32

[slarson]

OK closing this one and taking the rest of this issue over to the newly created openworm/CElegansNeuroML#57