Neuprint Executor - Labeling Edges by ROI

[jakobtroidl]

Hi Jordan,

Do you see an easy way to assign ROI labels to edges in the neuprint executor? Let's say I want to query something like this:

A -> B [weight > 20, ROI == "CX"]
A -> B [weight > 30, ROI == "CRE(L)"] 

So basically, there are two things here—multigraphs, which you address already in the docs, and encoding edge ROIs. I wonder if that's rather a hard thing to do or not. The data should be there as neuprint-python fetch_synapse_connections returns something like this

    bodyId_pre  bodyId_post roi_pre roi_post  x_pre  y_pre  z_pre  x_post  y_post  z_post  confidence_pre  confidence_post
0    792368888    754547386  PED(R)   PED(R)  14013  27747  19307   13992   27720   19313           0.996         0.401035
1    792368888    612742248  PED(R)   PED(R)  14049  27681  19417   14044   27662   19408           0.921         0.881487
2    792368888   5901225361  PED(R)   PED(R)  14049  27681  19417   14055   27653   19420
...

According to this issue it looks like it's possible. My observation is that the physical location of a connection between two neurons is an important feature of a motif. Looking forward to hearing what you say.

EDIT: Maybe an indirect way to support multiple edges between two nodes is by grouping edge attributes. Does something like this seem plausible. You are doing smth similar in the multigraph docs already: A -> B [synapse_count > 2]. But what exactly is synapse_count?

A -> B [[weight >= 20, ROI == "CX"], [weight > 30, ROI == "CRE(L)"]]

Best, Jakob

[j6k4m8]

Hey @jakobtroidl! Sorry, I was on vacation — just catching up on my GitHub notifications now :)

This "array-of-arrays" notation is interesting! Is the idea that BOTH of these edges must exist, or at least ONE of these edges must exist?

As for encoding ROI; anything that is in the edge object should already be queryable. I don't know if you can do

A -> B [roi_pre = "PED(R)"]

but I THINK that should work if these data are encoded in the Synapse edges in neuPrint?

[j6k4m8]

To answer your second question — synapse_count is just an example of how you can make a simple (non-multi) graph that still encodes some multigraph-like information. In other words, you can "collapse" edges so that this multigraph:

stateDiagram-v2
    a --> b : ntx=gaba
    a --> b : ntx=glu
    b --> c : ntx=ach
    c --> a : ntx=ach

becomes this simple graph:

stateDiagram-v2
    a --> b : ntx=gaba,glu\nsynapse_count=2
    b --> c : ntx=ach\nsynapse_count=1
    c --> a : ntx=ach\nsynapse_count=1

And then instead of having to search in a multigraph (slightly more computationally expensive), you can search for the motif in a simple graph:

x -> y [synapse_count >= 2, ntx contains "gaba", ntx contains "glu"]

...

This is a bit of a hack (and only works if you have control over the graph, so it's a non-starter for something like neuPrint) but it's a good way to simplify queries if you can do some pre-processing on the network.

[jakobtroidl]

Thanks for your answer, Jordan. Querying motifs with ROI edge constraints like this currently doesn't work for me. It returns an empty set. See this notebook for a minimal reproducible example.

A -> B [roi_pre = "PED(R)"]

This dotmotif query translate to this Cypher, which returns an empty set:

MATCH (A:Neuron)-[A_B:ConnectsTo]->(B:Neuron)
WHERE A_B["roi_pre"] = "CRE(L)"
RETURN DISTINCT A,B LIMIT 5

However, I could construct a Cypher query, that does exactly what I want to do. It looks like this:

MATCH (A:`Neuron`)-[A_B:ConnectsTo]->(B:`Neuron`)
WHERE (apoc.convert.fromJsonMap(A.roiInfo)["CRE(L)"].post + apoc.convert.fromJsonMap(B.roiInfo)["CRE(L)"].pre) > 10
RETURN DISTINCT A,B LIMIT 5

I'd be interested in contributing a feature to Dotmotif, that allows ROI edge attributes on motif queries. I would like to add a feature to the motif2cypher parser, that allows creating Cyphers queries like that. Do you think thats a reasonable approach & do you accept PRs for dotmotif?

Regarding the multigraph: That approach looks nice. I'll try it out once I use a graph that I have control over ;)

[j6k4m8]

Ah, I don't do anything with apoc right now (or JSON attributes at all) because I'm reusing the default Neo4j executor in DotMotif with custom entity names; the neuPrint executor doesn't really have that much in the way of neuPrint-specific logic since there hasn't been a use-case for that before (besides dataset selection etc).

So, short answer — DEFINITELY would love a PR to add this capability! I think it belongs in DotMotif (rather than motif2cypher) so that all motif searches can benefit from it; let's chat about implementation together! This MIGHT be pretty straightforward, but there might be some messy dotmotif guts involved 😬

[j6k4m8]

Another comment here — DotMotif has its own neo4j-provisioning service (through the tamarind package) that creates a neo4j docker container but doesn't install apoc. That's one of my current reasons for not depending upon apoc installations, but since we know that neuPrint DOES have it installed, perhaps we can update the neuprintexecutor to support these commands natively.

[jakobtroidl]

I am back from vacation and now thinking about how I should implement this feature.

I think it belongs in DotMotif (rather than motif2cypher) so that all motif searches can benefit from it.

What do you mean by it belongs to DotMotif? The NeuprintExecutor currently calls motif_to_cypher from the Neo4J executor. I can currently only think of one of the two approaches to add this feature? Which do you recommend or how should they be adapted?

1. Installing apoc for the Neo4J docker container and modifying motif_to_cypher in The Neo4J executor or
2. writing a new motif_to_cypher function in the NeuprintExecutor that handles queries requiring apoc?

Based on your comment above I assume option 2 is better as we can't assume all Neo4J executors to have json attributes to be in their data scheme that we parse with apoc.convert.fromJsonMap.

If you have a different architecture in mind for this feature, let me know.

[j6k4m8]

Yes I think # 2 is the right move, since we know that neuprint will always have apoc. We may need to think about how to format a query syntax in order to distinguish between regular attributes and JSON attributes... Maybe let's discuss via zoom?

…

On Tue, Aug 30, 2022, 4:06 PM Jakob Troidl ***@***.***> wrote: I am back from vacation and now thinking about how I should implement this feature. I think it belongs in DotMotif (rather than motif2cypher) so that all motif searches can benefit from it. What do you mean by it belongs to DotMotif? The NeuprintExecutor currently calls motif_to_cypher <https://github.com/aplbrain/dotmotif/blob/6b178af6fdc64a6426375912ef7549eaa2b1ad52/dotmotif/executors/NeuPrintExecutor.py#L121> from the Neo4J executor. I can currently only think of one of the two approaches to add this feature? Which do you recommend or how should these approaches be adapted? 1. Installing apoc for the Neo4J docker container and modifying motif_to_cypher in The Neo4J executor or 2. writing a new motif_to_cypher function in the NeuprintExecutor that handles queries requiring apoc? Based on your comment above I assume option 2 is better as we can't assume all Neo4J executors to have json attributes to be in their data scheme that we parse with apoc.convert.fromJsonMap. If you have a different architecture in mind for this feature, let me know. — Reply to this email directly, view it on GitHub <#127 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFJKB3IKPM64GEY5BREHDLV3ZSTJANCNFSM55NAJ5TQ> . You are receiving this because you commented.Message ID: ***@***.***>

[j6k4m8]

I think this is solved in #128 right?

[jakobtroidl]

Yeeees 🎉🎉