-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: edge attributes #35
Comments
Hi @MikeB2019x! It sounds like Grand should be great for your use-case. Indeed, Grand will handle attributes even if Networkit can't support them natively; Grand will offload network operations to Networkit, and then will add the attributes back in as you ask for them. (In other words, using Grand like NetworkX should solve your problem without you having to think about it.) There is a bit of overhead associated with my attribute manager, which runs as a layer on top of Networkit; depending on your use-case, this should be pretty unnoticeable, but can be larger for things like edge-attribute queries. I am just jotting things down from my phone, so I can't run this... But you should be able to do something like this: import grand
from grand.backends import NetworkitBackend
G = grand.Graph(backend=NetworkitBackend())
# G.nx is not really a networkx graph, but we can treat it like one:
G.nx.add_edge("A", "B", foo="bar", baz="luhrmann")
G.nx.edges(data=True)
# Can still get the secret underlying Networkit backend:
G.backend._nk_graph In the case of postgres, we will probably need to write a new Backend to support this optimally, but you may be able to get away with the existing SQLBackend: |
By the way, I am also very happy to answer questions about DotMotif as well :) |
Hi Jordan. Thanks for the super-quick response! I've been trolling around
your various repos and I couldn't have found them at a better time.
Outstanding quality of documentation btw, you use the 'wiki' feature which
seems to be underused.
I have many questions but am still reading through docs so just point me in
the right direction if the answer exists somewhere:
1. I'm unclear about the 'backends'. My impression is that I can have
only one but if that is the case then if I have say, the sql backend
imported, how do I specify the underlying graph tool? Is the default
'networkx'?
2. if I use the sql backend do I have to pre-construct a db to a
particular schema or can I use an existing one eg. node table (id, attr),
edge table (src, tgt, attr)
Note that I'm not sure I can contribute much in the way of coding but I am
happy to contribute to documentation.
Cheers,
Mike
…On Fri, Jun 3, 2022 at 11:43 AM Jordan Matelsky ***@***.***> wrote:
By the way, I am also very happy to answer questions about DotMotif as
well :)
—
Reply to this email directly, view it on GitHub
<#35 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALV3V7Y6E4B74EUACY74273VNIRZTANCNFSM5XZF2W2Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
<http://mindbridge.ai/>
Michael Bourassa
Data Scientist | *MindBridge*
+1 613-894-9189 <+1-613-894-9189>
Bourassa
mindbridge.ai
[image: twitter] <https://twitter.com/mindbridge_ai>
[image: linkedin] <https://www.linkedin.com/company/mindbridge-ai/>
[image: banner]
<https://www.mindbridge.ai/news/mindbridge-named-to-forbes-ai-50-list-of-most-promising-ai-companies>
|
Just reading through the backend code for the sql backend which answers my
second question. Interestingly it is similar to what I have been doing.
On Fri, Jun 3, 2022 at 12:07 PM Michael Bourassa <
***@***.***> wrote:
… Hi Jordan. Thanks for the super-quick response! I've been trolling
around your various repos and I couldn't have found them at a better time.
Outstanding quality of documentation btw, you use the 'wiki' feature which
seems to be underused.
I have many questions but am still reading through docs so just point me
in the right direction if the answer exists somewhere:
1. I'm unclear about the 'backends'. My impression is that I can have
only one but if that is the case then if I have say, the sql backend
imported, how do I specify the underlying graph tool? Is the default
'networkx'?
2. if I use the sql backend do I have to pre-construct a db to a
particular schema or can I use an existing one eg. node table (id, attr),
edge table (src, tgt, attr)
Note that I'm not sure I can contribute much in the way of coding but I am
happy to contribute to documentation.
Cheers,
Mike
On Fri, Jun 3, 2022 at 11:43 AM Jordan Matelsky ***@***.***>
wrote:
> By the way, I am also very happy to answer questions about DotMotif as
> well :)
>
> —
> Reply to this email directly, view it on GitHub
> <#35 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ALV3V7Y6E4B74EUACY74273VNIRZTANCNFSM5XZF2W2Q>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
<http://mindbridge.ai/>
Michael Bourassa
Data Scientist | *MindBridge*
+1 613-894-9189 <+1-613-894-9189>
Bourassa
mindbridge.ai
[image: twitter] <https://twitter.com/mindbridge_ai>
[image: linkedin] <https://www.linkedin.com/company/mindbridge-ai/>
[image: banner]
<https://www.mindbridge.ai/news/mindbridge-named-to-forbes-ai-50-list-of-most-promising-ai-companies>
--
<http://mindbridge.ai/>
Michael Bourassa
Data Scientist | *MindBridge*
+1 613-894-9189 <+1-613-894-9189>
Bourassa
mindbridge.ai
[image: twitter] <https://twitter.com/mindbridge_ai>
[image: linkedin] <https://www.linkedin.com/company/mindbridge-ai/>
[image: banner]
<https://www.mindbridge.ai/news/mindbridge-named-to-forbes-ai-50-list-of-most-promising-ai-companies>
|
I'm super super glad to hear that, and thank you for your kind words :)
Yes, that is correct — the data "live" in the backend, so to switch between backends, you need to either move the data between them or have a copy of the data in both.
The default is NetworkX if you create a graph without specifying a backend: from grand import Graph
g = Graph() This is the same as: from grand import Graph
from grand.backends import NetworkXBackend
g = Graph(backend=NetworkXBackend()) But you can also use a different backend, like this: import grand
from grand.backends import NetworkitBackend
g = grand.Graph(backend=NetworkitBackend()) In which case the data "live" in Networkit. Backends are a separate idea from "dialects," which are how you talk to the data. ALL dialects are available on ALL graphs, no matter what their backend is. You can see a full list of dialects here. For ANY of the graphs detailed above, you can talk to them as though they are NetworkX g.nx # pretends to be a networkx graph
g.nx.add_edge("node-1", "node-2") Or you can talk to them as though they were an igraph.Graph object: g.igraph.vs Grand handles the "rewriting" of these familiar operators into the language that the backend actually speaks. As far as you (as the user) are concerned, you are actually speaking to NetworkX, not Grand.
You do not have to pre-construct a database; in fact, you don't even need one to exist. Here's how I would create a SQLite graph with a few edges in it: import grand
from grand.backends import SQLBackend
g = grand.Graph(backend=SQLBackend(db_url="sqlite:///my-file.db")) You could also (I haven't done this before! I think it should work, though!) create a postgres connection like this: g = grand.Graph(backend=SQLBackend(db_url="postgresql://jordan:mypassword@localhost/mydatabase")) If you already have two database tables in your db (one for nodes and one for edges), you can tell Grand to connect to them like this: g = grand.Graph(
backend=SQLBackend(
db_url="postgresql://jordan:mypassword@localhost/mydatabase",
node_table_name="my_nodes",
edge_table_name="my_edges",
edge_table_source_column="src",
edge_table_target_column="tgt",
primary_key="id",
)
) This will look for a table called This starts to get a bit untested; I've done all of these things before, but I am curious to hear your experiences, especially if you wind up using a non-sqlite database! Contributions to documentation as you discover things would be AMAZING (even just issues saying 'this is under-documented' are helpful!); there are SO many interesting corners in this project that it's hard to tell what docs would be useful and used by people and which would just be extra work for me, without an audience. Some further reading: #19 talks about connecting to an existing database, with some commentary as well. |
I've been able to connect to the db, use existing tables, create node/edge tables. To speed up scaling some tests I tried to read/write a |
I think import networkx as nx
from grand import Graph
g = Graph(...)
nx.write_graphml(g.nx, "my.graphml") I would not be surprised if this works! But then again... I would not be surprised if this doesn't work :) One alternative would be to move the edges and nodes over to a "real" networkx object. For very large graphs this could take a while, but it might suit your purposes here: import networkx as nx
from grand import Graph
g = Graph(...)
real_g = nx.DiGraph()
for node, attrs in g.nx.nodes(data=True):
real_g.add_node(node, *attrs)
for u, v, attrs in g.nx.edges(data=True):
real_g.add_edge(u, v, attrs)
nx.write_graphml(real_g, "my.graphml") |
How goes it, @MikeB2019x? Would it be helpful to hop on a screenshare sometime? |
@j6k4m8 screenshare not required at the moment but I may take you up on that in the future. So trying to write out a graphml as suggested throws an error (stack trace below). If I compare a networkx graph's attributes and those of
|
Interesting — do you mind if I migrate this to a new issue to address? This would be a good capability for us to have in the Grand library, thank you for bringing it yup! |
@MikeB2019x — what is the status of this issue? Happy to discuss graph export separately in #39 if that's helpful; want to make sure edge attributes are working for you now! |
My situation is that I use
networkx
and have access to apostgres
db. I findnetworkx
to be quite slow and thought of using some of the alternatives esp.networkit
. The challenge I have is with attributes ie.networkit
seems to allow only a single numerical 'weight' for edge attributes. My graphs need pretty rich edge attributes andnetworkx
accommodates those. So:would
grand
allow me to usenetworkx
syntax/features and edge attribute functionality withnetworkit
eg. filter edges on rich attribute set but retain algorithms running at higher speeds?you mention
grand
interacting withdynamo db
. I'm not sure I understand that. Isgrand
using the db to store the graph structure and if so, could it do that with apostgres
db? Note: I had a look at this and it seems like this is what I had in mind but when I read yourreadme
.The text was updated successfully, but these errors were encountered: