Large Graph Processing

[cw00dw0rd]

Currently, we do not have a way to limit graph sizes. For example, if we have a large graph stored in ArangoDB that may take up 1TB if loaded into memory. It would be nice to explore limiting the extracted graph from ArangoDB via hard limits or sampling.

Additionally, this relates to optimizing vertex/edge dictionaries currently used to create the NetworkX graph.

[aMahanna]

Some things to consider:

• Large Graph sampling
• Parallel Batch Processing via multithreading in python
• Providing arangoexport and arangoimport as an option for ArangoDB to NetworkX & vice-versa
• GPU-enabled adapter operations

[aMahanna]

(low priority for now)

[nealrauhauser]

This really should be expanded so that there is a bridge to Graphistry - it's a fairly slick GPU enabled web based network visualization tool - similar in spirit to Gephi, but much more capable in terms of volume. They have a free plan that you can use for an hour, then you have to reset the API key. It's not much of a burden, an hour is a LONG time to be messing with any given graph visualization.