Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging multiple graphs? #38

Open
Getsumi3 opened this issue Jun 30, 2021 · 2 comments
Open

Merging multiple graphs? #38

Getsumi3 opened this issue Jun 30, 2021 · 2 comments

Comments

@Getsumi3
Copy link

Hi.
Thanks for this awesome library! I was able to visualize some pretty big datasets (5-10 millions of nodes and 15 millions of links).

So I tried to increase the number of nodes(somewhere around 50 millions).
I have 32GB of RAM btw.

My current issue is that creating a ngraph.graph instance is a bit difficult. I mean because of such a large dataset my PC memory gets full very fast.

So I was wondering if there is a way to save several ngraph.graph instances into files and then merge them (so the result next can be processed with ngraph.tobinary).
Or maybe there is another approach to this issue?

Thanks!

@Telokis
Copy link

Telokis commented Aug 14, 2023

I'm also wondering about that, yeah. I have several subgraphs that I would like to combine into one mega-graph for pathfinding.

@Getsumi3
Copy link
Author

I'm also wondering about that, yeah. I have several subgraphs that I would like to combine into one mega-graph for pathfinding.

I didn't manage to merge several subgraphs.
Instead I hit a bottleneck - browser per-tab memory.
Basically it means that if generated files are too big the browser wouldn't be able to handle your graph.
Different browsers have different limits. Google Chrome for example can handle up to 4GB data per tab, that's ~14 millions of nodes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants