-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Krowkee integration #2
Conversation
src/embed_ygm.cpp
Outdated
<< ", " << world.routing_protocol() << ", " << range_size | ||
<< ", " << vertex_count << ", " | ||
<< local_edge_count * world.size() << ", " | ||
<< compaction_threshold << ", " << promotion_threshold; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you print the seed after the promotion threshold to match the headers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops! fixed.
Does the For comparison, I tested embedding a graph with 2^26 vertices and 2^21 * 36 edges in the uniform case. However, the RMAT version appears to top out at 2^26 vertices and 2^14 * 36 edges. Adding more edges causes OOM errors. I am confident that the issue is not in In both cases I am storing the edges after generation in a Any ideas? |
The RMAT generator shouldn't be using much memory. It's essentially a handful of doubles, ints, and bools. I have been able to use the RMAT for experiments that amount to 2^26 vertices per compute node and 10M edges per MPI rank. Are you using the |
I am using the It appears that the I am still not sure why this would cause OOM issues when interfacing with |
I've illustrated the issue with changes in PR #3 |
Shoot. You're right. I was using In the case that |
…that it each iteration starts fresh.
So is the fix as simple as changing |
Yeah. I just played with the code from PR #3. The first trial has the local and global stats agree, but the second does not. Adding a bigger number is 'better' so each trial ends up with more distinct edges. Thanks for pointing out this bug. |
There is suddenly a new compile issue relating to the recent commit to the feature/routing branch.
I commented out the stats collection lines to get However, even when I comment out the I'm going to have to step away from this for now, but I don't think that I'll be able to get the rmat |
Sorry about that. I'm changing some stuff to get more stats of interest for looking at YGM performance. I'll get everything in a coherent state. |
…arameter from log to exact scale.
…is templated upon a functor providing a vector of edge updates.
…_generator to be very uniform, as skewed distributions cause out-of-memory errors for unknown reasons.
I went ahead and moved all of the krowkee test chassis into a header, so that the only differences between the uniform and rmat tests is the functor that generates the edge list. However, the RMAT test is still breaking with OOM errors. Now, however, it only breaks when the RMAT distribution is skewed. The benchmark runs correctly when all parameters are set to 0.25. As a consequence, we have to conclude that there is a problem somewhere in the |
…pear to have solved the OOM errors.
I finally revisited this, and concluded that there was an error in the way that I was handling |
No, I am still getting OOM errors. I am still unsure why. |
@steiltre I fixed this issue. It was a simple (and incredibly stupid) error on my part. The |
No description provided.