-
Notifications
You must be signed in to change notification settings - Fork 320
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modified Tarjan SCC to reduce memory usage. #413
Conversation
Reduced heap memory usage to a single vec allocation, improved cache locality, and made the code cleaner for a possible future rewrite to a more optimized imperative version. In addition, the new version has the property that it builds a lookup table for the components. If v belongs to the kth output component, then self.nodes[g.to_index(v)] == Some(k) . This can be used to save work in functions that may want to call it such as condensation.
Another quick comment: because of the way it builds up a component lookup table, this version would also enable a sort_scc_stable function (returning a list of SCCs where the nodes appear in the order given by the original graph or some iterator over its nodes), which would be very useful in some situations. Basically, since a node will be assigned to its proper SCC when visit returns from the top level loop, that means you can bucket sort nodes by their rindex value in the top level loop instead of when you pop them from the stack. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really nice !
In addition to the comments, it would be nice to have some benchmarks to compare the changes.
I ran #421's benchmarks, and I'm seeing some noticeable performance regressions (I include Kosaraju's for reference):
I'll try to do some profiling. |
Tests the visited flag before doing the recursive call.
I pushed a small change to your branch, it checks if the node has been visited before calling
|
Uhm, I'm getting pretty noisy benchmarks (full_kosaraju_sccs there got -15% with no code changes). If you are OK with this I think the PR is ready to merge. |
Sounds great! When I have more time I'll work on an imperative version that can't stack overflow and microoptimize it in a followup PR. There's also a few other functions that depend on the SCC's such as condensation that could be rewritten to take advantage of the new node_component_index method in order to avoid work. |
* Modified Tarjan SCC to reduce memory usage. Reduced heap memory usage to a single vec allocation, improved cache locality, and made the code cleaner for a possible future rewrite to a more optimized imperative version. In addition, the new version has the property that it builds a lookup table for the components. If v belongs to the kth output component, then self.nodes[g.to_index(v)] == Some(k) . This can be used to save work in functions that may want to call it such as condensation. * added std:: in front of std::usize::MAX * Rustfmt-ed the code. * Added node_component_index, adopted NonZeroUsize, reversed counters. * Minor change to node_component_index. * Docs update. Added citation to paper with link. * Improve tarjan scc performance Tests the visited flag before doing the recursive call. Co-authored-by: Agustin Borgna <agustinborgna@gmail.com>
* Modified Tarjan SCC to reduce memory usage. Reduced heap memory usage to a single vec allocation, improved cache locality, and made the code cleaner for a possible future rewrite to a more optimized imperative version. In addition, the new version has the property that it builds a lookup table for the components. If v belongs to the kth output component, then self.nodes[g.to_index(v)] == Some(k) . This can be used to save work in functions that may want to call it such as condensation. * added std:: in front of std::usize::MAX * Rustfmt-ed the code. * Added node_component_index, adopted NonZeroUsize, reversed counters. * Minor change to node_component_index. * Docs update. Added citation to paper with link. * Improve tarjan scc performance Tests the visited flag before doing the recursive call. Co-authored-by: Agustin Borgna <agustinborgna@gmail.com>
Reduced heap memory usage, improved cache locality, and made the code somewhat cleaner for a possible future rewrite to an imperative implementation. The changes to a more modern version of Tarjan's algorithm were directly based on algorithm 3 in "A Space-Efficient Algorithm for Finding Strongly Connected Components" by David J. Pearce, while also aiming to avoid changing the structure of the code too much.
In addition to saving memory, this version has the property that it builds a lookup
table for the components at the same time as it outputs them. If v belongs to the kth output component, then
self.nodes[g.to_index(v)] == Some(k)
. This could be used to save work infunctions that may want to call it such as condensation (where the work needed to make the node map can be completely avoided, while also creating a condensation that is topologically sorted).