Dynamic betweenness centrality #127

antepusic · 2022-03-15T08:00:11Z

Description

Dynamic betweenness centrality implementation.

Pull request type

Algorithm/Module

######################################

Reviewer checklist (the reviewer checks this part)

Module/Algorithm

######################################

jmatak

This is the first par of the review

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.cpp

jmatak

Just some idea for the E2E tests. Try out different examples with NetworkX and create test with Python

cpp/betweenness_centrality_module/CMakeLists.txt

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.cpp

cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp

jmatak · 2022-03-16T14:29:11Z

cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp

+        // Dynamically update betweenness centrality scores by each created edge
+        for (const auto created_edge : created_edges) {
+          edges.push_back({graph->GetInnerNodeId(created_edge.first), graph->GetInnerNodeId(created_edge.second)});
+          graph = mg_generate::BuildGraph(graph->Nodes().size(), edges, mg_graph::GraphType::kUndirectedGraph);


This looks extremely computationally heavy

The alternative is using the CreateEdge and EraseEdge methods. Whereas that will not mess up the inner IDs (with edge update, both the prior and the current graph have the same nodes), I'd appreciate a greenlight from you before going ahead with it.

Since we've discussed this separately, please make a TODO for that somewhere :D

jmatak · 2022-03-16T14:29:57Z

cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp

+      if (created_node_ids.size() == 0 && deleted_node_ids.size() == 0) {  // Use the online algorithm
+        // Construct the graph as before the update
+        std::vector<std::pair<std::uint64_t, std::uint64_t>> edges;
+        edges.reserve(graph->Edges().size());
+        for (const auto edge : graph->Edges()) {
+          std::pair<uint64_t, uint64_t> edge_memgraph_ids = {graph->GetMemgraphNodeId(edge.from),
+                                                             graph->GetMemgraphNodeId(edge.to)};
+          // Newly created edges aren’t part of the prior graph
+          if (std::find(created_edges.begin(), created_edges.end(), edge_memgraph_ids) == created_edges.end())
+            edges.push_back({edge.from, edge.to});
+        }
+        for (const auto edge : deleted_edges) {
+          edges.push_back({graph->GetInnerNodeId(edge.first), graph->GetInnerNodeId(edge.second)});
+        }
+        auto prior_graph = mg_generate::BuildGraph(graph->Nodes().size(), edges, mg_graph::GraphType::kUndirectedGraph);


Candidate for mg_utility?

I've understood that our utilities all get information from the current graph about its composition, and this code snippet is different in that it takes that data (the Edges call) and then transforms it. With this in mind, what would you suggest? 🤔

cpp/biconnected_components_module/algorithm/biconnected_components.hpp

…_centrality_online.cpp Co-authored-by: Josip Matak <31473001+jmatak@users.noreply.github.com>

… into T080-MAGE-dynamic-BC

cpp/mg_utility/data_structures/graph_view.hpp

cpp/mg_utility/mg_test_utils.hpp

e2e/betweenness_centrality_online_test/test_online_create_node/input.cyp

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.cpp

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.hpp

jmatak · 2022-04-08T12:12:03Z

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.hpp

+  ///@brief Performs a breadth-first search from a given node with the Brandes’ algorithm.
+  ///
+  ///@param graph Graph traversed the BFS
+  ///@param start_id Start node ID
+  ///@param compensate_for_deleted_node If true, guarantees correct results for the DETACH_DELETE_NODE operation
+  ///
+  ///@return Nº of shortest paths to each node from the start node
+  /// Each node’s immediate predecessors on the shortest paths from the start node
+  /// IDs of nodes visited by the BFS, in reverse order
+  std::tuple<std::unordered_map<std::uint64_t, std::uint64_t>,
+             std::unordered_map<std::uint64_t, std::set<std::uint64_t>>, std::vector<std::uint64_t>>
+  BrandesBFS(const mg_graph::GraphView<> &graph, const std::uint64_t start_id,
+             const bool compensate_for_deleted_node = false) const;


Where exactly do we use the information from the Brandes' algorithm

It is used in the cases handled by NodeEdgeUpdate, i.e. those where a single iteration of Brandes is enough to update BC values.

jmatak · 2022-04-08T12:16:00Z

cpp/betweenness_centrality_module/algorithm_online/betweenness_centrality_online.hpp

+  std::unordered_map<std::uint64_t, std::uint64_t> SSSPLengths(
+      const mg_graph::GraphView<> &graph, const std::uint64_t start_id,
+      const std::unordered_set<std::uint64_t> &bcc_nodes) const;


Could this be encapsulated in one of the BFS methods to not repeat the process, or it would be too much work?

Any of the BFS methods could be used for this, but at cost of making unnecessary data structures (e.g. predecessors, Ns of shortest paths) when the algorithm only needs the distances.

cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp

jmatak

🎉

antepusic added 2 commits March 15, 2022 08:58

Add extended biconnected components algorithm

cb2b1a5

Add tested online betweenness centrality algorithm

e543c28

antepusic self-assigned this Mar 15, 2022

antepusic requested a review from jmatak March 15, 2022 08:03

antepusic marked this pull request as draft March 15, 2022 08:04

antepusic added lang: cpp Issue on C++ codebase status: draft PR is in draft phase type: module type: algorithm Initial algorithm implementation labels Mar 15, 2022

Add query module and some e2e tests

38b7ec7

jmatak reviewed Mar 15, 2022

View reviewed changes

antepusic added 2 commits March 16, 2022 10:09

Transfer GetNeighboursMemgraphNodeIds to utilities

48c767c

Remove unused variable

3d1cf49

jmatak suggested changes Mar 16, 2022

View reviewed changes

antepusic and others added 7 commits March 16, 2022 16:00

Update cpp/betweenness_centrality_module/algorithm_online/betweenness…

1278dc9

…_centrality_online.cpp Co-authored-by: Josip Matak <31473001+jmatak@users.noreply.github.com>

Fix module and add support for node updates

5b8d037

Merge https://github.com/memgraph/mage into T080-MAGE-dynamic-BC

ccd3631

Add more tests

2c05145

Merge branch 'T080-MAGE-dynamic-BC' of https://github.com/memgraph/mage…

69f3ddf

… into T080-MAGE-dynamic-BC

Align method signature with header

0343e90

Add tests on more graphs and fix bug at detach&delete

94a4a23

antepusic mentioned this pull request Mar 18, 2022

[master < T082-MAGE] Dynamic betweenness centrality documentation memgraph/docs#321

Merged

2 tasks

Use auto

292fe9d

antepusic requested review from jbajic and jmatak March 18, 2022 15:40

Add partial BFS optimization

81afea1

antepusic added status: ready PR is ready for review and removed status: draft PR is in draft phase labels Mar 27, 2022

Make cast safe

70ba9d0

antepusic marked this pull request as ready for review April 6, 2022 07:59

Merge branch 'main' into T080-MAGE-dynamic-BC

c5bbb41

jmatak suggested changes Apr 8, 2022

View reviewed changes

antepusic added 3 commits April 8, 2022 18:29

Address Matak’s comments

d82bb4b

Make BFS return results bundled inside struct

75931e3

Handle new edges that make a graph connected

8a496fe

jmatak approved these changes Apr 14, 2022

View reviewed changes

Merge branch 'main' into T080-MAGE-dynamic-BC

60fdea0

jmatak mentioned this pull request Apr 15, 2022

Extend biconnected components for articulation points retrieval #123

Closed

15 tasks

jmatak merged commit 92f86eb into main Apr 15, 2022

jmatak deleted the T080-MAGE-dynamic-BC branch April 15, 2022 13:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dynamic betweenness centrality #127

Dynamic betweenness centrality #127

antepusic commented Mar 15, 2022 •

edited by jmatak

jmatak left a comment

jmatak left a comment

jmatak Mar 16, 2022

antepusic Mar 17, 2022 •

edited

jmatak Apr 10, 2022

jmatak Mar 16, 2022

antepusic Mar 17, 2022

jmatak Apr 8, 2022

antepusic Apr 8, 2022

jmatak Apr 8, 2022

antepusic Apr 8, 2022

jmatak left a comment

Dynamic betweenness centrality #127

Dynamic betweenness centrality #127

Conversation

antepusic commented Mar 15, 2022 • edited by jmatak

Description

Pull request type

Reviewer checklist (the reviewer checks this part)

Module/Algorithm

jmatak left a comment

Choose a reason for hiding this comment

jmatak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antepusic Mar 17, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmatak left a comment

Choose a reason for hiding this comment

antepusic commented Mar 15, 2022 •

edited by jmatak

antepusic Mar 17, 2022 •

edited