Broken graph after removing subnetworks, then adding nodes

[michaz]

Adding edges (or, at least, adding edges to new nodes) to a graph after having removed disconnected subnetworks results in a corrupted graph:

7f82541

[karussell]

This is ugly - thanks for finding and already writing the failing test :) 👍 !

We currently remove the nodes in the graph and do the compaction in-place to avoid holding two (potentially really big graphs) in memory. But we do this only for the nodes and leave the invalid edges in the graph.

BTW: Maybe we should do the same for the nodes (leaving them in the graph) and do the compaction only via the simpler graph copying mechanism as this in-place compaction made lot's of problems and changes the node-IDs and potentially does not save so much RAM that the problems are justified. (graph copying is already done if graph sorting is enabled then we call GHUtility.sortDFS(ghStorage, newGraph))

[michaz]

Yes, I must admit I had a moment of panic where I was afraid that we would be changing the edge ids. (I hold on to them and save them and do lookups on them!) But luckily we aren't.

(I don't care about nodes.)

[karussell]

I think the problem is that we do not reset the pointers to the edges of the invalidated nodes in the graph.optimize() call. So in the test the edge 7-8 is created and 8 is still linking to some old edges and we append now 7-8 to the linked list and reanimate an edge with invalid nodes.

The fix is here: 6be91f3

[karussell]

Strange, this does not make sense because we call baseGraph.trimToSize();

[karussell]

Ah, we keep only the necessary segment files but they could still partially filled with old data.