HDDS-7524. Compaction DAG node pruning

[hemantk-12]

What changes were proposed in this pull request?

To generate faster diff between snapshots, we maintain a compaction DAG in memory. Whenever compaction happens, related SST file nodes get added to the DAG. Over time, DAG will keep on increasing and may cause memory pressure or become a bottleneck. To solve this, we can prune the unnecessary SST file nodes from the DAG since we have a concept of the oldest snapshot with compaction history.
Related doc: https://docs.google.com/document/d/19ZOT-lY2WS8nDj1s9HGVyUY3kgnKIfv-Eck--SokaDU/

This change proposes the traversal and pruning of the DAG.
Idea here is to remove the nodes and arcs which were created before snapshot, to be deleted, was created because they are not needed to generate the diff anymore.
pruneBackwardDag prunes the backward DAG and removes the nodes and arcs from the upstream of the level at which snapshot was taken.
pruneForwardDag prunes the forward DAG and removes the nodes and arcs from the downstream of the level at which snapshot was taken.

Let's take an example of the following diagram (Backward DAG).

Snapshots were taken at level1, level-3 and level-5
Snapshot-1: 000015.sst, 000013.sst, 000011.sst, 000009.sst
Snapshot-2: 000027.sst, 000030.sst, 000028.sst, 000031.sst, 000029.sst, 000039.sst, 000037.sst, 000035.sst, 000033.sst
Snapshot-3: 000059.sst, 000055.sst, 000056.sst, 000060.sst, 000057.sst, 000058.sst

If Snapshot-1 and Snapshot-2 need to be pruned, we can simply prune upstream of level-3 in backward DAG and downstream of level-3 in Forward DAG. So all the nodes from level-1, level-2 and level-3 and arcs between level-1 & level-2, level-2 & level-3 and outgoing/incoming arcs from/to level-3 will be removed from both forward and backward DAGs.

What is the link to the Apache JIRA

• https://issues.apache.org/jira/browse/HDDS-7524

How was this patch tested?

• Unit tests.

[smengcl]

Thanks @hemantk-12 . Core logic looks good to me. Some comments inline.

[smengcl]

Thanks @hemantk-12 for the patch. Thanks @GeorgeJahad and @prashantpogde for reviewing this.