HDDS-8435. [Snapshot] Handle Directory renames for FSO Buckets.

[aswinshakil]

What changes were proposed in this pull request?

When directories are renamed, this information is not tracked. It is required by theSnapshotDeletingService to check the previous snapshot's directoryTable and check if the renamed key is present.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8435

How was this patch tested?

Existing tests.

[jojochuang]

HI i haven't reviewed the PR because i don't know much about the snapshot design.
But I know for a fact directory rename could be problematic for FSO. For example, what happens if a file is being written under a directory that is renamed? It will file to commit. Has this been considered?

[aswinshakil]

The snapshotRenamedTable only tracks the tableKey of the directory that was renamed and it is only used by SnapshotDeletingService. When a directory is renamed after a snapshot for a bucket is taken. The SnapshotDeletingService looks into the directoryTable of the previous snapshot, but when the directory is renamed the tableKey will be changed. The snapshotRenamedTable only acts as reference to check the renamed directories in between snapshot.

For example, what happens if a file is being written under a directory that is renamed? It will file to commit. Has this been considered?

This shouldn't be an issue, Everything would go through the original flow.

[smengcl]

lgtm overall

[aswinshakil]

Thanks @smengcl for the review. I have updated the PR, please take a look at it.

[smengcl]

lgtm so far.

@GeorgeJahad mentioned he would want to scrutinize the test cases.

Though we have the renameDir addition here. We might want to have more test cases around mixtures of key deletes, renames and snapshot deletes.

e.g.

1. create keys and dirs
2. create snapshot 1
3. rename keys and dirs
4. verify key and rename table states
5. create snapshot 2
6. rename some of the keys and dirs, delete some of the keys and dirs
7. verify table states
8. create snapshot 3
9. again rename some of the keys and dirs, delete some of the keys and dirs
10. create snapshot 4
11. verify table states
12. delete snapshot 3, trigger SDT, verify table states
13. delete snapshot 2, trigger SDT, verify table states
14. delete snapshot 1, trigger SDT, verify table states
15. delete snapshot 4, trigger SDT, verify table states

Parameterize for LEGACY, FSO and OBS.

Maybe in another jira.

[smengcl]

Thanks @ashishkumar50 for the patch. Thanks @devmadhuu and @ashishkumar50 for the reviews.