Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-7994. Expose information via the ClusterState endpoint on keys and Directories marked for deletion #4406

Merged
merged 4 commits into from Mar 29, 2023

Conversation

ArafatKhan2198
Copy link
Contributor

What changes were proposed in this pull request?

  • When a key is deleted, it is not immediately removed from the metadata but marked for deletion and moved to the DeletedTable. The KeyDeletingService sends requests to delete the corresponding data blocks associated with the keys. Once Recon detects the deletion, it updates its metadata store to reflect the change and retains a record of the deleted key in the DeletedTable for administrators to query.
  • This information can be useful for auditing, compliance, and troubleshooting purposes. Hence in short the main purpose is to get a quick idea of the deletion progress.
  • This information will be displayed on the overview page of recon which gets populated with the information provided by the ClusterStateEndpoint
  • The Jira is coming out of the following discussion :- HDDS-7926. [hsync] Recon throws ClassCastException. #4266 (comment)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7994

How was this patch tested?

Unit Testing

@ArafatKhan2198
Copy link
Contributor Author

@jojochuang @ChenSammi @devmadhuu @dombizita can you please take a look!

@kerneltime
Copy link
Contributor

@sumitagrawl can you please take a look?

@adoroszlai adoroszlai changed the title HDDS-7994. Expose information via the ClusterState endpoint on keys marked for deletion and moved to the DeletedTable. HDDS-7994. Expose information via the ClusterState endpoint on keys marked for deletion Mar 22, 2023
Copy link
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArafatKhan2198 Count of deleted keys is one of sub-part of features which is under analysis phase, as part of that, it need expose Recon Insight for deletedTable, deletedDirectoryTable, and other metadata, to show data in progress of deletion. So there can be changes in interface and details to be exposed.

I think we can hold this till that is designed. @devmadhuu is working over this.

Copy link
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArafatKhan2198 LGTM, plz add testcase for deletedDir Count also

@ArafatKhan2198 ArafatKhan2198 requested review from sumitagrawl and devmadhuu and removed request for duongkame, errose28, smengcl and sumitagrawl March 27, 2023 19:11
@ArafatKhan2198 ArafatKhan2198 changed the title HDDS-7994. Expose information via the ClusterState endpoint on keys marked for deletion HDDS-7994. Expose information via the ClusterState endpoint on keys and Directories marked for deletion Mar 28, 2023
Copy link
Contributor

@sumitagrawl sumitagrawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArafatKhan2198 Thanks for working, LGTM +1

Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 Merging it

@jojochuang jojochuang merged commit 05c8dbb into apache:master Mar 29, 2023
@jojochuang
Copy link
Contributor

Merged. Thanks @sumitagrawl for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants