Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add S3 rename tool #490

Merged
merged 5 commits into from
Jul 19, 2023
Merged

feat: Add S3 rename tool #490

merged 5 commits into from
Jul 19, 2023

Conversation

Alima777
Copy link
Collaborator

@Alima777 Alima777 commented Jul 14, 2023

This tool is used to delete table_uuid from the key of s3 objects. Only patterns like "root_prefix/table_uuid/part_uuid/key" will be renamed, other objects will keep like before.

@CLAassistant
Copy link

CLAassistant commented Jul 14, 2023

CLA assistant check
All committers have signed the CLA.

S3::Model::CopyObjectOutcome outcome = client.CopyObject(request);
if (!outcome.IsSuccess()) {
const S3::S3Error &err = outcome.GetError();
std::cerr << "Error: CopyObject: " <<
Copy link
Contributor

@linkwk7 linkwk7 Jul 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Poco::Logger to print logs

boost::asio::thread_pool thread_pool(threadNum);
for (S3::Model::Object &object: objects) {
// root_prefix/table_uuid/part_uuid/name
String from_key = object.GetKey();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key will have format of prefix/table_uuid/part_uuid/data, can check if table_uuid & part_uuid is a valid uuid here by ReaderHelpers.h:tryReadUUIDText

@hustnn
Copy link
Contributor

hustnn commented Jul 18, 2023

hi @Alima777 , pls sign the CLA

@Alima777
Copy link
Collaborator Author

Hi, @hustnn , it's signed now.

@linkwk7 linkwk7 merged commit 412abd4 into ByConity:master Jul 19, 2023
2 checks passed
@Alima777 Alima777 deleted the s3RenameTool branch August 14, 2023 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants