-
Notifications
You must be signed in to change notification settings - Fork 483
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDDS-10653. Support custom metadata for MPU key #6489
Conversation
@kerneltime @DaveTeng0 @SaketaChalamchala @vtutrinov Could you help take a look when you have time? |
cc @SaketaChalamchala @tanvipenumudy @vtutrinov can you'll please take a look? |
@ivandika3 thanks for the changes. LGTM at first glance, going to continue the review a little bit later |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ivandika3 for the patch.
import static org.mockito.ArgumentMatchers.anyMap; | ||
import static org.mockito.Mockito.any; | ||
import static org.mockito.Mockito.anyBoolean; | ||
import static org.mockito.Mockito.anyLong; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: import from Mockito
.
import static org.mockito.ArgumentMatchers.anyMap; | |
import static org.mockito.Mockito.any; | |
import static org.mockito.Mockito.anyBoolean; | |
import static org.mockito.Mockito.anyLong; | |
import static org.mockito.Mockito.any; | |
import static org.mockito.Mockito.anyBoolean; | |
import static org.mockito.Mockito.anyLong; | |
import static org.mockito.Mockito.anyMap; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the review. Updated.
for (Map.Entry<String, String> customEntry : customMetadata.entrySet()) { | ||
assertTrue(keyMetadata.containsKey(customEntry.getKey())); | ||
assertEquals(customEntry.getValue(), keyMetadata.get(customEntry.getKey())); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use assertThat
instead of assertTrue
for better message in case of failure. It also provides assertions for maps.
for (Map.Entry<String, String> customEntry : customMetadata.entrySet()) { | |
assertTrue(keyMetadata.containsKey(customEntry.getKey())); | |
assertEquals(customEntry.getValue(), keyMetadata.get(customEntry.getKey())); | |
} | |
assertThat(keyMetadata).containsAllEntriesOf(customMetadata); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the suggestion. Updated with assertThat
(also updated for other similar tests).
I'll try to use more assertThat
APIs in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some test failure due to the last commit, let me fix it first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
@ivandika3 thanks for the patch, LGTM, +1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @ivandika3 for updating the patch.
String multipartUploadID = keyToMultipartUpload.get(key).getUploadId(); | ||
if (multipartUploadID == null || !multipartUploadID.equals(uploadID)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't keyToMultipartUpload.get(key) == null
indicate "no such upload? We'd hit NPE in that case. I guess MultipartInfoStub#uploadID
would rarely be null
.
(createMultipartKey
seems to have the correct logic.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for catching it. I have updated it as the same logic as createMultipartKey
.
Thanks @ivandika3 for the patch, @vtutrinov for the review. |
Thank you @vtutrinov and @adoroszlai for the reviews. |
(cherry picked from commit d10a822)
(cherry picked from commit d10a822)
(cherry picked from commit d10a822)
(cherry picked from commit d10a822)
(cherry picked from commit d10a822)
What changes were proposed in this pull request?
HDDS-6440 added support for custom metadata for normal object creation.
We should also support custom metadata creation for multipart upload key.
The metadata should be set during the multipart upload initiation request, which will be committed during the multipart upload complete request.
See: https://docs.aws.amazon.com/cli/latest/reference/s3api/create-multipart-upload.html
Should be tested using unit tests and acceptance tests.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10653
How was this patch tested?
Unit, integration, and acceptance tests.
Clean CI: https://github.com/ivandika3/ozone/actions/runs/8581916553