HDDS-10324. Metadata are not updated when keys are overwritten.

[ArafatKhan2198]

What changes were proposed in this pull request?

A bug was identified where updating an existing key with new metadata does not lead to the expected update of the metadata; instead, the system retains the old metadata. These keys were created using the S3 CLI. To simulate this problem please take a look at the JIRA description.

Root Cause :-

When we create an object with the same key name as one already in the database, the request is forwarded to the OM side of the code, specifically to the OmKeyRequest class, which contains a method called prepareFileInfo(). This method persists the data to the openKeyTable. Initially, it checks if a key with the same name exists. If it does, it proceeds to update it with new data size, modification time, updateID, and replicationConfig. However, it fails to update the metadata of the overridden file. Consequently, the old metadata stored earlier is retained, which is the root cause of the problem and requires fixing.

Changes made :-

1. Extracting the new metadata from the KeyArgs object.
2. Comparing the new metadata with the existing metadata in the OmKeyInfo object.
3. Updating the OmKeyInfo object with any new or modified metadata entries.
4. Ensuring that metadata entries not mentioned in the overwrite operation are retained as is.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10324

How was this patch tested?

Tested it out Manually & Tested it out with UT's for the below two scenarios :-

1. Test Metadata Is Correctly Updated: Wrote tests to verify that when a key is created with metadata, and then overwritten with new or updated metadata, the metadata is correctly updated in the OmKeyInfo object that would be persisted. This involves:
2. Initial creation without metadata, followed by an overwrite that includes metadata.

[ivandika3]

@ArafatKhan2198 Will the bug also causes ETag field to not be updated when overwriting an object with another object with different content?

[ArafatKhan2198]

@ArafatKhan2198 Will the bug also causes ETag field to not be updated when overwriting an object with another object with different content?

@ivandika3
The ETag serves as a unique identifier for the content of a file at a given time. When you upload a file to S3 with a specific key, S3 generates an ETag based on the content of that file. If you upload the same file again (with the exact same content and under the same key), the ETag remains the same because the content has not changed.

If you modify the contents of the file, the ETag will change accordingly. I have tested this scenario, and when uploading a file with the same key name but different content, the ETag also changes.

[ArafatKhan2198]

@ChenSammi @sumitagrawl Could you please take a look!

[adoroszlai]

hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java
 26: Using the '.*' form of import should be avoided - java.util.*.
 35: Unused import - org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.

https://github.com/ArafatKhan2198/ozone/actions/runs/8045754649/job/21971634453

[adoroszlai]

Thanks @ArafatKhan2198 for the patch.

[ChenSammi]

The last patch looks good. Thanks @ArafatKhan2198 .

[adoroszlai]

Thanks @ArafatKhan2198 for the patch, @ChenSammi for the review.