HDDS-4102. Normalize Keypath for lookupKey.

[bharatviswa504]

What changes were proposed in this pull request?

When ozone.om.enable.filesystem.paths, OM normalizes path, and stores the Keyname.

Now when user tries to read the file from S3 using the keyName which user has used to create the Key, it will return error KEY_NOT_FOUND

The issue is, lookupKey also need to normalize path, when ozone.om.enable.filesystem.paths is enabled. This is a common API used by S3/FS.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4102

How was this patch tested?

Added a test.

[elek]

Thanks for the patch @bharatviswa504, I would suggest adding a related robot test, too, which would make it possible to test the AWS compatibility of the change.

What do you think?

[elek]

Tested and file/ and file are different:

$ aws s3api get-object --bucket=ozonetest --key=test1/file /tmp/www
{
    "AcceptRanges": "bytes",
    "LastModified": "Wed, 26 Aug 2020 14:24:06 GMT",
    "ContentLength": 79,
    "ETag": "\"6fcb0dbe8f0ef330b38f81a6a6a7a66e\"",
    "ContentType": "text/markdown",
    "Metadata": {}
}

aws s3api get-object --bucket=ozonetest --key=test1/file/ /tmp/www

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Just an idea: what do you think about normalizing the path on the client side? (o3fs/ofs). Would it make any problems?

[bharatviswa504]

Tested and file/ and file are different:

$ aws s3api get-object --bucket=ozonetest --key=test1/file /tmp/www
{
    "AcceptRanges": "bytes",
    "LastModified": "Wed, 26 Aug 2020 14:24:06 GMT",
    "ContentLength": 79,
    "ETag": "\"6fcb0dbe8f0ef330b38f81a6a6a7a66e\"",
    "ContentType": "text/markdown",
    "Metadata": {}
}

aws s3api get-object --bucket=ozonetest --key=test1/file/ /tmp/www

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Just an idea: what do you think about normalizing the path on the client side? (o3fs/ofs). Would it make any problems?

Thank You @elek for the test.

Uploaded design doc has provided reasoning to perform on the OM side.

And also when used via ofs/o3fs, it uses path Object, and paths are normalized already.

[bharatviswa504]

Thanks for the patch @bharatviswa504, I would suggest adding a related robot test, too, which would make it possible to test the AWS compatibility of the change.

What do you think?

Opened a jira HDDS-4154 to test S3 test suite with ozone.om.enable.filesystem.paths enabled.

[elek]

Opened a jira HDDS-4154 to test S3 test suite with ozone.om.enable.filesystem.paths enabled.

Thanks. I think we need additional s3 tests as the behavior what I tested above (and failing) is not part of the current test suite.

Uploaded design doc has provided reasoning to perform on the OM side.

Thanks to share it. Now I understood the motivation. But what is your opinion about the AWS compatibility? For me it seems to be a bigger problem compared to having the validation code in one at the OM or one place at the client side.

What is your opinion about AWS compatibility issues?

[bharatviswa504]

Opened a jira HDDS-4154 to test S3 test suite with ozone.om.enable.filesystem.paths enabled.

Thanks. I think we need additional s3 tests as the behavior what I tested above (and failing) is not part of the current test suite.

Uploaded design doc has provided reasoning to perform on the OM side.

Thanks to share it. Now I understood the motivation. But what is your opinion about the AWS compatibility? For me it seems to be a bigger problem compared to having the validation code in one at the OM or one place at the client side.

What is your opinion about AWS compatibility issues?

When this flag is enabled, as the config explains, we consider paths as unix paths and provide file system semantics. So when this is enabled we cannot gurantee 100% AWS compatibility. As said we will make this bucket level config to provide ease to users to have both on a cluster.

Opened a jira HDDS-4154 to test S3 test suite with ozone.om.enable.filesystem.paths enabled.

Thanks. I think we need additional s3 tests as the behavior what I tested above (and failing) is not part of the current test suite.

Uploaded design doc has provided reasoning to perform on the OM side.

Thanks to share it. Now I understood the motivation. But what is your opinion about the AWS compatibility? For me it seems to be a bigger problem compared to having the validation code in one at the OM or one place at the client side.

What is your opinion about AWS compatibility issues?

Currently, the test is simulated in the Integration test. As when this flag is enabled user has chosen to consider key as unix paths. I will add a test suite in HDDS-4154. For now, all tests are in integration tests.

  <property>
    <name>ozone.om.enable.filesystem.paths</name>
    <tag>OZONE, OM</tag>
    <value>false</value>
    <description>If true, key names will be interpreted as file system paths.
      "/" will be treated as a special character and paths will be normalized
      and must follow Unix filesystem path naming conventions. This flag will
      be helpful when objects created by S3G need to be accessed using OFS/O3Fs.
      If false, it will fallback to default behavior of Key/MPU create
      requests where key paths are not normalized and any intermediate
      directories will not be created or any file checks happens to check
      filesystem semantics.
    </description>
  </property>

So, test/file/ will be normalized to test/file. This PR provides user ability to read with the same file name user has used during the creation of key "test/file/".

[bharatviswa504]

@elek Any more comments?

[elek]

@elek Any more comments?

Looks like we don't have the consensus, yet, in HDDS-4097. It was discussed yesterday during the community meeting, and I felt that everybody agree more or less, but I am not sure if @arp7 fully agrees.

[arp7]

What is your opinion about AWS compatibility issues?

What exactly is the compatibility issue? If the paths are not interpreted (default behavior), then there is full AWS compatibility.

[bharatviswa504]

@elek
There is no issue with this, and I believe we have agreed on the part we need this like 100% HCFS with few changes to AWS S3 semantics. I don't see what is the problem in moving forward?.

As the other argument is when the flag is disabled to support compromised HCFS and 100% AWS.

Let me know if you have any concerns about this PR.

[elek]

I uploaded my proposal to https://issues.apache.org/jira/browse/HDDS-4097, please let me know your opinion.

I think the normalization should be optional and should be driven by a configuration which is independent of the intermediate directory creation (ozone.om.enable.filesystem.paths).

While intermediate directories are a must-have for providing services for both S3 and HCFS, normalization seems to be an optional (but very useful) feature.

[bharatviswa504]

I have rebased this PR.

cc @elek @arp7

[elek]

+1 LGTM

Based on my understanding this is exactly what we discussed: normalization is turned on when OZONE_OM_ENABLE_FILESYSTEM_PATHS_DEFAULT is enabled, but not in any other cases.