Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for OCI Object Storage #10194

Closed
cyberduck opened this issue Jan 12, 2018 · 19 comments
Closed

Support for OCI Object Storage #10194

cyberduck opened this issue Jan 12, 2018 · 19 comments
Assignees
Labels
enhancement fixed s3 version:6.3.4
Milestone

Comments

@cyberduck
Copy link
Collaborator

@cyberduck cyberduck commented Jan 12, 2018

57cbee5 created the issue

Oracle Cloud Infrastructure (OCI) Object Storage provides an S3 Compatible API. It would really be great if Cyberduck could also support it. Looks like it almost works but fails since OCI Object Storage uses different endpoints than S3 does. Cyberduck tries to use us-east-1 region identifier, which results in failure in authentication.

More information on OCI S3 Compatible API is available here: https://cloud.oracle.com/infrastructure/storage/object-storage/faq

As a side note, Cyberduck already supports OCI Object Storage Classic via Swift, which is different than OCI Object Storage.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 12, 2018

57cbee5 commented

Please see & review PR that addresses this issue:

https://bitbucket.org/dkocher/jets3t/pull-requests/13/master/diff

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 12, 2018

@dkocher commented

The hack is not required. Please create connection profile with a customRegion setting.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 12, 2018

57cbee5 commented

Thank you dkocher. I had tried it but it does not work. I use the following profile:

        <Key>Region</Key>
        <String>eu-frankfurt-1</String>
        <Key>Regions</Key>
        <array>
                <string>us-phoenix-1</string>
                <string>us-ashburn-1</string>
                <string>eu-frankfurt-1</string>
        </array>

When I try to authenticate, I get the following error message:
external image

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 12, 2018

57cbee5 commented

BTW, I think you meant Region setting as there doesn't seem to be a customRegion setting? Regardless, I also tried customRegion setting and the result is the same.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 23, 2018

57cbee5 commented

@dkocher, did you have a chance to look into this? Whether via the submitted PR or a configuration change, it will be great if Cyberduck can support OCI Object Storage.

Please kindly let me know what you think and if it is possible to put this into 7.0 release.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 23, 2018

@dkocher commented

Yes, this is planned. I will want to have a look into this to find a generic solution for any S3 provider.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 23, 2018

@dkocher commented

As a workaround it should be possible to use the OpenStack Swift endpoints.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 23, 2018

@dkocher commented

Fix in e417265 (upstream (https://bitbucket.org/dkocher/jets3t/commits/5b3b2d1b2a8199d6d6a676482e57e05ab2ecc5c4?at=bugfix/issue-47)). The Region setting in the connection profile should now be used as the default when signing requests with AWS4-HMAC-SHA256. I am unable to verify with OCI Object Storage – horrendous user interface to setup services I am unable to follow. Verified against objects-us-west-1.dream.io.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 23, 2018

@dkocher commented

@ilker It would be awesome if you could update our wiki page about Oracle Cloud Storage at https://trac.cyberduck.io/wiki/help/en/howto/oraclecloud and provide connection profiles. Please update to the latest snapshot build available for testing.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

57cbee5 commented

@dkocher Thank you. Unfortunately this didn't work but looks like we are pretty close. jets3t changes seem to do the job, but Region value from the configuration does not make it to jets3t.

This part of the code (

configuration.setProperty("storage-service.default-region", host.getRegion());
) seems to set the storage-service.default-region to null. Not entirely sure how host.getRegion() should get the Region out of the config but somewhere along the way it seems to be getting lost.

For reference here is the configuration I use: https://github.com/itemir/cyberduck/blob/master/profiles/OCI%20Object%20Storage%20eu-frankfurt-1.cyberduckprofile

P.S. Happy to update the wiki and upload the profiles once we get this running. I will also send a PR to merge the profiles to the main repo.

P.S. II A similar issue seems to exist for Regions Key Values as well. Regions as specified in the configuration do not seem to be parsed out. It might be caused by the same underlying issue so fixing this may as well fix it too. If not, I will file a separate ticket for it once we get this sorted out.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

@dkocher commented

Please make elements key and string in the connection profile lowercase.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

@dkocher commented

Please make sure the Vendor value is unique for all profiles.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

@dkocher commented

It looks like error messages in the XML response from the service include JSON in the Message element of the error response. This should be plain text instead.

<?xml version='1.0' encoding='UTF-8'?><Error><Message>AuthDataPlaneServerException{statusCode=404, body='{  "code" : "NotFound",  "message" : "Not Found"}'}</Message><Code>SignatureDoesNotMatch</Code></Error>

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

@dkocher commented

Reopen awaiting feedback from testing.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

57cbee5 commented

@dkocher, thank you. This did work. I will update the wiki page and link to updated connection profiles. I sent a PR for adding profiles to the main repo. Also ACK on json vs text error responses.

One last question, do you have any tentative timeline for the 7.0 release?

Thanks again and greetings to Switzerland.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Jan 24, 2018

@dkocher commented

We will retarget this to a minor update – we release about once a month.

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Feb 2, 2018

@dkocher commented

Milestone renamed

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Mar 14, 2021

22e47ae commented

Replying to [comment:16 dkocher]:

Reopen awaiting feedback from testing. would it be possible to give a sample XML of profils in order to help non technicians to use the workaround . I am an OVH storage user and since their outage in strasbourg i can't access my storage using cyberduck due to the bug - if someone can share a sample profil in order to help openstack swift ovh user to get connection back it would be great... if it can comes with an HOW-TO it would be better (what and where place tenant ID, Region,etc...
Thanks for any help you can provide

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Mar 14, 2021

@dkocher commented

Replying to [comment:21 keben]:

Replying to [comment:16 dkocher]:

Reopen awaiting feedback from testing. would it be possible to give a sample XML of profils in order to help non technicians to use the workaround . I am an OVH storage user and since their outage in strasbourg i can't access my storage using cyberduck due to the bug - if someone can share a sample profil in order to help openstack swift ovh user to get connection back it would be great... if it can comes with an HOW-TO it would be better (what and where place tenant ID, Region,etc...
Thanks for any help you can provide

See #11612.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement fixed s3 version:6.3.4
Projects
None yet
Development

No branches or pull requests

2 participants