Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interoperability with OpenStack Swift S3 middleware #10471

Closed
cyberduck opened this issue Sep 24, 2018 · 10 comments
Closed

Interoperability with OpenStack Swift S3 middleware #10471

cyberduck opened this issue Sep 24, 2018 · 10 comments

Comments

@cyberduck
Copy link
Collaborator

@cyberduck cyberduck commented Sep 24, 2018

fdfb60c created the issue

Hi, I have recently deployed an openstack cluster (Rocky version) using swift and keystone. I am testing my cluster with ec2 credentials for s3 support. Somehow, I am not able to see the objects inside containers, while using cyberduck, whereas, If I use other clients e.g. s3 browser etc., I am able to see all those objects. Cyberduck lists all containers and folder(s) inside those, but not objects.

Another thing, which I have noticed, If I connect via keystone using cyberduck, everything works great.

I have tried looking into code of s3api, and it seems to be returning correct object. I am not sure, why object listing is not happening with cyberduck client only....

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

fdfb60c commented

I was exploring more on this and noticed this in debug log

2018-09-25 12:23:54,073 [background-14] DEBUG org.jets3t.service.impl.rest.XmlResponsesSaxParser - **Parsing XML response document with handler: class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListBucketHandler**

if the function above gets called, it lists objects otherwise not.

It so happened, that cyberduck listed objects only once, and, I was able to capture it in debug logs, when I refreshed, objects were not getting listed again

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

@dkocher commented

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

fdfb60c commented

Replying to [comment:2 dkocher]:

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Hi David, I am connecting to openstack cluster swift proxy URL. I have created ec2 credentials of user and trying to connect via those. I am able to login and could see containers list, but when I try to go inside container and see objects, it doesn't list anything. Interestingly, if folders are present, those get displayed. Below is the sample response from s3api ( openstack swift middleware) for container's object and folders.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>testing-swapi-1</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>Tuning Your Swift Installation _ Software tuning.pdf</Key>
        <LastModified>2018-09-22T08:52:57.994Z</LastModified>
        <ETag>"0fa07e197c81907fdd9a20720ab82e51"</ETag>
        <Size>205721</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>cosbench_icpe13.pdf</Key>
        <LastModified>2018-09-22T07:25:37.525Z</LastModified>
        <ETag>"0b2d57b34ef2120cfb5bbf1a827af292"</ETag>
        <Size>761002</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>workload-config.xml</Key>
        <LastModified>2018-09-24T07:37:25.246Z</LastModified>
        <ETag>"f0581f6217d303d0504e96aa64c6f1c7"</ETag>
        <Size>7778</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <CommonPrefixes>
        <Prefix>folder1/</Prefix>
    </CommonPrefixes>
</ListBucketResult>

I do apologize for not formatting the contents, as I am very new to this.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

fdfb60c commented

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

@dkocher commented

Replying to [comment:4 pawan]:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

fdfb60c commented

Replying to [comment:5 dkocher]:

Replying to [comment:4 pawan]:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.
I have sent the same to given email address from my email address.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 25, 2018

@dkocher commented

The reply does not include the Version element we expect as we request the versions subresource in the GET request.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Sep 28, 2018

fdfb60c commented

as per this URL https://docs.openstack.org/swift/latest/s3_compat.html, openstack s3 api doesn't support versioning. It would be great, cyberduck could provide an option and it starts working without versions subresource in GET request.
Also, I would appreciate, if I would have an idea about timelines for this issue.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Oct 1, 2018

fdfb60c commented

Replying to [comment:10 dkocher]:
Thanks for updating the issue and setting the milestone.

Loading

@cyberduck
Copy link
Collaborator Author

@cyberduck cyberduck commented Oct 1, 2018

@dkocher commented

In 7326a14.

Loading

@cyberduck cyberduck closed this Oct 1, 2018
@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants