Update object storage access

[sebastian-luna-valero]

Changes to meet the requirements for a private object storage space, discussed in #17

[sebastian-luna-valero]

Could @tinaok or @guillaumeeb please test this and let me know your thoughts.

[guillaumeeb]

Thanks @sebastian-luna-valero, appart from my two commands above, I was able to list my already generated access and secret key for AWS S3 interface, and also to retrieve the Swift token.

Is the idea to stop using fedcloudclient package?

[sebastian-luna-valero]

@guillaumeeb you will need to create new S3 credentials to access the new OpenStack project with ID: 57102d3e06b7476088fe4924370ae170

We currently have one Virtual Organization (vo.pangeo.eu) and two OpenStack projects associated with it. fedcloudclient works well when there is only one project associated with a Virtual Organization (i.e. a one to one mapping between the two). We have created this new, separate OpenStack project to allow normal users (i.e. non admin users) of the vo.pangeo.eu VO to work with an object store. The openstack CLI allows to access any project in OpenStack. That's why we need to use it in replacement of the fedcloudclient. This is something that could be implemented in fedcloudclient, but it's kind of a strange use case, and there is limited effort to add this new functionality to the tool.

[sebastian-luna-valero]

By the way, I am discussing with CESNET alternatives. So we might go back to using fedcloudclient if we are able to restrict access to the object store in the initial OpenStack project.

[guillaumeeb]

We have created this new, separate OpenStack project to allow normal users (i.e. non admin users) of the vo.pangeo.eu VO to work with an object store

Right, I didn't get this part. So the current proposal is to have one project for compute resources, and the other for the storage containers, allowing finer grain access control. I can see through Horizon that I have now access to vo.pangeo.eu-swift project, this must be it!

[sebastian-luna-valero]

That's exactly right! :)

[guillaumeeb]

I was able to follow all commands and to use both ZarrSwiftStore and S3 interface with Read/write permission on the new Openstack project.

So this is perfectly fine, but I'd like to have just a sentence explaining that we create containers in a separate project than the default one, and this is why we do it like this.

[tinaok]

hi, can we merge this pull request?

[guillaumeeb]

@sebastian-luna-valero I tried to add your explanation with a bit of my words to this PR. If this is OK, we can then merge and continue working on the ACL aspects.

[sebastian-luna-valero]

Thanks!

[guillaumeeb]

I just saw you proposed to wait in #21 (comment). Do you still want to wait before merging?

[sebastian-luna-valero]

We can merge, it's just that the issue with private buckets hasn't been solved yet. Maybe we should add this warning to the PR until the issue is fixed.

See the current situation with permissions here

[tinaok]

@sebastian-luna-valero good idea for adding the warning!

[guillaumeeb]

I think if we put the really nice table of your comment in the other thread in this PR, this would be fine to merge!

[tinaok]

Let's add the table in the next merge. I merge the pull request now so that we can go on for clivar workshop.