rgw/CloudTransition: Transition objects to cloud endpoint

[soumyakoduri]

This PR contains series of patches for CloudTiering/CloudTransition feature support (Experimental)

Feature Doc:
https://docs.google.com/document/d/1IoeITPCF64A5W-UA-9Y3Vp2oSfz3xVQHu31GTu3u3Ug/edit#heading=h.4o1xyas8s23x

Tier-config Options:

• Tier storage class and the respective remote Cloud endpoint details can be configured via "zonegroup placement add <>" CLI.

• Existing tier-type & tier-config options are used to configure the same. Below are the config options made available -

  {
  "access_key": ,
  "secret": ,
  "endpoint": ,
  "region": "",
  "host_style": <path | virtual>,
  "acls": [ { "type": <id | email | uri>,
  "source_id": <source_id>,
  "dest_id": <dest_id> } ... ],
  "target_path": <target_path>,
  "target_storage_class": ,
  "multipart_sync_threshold": {object_size},
  "multipart_min_part_size": {part_size},
  "retain_object": <true | false>
  }

Object Transition:

1. Check whether the target_placement is tier-storage class
2. If yes, depending on object size, transition object as plain object or in multiparts.

• Few custom rgw xattrs will be added to target object (eg., x-amz-meta-rgwx-source: rgw)

3. Post transition, either delete the original source object (default behavior) or mark the object as CloudTiered and update its head object Manifest with cloud-tier details.

Multisite:
To avoid multiple zones transitioning same object, each lc thread checks whether the target endpoint (cloud) already contains the object to be transitioned by issuing HEAD request and matching its mtime attr.

Test cases:
ceph/s3-tests#380

TODO:

• Consolidate config and other common coroutine code with cloud-sync module.

To be handled in future:

• Send presigned url or read-through for GetObject on cloud transitioned object.
• Restore operation/policies to restore tiered object
• federation between RGW and cloud/AWS

Signed-off-by: Soumya Koduri skoduri@redhat.com

[soumyakoduri]

@yehudasa @mattbenjamin @cbodley @dang .. I have listed possible ways of configuring these cloud tier options (mainly to choose the right granularity - zonegroup/zone level) here - https://docs.google.com/document/d/1IoeITPCF64A5W-UA-9Y3Vp2oSfz3xVQHu31GTu3u3Ug/edit#heading=h.66scb8vgvd90

Please take a look and let me know your suggestions on which one seems fit for this use-case and if any changes/additions are required. Thanks!

[stale]

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

[soumyakoduri]

unstale

[github-actions]

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

[soumyakoduri]

jenkins render docs

[cbodley]

jenkins test dashboard cephadm

[cbodley]

jenkins test dashboard cephadm

[cbodley]

jenkins test windows