Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rgw: add the support for X-History-Location in swift API #15020

Closed
wants to merge 3 commits into from
Closed

rgw: add the support for X-History-Location in swift API #15020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
59e9d87
rgw: add the variable swift_his_location to be as archive bucket
Jing-Scott May 9, 2017
8347bc0
rgw: implement RGWRados::swift_versioning_history
Jing-Scott May 9, 2017
bd226b6
rgw: add the support for X-History-Location in swift API
Jing-Scott May 10, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions doc/radosgw/config-ref.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -872,13 +872,13 @@ Swift Settings
``rgw swift versioning enabled``

:Description: Enables the Object Versioning of OpenStack Object Storage API.
This allows clients to put the ``X-Versions-Location`` attribute
on containers that should be versioned. The attribute specifies
the name of container storing archived versions. It must be owned
by the same user that the versioned container due to access
control verification - ACLs are NOT taken into consideration.
Those containers cannot be versioned by the S3 object versioning
mechanism.
This allows clients to put the ``X-Versions-Location`` or
``X-History-Location`` attribute on containers that should be
versioned. The attribute specifies the name of container storing
archived versions. It must be owned by the same user that the
versioned container due to access control verification - ACLs
are NOT taken into consideration. Those containers cannot be
versioned by the S3 object versioning mechanism.
:Type: Boolean
:Default: ``false``

Expand Down
14 changes: 8 additions & 6 deletions src/rgw/rgw_common.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1180,6 +1180,7 @@ struct RGWBucketInfo

bool swift_versioning;
string swift_ver_location;
string swift_his_location;

map<string, uint32_t> mdsearch_config;

Expand All @@ -1188,7 +1189,7 @@ struct RGWBucketInfo
string new_bucket_instance_id;

void encode(bufferlist& bl) const {
ENCODE_START(19, 4, bl);
ENCODE_START(20, 4, bl);
::encode(bucket, bl);
::encode(owner.id, bl);
::encode(flags, bl);
Expand All @@ -1210,6 +1211,7 @@ struct RGWBucketInfo
::encode(swift_versioning, bl);
if (swift_versioning) {
::encode(swift_ver_location, bl);
::encode(swift_his_location, bl);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To preserve compatibility with older struct versions, we should encode new fields at the end, not in the middle. This is required during e.g. cluster upgrades when many instances of RadosGW, possibly from two different releases, may coexist in a single cluster.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rzarzynski seems to be fixed now?

}
::encode(creation_time, bl);
::encode(mdsearch_config, bl);
Expand All @@ -1218,7 +1220,7 @@ struct RGWBucketInfo
ENCODE_FINISH(bl);
}
void decode(bufferlist::iterator& bl) {
DECODE_START_LEGACY_COMPAT_LEN_32(19, 4, 4, bl);
DECODE_START_LEGACY_COMPAT_LEN_32(20, 4, 4, bl);
::decode(bucket, bl);
if (struct_v >= 2) {
string s;
Expand Down Expand Up @@ -1266,18 +1268,18 @@ struct RGWBucketInfo
}
swift_versioning = false;
swift_ver_location.clear();
if (struct_v >= 16) {
swift_his_location.clear();
if (struct_v >= 18) {
Copy link
Contributor

@rzarzynski rzarzynski Nov 3, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something weird happened here. IIUC swift_versioning and swift_ver_location can be present since struct_v 16th but now we expect it only since 18th. I believe we shouldn't touch this but instead introduce:

     swift_his_location.clear();
     if (struct_v >= 20 && swift_versioning) {
       ::decode(swift_his_location, bl);
     }

just before DECODE_FINISH(bl).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, @rzarzynski !

::decode(swift_versioning, bl);
::decode(mdsearch_config, bl);
if (swift_versioning) {
::decode(swift_ver_location, bl);
::decode(swift_his_location, bl);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field comes with version 20th, not 18th.

}
}
if (struct_v >= 17) {
::decode(creation_time, bl);
}
if (struct_v >= 18) {
::decode(mdsearch_config, bl);
}
if (struct_v >= 19) {
::decode(reshard_status, bl);
::decode(new_bucket_instance_id, bl);
Expand Down
2 changes: 2 additions & 0 deletions src/rgw/rgw_json_enc.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,7 @@ void RGWBucketInfo::dump(Formatter *f) const
}
encode_json("swift_versioning", swift_versioning, f);
encode_json("swift_ver_location", swift_ver_location, f);
encode_json("swift_his_location", swift_his_location, f);
encode_json("index_type", (uint32_t)index_type, f);
encode_json("mdsearch_config", mdsearch_config, f);
encode_json("reshard_status", (int)reshard_status, f);
Expand Down Expand Up @@ -768,6 +769,7 @@ void RGWBucketInfo::decode_json(JSONObj *obj) {
}
JSONDecoder::decode_json("swift_versioning", swift_versioning, obj);
JSONDecoder::decode_json("swift_ver_location", swift_ver_location, obj);
JSONDecoder::decode_json("swift_his_location", swift_his_location, obj);
uint32_t it;
JSONDecoder::decode_json("index_type", it, obj);
index_type = (RGWBucketIndexType)it;
Expand Down
29 changes: 26 additions & 3 deletions src/rgw/rgw_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2512,11 +2512,16 @@ void RGWCreateBucket::execute()
if (swift_ver_location) {
s->bucket_info.swift_ver_location = *swift_ver_location;
s->bucket_info.swift_versioning = (! swift_ver_location->empty());
s->bucket_info.swift_his_location.clear();
} else if (swift_his_location) {
s->bucket_info.swift_his_location = *swift_his_location;
s->bucket_info.swift_versioning = (! swift_his_location->empty());
s->bucket_info.swift_ver_location.clear();
}

op_ret = store->create_bucket(*(s->user), s->bucket, zonegroup_id,
placement_rule, s->bucket_info.swift_ver_location,
pquota_info, attrs,
s->bucket_info.swift_his_location, pquota_info, attrs,
info, pobjv, &ep_objv, creation_time,
pmaster_bucket, pmaster_num_shards, true);
/* continue if EEXIST and create_bucket will fail below. this way we can
Expand Down Expand Up @@ -2594,6 +2599,11 @@ void RGWCreateBucket::execute()
if (swift_ver_location) {
s->bucket_info.swift_ver_location = *swift_ver_location;
s->bucket_info.swift_versioning = (! swift_ver_location->empty());
s->bucket_info.swift_his_location.clear();
} else if(swift_his_location) {
s->bucket_info.swift_his_location = *swift_his_location;
s->bucket_info.swift_versioning = (! swift_his_location->empty());
s->bucket_info.swift_ver_location.clear();
}

/* Web site of Swift API. */
Expand Down Expand Up @@ -3816,6 +3826,11 @@ void RGWPutMetadataBucket::execute()
if (swift_ver_location) {
s->bucket_info.swift_ver_location = *swift_ver_location;
s->bucket_info.swift_versioning = (! swift_ver_location->empty());
s->bucket_info.swift_his_location.clear();
} else if(swift_his_location) {
s->bucket_info.swift_his_location = *swift_his_location;
s->bucket_info.swift_versioning = (! swift_his_location->empty());
s->bucket_info.swift_ver_location.clear();
}

/* Web site of Swift API. */
Expand Down Expand Up @@ -3984,6 +3999,14 @@ void RGWDeleteObj::execute()


if (!s->object.empty()) {
bool his_deleted = false;
/* If obj does not exits, the "delete marker" still
* can be written in swift history location. */
op_ret = store->swift_versioning_history(s, user_quota, bucket_quota, obj, his_deleted);
if (op_ret < 0) {
return;
}

if (need_object_expiration() || multipart_delete) {
/* check if obj exists, read orig attrs */
op_ret = get_obj_attrs(store, s, obj, attrs);
Expand Down Expand Up @@ -4017,7 +4040,7 @@ void RGWDeleteObj::execute()
return;
}

if (!ver_restored) {
if (!ver_restored && !his_deleted) {
/* Swift's versioning mechanism hasn't found any previous version of
* the object that could be restored. This means we should proceed
* with the regular delete path. */
Expand Down Expand Up @@ -5902,7 +5925,7 @@ int RGWBulkUploadOp::handle_dir(const boost::string_ref path)
bucket,
store->get_zonegroup().get_id(),
placement_rule, binfo.swift_ver_location,
pquota_info, attrs,
binfo.swift_his_location, pquota_info, attrs,
out_info, pobjv, &ep_objv, creation_time,
pmaster_bucket, pmaster_num_shards, true);
/* continue if EEXIST and create_bucket will fail below. this way we can
Expand Down
2 changes: 2 additions & 0 deletions src/rgw/rgw_op.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -806,6 +806,7 @@ class RGWCreateBucket : public RGWOp {
bool has_cors;
RGWCORSConfiguration cors_config;
boost::optional<std::string> swift_ver_location;
boost::optional<std::string> swift_his_location;
map<string, buffer::list> attrs;
set<string> rmattr_names;

Expand Down Expand Up @@ -1118,6 +1119,7 @@ class RGWPutMetadataBucket : public RGWOp {
RGWCORSConfiguration cors_config;
string placement_rule;
boost::optional<std::string> swift_ver_location;
boost::optional<std::string> swift_his_location;

public:
RGWPutMetadataBucket()
Expand Down
Loading