New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw/multisite: validate bucket location during bucket creation #15333

Merged
merged 3 commits into from Jun 19, 2017

Conversation

Projects
None yet
2 participants
@mikulely
Contributor

mikulely commented May 27, 2017

No description provided.

std::string current_period_id = store->get_current_period_id();
if (!current_period_id.empty()) {
RGWPeriod current_period(current_period_id);
auto zonegroups_api = current_period.get_map().zonegroups_by_api;

This comment has been minimized.

@cbodley

cbodley May 30, 2017

Contributor

for this to work, the period object would have to be read from rados with RGWPeriod::init(). but this period is already stored in RGWRados::current_period, so we should use that instead. it's private, so we'd need to add a helper function like get_zonegroup_by_api() (similar to RGWRados::find_zone_id_by_name())

This comment has been minimized.

@mikulely

mikulely May 31, 2017

Contributor

Reasonable, we can use RGWRados::current_period ,repushed .

@cbodley cbodley added the rgw label May 30, 2017

@cbodley cbodley self-assigned this May 31, 2017

@mikulely

This comment has been minimized.

Contributor

mikulely commented Jun 1, 2017

Jenkins retest this please

}
return ret;
}

This comment has been minimized.

@cbodley

cbodley Jun 1, 2017

Contributor

thanks. can we find a way to avoid copying the whole map here? it's especially expensive because each RGWZoneGroup itself has a few lists/maps. there are a few options, but the simplest is probably to narrow the interface to just search for a given api:

bool validate_zonegroup_api(const std::string& api) const {

what do you think?

This comment has been minimized.

@mikulely

mikulely Jun 5, 2017

Contributor

reasonable, but here we are only checking the existence of location_constraint , then checking whether location_constraint match the region(is_zonegroup_api_exist). Maybe is_zonegroup_api_exist is more accurate.

repushed.

bool is_zonegroup_api_exist(const std::string& api) const {
if (!current_period.get_id().empty()) {
auto zonegroups_by_api = current_period.get_map().zonegroups_by_api;

This comment has been minimized.

@cbodley

cbodley Jun 5, 2017

Contributor

careful with auto, this will still make a copy. try const auto&

re: function name, maybe has_zonegroup_api or does_zonegroup_api_exist?

This comment has been minimized.

@mikulely

mikulely Jun 6, 2017

Contributor

Thanks! repushed.

@cbodley

cbodley approved these changes Jun 6, 2017

looks good, thanks. verified with s3cmd against a multisite configuration with two zonegroups 'na' and 'sa':

$ s3cmd -c c1.s3cfg --bucket-location=na mb s3://bucket1
Bucket 's3://bucket1/' created
$ s3cmd -c c1.s3cfg --bucket-location=sa mb s3://bucket2
Bucket 's3://bucket2/' created
$ s3cmd -c c1.s3cfg --bucket-location=qa mb s3://bucket3
ERROR: S3 error: 400 (InvalidLocationConstraint): The specified location-constraint is not valid

@cbodley

This comment has been minimized.

Contributor

cbodley commented Jun 16, 2017

@mikulely please rebase:

<<<<<<< HEAD
#define ERR_NO_SUCH_BUCKET_POLICY  2207
=======
#define ERR_INVALID_LOCATION_CONSTRAINT 2207

>>>>>>> b77dcc4... rgw: fix bucket location s3 compatible issue

mikulely added some commits May 25, 2017

rgw: fix bucket location s3 compatible issue
create bucket with location constraint doesn't exist in AWS S3
will response like the following:

{'data': '<?xml version="1.0" encoding="UTF-8"?>\n
<Error><Code>InvalidLocationConstraint</Code>
<Message>The specified location-constraint is not valid</Message>
<LocationConstraint>ap-southeast-4</LocationConstraint>
<RequestId>15D886DFEF49765A</RequestId>
<HostId>IroSlmBZq7iSluYIgea7y+72OXbOSmjX2eVYING0tUuGR8BAHLcYPOgUWUIJUuG1z/1FhfaxJUo=</HostId></Error>',
 'headers': {'connection': 'close',
             'content-type': 'application/xml',
             'date': 'Thu, 25 May 2017 03:45:49 GMT',
             'server': 'AmazonS3',
             'transfer-encoding': 'chunked',
             'x-amz-id-2': 'IroSlmBZq7iSluYIgea7y+72OXbOSmjX2eVYING0tUuGR8BAHLcYPOgUWUIJUuG1z/1FhfaxJUo=',
             'x-amz-request-id': '15D886DFEF49765A'},
 'reason': 'Bad Request',
 'status': 400}

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
rgw/multisite: unify zg is_master judge
Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
rgw/multisite: check location constraint existness
to match the behavior of AWS S3

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
@mikulely

This comment has been minimized.

Contributor

mikulely commented Jun 19, 2017

@cbodley rebased!

@cbodley

This comment has been minimized.

@cbodley cbodley merged commit ac3c6c5 into ceph:master Jun 19, 2017

4 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
arm64 make check arm64 make check succeeded
Details
make check make check succeeded
Details

@mikulely mikulely deleted the mikulely:validate-bucket-location branch Jun 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment