Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rgw: check object locks in multi-object delete #37933

Merged
merged 6 commits into from Dec 1, 2020

Conversation

markhoughton-microfocus
Copy link
Contributor

@markhoughton-microfocus markhoughton-microfocus commented Nov 3, 2020

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Add checks on object lock status before allowing an object to be deleted by the S3 multi-object delete API.
Also fixes some issues with verifying permissions when a bucket policy is in place. This part includes a fix which overlaps with #36583 (PR in review at the time of writing).

Fixes https://tracker.ceph.com/issues/47586

@markhoughton-microfocus
Copy link
Contributor Author

Tests for this: ceph/s3-tests#365

@mattbenjamin mattbenjamin self-assigned this Nov 3, 2020
Copy link
Contributor

@mattbenjamin mattbenjamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks very good, have one question for @dang

}
}
}
int object_lock_response = verify_object_lock(this, obj->get_attrs(), false, false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this function is really going to take two booleans as arguments, maybe /* */ comment them where it's called

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was changed in a later commit.

@@ -6392,9 +6397,12 @@ int RGWDeleteMultiObj::verify_permission()
}
}
}

bool empty = rgw::sal::RGWObject::empty(s->object.get()) || s->object->get_instance().empty();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dang is this typical usage?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right. It's technically valid, since empty() checks that the name is empty but not the instance, but it should be valid to have an un-instanced object, and this check will consider that to be an empty object.

If this check is as intended, can we change the name of the variable from "empty" to something more descriptive, since an object with a name but no instance is definitely not empty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing I'm trying to solve here is that while testing, I found that for multi-object delete, s->object.get() was returning null. The other side of the || is just what was there already, so I assume it has a purpose but I'm not very familiar with this code. #36583 has a very similar change, but calls it is_versioned. Would that be more appropriate? (I generally don't like putting in changes that are already in a separate PR, but I needed this part of the fix to test that the core change was working).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, now I'm understanding. Let's not call the variable "empty". "is_versioned" would work, but you'll have to negate it. Maybe "not_versioned" would be simplest?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ah, getting it now); yes, naming this var not_versioned or similar would assist readability

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Changed to not_versioned.

Copy link
Contributor

@mattbenjamin mattbenjamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, includes tests

Copy link
Member

@theanalyst theanalyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, includes fix for https://tracker.ceph.com/issues/46567 / pr #36583 as well

@dang
Copy link
Contributor

dang commented Nov 20, 2020

Needs a rebase

Multi-object delete (via the S3 API) will now check each object's retention date in the same way as single object delet does.

Fixes: http://tracker.ceph.com/issues/47586
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
Allow governance  retention to be overridden by a suitably privileged user.

Fixes: http://tracker.ceph.com/issues/47586
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
…object delete.

fixes: https://tracker.ceph.com/issues/47586
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
Signed-off-by: Mark Houghton <mhoughton@microfocus.com>
@smithfarm
Copy link
Contributor

The following tests FAILED:
	198 - unittest-seastar-messenger (Failed)

@smithfarm
Copy link
Contributor

jenkins test make check

@markhoughton-microfocus
Copy link
Contributor Author

Any idea why it's failing? I haven't got it to run locally yet and the Jenkins logs don't reveal much.

@mattbenjamin
Copy link
Contributor

@markhoughton-microfocus I don't believe a failure in seastar messenger could be traced to this PR

@smithfarm
Copy link
Contributor

@markhoughton-microfocus Some of the "make check" tests suffer from transient failures. I triggered a re-run, and now I see the tests all succeeded.

@markhoughton-microfocus
Copy link
Contributor Author

@smithfarm thank you. In that case, from my side this is done and ready to be re-reviewed following the rebase (a very small change to pass on the new yield parameter).

@markhoughton-microfocus
Copy link
Contributor Author

What are the next steps to get this merged?

@mattbenjamin
Copy link
Contributor

@markhoughton-microfocus There needs to be a successful teuthology run. I think that's the only obstacle to merging this. Sorry for the delay.

@ivancich ivancich added the wip-eric-testing-1 for ivancich testing label Nov 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants