Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

quincy: rgw multisite: replicate metadata for iam roles #48030

Merged
merged 55 commits into from
Feb 21, 2023
Merged

quincy: rgw multisite: replicate metadata for iam roles #48030

merged 55 commits into from
Feb 21, 2023

Conversation

pritha-srivastava
Copy link
Contributor

backport tracker: https://tracker.ceph.com/issues/57364

backport of #43597
parent tracker: https://tracker.ceph.com/issues/51068

this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/master/src/script/ceph-backport.sh

theanalyst and others added 30 commits September 1, 2022 13:17
@theanalyst @pritha-srivastava
rgw: svc: introduce a service for rgw_role
608da8c 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 1584fe5)
@theanalyst @pritha-srivastava
rgw: introduce RGW Role Rados service
87d24ac 
This handles the metadata for RGW Roles with the RADOS backend

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 3fe4307)
@theanalyst @pritha-srivastava
rgw: RGWRole: add a missing RGWRados forward declaration
ff4c2fc 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit cff18a2)
@theanalyst @pritha-srivastava
svc: role: fix real_time put param
4d3550f 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4f8b4b5)
@theanalyst @pritha-srivastava
svc: role: implement get/put/delete Roles
f72052b 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 842f74a)
@theanalyst @pritha-srivastava
svc: role: implement read/write role path & name
aff3cdb 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 00dbe32)
@theanalyst @pritha-srivastava
rgw: implement delete oids
491c608 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit cd69cfc)
@theanalyst @pritha-srivastava
rgw: split RGWRole into an Info Class containing members
a43e8c3 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0cabcf6)
@theanalyst @pritha-srivastava
rgw: role: basic ctl class defn
a777a76 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 2335a4b)
@theanalyst @pritha-srivastava
rgw_role: define RGWRoleCtl
03405b3 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 150f78b)
@theanalyst @pritha-srivastava
rgw: svc: role_rados: stub out get_path
6465d2f 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0569b7d)
@theanalyst @pritha-srivastava
rgw: start using role_ctl
9a9914e 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 7a2795f)
@theanalyst @pritha-srivastava
svc: introduce role_rados service
343e538 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 3c7d027)
@theanalyst @pritha-srivastava
rgw: base role md handler decl
e6a683d 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit e2b8006)
@theanalyst @pritha-srivastava
rgw: role: modify store_name & path
5a11914 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit f8189c6)
@theanalyst @pritha-srivastava
rgw: finish role class impl
f935cf7 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit ccebb5c)
@theanalyst @pritha-srivastava
rgw: role: drop rados_ctl and use only role_ctl
fae7c10 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit e41629f)
@theanalyst @pritha-srivastava
rgw: metadata: provide a constructor for RGWMetadataObject
a42647b 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 0ca6756)
@theanalyst @pritha-srivastava
rgw: role: add RGWRoleMetadataObject
ca87257 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 90b6642)
@theanalyst @pritha-srivastava
rgw: role: define md handler for roles
c3982c6 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit d1fd01c)
@theanalyst @pritha-srivastava
rgw: role: md handler get_meta_obj defn
a34b3ca 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 59b072a)
@theanalyst @pritha-srivastava
rgw: role: backend handler definition
2cd67ca 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit ceb7cf9)
@theanalyst @pritha-srivastava
rgw: role fixes
b4d3393 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit feb4754)
@theanalyst @pritha-srivastava
svc: role: move prefix to Module
25e943b 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 83e30f6)
@theanalyst @pritha-srivastava
rgw: role: use ot
05a4a09 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 7fb221c)
@theanalyst @pritha-srivastava
rgw: role: json fixes
a8ebb22 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 93375dd)
@pritha-srivastava
svc: role: introduce create role
4a1f743 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 6e46243)
@theanalyst @pritha-srivastava
rgw: role: use create_role from svc
7c99dea 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 932807b)
@theanalyst @pritha-srivastava
rgw: role: move populate from svc->role
8d0509a 
As populate creates uuids etc which are only needed in direct calls to rgw_admin

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 5f9f977)
@theanalyst @pritha-srivastava
svc: roles: set objv_tracker to null for sysobj calls
3fc36c9 
Since these aren't tracked via MDLog and merely ancillary objects

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit a92752b)
theanalyst and others added 21 commits September 6, 2022 09:24
@theanalyst @pritha-srivastava
svc: role: introduce list roles by path
af8311b 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit e078dea)
@theanalyst @pritha-srivastava
svc: role: make string consts static
f7df0fa 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 35c0e13)
@theanalyst @pritha-srivastava
rgw: role: use the list roles from role svc classes
d334051 
This keeps the implementation of storage seperate from the RGWRole class. Also
drop the get_roles_by_path_prefix static function as this is currently
implemented in the backend and can be directly accessed from role ctl classes.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 846a0e4)
@theanalyst @pritha-srivastava
rgw: role/svc cosmetic comments & ws cleanup
0f3bd9c 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit d8c2a89)
@theanalyst @pritha-srivastava
rgw: tools: introduce rgw_stat_system_obj
3fd2de8 
Similar to the read and write system_obj interfaces this function stats an
object and returns the mtime & attrs if set.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 8f4e00c)
@theanalyst @pritha-srivastava
svc: role: use the stat system obj call for checking presence
7fdda5c 
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4aa53ff)
@pritha-srivastava
rgw: role metadata handler invokes RGWRole interfaces.
a954e1f 
Removing references to RGW services and RGW Ctl.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 04913eb)
@pritha-srivastava
rgw: removing do_start() method from
6b0577b 
RGWRoleMetadataHandler and class RGWSI_Role_Module.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 3f574a3)
@pritha-srivastava
rgw: code for handling mtime, object tracker and attrs
bab45d2 
needed for RGWRoleMetadataHandler.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit a100026)
@pritha-srivastava
rgw: cleanup.
1c93903 
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 510db31)
@pritha-srivastava
rgw: using put() method of metadata backend to ensure that role
c1e15ee 
metadata is written to md log in method store_info().

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 36169ca)
@pritha-srivastava
rgw: shifting back be_handler and module initialization to svc role
020054b 
rados.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit fb1f670)
@pritha-srivastava
rgw: moving data members of RGWRole into RGWRoleInfo.
5ef354a 
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 520e4ff)
@pritha-srivastava
rgw: adding a metadata section for roles.
680ff24 
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 9549d2c)
@pritha-srivastava
rgw: oids must not be prefixed in case of read_info
bd2e15e 
and store_info since that gets taken care of the meta_be->put()
and meta_be->get().

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit f272a57)
@pritha-srivastava
rgw: removing svc_role.h and svc_role.cc, and
095a7f1 
references to RGWSI_Role and cleaning up code.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit e57e8da)
@pritha-srivastava
rgw: adding roles to non master ops list.
7613470 
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit a4caa0b)
@pritha-srivastava
rgw: code changes to include tenant in replicated metadata
a214864 
for roles.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 0d96b29)
@pritha-srivastava
rgw: implementation of forward_request_to_master()
f9e8443 
for role REST APIs.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit c129eda)
@pritha-srivastava
rgw: code for correctly decoding permission policy and tags
6f48179 
in a JSON object for a role.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 4bae137)
@pritha-srivastava
rgw: adding a test to check if role metadata syncs fine.
7e7fb04 
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit 7140e82)
@pritha-srivastava pritha-srivastava added this to the quincy milestone Sep 9, 2022
mattbenjamin
mattbenjamin approved these changes Jan 10, 2023
View reviewed changes
Copy link
Contributor

@mattbenjamin mattbenjamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, and validated (on 5.3) downstream

@yuriw yuriw merged commit d2832d3 into ceph:quincy Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattbenjamin mattbenjamin mattbenjamin approved these changes

Assignees
No one assigned
Labels
build/ops needs-qa rgw tests wip-yuri5-testing
Projects
None yet
Milestone
quincy
Development

Successfully merging this pull request may close these issues.
5 participants
@pritha-srivastava @mattbenjamin @cbodley @yuriw @theanalyst