mgr/cephadm: automatically configure dashboard <-> RGW connection #41590

liewegas · 2021-05-29T14:24:07Z

Automatically configure the credential(s) for dashboard to talk to RGW.

We do not provide any endpoint information--we assume that dashboard can figure that out on its own.

Note that this path is triggered any time we add or remove an rgw daemon, which isn't quite right--what we really want is to trigger this change when teh first or last rgw daemon is deployed, and/or when the realm configuration is modified. Once a mgr/rgw module exists, we should probably migrate this code there.

configure credentials per-realm if multisite is enabled
trigger this update on upgrade

Fixes: https://tracker.ceph.com/issues/51302

src/pybind/mgr/cephadm/module.py

src/pybind/mgr/cephadm/services/cephadmservice.py

src/pybind/mgr/cephadm/serve.py

sebastian-philipp · 2021-06-07T14:08:13Z

src/pybind/mgr/cephadm/serve.py

+                if self.mgr.need_connect_dashboard_rgw and self.mgr.config_dashboard:
+                    self.mgr.need_connect_dashboard_rgw = False


This is not persistent, which means there is a race between MGR failover and making this connection. Feeling slightly uneasy, but I guess we can fix this later, if it turns out to be a problem

src/pybind/mgr/cephadm/services/cephadmservice.py

epuertat · 2021-06-07T15:19:40Z

src/pybind/mgr/cephadm/services/cephadmservice.py

+        def update_dashboard(what: str, value: str) -> None:
+            _, out, _ = self.mgr.check_mon_command({'prefix': f'dashboard get-{what}'})
+            if out.strip() != value:
+                if what.endswith('-key'):
+                    self.mgr.check_mon_command(
+                        {'prefix': f'dashboard set-{what}'},
+                        inbuf=value
+                    )
+                else:
+                    self.mgr.check_mon_command({'prefix': f'dashboard set-{what}',
+                                                "value": value})
+                self.mgr.log.info(f'Updated dashboard {what}')
+
+        completion = self.mgr.list_daemons(daemon_type='rgw')
+        raise_if_exception(completion)
+        daemons = completion.result
+        if not daemons:
+            self.mgr.log.info('No remaining RGW daemons; disconnecting dashboard')
+            self.mgr.check_mon_command({'prefix': 'dashboard reset-rgw-api-host'})
+            self.mgr.check_mon_command({'prefix': 'dashboard reset-rgw-api-port'})
+            self.mgr.check_mon_command({'prefix': 'dashboard reset-rgw-api-scheme'})
+            return


@alfonsomthd you'd be interested in reviewing this (I meant the whole file, not just this section, but my ability to drag & scroll is affected 🙈 ).

@liewegas, as @Daniel-Pivonka pointed out, there are 2 ways of feeding these params:

single RGW daemon deployment: where access key and secret key contain each a single plain value (key)

multi-daemon (zone, zg, realm) deployment: where a dict-like syntax needs to be used for each: api-access-key=$ echo -n "{'<daemon1.id>': '', '<daemon2.id>': '', ...}"

We'd really like to see this simplified, but we couldn't find a better way to locate and configure connection to multiple daemons.

in the multisite case, do you need a separate keypair for each zone, and the daemon is any daemon for that zone? or one per realm? (I thought the user creds were all per-realm...)

@liewegas Currently we need a dict-like syntax per daemon. Form the mgr service map we can see the zone/zonegroup of the daemon but not the realm (and without first creds we cannot consume the admin ops API), so for now the credentials are per-daemon.

#41739

I'm still confused about the limitation, though: what can/can't you see depending on which daemon you connect to? Is the issue that you only see the zonegroup? Or everything in the realm?

@cbodley @yehudasa

@liewegas @alfonsomthd if any, credentials would be per zone and not per gateway. In the stable state all users will be synced from the main/primary zone. During its bootstrap, a zone would not have any users defined (it will only hold credentials to connect to the primary as a client).

@yehudasa Does this mean that the Dashboard dropdown menu for selecting gateways should instead allow a user to select zones (not gateways, not realms) and then connect to the first daemon that matches the <zone_id> ?

@alfonsomthd I'm not sure what would make the most sense, maybe gateway selection can be a secondary and optional/advanced selection?

Yeah it seems like in most cases the important part is to select the realm? It should be rare that a specific zone needs to be selected?

@liewegas A daemon cannot operate on a zone other than its zone, so we need to select a zone.
Regarding credentials, after orch. weekly meeting the agreement is to set credentials on a per-realm basis. In order to do that on Dashboard side (to identify the daemons that run on zones belonging to a certain realm), this is required:
#41739

Automatically configure the dashboard creds to talk to RGW. Fixes: https://tracker.ceph.com/issues/44605 Signed-off-by: Sage Weil <sage@newdream.net>

liewegas · 2021-06-16T14:32:08Z

jenkins test make check

github-actions · 2021-07-02T14:22:23Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

liewegas · 2021-07-08T20:11:57Z

replaced by #42252

liewegas requested a review from a team as a code owner May 29, 2021 14:24

github-actions bot added cephadm pybind labels May 29, 2021

liewegas force-pushed the cephadm-rgw-dashboard branch from 1d82ddb to 8ee1ee2 Compare May 29, 2021 14:24

sebastian-philipp reviewed May 29, 2021

View reviewed changes

src/pybind/mgr/cephadm/module.py Outdated Show resolved Hide resolved

liewegas force-pushed the cephadm-rgw-dashboard branch from 8ee1ee2 to b0bb373 Compare May 29, 2021 18:12

sebastian-philipp reviewed Jun 1, 2021

View reviewed changes

src/pybind/mgr/cephadm/services/cephadmservice.py Show resolved Hide resolved

src/pybind/mgr/cephadm/serve.py Outdated Show resolved Hide resolved

liewegas force-pushed the cephadm-rgw-dashboard branch 2 times, most recently from b235e8a to a69ea64 Compare June 1, 2021 17:22

liewegas added the wip-sage-testing label Jun 1, 2021

liewegas force-pushed the cephadm-rgw-dashboard branch 2 times, most recently from f83023b to 9f7e87f Compare June 2, 2021 23:21

liewegas removed the wip-sage-testing label Jun 3, 2021

liewegas force-pushed the cephadm-rgw-dashboard branch from 9f7e87f to 1e8a4c8 Compare June 3, 2021 14:01

liewegas added the wip-sage-testing label Jun 4, 2021

sebastian-philipp reviewed Jun 7, 2021

View reviewed changes

Daniel-Pivonka reviewed Jun 7, 2021

View reviewed changes

src/pybind/mgr/cephadm/services/cephadmservice.py Show resolved Hide resolved

epuertat reviewed Jun 7, 2021

View reviewed changes

epuertat requested a review from alfonsomthd June 7, 2021 15:21

liewegas removed the wip-sage-testing label Jun 7, 2021

epuertat added the dashboard label Jun 8, 2021

liewegas changed the title ~~mgr/cephadm: automatically configure dashboard <-> mgr connection~~ mgr/cephadm: automatically configure dashboard <-> RGW connection Jun 15, 2021

liewegas force-pushed the cephadm-rgw-dashboard branch from 1e8a4c8 to 3cf1c66 Compare June 15, 2021 16:31

liewegas requested a review from a team as a code owner June 15, 2021 16:31

liewegas requested review from avanthakkar and removed request for a team June 15, 2021 16:31

liewegas force-pushed the cephadm-rgw-dashboard branch from 3cf1c66 to 3d77100 Compare June 15, 2021 16:49

liewegas added the wip-sage3-testing label Jun 15, 2021

mgr/cephadm: automatically configure dashboard <-> mgr connection

310c1b8

Automatically configure the dashboard creds to talk to RGW. Fixes: https://tracker.ceph.com/issues/44605 Signed-off-by: Sage Weil <sage@newdream.net>

liewegas force-pushed the cephadm-rgw-dashboard branch from 3d77100 to 310c1b8 Compare June 15, 2021 19:11

sebastian-philipp added the rgw label Jun 16, 2021

liewegas removed the wip-sage3-testing label Jul 2, 2021

github-actions bot added the needs-rebase label Jul 2, 2021

liewegas closed this Jul 8, 2021

alfonsomthd mentioned this pull request Jul 28, 2021

mgr/dashboard,cephadm: automate setup of RGW credentials for dashboard #42252

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/cephadm: automatically configure dashboard <-> RGW connection #41590

mgr/cephadm: automatically configure dashboard <-> RGW connection #41590

liewegas commented May 29, 2021 •

edited by alfonsomthd

sebastian-philipp Jun 7, 2021

epuertat Jun 7, 2021 •

edited

liewegas Jun 7, 2021

alfonsomthd Jun 7, 2021

liewegas Jun 7, 2021

liewegas Jun 7, 2021

yehudasa Jun 8, 2021

alfonsomthd Jun 9, 2021 •

edited

yehudasa Jun 9, 2021

liewegas Jun 15, 2021

alfonsomthd Jun 15, 2021

liewegas commented Jun 16, 2021

github-actions bot commented Jul 2, 2021

liewegas commented Jul 8, 2021

		if self.mgr.need_connect_dashboard_rgw and self.mgr.config_dashboard:
		self.mgr.need_connect_dashboard_rgw = False

mgr/cephadm: automatically configure dashboard <-> RGW connection #41590

mgr/cephadm: automatically configure dashboard <-> RGW connection #41590

Conversation

liewegas commented May 29, 2021 • edited by alfonsomthd

Choose a reason for hiding this comment

epuertat Jun 7, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alfonsomthd Jun 9, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liewegas commented Jun 16, 2021

github-actions bot commented Jul 2, 2021

liewegas commented Jul 8, 2021

liewegas commented May 29, 2021 •

edited by alfonsomthd

epuertat Jun 7, 2021 •

edited

alfonsomthd Jun 9, 2021 •

edited