mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #52317

aaSharma14 · 2023-07-05T14:17:34Z

Add Daemons, Zoning, Buckets, Users, Used capacity(capacity used by all the pools in the cluster), Avg object Size(yet to be implemented) cards to the rgw overview dashboard.

Fixes: https://tracker.ceph.com/issues/62258

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

nizamial09

Thanks @aaSharma14, I haven't tested it locally. Will do it and give more reviews here

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss

...rd/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.spec.ts

src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html

src/pybind/mgr/dashboard/services/cluster.py

github-actions · 2023-07-12T08:55:12Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

nizamial09

few nit picks

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.html

src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html

nizamial09 · 2023-07-14T06:02:28Z

src/pybind/mgr/dashboard/services/cluster.py

+            total_objects = total_objects + pool['stats']['objects']
+            total_pool_bytes_used = total_pool_bytes_used + pool['stats']['bytes_used']


we should review if this is correct with rgw folks.

Yeah I will confirm this with them

nizamial09 · 2023-07-14T08:15:41Z

unit-test failure

[lint:html    ] 
[lint:html    ] [htmllint] found 0 errors out of 194 files
[lint:html    ] 
[lint:html    ] ERROR in /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
[lint:html    ] 
[lint:html    ] src/app/core/navigation/navigation/navigation.component.html
[lint:html    ]   274:15  Attributes should vertically align
[lint:html    ] 
[lint:html    ] 
[lint:eslint  ] 
[lint:eslint  ] Linting "ceph-dashboard"...
[lint:eslint  ] 
[lint:eslint  ] All files pass linting.
[lint:eslint  ]

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss

cloudbehl · 2023-07-14T12:10:29Z

Check if Avg object size is available from backend. If not we can remove the till or comment it for now.

src/pybind/mgr/dashboard/services/cluster.py

avanthakkar

LGTM!

epuertat

Given the new landing page (v3) no longer uses cards in this way, why not adopting that approach for depicting this info too:

Inventory card: could show the count and status of all the RGW items:
- Daemons
  - Realms
  - Zonegroups
  - Zones
- Users
- Buckets
- Objects
Capacity: a pie-chart like the one used for the main landing page Capacity.

The only piece of info missing from this proposal would be the Avg. Object Size, but my question here is: how's this info actionable? Are there any recommendations about an ideal avg. object size? If there's some threshold/s under/beyond what the avg. object size should be kept, then we could use alerts/health check warnings for that purpose?

epuertat · 2023-07-17T09:53:44Z

...shboard/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.ts

+    private rgwBucketService: RgwBucketService,
+    private rgwUserService: RgwUserService
+  ) {
+    this.permissions = this.authStorageService.getPermissions();


If I recall it correctly, @nSedrickm created long time ago an Angular directive to avoid having to deal directly with the Permissions (it'd be desirable to have a similar thing for feature toggles, or reuse the cdScope, given both authorization/RBAC and feature-toggles rely on mapping HTML/DOM to specific features/scope), which allowed to simply tag HTML elements with specific permissions/feature toggles.

<div cdScope='"rgw"'> ...

@aaSharma14 can you add *cdScope to the first div. Then we can remove the permissions thing.

aaSharma14 · 2023-07-18T06:29:37Z

Given the new landing page (v3) no longer uses cards in this way, why not adopting that approach for depicting this info too:
* **Inventory card**: could show the count and status of all the RGW items:
  
  * Daemons
    
    * Realms
    * Zonegroups
    * Zones
  * Users
  * Buckets
  * Objects

* **Capacity**: a pie-chart like the one used for the main landing page Capacity.
The only piece of info missing from this proposal would be the Avg. Object Size, but my question here is: how's this info actionable? Are there any recommendations about an ideal avg. object size? If there's some threshold/s under/beyond what the avg. object size should be kept, then we could use alerts/health check warnings for that purpose?

Thanks @epuertat , We initially went with a design similar to the dashboard v3 landing page , a sample design for that approach was something like this -

The issues that we identified with this approach were -

As RGW doesn’t have and overall status, so we can’t have status panel.
What details should be shown in the details card was not clear.
Since we just wanted to show the used capacity by rgw and am not sure if we have any Total Capacity in this case, so a pie chart was not fitting here as well
For the inventory card most of the things that we are going to show such as - Realms,Zonegroups,Zones,Users, Buckets, Objects do not have any status property so we could just show their number and the status badge along with it didn't make any sense.

We also have this other PR - #52405, which adds some graphs to the overview page, the overall view looks like this for now -

The space left below is to show the multi-site sync status, for which we are planning to use a design similar to this -

cloudbehl · 2023-07-18T12:00:23Z

@epuertat I agree we thought of going with the inventory approach and discussed it with the UX team as well. Things were not fitting well other details we wanted for us with inventory card. Also we had timing constraints we have to choose for the approach and start the implementation. As lot of the data from the cluster dashboard is not present in RGW dashboard. We went with this for now and maybe based on feedback we can adapt. WDYT?

epuertat · 2023-07-18T15:18:33Z

Thanks @epuertat , We initially went with a design similar to the dashboard v3 landing page , a sample design for that approach was something like this -

That's great! I didn't know about this design.

The issues that we identified with this approach were -

As RGW doesn’t have and overall status, so we can’t have status panel.

First of all, there's no need to replicate exactly the Landing Page Dashboard. We can simply pick the widgets that are required.

That said, even if there's no "native" concept of overall status in RGW, we can infer that. What does it mean that everything is OK? E.g.: All services up & running. The idea of having a single indicator of a service status is to provide a single place at which operators should pay attention.

What details should be shown in the details card was not clear.

Same: we can remove if if not required.

Since we just wanted to show the used capacity by rgw and am not sure if we have any Total Capacity in this case, so a pie chart was not fitting here as well

You can still display:

total available storage,
storage used by RGW,
storage used by other interfaces.

For the inventory card most of the things that we are going to show such as - Realms,Zonegroups,Zones,Users, Buckets, Objects do not have any status property so we could just show their number and the status badge along with it didn't make any sense.

It doesn't matter whether they provide "status", what we want to depict is whether everything is ok or not: are all daemons up? are all realms/zonegroups/zones up? ...

We also have this other PR - #52405, which adds some graphs to the overview page, the overall view looks like this for now -

I already reviewed this one too, and my suggestion is the same: to deliver the existing chart widget.

epuertat · 2023-07-18T15:20:49Z

@epuertat I agree we thought of going with the inventory approach and discussed it with the UX team as well. Things were not fitting well other details we wanted for us with inventory card. Also we had timing constraints we have to choose for the approach and start the implementation. As lot of the data from the cluster dashboard is not present in RGW dashboard. We went with this for now and maybe based on feedback we can adapt. WDYT?

As I replied to @aaSharma14 there would be no need to replicate the same exact dashboard, but if the information can fit in an existing widget (like the inventory), I think it's good to leverage that. In any case, this can be implemented incrementally.

aaSharma14 · 2023-07-19T05:31:35Z

jenkins test make check

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

aaSharma14 · 2023-07-21T05:28:58Z

jenkins test make check

aaSharma14 · 2023-07-21T12:03:19Z

jenkins test dashboard cephadm

cloudbehl · 2023-07-25T06:32:37Z

Few things:

Inventory:

Daemons -> Gateways
if we have 0 objects -> it row gets hidden from inventory

aaSharma14 · 2023-07-25T06:41:23Z

Few things:

Inventory:

1. Daemons -> Gateways

2. if we have 0 objects -> it row gets hidden from inventory

Thanks @cloudbehl , changed Daemons -> Gateways. We are not hiding the object count if its 0..we are hiding it if it is null

cloudbehl · 2023-07-25T06:55:50Z

Few things:
Inventory:
1. Daemons -> Gateways

2. if we have 0 objects -> it row gets hidden from inventory
Thanks @cloudbehl , changed Daemons -> Gateways. We are not hiding the object count if its 0..we are hiding it if it is null

If its null we can assume its 0. As there are not objects that's why it returns null. We should not hide it

nizamial09

The layout is broken for me because of the improper css hacks! As per the design I can see that there's a row and 3 columns, so why not divide the layout like that in html?

I tried to fix it. Maybe you can make it more optimizes

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.html

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss

nizamial09 · 2023-07-25T11:49:42Z

...shboard/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.ts

+    private rgwBucketService: RgwBucketService,
+    private rgwUserService: RgwUserService
+  ) {
+    this.permissions = this.authStorageService.getPermissions();


@aaSharma14 can you add *cdScope to the first div. Then we can remove the permissions thing.

epuertat · 2023-07-25T17:37:53Z

The layout is broken for me because of the improper css hacks! As per the design I can see that there's a row and 3 columns, so why not divide the layout like that in html?

I tried to fix it. Maybe you can make it more optimizes

What CSS grid/layout are we using there? It looked like we were manually placing components 🤔 ?

overview dashboard Signed-off-by: Aashish Sharma <aasharma@redhat.com>

aaSharma14 · 2023-07-27T08:51:41Z

jenkins test dashboard

aaSharma14 · 2023-07-27T08:51:50Z

jenkins test windows

aaSharma14 · 2023-07-28T09:04:53Z

jenkins test dashboard

nizamial09 · 2023-07-28T13:08:01Z

The layout is broken for me because of the improper css hacks! As per the design I can see that there's a row and 3 columns, so why not divide the layout like that in html?
I tried to fix it. Maybe you can make it more optimizes

What CSS grid/layout are we using there? It looked like we were manually placing components thinking ?

now its using row-col. 1 row 3 col

nizamial09 · 2023-07-28T13:09:13Z

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss

+.pb-5 {
+  padding-bottom: 3.5rem !important;
+}


is this necessary? this could break the layout in some views. You can test it out by resizing your window.

Yes, this is necessary..tested with resizing the window..works fine

nizamial09

mostly looks good! just minor comment on layout!

aaSharma14 added the dashboard label Jul 5, 2023

aaSharma14 requested review from cloudbehl, avanthakkar, nizamial09 and Pegonzal July 5, 2023 14:17

aaSharma14 requested a review from a team as a code owner July 5, 2023 14:17

github-actions bot added the pybind label Jul 5, 2023

nizamial09 reviewed Jul 7, 2023

View reviewed changes

github-actions bot added the needs-rebase label Jul 12, 2023

aaSharma14 force-pushed the rgw-overview-dashboard branch 2 times, most recently from b5a7436 to 427d800 Compare July 13, 2023 08:16

github-actions bot removed the needs-rebase label Jul 13, 2023

nizamial09 reviewed Jul 14, 2023

View reviewed changes

aaSharma14 force-pushed the rgw-overview-dashboard branch from 427d800 to d93e4b2 Compare July 14, 2023 07:12

aaSharma14 requested a review from nizamial09 July 14, 2023 07:13

cloudbehl reviewed Jul 14, 2023

View reviewed changes

...board/frontend/src/app/ceph/rgw/rgw-overview-dashboard/rgw-overview-dashboard.component.scss Outdated Show resolved Hide resolved

aaSharma14 force-pushed the rgw-overview-dashboard branch 2 times, most recently from 8937b65 to f2ad547 Compare July 17, 2023 07:46

avanthakkar reviewed Jul 17, 2023

View reviewed changes

src/pybind/mgr/dashboard/services/cluster.py Outdated Show resolved Hide resolved

src/pybind/mgr/dashboard/services/cluster.py Outdated Show resolved Hide resolved

aaSharma14 force-pushed the rgw-overview-dashboard branch 2 times, most recently from 1884beb to 3248576 Compare July 17, 2023 08:54

avanthakkar approved these changes Jul 17, 2023

View reviewed changes

epuertat reviewed Jul 17, 2023

View reviewed changes

mgr/dashboard: add single stat cards to rgw overview dashboard

a685dde

Signed-off-by: Aashish Sharma <aasharma@redhat.com>

aaSharma14 force-pushed the rgw-overview-dashboard branch from 3248576 to e3ed882 Compare July 20, 2023 07:03

aaSharma14 changed the title ~~mgr/dashboard: add single stat cards to rgw overview dashboard~~ mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard Jul 20, 2023

aaSharma14 force-pushed the rgw-overview-dashboard branch from e3ed882 to 224746c Compare July 21, 2023 08:43

cloudbehl approved these changes Jul 25, 2023

View reviewed changes

aaSharma14 force-pushed the rgw-overview-dashboard branch from 224746c to 803602d Compare July 25, 2023 06:40

nizamial09 reviewed Jul 25, 2023

View reviewed changes

aaSharma14 force-pushed the rgw-overview-dashboard branch from 9f7112f to 0b84f31 Compare July 25, 2023 12:57

aaSharma14 requested a review from nizamial09 July 25, 2023 12:57

mgr/dashboard: Add inventory card and two single stat panels to rgw

6a21d60

overview dashboard Signed-off-by: Aashish Sharma <aasharma@redhat.com>

aaSharma14 force-pushed the rgw-overview-dashboard branch from 0b84f31 to 6a21d60 Compare July 27, 2023 06:02

nizamial09 reviewed Jul 28, 2023

View reviewed changes

nizamial09 approved these changes Jul 28, 2023

View reviewed changes

aaSharma14 merged commit d32b6c2 into ceph:main Jul 31, 2023
10 of 13 checks passed

aaSharma14 deleted the rgw-overview-dashboard branch July 31, 2023 05:36

aaSharma14 mentioned this pull request Aug 10, 2023

reef: mgr/dashboard: Dashboard RGW multisite configuration #52922

Merged

This was referenced Aug 21, 2023

reef: Rgw overview dashboard backport #53065

Merged

reef: mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #53067

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #52317

mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #52317

aaSharma14 commented Jul 5, 2023 •

edited

nizamial09 left a comment

github-actions bot commented Jul 12, 2023

nizamial09 left a comment

nizamial09 Jul 14, 2023

aaSharma14 Jul 14, 2023

nizamial09 commented Jul 14, 2023

cloudbehl commented Jul 14, 2023

avanthakkar left a comment

epuertat left a comment •

edited

epuertat Jul 17, 2023

nizamial09 Jul 25, 2023

aaSharma14 commented Jul 18, 2023

cloudbehl commented Jul 18, 2023

epuertat commented Jul 18, 2023

epuertat commented Jul 18, 2023

aaSharma14 commented Jul 19, 2023

aaSharma14 commented Jul 21, 2023

aaSharma14 commented Jul 21, 2023

cloudbehl commented Jul 25, 2023

aaSharma14 commented Jul 25, 2023

cloudbehl commented Jul 25, 2023

nizamial09 left a comment

nizamial09 Jul 25, 2023

epuertat commented Jul 25, 2023

aaSharma14 commented Jul 27, 2023

aaSharma14 commented Jul 27, 2023

aaSharma14 commented Jul 28, 2023

nizamial09 commented Jul 28, 2023

nizamial09 Jul 28, 2023

aaSharma14 Jul 31, 2023

nizamial09 left a comment

		total_objects = total_objects + pool['stats']['objects']
		total_pool_bytes_used = total_pool_bytes_used + pool['stats']['bytes_used']

mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #52317

mgr/dashboard: add inventory card and single stat cards to rgw overview dashboard #52317

Conversation

aaSharma14 commented Jul 5, 2023 • edited

Contribution Guidelines

Checklist

nizamial09 left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 12, 2023

nizamial09 left a comment

Choose a reason for hiding this comment

nizamial09 Jul 14, 2023

Choose a reason for hiding this comment

aaSharma14 Jul 14, 2023

Choose a reason for hiding this comment

nizamial09 commented Jul 14, 2023

cloudbehl commented Jul 14, 2023

avanthakkar left a comment

Choose a reason for hiding this comment

epuertat left a comment • edited

Choose a reason for hiding this comment

epuertat Jul 17, 2023

Choose a reason for hiding this comment

nizamial09 Jul 25, 2023

Choose a reason for hiding this comment

aaSharma14 commented Jul 18, 2023

cloudbehl commented Jul 18, 2023

epuertat commented Jul 18, 2023

epuertat commented Jul 18, 2023

aaSharma14 commented Jul 19, 2023

aaSharma14 commented Jul 21, 2023

aaSharma14 commented Jul 21, 2023

cloudbehl commented Jul 25, 2023

aaSharma14 commented Jul 25, 2023

cloudbehl commented Jul 25, 2023

nizamial09 left a comment

Choose a reason for hiding this comment

nizamial09 Jul 25, 2023

Choose a reason for hiding this comment

epuertat commented Jul 25, 2023

aaSharma14 commented Jul 27, 2023

aaSharma14 commented Jul 27, 2023

aaSharma14 commented Jul 28, 2023

nizamial09 commented Jul 28, 2023

nizamial09 Jul 28, 2023

Choose a reason for hiding this comment

aaSharma14 Jul 31, 2023

Choose a reason for hiding this comment

nizamial09 left a comment

Choose a reason for hiding this comment

aaSharma14 commented Jul 5, 2023 •

edited

epuertat left a comment •

edited