mgr/dashboard: RGW page #19512

liuchang0812 · 2017-12-14T09:22:36Z

Signed-off-by: Chang Liu liuchang0812@gmail.com

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 · 2017-12-14T09:24:21Z

List all RGW daemons:

Metadata and Status of one RGW daemon:

mattbenjamin · 2017-12-14T10:27:06Z

@liuchang0812 looks reasonable offhand, thanks very much; note that this area may evolve quickly

liuchang0812 · 2017-12-14T12:20:25Z

@mattbenjamin thank you. as we discussed last CDM, we are going to create a RGW page. actually this is a early version, and a lot of work need to do. I wonder that we could show more information :

RGW-NFS daemons
RGW request statistics: qps/average latency
RGW sync status: data sync status/metadata sync status/sync error
user/bucket informations: i afraid that we need to pythonize RGWRados class

liuchang0812 · 2017-12-15T03:37:22Z

I find that we have already implemented pybind/rgw.

mattbenjamin · 2017-12-15T03:50:02Z

@liuchang0812 for things like user and bucket info--and day2 mgmt operations, the consensus within the RGW team was to delegate to a running instance of the RGW restful admin server, rather than to embed an RGW instance in ceph-mgr via librgw; There are already community bindings to the currently exported methods here: #19164
@liuchang0812 would you be able to come to some regular meetings of the upstream RGW team to discuss ideas and share progress?

cbodley · 2017-12-15T17:23:39Z

user/bucket informations: i afraid that we need to pythonize RGWRados class

this is indeed terrifying :) ideally we would just send s3 requests to a running gateway for this

similarly, we have admin apis that can serve sync status for 3.

liuchang0812 · 2017-12-18T13:42:22Z

@Liuchang0812 would you be able to come to some regular meetings of the upstream RGW team to discuss ideas and share progress?

Of course. I'm very glad to discuss it with you all. My colleague @Leeshine is also working on it. We will prepare a simple slide. Would you mind giving me more information about the meeting? what time is the meeting and how could we join it?

jcsp

This is great, I just have minor suggested changes

jcsp · 2017-12-19T12:29:47Z

src/pybind/mgr/dashboard/base.html

+
+        <li class="{%if path_info.startswith('/rgw')%}active{%endif%}">          
+            <a href="{{ url_prefix }}/rgw">                                                        
+ <i class="fa fa-files-o" aria-hidden="true"></i>


There is no "right" icon for RGW (as far as I know), but what about cloud-download (http://fontawesome.io/icon/cloud-download/) instead of a file-related icon?

jcsp · 2017-12-19T12:32:15Z

src/pybind/mgr/dashboard/rgw.py

+                        'server': server,
+                        'metadata': metadata,
+                        'status': status,
+                        'url': "/rgw/detail/{0}".format(service['id'])


I suggest leaving out the /detail/ part, and just use /rgw for the list and /rgw/<id> for the detail

sorry, John, how could I get this <id> in RGWEndpoint::index function. I tested with index(self, rgw_id=None) and index(self, rgw_id), cherrypy could not dispatch request correctly as expected. we use /osd/perf/<id> in OSDEndpoint, not /osd/<id>. I wonder whether cherrypy could support this url pattern. I 'm looking cherrypy's document.

It's a little awkward, but I think in this case you can prefix a @cherrypy.popargs('rgw_id') to the RGWEndpoint class (http://docs.cherrypy.org/en/latest/advanced.html#restful-style-dispatching), and then the rgw_id argument to index will work.

jcsp · 2017-12-19T12:36:04Z

src/pybind/mgr/dashboard/rgw_detail.html

+        rgw.{rgw_id}
+    </h1>
+</section>
+


Start a section class="content" here, to get proper padding/margins on the main part with the tables.

jcsp · 2017-12-19T12:41:52Z

src/pybind/mgr/dashboard/rgw_detail.html

+                    });
+                });
+
+                if (content_data.rgw_status_list.length() == 0) {


You can use rv-show="rgw_status_list | length" in the markup, to avoid the need to explicitly set a _show variable up here (the length helper is defined in base.html so you can use it anywhere)

jcsp · 2017-12-19T12:43:37Z

src/pybind/mgr/dashboard/rgw_detail.html

+            var post_load = function() {
+                content_data.rgw_metadata_list = [];
+                content_data.rgw_status_list = [];
+                _.each(content_data.rgw_metadata, function(v, k) {


It would also be fine to do this on the python side, so that the javascript just gets the data in the form that it prefers (a list).

These items are currently coming out in arbitrary order, which makes it a bit difficult to scan visually -- it would be nice to sort them alphabetically by key (could also do this server side if passing a list instead of a dict)

If you choose to keep it on the javascript side, you'll also need to make sure that post_load is called after each refresh()

jcsp · 2017-12-19T12:45:33Z

src/pybind/mgr/dashboard/rgw_detail.html

+            var refresh = function() {
+                $.get("{{ url_prefix }}/rgw/rgw_data/" + content_data.rgw_id + "/", function(data) {
+                   content_data = data;
+                   setTimeout(refresh, 3000);


Let's make this period the same as the one on line 16 (probably both 5000)

jcsp · 2017-12-19T12:51:52Z

src/pybind/mgr/dashboard/rgw.py

+
+log = logging.getLogger("dashboard")
+
+class RGWDaemons(RemoteViewCache):


Please leave two blank lines before class definition for PEP8 style

jcsp · 2017-12-19T12:52:03Z

src/pybind/mgr/dashboard/rgw.py

+
+import json
+import re
+import rados


the re and rados imports are unused and can be removed

jcsp · 2017-12-19T12:53:08Z

src/pybind/mgr/dashboard/rgw.py

+                    try:
+                        status = json.loads(status['json'])
+                    except:
+                        status = {}


Let's do a log.warn here to say the service['id'] which had invalid status json

cbodley · 2017-12-19T15:10:59Z

what time is the meeting and how could we join it?

the rgw team does informal 15-20min standup meetings in https://bluejeans.com/3919548994. they happen on Monday at 9:45am EST, and Tues/Thurs/Fri at 11:45am EST (sorry, we need to publish this stuff on ceph.com somewhere). you're all welcome to join as often as you like!

LenzGr · 2017-12-20T11:58:21Z

src/pybind/mgr/dashboard/base.html

+        <li class="{%if path_info.startswith('/rgw')%}active{%endif%}">          
+            <a href="{{ url_prefix }}/rgw">                                                        
+ <i class="fa fa-files-o" aria-hidden="true"></i>
+            <span>RGW</span></a>                                                           


Isn't the "official" name "Object Store" or "Object Gateway"? At least that's what the documentation refers to nowadays (albeit not consistently).

The docs do have object gateway. I think that's pretty formal, and RGW is the shortform everywhere. Just offhand, what do you prefer?

We've settled to using "Object Gateway" in our project - we found "RGW" or "RADOS Gateway" to be too non-descriptive in a Web UI.

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 · 2017-12-21T12:09:29Z

@cbodley @jcsp thank you all for the information and reviewing.

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 · 2017-12-22T07:23:33Z

@jcsp aha. cherrypy.popargs is awesome! It seems I have fixed all your comments. RGW page is more beautiful now.

liuchang0812 · 2017-12-22T07:26:42Z

src/pybind/mgr/dashboard/rgw.py

+                        'server': server,
+                        'metadata': metadata,
+                        'status': status,
+                        'url': "/rgw/{0}".format(service['id'])


seems that I missed url_prefix here

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

jcsp · 2018-01-02T17:57:44Z

Tested this locally with a vstart cluster and it looks good!

jcsp · 2018-01-02T17:58:09Z

Thank you @liuchang0812

mgr/dashboard: create RGW page

10578ba

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

mattbenjamin added the rgw label Dec 14, 2017

mattbenjamin self-assigned this Dec 14, 2017

mattbenjamin added the mgr label Dec 14, 2017

mattbenjamin requested a review from jcsp December 14, 2017 10:24

mattbenjamin requested a review from yehudasa December 14, 2017 10:27

mattbenjamin requested a review from cbodley December 15, 2017 04:49

jcsp reviewed Dec 19, 2017

View reviewed changes

LenzGr reviewed Dec 20, 2017

View reviewed changes

liuchang0812 added 3 commits December 21, 2017 17:24

mgr/dashbaord: uses fa-cloud-download as RGW's icon

c16472d

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

mgr/dashboard: remove redundant python modules re/rados

d6bc960

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

mgr/dashboard: logging when loads json failed

f5b3bb3

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 added 2 commits December 22, 2017 15:19

mgr/dashboard: simplify URL routing, /rgw/detail/<id> -> /rgw/<id>

c7e2ab5

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

rgw/dashboard: move some logic from JS to Python side, some cleanups

ffdc050

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 force-pushed the wip-dashboard-rgw branch from 9230bc5 to ffdc050 Compare December 22, 2017 07:20

liuchang0812 commented Dec 22, 2017

View reviewed changes

rgw/dashboard: supports url_prefix in rgw page

0741dbd

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

liuchang0812 changed the title ~~[rfc]mgr/dashboard: RGW page~~ mgr/dashboard: RGW page Dec 22, 2017

jcsp approved these changes Jan 2, 2018

View reviewed changes

jcsp merged commit 81fc8e0 into ceph:master Jan 2, 2018

liuchang0812 deleted the wip-dashboard-rgw branch January 3, 2018 02:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/dashboard: RGW page #19512

mgr/dashboard: RGW page #19512

liuchang0812 commented Dec 14, 2017

liuchang0812 commented Dec 14, 2017

mattbenjamin commented Dec 14, 2017

liuchang0812 commented Dec 14, 2017

liuchang0812 commented Dec 15, 2017

mattbenjamin commented Dec 15, 2017

cbodley commented Dec 15, 2017

liuchang0812 commented Dec 18, 2017

jcsp left a comment

jcsp Dec 19, 2017

jcsp Dec 19, 2017

liuchang0812 Dec 21, 2017

jcsp Dec 21, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

jcsp Dec 19, 2017

cbodley commented Dec 19, 2017

LenzGr Dec 20, 2017

mattbenjamin Dec 20, 2017

LenzGr Dec 20, 2017

liuchang0812 commented Dec 21, 2017

liuchang0812 commented Dec 22, 2017

liuchang0812 Dec 22, 2017

jcsp commented Jan 2, 2018

jcsp commented Jan 2, 2018


		log = logging.getLogger("dashboard")

		class RGWDaemons(RemoteViewCache):

mgr/dashboard: RGW page #19512

mgr/dashboard: RGW page #19512

Conversation

liuchang0812 commented Dec 14, 2017

liuchang0812 commented Dec 14, 2017

mattbenjamin commented Dec 14, 2017

liuchang0812 commented Dec 14, 2017

liuchang0812 commented Dec 15, 2017

mattbenjamin commented Dec 15, 2017

cbodley commented Dec 15, 2017

liuchang0812 commented Dec 18, 2017

jcsp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cbodley commented Dec 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuchang0812 commented Dec 21, 2017

liuchang0812 commented Dec 22, 2017

Choose a reason for hiding this comment

jcsp commented Jan 2, 2018

jcsp commented Jan 2, 2018