mgr: dashboard GUI module #14946

dmick · 2017-05-04T03:13:56Z

Cleanup of work from @jcsp to add a ceph-mgr Python module to provide a web service that displays a simple GUI status, with one feature added to allow configuration of the listen addr and port for the server.

Configuration is done with ceph config-key put; the key names are

mgr.dashboard.server_addr
mgr.dashboard.server_port

liewegas · 2017-05-04T13:07:55Z

Since you can have multiple mgr's on different hosts, I think we need to qualify these with the mgr id. Something like

mgr/x/dashboard/server_addr
mgr/x/dashboard/server_port
mgr/y/dashboard/server_addr
mgr/y/dashboard/server_port

(The dm-crypt stuff in config-key is currently using / instead of . as delimiter; we may as well stick with that?)

dmick · 2017-05-04T18:45:35Z

ok, will hack

dmick · 2017-05-06T03:07:54Z

Working on Yet Another Rebase

dmick · 2017-05-08T21:46:10Z

Removing some commits that had made it into master in the meantime made this merge more cleanly; this branch appears to pass dumb functional tests again.

dmick · 2017-05-08T22:52:39Z

uhh....there are apparently a few more files than I wanted here. Edit: ah, no, all the JS and CSS. My bad.

jcsp · 2017-05-10T12:57:46Z

@dmick you could probably cull a lot of the AdminLTE guff, I think there are various optional bits (themes/plugins) in there where we only need the files for the ones we use.

dmick · 2017-05-16T02:27:04Z

rebased again

liewegas · 2017-05-17T22:08:39Z

weird thing on this run: http://pulpito.ceph.com/sage-2017-05-17_15:24:19-rados-wip-sage-testing---basic-smithi/1189010

smithi055 and smithi067 are both trusty, but the cherrypy error only came up on one of them. not a package dependency for deb yet?

liewegas · 2017-05-17T22:13:56Z

http://pulpito.ceph.com/sage-2017-05-17_15:24:19-rados-wip-sage-testing---basic-smithi/
looks fine in rados suite, fwiw.

tchaikov

yeah, we can remove the unused AdminLTE bits later on. and move this into a separated package probably.

tchaikov · 2017-05-18T14:49:24Z

src/mgr/PyState.cc

@@ -196,10 +196,10 @@ ceph_config_get(PyObject *self, PyObject *args)
  std::string value;
  bool found = global_handle->get_config(handle, what, &value);
  if (found) {
-    derr << "Found" << dendl;
+    derr << "ceph_config_get " << what << " found: " << value.c_str() << dendl;


could you use __func__ instead here?

tchaikov · 2017-05-18T14:49:32Z

src/mgr/PyState.cc

    return PyString_FromString(value.c_str());
  } else {
-    derr << "Not found" << dendl;
+    derr << "ceph_config_get " << what << " not found " << dendl;


tchaikov · 2017-05-18T14:51:49Z

@dmick please feel free to merge this when you feel appropriate.

...so that mgr modules can get at it. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: John Spray <john.spray@redhat.com>

A read only display of some of the cluster's state, plus some cephfs/rbd/osd pages that demonstrate how to go read state remotely from daemons. Signed-off-by: John Spray <john.spray@redhat.com>

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Otherwise, the incomplete ServeThread type causes problems for others who include PyModules.h Signed-off-by: Dan Mick <dan.mick@redhat.com>

…y '/' dm_crypt config-key usage has established '/' as the separator; keep that as a convention for now Signed-off-by: Dan Mick <dan.mick@redhat.com>

Signed-off-by: Dan Mick <dan.mick@redhat.com>

dmick · 2017-05-19T05:17:09Z

Dependencies added, very simple smoke test added to make check

liewegas · 2017-05-19T13:14:59Z

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:45: run:  sleep 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:39: run:  [[ 4 < 30 ]]
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:47: run:  curl -s http://127.0.0.1:7001/toplevel_data
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:48: run:  jq .health.overall_status
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:48: run:  grep HEALTH_
parse error: Invalid numeric literal at line 1, column 10
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mgr/mgr-dashboard-smoke.sh:48: run:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:1692: main:  display_logs td/mgr-dashboard-smoke

dmick · 2017-05-19T17:54:20Z

Yep; internal failure in dashboard. Trying to reproduce now.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

dmick · 2017-05-19T21:49:28Z

It was intermittent; sometimes after the mgr said 'available', dashboard still wasn't quite ready. Added a second retry loop; running with ctest --output-on-failure --repeat-until-fail 100 -R mgr-dash looks solid now

liewegas · 2017-05-19T21:59:35Z

excellent, thanks!

dmick · 2017-05-19T22:36:42Z

Sorry @tchaikov, missed your comments about func; I wasn't ignoring you on purpose

tchaikov · 2017-05-20T02:06:02Z

@dmick no worries, we can always improve it once we are at it.

dmick requested a review from jcsp May 4, 2017 03:14

dmick force-pushed the wip-mgr-dashboard branch from ba7c2f6 to 38a1847 Compare May 4, 2017 03:28

smithfarm added feature mgr labels May 4, 2017

liewegas changed the title ~~Manager dashboard GUI module~~ mgr: dashboard GUI module May 4, 2017

dmick force-pushed the wip-mgr-dashboard branch from 781c6c3 to e5d6791 Compare May 4, 2017 21:22

dmick changed the title ~~mgr: dashboard GUI module~~ DNM: mgr: dashboard GUI module May 4, 2017

dmick force-pushed the wip-mgr-dashboard branch 5 times, most recently from ac10336 to 1217a29 Compare May 4, 2017 23:53

dmick changed the title ~~DNM: mgr: dashboard GUI module~~ mgr: dashboard GUI module May 5, 2017

tserong mentioned this pull request May 5, 2017

mgr: load modules in separate python sub-interpreters #14971

Merged

dmick force-pushed the wip-mgr-dashboard branch from 1217a29 to 36686ce Compare May 8, 2017 18:26

dmick changed the title ~~mgr: dashboard GUI module~~ DNM: mgr: dashboard GUI module May 8, 2017

dmick force-pushed the wip-mgr-dashboard branch from 36686ce to 8bc3fdb Compare May 8, 2017 22:53

dmick changed the title ~~DNM: mgr: dashboard GUI module~~ mgr: dashboard GUI module May 8, 2017

liewegas added the needs-qa label May 15, 2017

dmick force-pushed the wip-mgr-dashboard branch from 8bc3fdb to ea3e37c Compare May 16, 2017 02:27

liewegas added the wip-sage-testing label May 16, 2017

tchaikov approved these changes May 18, 2017

View reviewed changes

liewegas added the wip-sage-testing label May 18, 2017

tchaikov approved these changes May 18, 2017

View reviewed changes

John Spray and others added 11 commits May 18, 2017 22:09

osd: wire up "histogram dump" to tell

80dc6a1

...so that mgr modules can get at it. Signed-off-by: John Spray <john.spray@redhat.com>

vstart: enable gui plugin

b83fe0e

Signed-off-by: John Spray <john.spray@redhat.com>

pybind/mgr: web dashboard

c249366

A read only display of some of the cluster's state, plus some cephfs/rbd/osd pages that demonstrate how to go read state remotely from daemons. Signed-off-by: John Spray <john.spray@redhat.com>

ServeThread: add 'running' flag, don't notify non-running threads

f4bc4f6

Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph_syn.cc, client/SyntheticClient.cc: Client -> StandaloneClient

e146b85

Signed-off-by: Dan Mick <dan.mick@redhat.com>

client/SyntheticClient.cc: filer is now a std::unique_ptr

8abd69d

Signed-off-by: Dan Mick <dan.mick@redhat.com>

mgr/PyModules.cc,PyState.cc: improve config access debug logging

f2d29d2

Signed-off-by: Dan Mick <dan.mick@redhat.com>

pybind/mgr/dashboard/module.py: allow custom server addr, port

f69ce19

Signed-off-by: Dan Mick <dan.mick@redhat.com>

mgr/PyModules: move constructor definition to .cc

987211a

Otherwise, the incomplete ServeThread type causes problems for others who include PyModules.h Signed-off-by: Dan Mick <dan.mick@redhat.com>

mgr: change config_prefix to be 'mgr/<id>/' and separate handle/key b…

dc15cd6

…y '/' dm_crypt config-key usage has established '/' as the separator; keep that as a convention for now Signed-off-by: Dan Mick <dan.mick@redhat.com>

ceph.spec.in, debian/control: build/runtime dependency for cherrypy

bd6f2d9

Signed-off-by: Dan Mick <dan.mick@redhat.com>

dmick force-pushed the wip-mgr-dashboard branch from c8ab383 to 719e7e6 Compare May 19, 2017 05:10

liewegas added wip-sage2-testing and removed wip-sage-testing wip-sage2-testing labels May 19, 2017

test: add mgr/ and smoke test for mgr dashboard

10b1fa2

Signed-off-by: Dan Mick <dan.mick@redhat.com>

dmick force-pushed the wip-mgr-dashboard branch from 719e7e6 to 10b1fa2 Compare May 19, 2017 21:47

liewegas merged commit de6c0bd into ceph:master May 19, 2017

dmick deleted the wip-mgr-dashboard branch May 19, 2017 22:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr: dashboard GUI module #14946

mgr: dashboard GUI module #14946

dmick commented May 4, 2017

liewegas commented May 4, 2017

dmick commented May 4, 2017

dmick commented May 6, 2017

dmick commented May 8, 2017

dmick commented May 8, 2017 •

edited

jcsp commented May 10, 2017

dmick commented May 16, 2017

liewegas commented May 17, 2017

liewegas commented May 17, 2017

tchaikov left a comment

tchaikov May 18, 2017

tchaikov May 18, 2017

tchaikov commented May 18, 2017

dmick commented May 19, 2017

liewegas commented May 19, 2017

dmick commented May 19, 2017

dmick commented May 19, 2017

liewegas commented May 19, 2017

dmick commented May 19, 2017

tchaikov commented May 20, 2017

mgr: dashboard GUI module #14946

mgr: dashboard GUI module #14946

Conversation

dmick commented May 4, 2017

liewegas commented May 4, 2017

dmick commented May 4, 2017

dmick commented May 6, 2017

dmick commented May 8, 2017

dmick commented May 8, 2017 • edited

jcsp commented May 10, 2017

dmick commented May 16, 2017

liewegas commented May 17, 2017

liewegas commented May 17, 2017

tchaikov left a comment

Choose a reason for hiding this comment

tchaikov May 18, 2017

Choose a reason for hiding this comment

tchaikov May 18, 2017

Choose a reason for hiding this comment

tchaikov commented May 18, 2017

dmick commented May 19, 2017

liewegas commented May 19, 2017

dmick commented May 19, 2017

dmick commented May 19, 2017

liewegas commented May 19, 2017

dmick commented May 19, 2017

tchaikov commented May 20, 2017

dmick commented May 8, 2017 •

edited