mgr/dashboard: list services and daemons #33531

bk201 · 2020-02-25T10:14:14Z

Display services and daemons in the cluster/services page.
Display daemons in the cluster/hosts/host-detail page (Daemons tab).

This PR also partially addresses https://tracker.ceph.com/issues/43165:
The endpoint /api/orchestrator/service is removed.

Create new endpoints:

/api/service: listing all services in the Ceph cluster.
/api/service/<service_name>/daemons: listing daemons for a
service. e.g. daemons of OSD.
/api/host/<hostname>/daemons: listing daemons of a host.

Fixes: https://tracker.ceph.com/issues/44221

Signed-off-by: Kiefer Chang kiefer.chang@suse.com

How to test

This PR needs to be tested in cephadm spawned cluster.

Screenshots

Cluster->Services

Cluster->Hosts (Daemons tab)

Checklist

References tracker ticket
Updates documentation if necessary
Includes tests for new functionality or reproducer for bug

Show available Jenkins commands

jenkins retest this please
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard backend
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox

sebastian-philipp

ack

src/pybind/mgr/dashboard/controllers/host.py

src/pybind/mgr/dashboard/controllers/service.py

votdev · 2020-02-25T12:14:26Z

Is it possible to add service information to dummy_data.json? This will make it easier to test the PR and such things in future.

bk201 · 2020-02-27T07:56:28Z

Is it possible to add service information to dummy_data.json? This will make it easier to test the PR and such things in future.

Added. I also refactor test_orchestrator a bit.

votdev · 2020-02-27T08:34:38Z

src/pybind/mgr/dashboard/controllers/orchestrator.py

@@ -36,7 +36,7 @@ def get_device_osd_map():
             }
    :rtype: dict
    """
-    result = {}
+    result: dict = {}


This is PY3 type hinting syntax. Can we use that or should we use # type: dict instead? This may be a concern when backporting Octopus PRs to a PY2 based release.

@sebastian-philipp What do you think?

Thanks for bringing this up. Orchestrator integration will not be backported, but considering we are encouraged to add typing info in Dashboard (are we?), we should have a guideline to follow.

PY3 type hinting syntax will make backport a nightmare.

In today's standup we agree:

Use the PY3-only syntax if the codes are not going to be backported.

Stick to the comments way # type: xxxx if codes are going to be backported potentially.

If you don't know, stick to the comments way.

bk201 · 2020-02-27T10:39:27Z

jenkins test dashboard

callithea · 2020-02-27T14:05:46Z

jenkins test dashboard

- Display services and daemons in the cluster/services page. - Display daemons in the cluster/hosts/host-detail page (Daemons tab). This PR also partially addresses https://tracker.ceph.com/issues/43165: The endpoint `/api/orchestrator/service` is removed. Create new endpoints: - `/api/service`: listing all services in the Ceph cluster. - `/api/service/<service_name>/daemons`: listing daemons for a service. e.g. daemons of OSD. - `/api/host/<hostname>/daemons`: listing daemons of a host. Fixes: https://tracker.ceph.com/issues/44221 Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

- Refactor list_daemons: return ceph-xxx processes on host. - Refactor describe_service: return services by grouping ceph-xxx processes on host. - Add dummy data. Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

tspmelo · 2020-03-02T20:36:19Z

jenkins test make check

tspmelo · 2020-03-02T20:36:47Z

jenkins test dashboard

bk201 · 2020-03-03T01:59:55Z

jenkins test dashboard

jschmid1 · 2020-03-03T08:32:22Z

there was a column newly added with #33553 (SPEC) which you might include here?

bk201 · 2020-03-03T10:13:31Z

there was a column newly added with #33553 (SPEC) which you might include here?

Sure. But need this PR to display the value: #33685

tspmelo

lgtm

bk201 · 2020-03-04T03:24:21Z

there was a column newly added with #33553 (SPEC) which you might include here?

Sure. But need this PR to display the value: #33685

Let's add the Placement column in another PR and get this merged first.
Created an issue to track: https://tracker.ceph.com/issues/44404

liewegas · 2020-03-04T12:29:47Z

The spec_presence was changed to an actual spec in #33667, merged yesterday

liewegas · 2020-03-05T13:16:45Z

these tests need to be fixed:

    def test_ps(self):
        ret = self._orch_cmd("ps")
        self.assertIn("ceph-mgr", ret)

    def test_ps_json(self):
        ret = self._orch_cmd("ps", "--format", "json")
        self.assertIsInstance(json.loads(ret), list)
        self.assertIn("ceph-mgr", ret)

now that the ps implementation is not bonkers it should check for mgr not ceph-mgr

I didn't realize test_orchestator was fabricating a result for ps aux... eek!

http://pulpito.ceph.com/sage-2020-03-05_03:05:51-rados-wip-sage2-testing-2020-03-04-1543-distro-basic-smithi/4826342

The ps output names daemons like 'type.foo', e.g., 'mgr.x'. Now that the test_orchestrator impl is less bonkers this needs to be adjusted to match reality. Signed-off-by: Sage Weil <sage@redhat.com>

bk201 added bug-fix dashboard labels Feb 25, 2020

bk201 requested a review from a team as a code owner February 25, 2020 10:14

sebastian-philipp reviewed Feb 25, 2020

View reviewed changes

votdev requested changes Feb 25, 2020

View reviewed changes

LenzGr added this to the octopus milestone Feb 26, 2020

LenzGr added the needs-review label Feb 26, 2020

bk201 force-pushed the wip-44221 branch from 175b3de to a56eb3f Compare February 27, 2020 07:54

bk201 requested a review from a team as a code owner February 27, 2020 07:54

votdev reviewed Feb 27, 2020

View reviewed changes

bk201 added 2 commits March 2, 2020 10:20

mgr/test_orchestrator: refactor listing services/daemons

229c081

- Refactor list_daemons: return ceph-xxx processes on host. - Refactor describe_service: return services by grouping ceph-xxx processes on host. - Add dummy data. Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

bk201 force-pushed the wip-44221 branch from a56eb3f to 229c081 Compare March 2, 2020 04:20

bk201 added the orchestrator label Mar 2, 2020

tspmelo approved these changes Mar 3, 2020

View reviewed changes

liewegas added needs-qa wip-sage2-testing labels Mar 4, 2020

liewegas mentioned this pull request Mar 4, 2020

mgr/dashboard: support multiple DriveGroups when creating OSDs #32678

Merged

5 tasks

liewegas removed needs-qa wip-sage2-testing labels Mar 5, 2020

qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests

bb126cf

The ps output names daemons like 'type.foo', e.g., 'mgr.x'. Now that the test_orchestrator impl is less bonkers this needs to be adjusted to match reality. Signed-off-by: Sage Weil <sage@redhat.com>

liewegas added the wip-sage2-testing label Mar 5, 2020

votdev approved these changes Mar 5, 2020

View reviewed changes

liewegas merged commit 8a84dde into ceph:master Mar 5, 2020

bk201 deleted the wip-44221 branch March 11, 2020 03:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/dashboard: list services and daemons #33531

mgr/dashboard: list services and daemons #33531

bk201 commented Feb 25, 2020

sebastian-philipp left a comment

votdev commented Feb 25, 2020 •

edited

bk201 commented Feb 27, 2020

votdev Feb 27, 2020

bk201 Feb 27, 2020

bk201 Feb 27, 2020

bk201 commented Feb 27, 2020

callithea commented Feb 27, 2020

tspmelo commented Mar 2, 2020

tspmelo commented Mar 2, 2020

bk201 commented Mar 3, 2020

jschmid1 commented Mar 3, 2020 •

edited

bk201 commented Mar 3, 2020

tspmelo left a comment

bk201 commented Mar 4, 2020

liewegas commented Mar 4, 2020

liewegas commented Mar 5, 2020 •

edited

mgr/dashboard: list services and daemons #33531

mgr/dashboard: list services and daemons #33531

Conversation

bk201 commented Feb 25, 2020

How to test

Screenshots

Checklist

sebastian-philipp left a comment

Choose a reason for hiding this comment

votdev commented Feb 25, 2020 • edited

bk201 commented Feb 27, 2020

votdev Feb 27, 2020

Choose a reason for hiding this comment

bk201 Feb 27, 2020

Choose a reason for hiding this comment

bk201 Feb 27, 2020

Choose a reason for hiding this comment

bk201 commented Feb 27, 2020

callithea commented Feb 27, 2020

tspmelo commented Mar 2, 2020

tspmelo commented Mar 2, 2020

bk201 commented Mar 3, 2020

jschmid1 commented Mar 3, 2020 • edited

bk201 commented Mar 3, 2020

tspmelo left a comment

Choose a reason for hiding this comment

bk201 commented Mar 4, 2020

liewegas commented Mar 4, 2020

liewegas commented Mar 5, 2020 • edited

votdev commented Feb 25, 2020 •

edited

jschmid1 commented Mar 3, 2020 •

edited

liewegas commented Mar 5, 2020 •

edited