mgr/dashboard: CRUSH map viewer #24766

familyuu · 2018-10-26T06:50:13Z

mgr/dashboard: Add CRUSH map viewer
Fixes: http://tracker.ceph.com/issues/35684
Signed-off-by: familyuu guodan1@lenovo.com

tspmelo · 2018-10-26T14:34:55Z

I would prefer not to include jquery.
Could try to use ng2-tree instead of the bootstrap-treeview?

familyuu · 2018-10-28T07:17:36Z

@tspmelo Ok, I will try, and give the implementation as soon as possible.

votdev · 2018-10-29T08:42:54Z

Would be great to use the TableKeyValueComponent data table to display the values.

familyuu · 2018-10-29T09:16:35Z

@votdev Yeah, I think so, for the perfect display, I will implement it as soon as possible.

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.scss

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts

Devp00l · 2018-10-29T12:39:47Z

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

+      return { value: 'No nodes!' };
+    }
+
+    for (let i = nodesLength - 1; i > -1; i--) {


How about using forEach instead?

Suggested change

for (let i = nodesLength - 1; i > -1; i--) {

cephNodes.reverse().forEach((node) => {

Your suggestion is better, I have modified it.

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

LenzGr · 2018-10-30T17:15:53Z

Thanks for addressing the review feedback so far. Good progress! Displaying the metadata in table format looks much better, but I wonder if it should be displayed below the tree view, to be in line with how we display an object's detail on other pages?

I tested your latest changes:

Some comments:

Would it make sense to reverse the order in which the OSDs are sorted?
Not sure if I would display the crush weight in the tree view on the left

What's next? Do you have any other enhancements in mind, or is this good to go as the first implementation and we enhance this feature with follow-up pull requests?

familyuu · 2018-10-31T03:41:30Z

@LenzGr Thanks for your suggestion, and I have modified code in terms of your comments, including: changing the location to display the metadata, replacing the "crush weight" information with the "status" information of osds.
For the order of OSDS, however, I have to say it is a vice effect that came with the implementation of build the data structure of tree-view. As we known, we need build a special data object from the metadata returned from python controller, like this:
{ value: 'parent', children:[ { value: 'osd1' }, { value: 'osd2' }] }
For build an object like this structure, that mean, when we build a node which have children, all children node have been built already. So we need to traverse the "nodes" list from the end, and so, the order of OSDS is reverse. While, for now, I don't think this order will affect the tree view of the CRUSH map.
Finally, I think this can be the first implementation of CRUSH map, what do you think about it?

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

Devp00l · 2018-10-31T09:33:40Z

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

+    return treeNodeMap[rootNodeId];
+  }
+
+  private generateNode(node: any, treeNodeMap) {


I'm would rename it to generateNodeLeaf or generateTreeLeaf as you are not generating a node.

Hmm, I will check it and find a better name for it.

tspmelo · 2018-10-31T11:00:23Z

@LenzGr @familyuu I would prefer something like this:

This situation is a bit diferent from the tables, since we have lots of empty space to the right and the crush list will keep growing vertically.
By using the rigth side we can see the info without having to scroll.

I would prefer to have the table inside the crush panel, otherwise it seems to separate things.

LenzGr · 2018-11-01T13:43:34Z

I would prefer to have the table inside the crush panel, otherwise it seems to separate things.

Good point, @tspmelo - I'm fine with this approach. Sorry for the back-and-forth about this, @familyuu :/

LenzGr · 2018-11-01T13:52:32Z

@LenzGr Thanks for your suggestion, and I have modified code in terms of your comments, including: changing the location to display the metadata, replacing the "crush weight" information with the "status" information of osds.

Thank you!

For the order of OSDS, however, I have to say it is a vice effect that came with the implementation of build the data structure of tree-view. As we known, we need build a special data object from the metadata returned from python controller, like this:
{ value: 'parent', children:[ { value: 'osd1' }, { value: 'osd2' }] }
For build an object like this structure, that mean, when we build a node which have children, all children node have been built already. So we need to traverse the "nodes" list from the end, and so, the order of OSDS is reverse. While, for now, I don't think this order will affect the tree view of the CRUSH map.

OK, thanks for the explanation. I'm fine with keeping the current order for now, it it's too involved to sort it.

Finally, I think this can be the first implementation of CRUSH map, what do you think about it?

I agree, this is a good start. Would you mind changing the layout as suggested by @tspmelo in #24766 (comment) and adding a few tests as requested? From a functionality POV, this would be a good first step (a "CRUSH map viewer") to get merged.

familyuu · 2018-11-02T02:31:25Z

@LenzGr @tspmelo @Devp00l Hi, I've modified the code as your suggested, include: add a test for create the correct tree, put the metadata table to the left of the tree view, and change the misleading function name in the code. Please review again. Thank you for your patience and advises.

Devp00l · 2018-11-02T12:00:03Z

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

+
+    const children: any[] = [];
+    if (node.children) {
+      node.children.forEach((childId) => {


How about sorting the children by ID, using node.children.sort() ?

Suggested change

node.children.forEach((childId) => {

node.children.sort().forEach((childId) => {

Devp00l · 2018-11-02T12:06:30Z

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts

+    };
+    const data = { osd_map };
+    spyOn(dashboardService, 'getHealth').and.returnValue(of(data));
+    fixture.detectChanges();


Hi, I looked through your tests and refactored and extend them a bit, hopefully you will like my suggestion, to minimize the preparation per test.

diff --git a/src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts b/src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts index 3395120b72..704416c47e 100644 --- a/src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts +++ b/src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.spec.ts @@ -16,6 +16,7 @@ describe('CrushmapComponent', () => { let component: CrushmapComponent; let fixture: ComponentFixture<CrushmapComponent>; let debugElement: DebugElement; + configureTestBed({ imports: [HttpClientTestingModule, TreeModule, TabsModule.forRoot(), SharedModule], declarations: [CrushmapComponent], @@ -38,33 +39,52 @@ describe('CrushmapComponent', () => { expect(span.textContent).toContain(component.panelTitle); }); - it('should display "No nodes!" if ceph tree nodes is empty array', () => { - const dashboardService = debugElement.injector.get(DashboardService); - const osd_map = { tree: {} }; - const data = { osd_map }; - const getHealthSpy = spyOn(dashboardService, 'getHealth'); - getHealthSpy.and.returnValue(of(data)); - fixture.detectChanges(); + describe('tests tree', () => { + let dashboardService: DashboardService; - expect(getHealthSpy).toHaveBeenCalled(); - expect(component.tree.value).toEqual('No nodes!'); - }); - - it('should has correct tree data structure after transform the metadata', () => { - const dashboardService = debugElement.injector.get(DashboardService); - const osd_map = { - tree: { - nodes: [ - { children: [-1], type: 'root', name: 'Root', id: 1 }, - { children: [-2], type: 'host', name: 'Host', id: -1 }, - { status: 'up', type: 'osd', name: 'Osd', id: -2 } - ] - } + const prepareGetHealth = (nodes: object[]) => { + spyOn(dashboardService, 'getHealth').and.returnValue( + of({ osd_map: { tree: { nodes: nodes } } }) + ); + fixture.detectChanges(); }; - const data = { osd_map }; - spyOn(dashboardService, 'getHealth').and.returnValue(of(data)); - fixture.detectChanges(); - expect(component.tree.children[0].children[0].value).toBe('Osd (osd)--up'); + beforeEach(() => { + dashboardService = debugElement.injector.get(DashboardService); + }); + + it('should display "No nodes!" if ceph tree nodes is empty array', () => { + prepareGetHealth([]); + expect(dashboardService.getHealth).toHaveBeenCalled(); + expect(component.tree.value).toEqual('No nodes!'); + }); + + describe('with data', () => { + beforeEach(() => { + prepareGetHealth([ + { children: [-2], type: 'root', name: 'default', id: -1 }, + { children: [1, 0, 2], type: 'host', name: 'my-host', id: -2 }, + { status: 'up', type: 'osd', name: 'osd.0', id: 0 }, + { status: 'down', type: 'osd', name: 'osd.1', id: 1 }, + { status: 'up', type: 'osd', name: 'osd.2', id: 2 } + ]); + }); + + it('has a tree structure deriving from root', () => { + expect(component.tree.value).toBe('default (root)'); + }); + + it('has a tree structure with one host child with 3 children', () => { + expect(component.tree.children.length).toBe(1); + expect(component.tree.children[0].value).toBe('my-host (host)'); + expect(component.tree.children[0].children.length).toBe(3); + }); + + it('has sorted all host children by id', () => { + expect(component.tree.children[0].children[0].value).toBe('osd.0 (osd)--up'); + expect(component.tree.children[0].children[1].value).toBe('osd.1 (osd)--down'); + expect(component.tree.children[0].children[2].value).toBe('osd.2 (osd)--up'); + }); + }); }); });

Thanks for your suggestion. Actually, I am not good at write tests, so, just forgive my poor test cases, and I have modified the code as you suggested, thanks again for your advise.

LenzGr · 2018-11-02T12:31:21Z

Thank you, looks good to me. Can you please amend the git commit message to match the title and description of this PR and to include a reference to the tracker issue?

Some minor observations:

For consistency, I suggest to always print "CRUSH" in capital letters (as done in the documentation), e.g. in the breadcrumb text or heading above the tree view.
There seems to be an extra empty row on the bottom of the table?

votdev · 2018-11-02T13:25:58Z

* There seems to be an extra empty row on the bottom of the table?

This is something we can not fix in the Swimlane ngx datatable. We have many tables (e.g. in the details tables) in the UI with the same behaviour.

familyuu · 2018-11-03T13:18:23Z

Thank you, looks good to me. Can you please amend the git commit message to match the title and description of this PR and to include a reference to the tracker issue?

Ok, no problem.

Devp00l

LGTM :-)

votdev · 2018-11-05T11:00:43Z

src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts

+
+    let value: string = node.name + ' (' + node.type + ')';
+    if (node.status) {
+      value += '--' + node.status;


Is this a special Ceph convention? What about ods.1 (osd) - up or ods.1 (osd) [up]? Using colors might be the best solution in my opinion.

LenzGr · 2018-11-05T12:15:40Z

Ok, no problem.

Sorry to be such a pain, @familyuu - but the git commit message is still not in the correct format :/

Can you please amend it so it also contains a reference to the tracker issue?

mgr/dashboard: Add CRUSH map viewer
Fixes: http://tracker.ceph.com/issues/35684
Signed-off-by: familyuu <guodan1@lenovo.com>

votdev · 2018-11-05T12:24:58Z

What about this?

--- src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.html	(date 1540782268000)
+++ src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.html	(date 1541420464000)
@@ -9,7 +9,14 @@
       <div class="panel-body">
         <div class="col-sm-6 col-lg-6">
           <tree [tree]="tree"
-                (nodeSelected)="onNodeSelected($event)"></tree>
+                (nodeSelected)="onNodeSelected($event)">
+            <ng-template let-node>
+              <span class="node-name" [innerHTML]="node.value"></span>
+              <span>&nbsp;</span>
+              <span class="label"
+                    [ngClass]="{'label-success': ['in', 'up'].includes(node.status), 'label-danger': ['down', 'out'].includes(node.status)}">{{ node.status }}</span>
+            </ng-template>
+          </tree>
         </div>
         <div class="col-sm-6 col-lg-6 metadata">
           <cd-table-key-value *ngIf="metadata"
@@ -19,4 +26,4 @@
       </div>
     </div>
   </div>

--- src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts	(date 1540782268000)
+++ src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/crushmap/crushmap.component.ts	(date 1541419606000)
@@ -50,9 +50,7 @@
     const settings = { static: true };
 
     let value: string = node.name + ' (' + node.type + ')';
-    if (node.status) {
-      value += '--' + node.status;
-    }
+    let status: string = node.status;
 
     const children: any[] = [];
     if (node.children) {
@@ -60,10 +58,10 @@
         children.push(treeNodeMap[childId]);
       });
 
-      return { value, settings, id, children };
+      return { value, status, settings, id, children };
     }
 
-    return { value, settings, id };
+    return { value, status, settings, id };
   }
 
   onNodeSelected(e: NodeEvent) {

LenzGr · 2018-11-05T12:44:18Z

What about this?

Looks good to me @votdev - thanks for the suggestion. @familyuu are you fine with including this suggestion?

Please let's focus on getting the existing implementation reviewed and merged and refrain from making additional visual/functionality improvements in this PR; let's open new PRs for additional changes/enhancements instead. Thanks!

votdev · 2018-11-05T15:45:34Z

What about this?

Looks good to me @votdev - thanks for the suggestion. @familyuu are you fine with including this suggestion?

Please let's focus on getting the existing implementation reviewed and merged and refrain from making additional visual/functionality improvements in this PR; let's open new PRs for additional changes/enhancements instead. Thanks!

To get the PR merged asap i've created a separate PR with my suggestions.

Fixes: http://tracker.ceph.com/issues/35684 Signed-off-by: familyuu <guodan1@lenovo.com>

familyuu · 2018-11-06T02:39:22Z

Ok, no problem.

Sorry to be such a pain, @familyuu - but the git commit message is still not in the correct format :/

Can you please amend it so it also contains a reference to the tracker issue?
mgr/dashboard: Add CRUSH map viewer
Fixes: http://tracker.ceph.com/issues/35684
Signed-off-by: familyuu <guodan1@lenovo.com>

Oh, it looks like I misunderstood some of the things you talked about, I modified the git commit message and it looks right now. I am sorry for my carelessness.

familyuu · 2018-11-06T03:15:48Z

What about this?

Looks good to me @votdev - thanks for the suggestion. @familyuu are you fine with including this suggestion?
Please let's focus on getting the existing implementation reviewed and merged and refrain from making additional visual/functionality improvements in this PR; let's open new PRs for additional changes/enhancements instead. Thanks!

To get the PR merged asap i've created a separate PR with my suggestions.

Sounds like I don't need to add code about this topic to my PR. Thank you very much for your advice that make me more capable of developing better apps. I think I have learned a lot of skills in this progress.

LenzGr · 2018-11-06T08:08:42Z

I am sorry for my carelessness.

Nothing to be sorry about! Thanks a lot for addressing all of the suggestions and for your contribution! Much appreciated.

LenzGr requested review from tspmelo, votdev, ricardoasmarques and Devp00l October 26, 2018 13:18

LenzGr added feature needs-review dashboard needs-rebase labels Oct 26, 2018

votdev removed the needs-rebase label Oct 29, 2018

tspmelo suggested changes Oct 29, 2018

View reviewed changes

Devp00l suggested changes Oct 29, 2018

View reviewed changes

Devp00l reviewed Oct 31, 2018

View reviewed changes

LenzGr changed the title ~~mgr/dashboard: crush map~~ mgr/dashboard: CRUSH map viewer Nov 2, 2018

Devp00l reviewed Nov 2, 2018

View reviewed changes

Devp00l approved these changes Nov 5, 2018

View reviewed changes

votdev requested changes Nov 5, 2018

View reviewed changes

tspmelo approved these changes Nov 5, 2018

View reviewed changes

votdev mentioned this pull request Nov 5, 2018

mgr/dashboard: Improve CRUSH map viewer #24934

Merged

3 tasks

mgr/dashboard: Add CRUSH map viewer

53fa735

Fixes: http://tracker.ceph.com/issues/35684 Signed-off-by: familyuu <guodan1@lenovo.com>

votdev approved these changes Nov 6, 2018

View reviewed changes

LenzGr removed the needs-review label Nov 6, 2018

LenzGr merged commit 100e15f into ceph:master Nov 6, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/dashboard: CRUSH map viewer #24766

mgr/dashboard: CRUSH map viewer #24766

familyuu commented Oct 26, 2018 •

edited

Loading

tspmelo commented Oct 26, 2018

familyuu commented Oct 28, 2018

votdev commented Oct 29, 2018

familyuu commented Oct 29, 2018

Devp00l Oct 29, 2018

familyuu Oct 30, 2018

LenzGr commented Oct 30, 2018 •

edited

Loading

familyuu commented Oct 31, 2018 •

edited

Loading

Devp00l Oct 31, 2018

familyuu Oct 31, 2018

tspmelo commented Oct 31, 2018

LenzGr commented Nov 1, 2018

LenzGr commented Nov 1, 2018

familyuu commented Nov 2, 2018

Devp00l Nov 2, 2018

Devp00l Nov 2, 2018

familyuu Nov 3, 2018

LenzGr commented Nov 2, 2018

votdev commented Nov 2, 2018

familyuu commented Nov 3, 2018

Devp00l left a comment

votdev Nov 5, 2018

LenzGr commented Nov 5, 2018

votdev commented Nov 5, 2018

LenzGr commented Nov 5, 2018

votdev commented Nov 5, 2018 •

edited

Loading

familyuu commented Nov 6, 2018

familyuu commented Nov 6, 2018

LenzGr commented Nov 6, 2018

	for (let i = nodesLength - 1; i > -1; i--) {
	cephNodes.reverse().forEach((node) => {

	node.children.forEach((childId) => {
	node.children.sort().forEach((childId) => {

mgr/dashboard: CRUSH map viewer #24766

mgr/dashboard: CRUSH map viewer #24766

Conversation

familyuu commented Oct 26, 2018 • edited Loading

tspmelo commented Oct 26, 2018

familyuu commented Oct 28, 2018

votdev commented Oct 29, 2018

familyuu commented Oct 29, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LenzGr commented Oct 30, 2018 • edited Loading

familyuu commented Oct 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tspmelo commented Oct 31, 2018

LenzGr commented Nov 1, 2018

LenzGr commented Nov 1, 2018

familyuu commented Nov 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LenzGr commented Nov 2, 2018

votdev commented Nov 2, 2018

familyuu commented Nov 3, 2018

Devp00l left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LenzGr commented Nov 5, 2018

votdev commented Nov 5, 2018

LenzGr commented Nov 5, 2018

votdev commented Nov 5, 2018 • edited Loading

familyuu commented Nov 6, 2018

familyuu commented Nov 6, 2018

LenzGr commented Nov 6, 2018

familyuu commented Oct 26, 2018 •

edited

Loading

LenzGr commented Oct 30, 2018 •

edited

Loading

familyuu commented Oct 31, 2018 •

edited

Loading

votdev commented Nov 5, 2018 •

edited

Loading