Resource Usage UI #1592

kwm4385 · 2017-07-14T19:38:28Z

New Features

Cluster-level utilization data added to /utilization page (formerly /slave-usage). Max over/under CPU and memory numbers are linked to the corresponding request. Colored sections of the breakdown link to the appropriate filter on the Requests page.
Resource Usage section on Request Detail page added underneath Task History. Will be shown open by default if CPU has been over-allocated in the past 24 hours.
Over/under CPU/memory filters on the Requests page allow searching of requests based on resource utilization.

Modifications

/slave-usage page has been renamed to /utilization. The tiles within the Slave Health section have had their details replaced by popovers to eliminate them being covered by the tooltip component. Overall layout was improved and code refactored to better align with project standards.

TODO:

Add filters for utilization levels on Requests page
Display 24-hr total CPU/mem stats on utilization page

kwm4385 · 2017-07-14T21:07:56Z

/cc @darcatron

kwm4385 · 2017-07-17T17:04:52Z

I changed the colors used by the bar chart/Breakdown component to use the bootstrap contextual backgrounds (http://getbootstrap.com/css/#helper-classes-backgrounds) when possible, supplemented by similar generic colors:

Status Page:

These match much better with our newer design elements.

As far as what information they are conveying on the utilization page, each section is the actual number of requests that are under or over utilizing CPU/memory out of the total number of requests. I'm also adding a section showing the 24 average for overall cpu/mem utilization.

kwm4385 · 2017-07-17T17:44:36Z

Top section now includes data from

// across all slaves
long totalMemBytesUsed;
long totalMemBytesAvailable;
double totalCpuUsed;
double totalCpuAvailable;

andyhuang91 · 2017-07-17T20:28:48Z

SingularityUI/app/components/machines/usage/ClusterAggregates.jsx

+        <div className="col-md-10">
+          <div className="row">
+            <div className="col-md-3">
+              <div className="aggregate">


This seems like a good thing to break out into a separate component.

andyhuang91 · 2017-07-17T20:43:59Z

SingularityUI/app/components/machines/usage/SlaveAggregates.jsx

+import Utils from '../../../utils';
+import { STAT_NAMES, HUNDREDTHS_PLACE } from '../Constants';
+
+const getPctSlaveUsage = (slaves, slaveUsages, usageCallback, resourceCallback) => {


Mapper might be a more descriptive suffix than Callback for the parameter names. You could also avoid passing around the slaves and slaveUsages parameters around with

const getPctSlaveUsage = (usageMapper, resourceMapper) => (slaves, slaveUsages) => { //...} const getCpuUtilizationPct = getPctSlaveUsage( usage => usage.cpusUsed, slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.cpusUsedStat) ); const getMemUtilizationPct = getPctSlaveUsage( usage => usage.memoryBytesUsed, slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.memoryBytesUsedStat) );

andyhuang91 · 2017-07-17T20:45:25Z

SingularityUI/app/components/machines/usage/SlaveAggregates.jsx

+  const totalResource = slaves.map(resourceCallback)
+                              .reduce((acc, val) => acc + parseFloat(val), 0);
+
+  return Utils.roundTo((totalUsage / totalResource) * 100, HUNDREDTHS_PLACE);


This line might be good to abstract out into a helper function since it does one very specific thing.

andyhuang91 · 2017-07-17T20:48:38Z

SingularityUI/app/components/machines/usage/SlaveAggregates.jsx

+        <h4>Current</h4>
+        <div className="row">
+          <div className="col-xs-12">
+            <div className="aggregate graph col-xs-2">


This is very similar to ClusterAggregates.jsx. I'd strongly suggest creating a new component.

andyhuang91 · 2017-07-17T20:50:45Z

SingularityUI/app/components/machines/usage/SlaveResourceHealthMenuItems.js

@@ -56,6 +56,7 @@ const maybeLink = (name, value) => {
 };

 const SlaveResourceHealthMenuItems = ({stats}) => {
+  stats = _.filter(stats, (stat) => _.values(STAT_NAMES).includes(stat.name));


If you're going to use includes (ES2016), I'd suggest that you don't use _.contains elsewhere in your code. You can also directly use Array.prototype.filter

andyhuang91 · 2017-07-17T20:52:46Z

SingularityUI/app/components/requestDetail/RequestUtilization.jsx

+            used={utilization.cpuUsed / utilization.numTasks}
+            style={utilization.cpuUsed >= utilization.cpuReserved ? 'danger' : null}
+          >
+            <p>{Utils.roundTo(utilization.cpuUsed / utilization.numTasks, 2)} of {utilization.cpuReserved / utilization.numTasks} CPU reserved</p>


Move these percentage calculations into a separate utility function instead of using the magic number 2 all over the place.

andyhuang91 · 2017-07-17T20:54:09Z

SingularityUI/app/components/requestDetail/RequestUtilization.jsx

+
+  return utilization ? (
+    <CollapsableSection id="request-utilization" title="Resource usage" subtitle="(past 24 hours)" defaultExpanded={isCpuOverAllocated}>
+      {isFetching ? <div className="page-loader fixed" /> : attributes}


This loading component would be good to abstract out too.

andyhuang91 · 2017-07-17T20:54:50Z

SingularityUI/app/components/requests/RequestFilters.jsx

@@ -80,7 +92,7 @@ export default class RequestFilters extends React.Component {
      return (
        <NavItem
          key={index}
-          className={classNames({'separator-pill': _.contains([3, 5], index)})}
+          className={classNames({'separator-pill': _.contains([3, 5, 7], index)})}


andyhuang91 · 2017-07-17T20:57:42Z

SingularityUI/app/styles/stylus/slaveUsage.styl

 	margin-left : 10px
 	width : 160px

+small


This would override the color of <small> tags everywhere. If you actually want to do that, move this rule out of the slaveUsage.styl, otherwise scope this override to just this page.

css

'

ssalinas · 2017-07-24T16:46:51Z

This one is looking good. Already found a few requests that were really over-allocated using it. Going to merge so @baconmania can work off of some of this to eventually add disk utilization details once we gather them as well

kwm4385 added 16 commits July 13, 2017 13:47

Fetch utilization and add section to request page

414aaaf

Use table for min/max values

3bafab7

Group cpu and mem

d5cd329

Red progressbar if overutilized

31ee882

Fix null cases

5cb660a

Simplify slave resource health into popover instead of dropdown

44d22c9

Remove focus outline

3d6ffef

Refactor aggregate css

7385fcb

Add utilization aggregates

b6283ae

copy

a2ea121

Move slave tooltip to top

54a24c0

Shorten host name in popover title

8fb08f2

Reposition request section

5464ec8

Add breakdowns for request utilizations

4d93c37

Add utilization stats

5332965

Add min/max data

9e039cc

kwm4385 added the UI label Jul 14, 2017

Cpu -> cpus

bfc0cb4

kwm4385 added the hs_staging label Jul 14, 2017

ssalinas modified the milestone: 0.17.0 Jul 14, 2017

kwm4385 added 6 commits July 14, 2017 15:50

Reorder breakdown

3af3fdc

No colon

acc7a32

Merge branch 'master' into resource-usage-ui

6d0b2c6

Add under and over utilized cpu requests filters

789da01

Make breakdown link optional

b8efb46

Add underutilized memory request filter

e6c7d7e

kwm4385 requested a review from ssalinas July 14, 2017 21:07

petrovicboban mentioned this pull request Jul 15, 2017

Resource Usage bug #1509

Closed

kwm4385 added 3 commits July 17, 2017 13:06

Use green for normal requests

7d9e05d

Generify color names and move to boostrap-tweaks

c4b8439

Add avg slave usage

f4c8004

Fix rounding

1ca6c2a

kwm4385 changed the title ~~(WIP) Resource Usage UI~~ Resource Usage UI Jul 17, 2017

Fix devtools

576b679

kwm4385 mentioned this pull request Jul 17, 2017

Fix check if redux devtools exist #1596

Closed

andyhuang91 reviewed Jul 17, 2017

View reviewed changes

kwm4385 added 10 commits July 18, 2017 09:32

Use redux devtools composer

25bf7ca

Abstract out reusables

c300e0d

Scope last updated

77e8e3e

css

Style fixes and factor out loader component

a541877

Use labeledcolumn

b04c2f7

Move separator to state definition

eb04074

zero is an acceptable value

13ee5e2

Don't show progressbar if no CPU allocation

a906220

Create aggregate component

e2784c1

lint

ffed8e6

kwm4385 added the hs_qa label Jul 19, 2017

kwm4385 added 2 commits July 20, 2017 10:56

Gate redux extension behind localStorage.enableReduxExtension

f4775ad

'

wrong logger version

8cd518a

ssalinas approved these changes Jul 21, 2017

View reviewed changes

kwm4385 added the hs_stable label Jul 21, 2017

ssalinas merged commit 8730005 into master Jul 24, 2017

ssalinas deleted the resource-usage-ui branch July 24, 2017 16:46

baconmania modified the milestones: 0.18.0, 0.17.0 Sep 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resource Usage UI #1592

Resource Usage UI #1592

kwm4385 commented Jul 14, 2017 •

edited

Loading

kwm4385 commented Jul 14, 2017

kwm4385 commented Jul 17, 2017 •

edited

Loading

kwm4385 commented Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

andyhuang91 Jul 17, 2017

ssalinas commented Jul 24, 2017

Resource Usage UI #1592

Resource Usage UI #1592

Conversation

kwm4385 commented Jul 14, 2017 • edited Loading

New Features

Modifications

TODO:

kwm4385 commented Jul 14, 2017

kwm4385 commented Jul 17, 2017 • edited Loading

kwm4385 commented Jul 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssalinas commented Jul 24, 2017

kwm4385 commented Jul 14, 2017 •

edited

Loading

kwm4385 commented Jul 17, 2017 •

edited

Loading