Skip to content

Resource Usage UI#1592

Merged
ssalinas merged 42 commits intomasterfrom
resource-usage-ui
Jul 24, 2017
Merged

Resource Usage UI#1592
ssalinas merged 42 commits intomasterfrom
resource-usage-ui

Conversation

@kwm4385
Copy link
Copy Markdown
Contributor

@kwm4385 kwm4385 commented Jul 14, 2017

New Features

  • Cluster-level utilization data added to /utilization page (formerly /slave-usage). Max over/under CPU and memory numbers are linked to the corresponding request. Colored sections of the breakdown link to the appropriate filter on the Requests page.
    local hubteamqa com-3334-singularity-ui-utilization
  • Resource Usage section on Request Detail page added underneath Task History. Will be shown open by default if CPU has been over-allocated in the past 24 hours.
    image
  • Over/under CPU/memory filters on the Requests page allow searching of requests based on resource utilization.

Modifications

  • /slave-usage page has been renamed to /utilization. The tiles within the Slave Health section have had their details replaced by popovers to eliminate them being covered by the tooltip component. Overall layout was improved and code refactored to better align with project standards.

TODO:

  • Add filters for utilization levels on Requests page
  • Display 24-hr total CPU/mem stats on utilization page

@kwm4385 kwm4385 added the UI label Jul 14, 2017
@ssalinas ssalinas modified the milestone: 0.17.0 Jul 14, 2017
@kwm4385 kwm4385 requested a review from ssalinas July 14, 2017 21:07
@kwm4385
Copy link
Copy Markdown
Contributor Author

kwm4385 commented Jul 14, 2017

/cc @darcatron

@kwm4385
Copy link
Copy Markdown
Contributor Author

kwm4385 commented Jul 17, 2017

I changed the colors used by the bar chart/Breakdown component to use the bootstrap contextual backgrounds (http://getbootstrap.com/css/#helper-classes-backgrounds) when possible, supplemented by similar generic colors:
image

Status Page:
image

These match much better with our newer design elements.

As far as what information they are conveying on the utilization page, each section is the actual number of requests that are under or over utilizing CPU/memory out of the total number of requests. I'm also adding a section showing the 24 average for overall cpu/mem utilization.

@kwm4385
Copy link
Copy Markdown
Contributor Author

kwm4385 commented Jul 17, 2017

Top section now includes data from

// across all slaves
long totalMemBytesUsed;
long totalMemBytesAvailable;
double totalCpuUsed;
double totalCpuAvailable;

image

@kwm4385 kwm4385 changed the title (WIP) Resource Usage UI Resource Usage UI Jul 17, 2017
<div className="col-md-10">
<div className="row">
<div className="col-md-3">
<div className="aggregate">
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good thing to break out into a separate component.

import Utils from '../../../utils';
import { STAT_NAMES, HUNDREDTHS_PLACE } from '../Constants';

const getPctSlaveUsage = (slaves, slaveUsages, usageCallback, resourceCallback) => {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mapper might be a more descriptive suffix than Callback for the parameter names. You could also avoid passing around the slaves and slaveUsages parameters around with

const getPctSlaveUsage = (usageMapper, resourceMapper) => (slaves, slaveUsages) => { //...}

const getCpuUtilizationPct = getPctSlaveUsage(
  usage => usage.cpusUsed,
  slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.cpusUsedStat)
);

const getMemUtilizationPct = getPctSlaveUsage(
  usage => usage.memoryBytesUsed,
  slave => Utils.getMaxAvailableResource(slave, STAT_NAMES.memoryBytesUsedStat)
);

const totalResource = slaves.map(resourceCallback)
.reduce((acc, val) => acc + parseFloat(val), 0);

return Utils.roundTo((totalUsage / totalResource) * 100, HUNDREDTHS_PLACE);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line might be good to abstract out into a helper function since it does one very specific thing.

<h4>Current</h4>
<div className="row">
<div className="col-xs-12">
<div className="aggregate graph col-xs-2">
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very similar to ClusterAggregates.jsx. I'd strongly suggest creating a new component.

};

const SlaveResourceHealthMenuItems = ({stats}) => {
stats = _.filter(stats, (stat) => _.values(STAT_NAMES).includes(stat.name));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're going to use includes (ES2016), I'd suggest that you don't use _.contains elsewhere in your code. You can also directly use Array.prototype.filter

used={utilization.cpuUsed / utilization.numTasks}
style={utilization.cpuUsed >= utilization.cpuReserved ? 'danger' : null}
>
<p>{Utils.roundTo(utilization.cpuUsed / utilization.numTasks, 2)} of {utilization.cpuReserved / utilization.numTasks} CPU reserved</p>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move these percentage calculations into a separate utility function instead of using the magic number 2 all over the place.


return utilization ? (
<CollapsableSection id="request-utilization" title="Resource usage" subtitle="(past 24 hours)" defaultExpanded={isCpuOverAllocated}>
{isFetching ? <div className="page-loader fixed" /> : attributes}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loading component would be good to abstract out too.

<NavItem
key={index}
className={classNames({'separator-pill': _.contains([3, 5], index)})}
className={classNames({'separator-pill': _.contains([3, 5, 7], index)})}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

???

margin-left : 10px
width : 160px

small
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would override the color of <small> tags everywhere. If you actually want to do that, move this rule out of the slaveUsage.styl, otherwise scope this override to just this page.

@kwm4385 kwm4385 added the hs_qa label Jul 19, 2017
@ssalinas
Copy link
Copy Markdown
Contributor

This one is looking good. Already found a few requests that were really over-allocated using it. Going to merge so @baconmania can work off of some of this to eventually add disk utilization details once we gather them as well

@ssalinas ssalinas merged commit 8730005 into master Jul 24, 2017
@ssalinas ssalinas deleted the resource-usage-ui branch July 24, 2017 16:46
@baconmania baconmania modified the milestones: 0.18.0, 0.17.0 Sep 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants